I am creating a web stock apps.
I want to read this Excel or Google Sheet File in this link:
Link To Report
Without need to download the file first.
Is it possible for PHP to read and choose the sheet then read entire row and column in online ?
What library I should use ?
I try to use phpspreadsheet but I cannot make it. Not sure if i do it wrong or not.
Also i try to use Google Sheet API. But in the link i cannot find the google spreadsheet id.
So what is the solution in here to read the excel or google spreadsheet online using php ?
Update :
Suggest from #ADyson to use a same header to get the data download by php.
So this is my code :
function dfCurl($url){
$ch = curl_init($url);
$dir = '../';
$fileName = basename($url);
$saveFilePath = $dir . $fileName;
$fp = fopen($saveFilePath, 'wb');
$headers = [
':authority: www.idx.co.id',
':method: GET',
':path: /Portals/0/StaticData/ListedCompanies/Corporate_Actions/New_Info_JSX/Jenis_Informasi/01_Laporan_Keuangan/02_Soft_Copy_Laporan_Keuangan//Laporan%20Keuangan%20Tahun%202021/TW1/AALI/FinancialStatement-2021-I-AALI.xlsx',
':scheme: https',
'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'accept-encoding: gzip, deflate, br',
'accept-language: en-AU,en;q=0.9,id-ID;q=0.8,id;q=0.7,zh-CN;q=0.6,zh;q=0.5,ja-JP;q=0.4,ja;q=0.3,en-GB;q=0.2,en-US;q=0.1',
'cache-control: no-cache',
'cookie: _ga=GA1.3.1396311216.1625813381; skipFeedback=1; __cf_bm=mF3lXNPMKeLe1xFzY5eKQM1TFN.yz7lxgWMJEYWBcjA-1635412963-0-AUMv9HWMpGhIN6c7eDVT++ok0dYE1NL+PyRJBpNUfZviT8bNS5Zm4UQjAm7gUJad4Qv7h+Q9ak+u/Q18tGDx0pU=',
'pragma: no-cache',
'referer: https://www.idx.co.id/perusahaan-tercatat/laporan-keuangan-dan-tahunan/',
'sec-ch-ua: "Google Chrome";v="95", "Chromium";v="95", ";Not A Brand";v="99"',
'sec-ch-ua-mobile: ?1',
'sec-ch-ua-platform: "Android"',
'sec-fetch-dest: document',
'sec-fetch-mode: navigate',
'sec-fetch-site: same-origin',
'sec-fetch-user: ?1',
'upgrade-insecure-requests: 1',
'user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Mobile Safari/537.36'
];
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, $headers);
curl_exec($ch);
curl_close($ch);
fclose($fp);
}
And I call using :
public function Export() {
// Initialize a file URL to the variable
$url = 'https://www.idx.co.id/Portals/0/StaticData/ListedCompanies/Corporate_Actions/New_Info_JSX/Jenis_Informasi/01_Laporan_Keuangan/02_Soft_Copy_Laporan_Keuangan//Laporan%20Keuangan%20Tahun%202021/TW1/AALI/FinancialStatement-2021-I-AALI.xlsx';
dfCurl($url);
}
But unfortunetly I still cannot download the xlsx file properly.
I just got a 1 kb file and when open the file i got error from excel.
So what is the correct way to download from url using php ?
The link is :
https://www.idx.co.id/Portals/0/StaticData/ListedCompanies/Corporate_Actions/New_Info_JSX/Jenis_Informasi/01_Laporan_Keuangan/02_Soft_Copy_Laporan_Keuangan//Laporan%20Keuangan%20Tahun%202021/TW1/AALI/FinancialStatement-2021-I-AALI.xlsx
Thank You
Finally I can download using php from that site.
All I want to do is use a curl :
function dfCurl($url){
$ch = curl_init($url);
$dir = '../';
$fileName = basename($url);
$saveFilePath = $dir . $fileName;
$fp = fopen($saveFilePath, 'wb');
$headers = [
'authority: www.idx.co.id',
'method: GET',
'scheme: https',
'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'cache-control: no-cache',
'pragma: no-cache',
'user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Mobile Safari/537.36',
'Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
];
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_exec($ch);
curl_close($ch);
fclose($fp);
}
And that's all.
Related
Hello everyone I have this CURL that works if I put the "wp_create_nonce" cookie user manually.
And when I execute the php from my browser with the user logged in.
The problem is to get the "wp_create_nonce" of the user from the cookies, is there any way to get it or simulate it?
The problem is when I run the php from the Cron it doesn't take the current session from the CURL to create the wp_create_nonce
Code CURL PHP
//We get wp_create_nonce from the current user logged
//$fb_product_id = 27540 is the ID of the product.
$nemo = wp_create_nonce( 'update-post_'.$fb_product_id);
$nemowoo = wp_create_nonce( 'woocommerce_save_data', 'woocommerce_meta_nonce' );
//Start CURL
$chanelfacebook = curl_init();
curl_setopt($chanelfacebook, CURLOPT_URL, 'https://myserverexample.com/wp-admin/post.php');
curl_setopt($chanelfacebook, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($chanelfacebook, CURLOPT_POST, 1);
curl_setopt($chanelfacebook, CURLOPT_POSTFIELDS, "_wpnonce=".$nemo."&user_ID=1&action=editpost&post_ID=".$fb_product_id."&woocommerce_meta_nonce=".$nemowoo."&_wc_pre_orders_enabled=yes&facebookpreloadedu=yes&save=Actualizar");
curl_setopt($chanelfacebook, CURLOPT_ENCODING, 'gzip, deflate');
$headersfacebook = array();
$headersfacebook[] = 'Authority: miserverexample.com';
$headersfacebook[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9';
$headersfacebook[] = 'Accept-Language: es-419,es;q=0.9,en;q=0.8';
$headersfacebook[] = 'Cache-Control: max-age=0';
$headersfacebook[] = 'Content-Type: application/x-www-form-urlencoded';
$headersfacebook[] = 'Cookie: //cookie deleted for security//';
$headersfacebook[] = 'Origin: https://myserverexample.com';
$headersfacebook[] = 'Referer: https://myserverexample.com/wp-admin/post.php?post='.$fb_product_id.'&action=edit';
$headersfacebook[] = 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36';
curl_setopt($chanelfacebook, CURLOPT_HTTPHEADER, $headersfacebook);
$result = curl_exec($chanelfacebook);
if (curl_errno($chanelfacebook)) { error_log( print_r( "Error de Curl EN STOCK: ".curl_error($chanelfacebook), true ) ); }
curl_close($chanelfacebook);
The problem is creating the wp_create_nonce by doing it manually in the cron from the browser where the user is logged in works.
But if it is done from the cron when there is no user, it does not work.
How to solve it?
I can think of maybe an impersonate user to generate the code "wp_create_nonce"
P.D:
1.- I am doing CURL to the same origin server.
2.- CURL works fine if I run it from the browser where the curl cookie user is logged in.
In short I need to simulate the "wp_create_nonce" by user_id is this possible?
Example:
wp_create_nonce( 'update-post_'.$fb_product_id, $simulateiduser);
I have a cURL command like this:
curl 'https://www.example.com' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36' \
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3' \
-H 'accept-language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7' \
-H 'authority: www.example.com'
Executing this in a command line like in Terminal app on my Mac, results to the expected output.
(In case you test it yourself: If this output contains the word Sicherheitsüberprüfung it's geo blocked and you have to use a German IP to test it.)
I transferred the exact command to PHP cURL like this:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.example.com');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
$headers = array();
$headers[] = 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3';
$headers[] = 'Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7';
$headers[] = 'Authority: www.example.com';
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
curl_close($ch);
echo $result;
?>
When I run this code I'm getting a message that my request was recognized as automated request/robot: It says Sicherheitsüberprüfung, means security check.
Of course, I'm using the same IP for both, command line and PHP cURL request.
Why that? Isn't command line cURL the same as PHP cURL?
Or is there anything wrong with my PHP script?
UPDATE
I fortuitously found out the following: I'm using Coda as code editor on my Mac. This has a build-in PHP rendering engine. Using this with my PHP script, the result is as expected. It's the same result I'm getting in the command line.
UPDATE 2
I made what Jannes Botis suggested in his answer. I then ran the PHP script in my Coda code editor app (what output the expected) and with MAMP as localhost (what is always recognized as automated request).
I figured out that the the code executed with MAMP was using HTTP/2 while the code executed in Coda is using HTTP/1.1. To solve this, I added the following to the script:
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
Now, both output exact the same string:
GET / HTTP/1.1
Host: www.example.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Authority: www.example.com
But, it's still the same: The one is working, the other is recognized as automated request.
Try to debug the request in both cases:
a) Terminal: use curl verbose mode: curl -v and check the http request sent, especially check the header list
b) php curl: print the http request using CURLINFO_HEADER_OUT:
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_exec($ch);
$info = curl_getinfo($ch);
print_r($info['request_header']);
Testing the different headers, what made it work was adding "Pragma: no-cache" header to the request:
$headers[] = 'Pragma: no-cache';
On the other hand, in terminal curl, I had to uppercase the request headers, e.g. User-Agent etc.
Try to create a tcp connection with fsockopen:
$fp = fsockopen("ssl://"."www.example.com", 443, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
$out = "GET / HTTP/1.1\r\n";
$out .= "Host: www.example.com\r\n";
$headers = array();
$headers[] = 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3';
$headers[] = 'Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7';
$headers[] = 'Authority: www.example.com';
$out .= $headers;
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
while (!feof($fp)) {
echo fgets($fp, 1024);
}
fclose($fp);
and test if this works. Maybe the issue is either that php curl adds some info to the http request or the problem is on the tcp connection level, some info added there.
References
cURL works from Terminal, but not from PHP
PHP cURL: modify/overwrite Connection header
Sending TCP Data with PHP
Command line curl :
It is a tool to transfer data to or from a server, using any of the supported protocols (HTTP, FTP, IMAP, POP3, SCP, SFTP, SMTP, TFTP, TELNET, LDAP or FILE). curl is powered by Libcurl. This tool is preferred for automation, since it is designed to work without user interaction. curl can transfer multiple file at once.
For more details for Command line curl
Syntax:
curl [options] [URL...]
Example:
curl http://site.{one, two, three}.com
PHP cURL
$ch = curl_init('http://example.com/wp-login.php');
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 3);
if($this->getRequestType() == 'POST')
{
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS,
array(
'user[name]' => 'Generic+Username',
'user[email]' => 'mahekpatel04#gmail.com'
);
);
}
$response = curl_exec($ch);
The issue is with ciphers selected by PHP's cURL by default.
Running curl command with -Ivs options allows us to see what ciphers it uses:
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
Setting them in PHP allows it to bypass this mysterious check:
curl_setopt($ch,
CURLOPT_SSL_CIPHER_LIST,
'ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH'
);
Also, it seems that Host header and using HTTPv2 should be added:
$headers[] = 'Host: www.11880.com';
// ...
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_2_0);
Hi So I am setting up a system where you enter an isntagram username and then the website get the informations about this account (username, profilepic, followers and following...)
So i am doing this with simple php code with file_get_content and fetching to get the id of the suer and then go to the info page with this url and the preset instagram info link.
$username = $_POST['username'];
$html =file_get_contents('https://instagram.com/'.$username);
$subData=substr($html, strpos($html, 'window._sharedData'), strpos($html,
'};'));
$userid=strstr($subData, '"id":"');
$userid=str_replace('"id":"', '', $userid);
$userid=strstr($userid, '"', true);
$userData =
file_get_contents('https://i.instagram.com/api/v1/users/'.$userid.'/
info/');
$userDecodedData=json_decode($userData);
session_start();
$username = $userDecodedData->user->username;
$profilepicurl = $userDecodedData->user->hd_profile_pic_url_info->url;
$followers = $userDecodedData->user->follower_count;
$following = $userDecodedData->user->following_count;
$bio = $userDecodedData->user->biography;
$_SESSION['scoreinsta'] = $followers - $following;
So this works just fine when type my instagram username or my friend's but not when I try with kylie jenner's username or Instagram's or Ariana Grande, i've tried with cristiano ronaldo account to see if instagram was blocking all the most famous people but it works with his account :/ I'm kinda lost...
file_get_contents(https://i.instagram.com/api/v1/users/12281817/info/): failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in C:\wamp64\www\Fame\addinsta.php on line 11
(error message I get when I trie with Kylie Jenner).
This is the error message but what I dont understand is that you can try the url it gives and it works jsut fine (you can see the info in an array or whathever) but the error message says he cant access it.
Edit: I'm currently trying with every most followed accounts I can and it doesnt work with taylor swift also.
You lack cookies to be able to retrieve data from the url:
you can try (ex: for case me):
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://i.instagram.com/api/v1/users/12281817/info/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
$headers = array();
$headers[] = 'Authority: i.instagram.com';
$headers[] = 'Pragma: no-cache';
$headers[] = 'Cache-Control: no-cache';
$headers[] = 'Upgrade-Insecure-Requests: 1';
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3';
// $headers[] = 'Accept-Encoding: gzip, deflate, br';
$headers[] = 'Accept-Language: vi-VN,vi;q=0.9,fr-FR;q=0.8,fr;q=0.7,en-US;q=0.6,en;q=0.5';
$headers[] = 'Cookie: mid=XS3ghgALAAGXu10Eb58jOsW7SAEi; fbm_124024574287414=base_domain=.instagram.com; csrftoken=tNVs4niJr2fLiLh76dPPGJuFaMlihIEd; ds_user_id=3043596499; sessionid=3043596499%3ArcZihGFrIEdqkX%3A6; shbid=14335; shbts=1565197428.8656168; rur=FTW; urlgen=^^^{\"113.177.118.128\":';
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close($ch);
var_dump($result);
I'm trying to use CURL to assess the visitors on my site. I'd like to see if they are being linked from a bad neighborhood or not. Most of the time my current code works, but not always.
I'm having a bit of trouble making my CURL able to fool all servers. How do I make my CURL headers totally convincing, and remove any possible clues that I'm using CURL?
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"http://www.example.com");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$vars); //Post Fields
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$headers = array();
$headers[] = 'X-Apple-Tz: 0';
$headers[] = 'X-Apple-Store-Front: 143444,12';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
$headers[] = 'Accept-Encoding: gzip, deflate';
$headers[] = 'Accept-Language: en-US,en;q=0.5';
$headers[] = 'Cache-Control: no-cache';
$headers[] = 'Content-Type: application/x-www-form-urlencoded; charset=utf-8';
$headers[] = 'Host: www.example.com';
$headers[] = 'Referer: http://www.example.com/index.php'; //Your referrer address
$headers[] = 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0';
$headers[] = 'X-MicrosoftAjax: Delta=true';
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$server_output = curl_exec ($ch);
print $server_output;
curl_close ($ch);
if (strpos($output,'sex') !== false)
{
echo 'sex';
}
?>
For example, a certain well known adult video site with an orange logo that looks a lot like the YouTube logo (maybe you guys know the one) responded with this:
403 Forbidden
Request forbidden by administrative rules.
__SERVERNAME__
In chrome dev tools, you can obtain the full HTTP request chrome used against a URL by:
opening dev tools
goto the "network" tab
request the URL you want - if you are already on the target page - hit F5 or reload.
Dev tools will then generate a list of HTTP requests (and responses) made
right click on the HTTP request / URL you are interested in
click the "save as curl" option and you will now have the full HTTP request details (for command line curl ) in your clipboard.
By using these values when sending an HTTP request your request will ostensibly appear to be made by a Chrome web browser.
i have script this code:
require_once 'Request2.php';
$request = new HTTP_Request2('http://fakevisit.blogfa.com/');
$request->setHeader(array(
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding: gzip, deflate',
'Accept-Language: en-US,en;q=0.5',
'Connection: keep-alive',
'Ip: 88.168.25.98',
'SomeBull: BeingIgnored',
'Referer: http://www.google.com',
'User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)'
));
$response = $request->send();
echo "Response body:\n" . $response->getBody();
The program is run once every 1 minute Is working correctly.
but in google analytics And other counter scripts Does not add to the visitors!
at this line
echo "Response body:\n" . $response->getBody();
The entire contents of the page with Script to display the number of visitors are displayed correctly in page.
Google Analytics and most others use Javascript to populate the hit. If you're calling this via Cron/Scheduled Tasks then you're doing a HTTP request and getting the response, but you aren't executing the Javascript for GA, therefore no hit is added.