I'm trying to download and parse a google spreadsheet directly via PHP.
Do i need to download the exported CSV file from google docs a different way?
$csv = file_get_contents('https://docs.google.com/spreadsheets/d/KEY/export?format=csv');
print_r($csv);
I tried it with file too:
$csv = file('https://docs.google.com/spreadsheets/d/KEY/export?format=csv');
And with cURL
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://docs.google.com/spreadsheets/d/KEY/export?format=csv");
curl_setopt($ch, CURLOPT_REFERER, "http://www.example.org/yay.htm");
curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$csv = curl_exec($ch);
curl_close($ch);
var_dump($csv);
When i call the URL from a browser i get a nice tiny CSV file. This way i get the source of the google docs website.
Related
I'm having trouble understanding how CURL handles headers.
I have a site.com/page1 that I want to access with CURL it does a 308 redirect to site.com/page2/file.zip
What I need is to go through site.com/page1 with CURL but download site.com/page2/file.zip directly from site.com
I'm using this code but it does not work as expected. It accesses site.com/page1 redirects to site.com/page2/file.zip but opens the file in the browser
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_cookie_file);
curl_setopt($ch, CURLOPT_REFERER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_exec($ch);
$error = curl_getinfo($ch);
curl_close($ch);
I guess if I can keep the response headers I'll fix the problem. But how do I do it ?? How do I use the same headers for the CURL visitor that the site I am accessing is sending me.
You want the cURL option RETURNTRANSFER set to true so what is returned comes back to you. Since you are trying to save a ZIP file you'll also need to open a file and use the CURLOPT_FILE option to tell cURL where to save your ZIP file.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt ($ch, CURLOPT_TIMEOUT,'180'); # 3 minute timeout
$FileOut = fopen('MyZIP_File.zip','w') or die('Could not open the output data file');
curl_setopt ($ch, CURLOPT_FILE,$FileOut);
curl_exec ($ch);
fclose($FileOut) or die('We ran into a problem saving data file');
This solved the problem.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_cookie_file);
curl_setopt($ch, CURLOPT_REFERER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
$result = curl_exec($ch);
if (preg_match('~Location: (.*)~i', $result, $match)) {
$location = trim($match[1]);
header('Location:' . $location);
}
I can not do file_get_contents to a m3u8 link, I need it to then make a str_replace in its content, the m3u8 is downloaded without problems from the browser
<?php $url="https://neunlds120dal.akamaized.net/nlds/univisionnow2/univision_mia/as/live/univision_mia_hd_3000_pc.m3u8"; $url2= file_get_contents($url); echo $url2; >
Note that file_get_contents does not work with HTTPS well, try using cURL. For example:
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, "https://neunlds120dal.akamaized.net/nlds/univisionnow2/univision_mia/as/live/univision_mia_hd_3000_pc.m3u8");
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
echo $result;
I am struck with this. I want to download a csv file from url using curl. I have referred all the answer in stackoverflow and tried all. But not getting what i am expected. i have the following code.
define("COOKIE_FILE", "cookie.txt");
$path = "settlement_file/test.csv";
set_time_limit(0);
$fp = fopen ($path, 'w+');//This is the file where we save the information
$ch = curl_init(str_replace(" ","%20",$url));//Here is the file we are downloading, replace spaces with %20
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp); // write curl response to file
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,false);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSLVERSION,3);
curl_exec($ch); // get curl response
curl_close($ch);
fclose($fp);
You have to remove the curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); statement. It makes curl_exec return the data instead of writing it to a file. Since it comes after the curl_setopt($ch, CURLOPT_FILE, $fp); it overrides that, so just remove the former line.
I'm trying to get content from bank site using curl.
http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna
Site is specific becouse it using ajax to fill currency exchange table. There is a link for download data in to file but you have to have same session id to able to do that.
Im trying this code:
$url="http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna";
$useragent = $_SERVER['HTTP_USER_AGENT'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_URL,$url);
$cl = curl_exec($ch);
$dom = new DOMDocument();
#$dom->loadHTML($cl);
#$link = $dom->getElementById('tecajPrn');
echo $suburl = "http://www.zaba.hr".$link->getAttribute('href');
After this I got link to file but I can't open it.
Another strange situation is that link I got with curl is http://www.zaba.hr/home/ZabaUtilsWeb/utils/tecaj/danasPrn but real link when I click on icon is http://www.zaba.hr/ZabaUtilsWeb/utils/tecaj/prn/62/2014
You are messing with cookie and ajax(may be!). Here is the lookaround. Try this:
First send a request to the page to obtain the cookie.
$url="http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna";
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "mozilla 5.0");
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_COOKIEFILE,"cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR,"cookie.txt");
$cl = curl_exec($ch);
curl_close($ch);
After that make another curl request. This time to obtain the json data:
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "mozilla 5.0");
curl_setopt($ch, CURLOPT_HTTPHEADER, array("X-Requested-With: XMLHttpRequest", "Referer: http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna"));
curl_setopt($ch, CURLOPT_URL,"http://www.zaba.hr/ZabaUtilsWeb/utils/tecaj/danas");
curl_setopt($ch, CURLOPT_COOKIEFILE,"cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR,"cookie.txt");
$cl = curl_exec($ch);
curl_close($ch);
Your json is available at this variable. Parse it using json_decode()
// now parse json from $cl
print $cl;
Anything required, help yourself straightway!
Note: Make sure you have write permission for the cookie.txt file. Also, its better to use absolute path like c:/test/cookie.txt or /var/tmp/cookie.txt.
I'm using the Amazon AIMS API to upload a an inventory file and I'm having an issue with the cURL call to upload the file. The documentation is very limited, so there is no example code that helps out in this.
This is what I have so far of the cURL call:
// $FILENAME is filename of the CSV file being uploaded:
$inventory = fopen($FILENAME, 'r') or die("Can't open file!");
echo $inventory;
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_INFILE, $inventory);
curl_setopt($ch, CURLOPT_POSTFIELDS, '');
curl_setopt($ch, CURLOPT_INFILESIZE, filesize($filename));
curl_setopt($ch, CUROPT_PUT, TRUE);
$response = curl_exec($ch);
curl_close($ch);
I've try putting $inventory into the CURLOPT_POSTFIELDS and not having an INFILE, but I get the same error.
On the XML response, I'm getting "NO_FILE_ATTACHED" so the obvious issue is getting the file to be attached to the XML call.
I also tried uploading as the first responder said using the Example #2 on the curl_setopt page on php.net.
For that, I used the following code:
$data = array('#/tmp/amazon_export.csv');
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$response = curl_exec($ch);
curl_close($ch);
I got the same NO_FILE_ATTACHED response back.
Any ideas?
This works for me:
$hCurl = curl_init();
curl_setopt($hCurl, CURLOPT_PUT, true);
curl_setopt($hCurl, CURLOPT_HEADER, true);
curl_setopt($hCurl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($hCurl, CURLOPT_CONNECTTIMEOUT, CURL_TIMEOUT_SECS);
curl_setopt($hCurl, CURLOPT_URL, "$oMessage->url/att/$fid");
curl_setopt($hCurl, CURLOPT_HTTPHEADER, $aCurlHeaders);
// TODO it could be possible that fopen() would return an invalid handle or not work altogether. Should handle that
$fp = fopen ($finfo['tmp_name'], "r");
curl_setopt($hCurl, CURLOPT_INFILE, $fp);
curl_setopt($hCurl, CURLOPT_INFILESIZE, $finfo['size']);
$sResp = curl_exec($hCurl);
You are combining PUT and POST in a single curl operation, which will not work. Refer to example #2 of the curl_setopt manual page for an example on how to upload a file using POST.
you have $filename and $FILENAME, the filesize call should be in your case filesize($FILENAME)...
Hope thats helps