PHP curl downloading a Zipped CSV from a caspio driven website - php

Question

I need to download a zipped .csv file from this website. http://www.phrfsocal.org/web-lookup-2/ The file is the link Download Data above the table on the right.
The gotcha is the link is created dynamically. So I need to extract it first.
That part seems to work fine. I get this link for the href.
https://b6.caspio.com/dp.asp?appSession=68982476236455965042483715808486764445346819370685922723164994812296661481433499615115137717633929851735433386281180144919150987&RecordID=&PageID=2&PrevPageID=&cpipage=&download=1
When I paste that link into a new browser tab, the browser downloads the zip file containing the csv that I am interested in.
However when a use CURL to try to get the zip, it instead gets the html of the table below the link. Can't seem to figure out how to grab the .zip.
Below is my code the first part finds the link and seems to be working.
The second part is where I having trouble.
PS I have permission from the owner of this page to download this data nightly using a Cron job.
thanks in advance,
Dave
$url = "http://www.phrfsocal.org/web-lookup-2/";
// url to the dynamic content doesn't seem to change.
$url = "https://b6.caspio.com/dp.asp?AppKey=0dc330000cbc1d03fd244fea82b4";
$header = get_web_page($url);
// Find the location of the Download Data link and extract the href
$strpos = strpos($header['content'], 'Download Data');
$link = substr($header['content'], $strpos, 300);
$link = explode(" ", $link);
$link = explode('"', $link[2]);
$url1 = $link[1];
print_r($url1);
print "<p>";
// Now Go get the zip file.
$zipFile = "temp/SoCalzipfile.zip"; // Local Zip File Path
$zipResource = fopen($zipFile, "w+");
// Get The Zip File From Server
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url1);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FILE, $zipResource);
$page = curl_exec($ch);
if (!$page) {
echo "Error :- " . curl_error($ch);
}
curl_close($ch);
echo "zip file recieved";
/* Open the Zip file */
$zip = new ZipArchive;
$extractPath = "temp";
if ($zip->open($zipFile) != "true") {
echo "Error :- Unable to open the Zip File";
}emphasized text
/* Extract Zip File */
$zip->extractTo($extractPath);
$zip->close();

Answer 1

The following code will download the zip file and unzip it into the given folder. Make sure that the folder is writable. So in this example make sure the temp folder has write permission.
You also don't need to fetch the html version of the page to extract the link. I had a play around with the URLs and you can get the zip file for each page by using the cpipage variable. You can change the $page_num variable to grab the zip from the given page.
$page_num = 1;
$url = 'https://b6.caspio.com/dp.asp?AppKey=0dc330000cbc1d03fd244fea82b4&RecordID=&PageID=2&PrevPageID=&cpipage=' .$page_num. '&download=1';
$zipFile = "temp/SoCalzipfile.zip"; // Local Zip File Path
$zipResource = fopen($zipFile, "w");
// Get The Zip File From Server
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FILE, $zipResource);
$page = curl_exec($ch);
if(!$page) {
echo "Error :- ".curl_error($ch);
}
curl_close($ch);
$zip = new ZipArchive;
$extractPath = "temp";
if($zip->open($zipFile) != "true"){
echo "Error :- Unable to open the Zip File";
}
/* Extract Zip File */
$zip->extractTo($extractPath);
$zip->close();