I'm using PHP to grab CSV files from Yahoo Finance based on an array of stock tickers.
My code is simple and is as follows:
$file = 'http://ichart.yahoo.com/table.csv?s='.$this->symbol.'&a='.$a.'&b='.$b.'&c='.$c.'&d='.$d.'&e='.$e.'&f='.$f.'&g='.$g.'&q=q&y=0&z='.$this->symbol.'&x=.csv';
$handle = fopen($file, "r");
//Then use fgetcsv to grab data,
fclose($handle);
I'm getting the following error:
Warning: fopen(http://ichart.yahoo.com/table.csv?s=ANA.L&a=2&b=16&c=2016&d=2&e=19&f=2016&g=d&q=q&y=0&z=ANA.L&x=.csv): failed to open stream: HTTP request failed! HTTP/1.0 404 Not Found in y.php on line x
Manually entering the previous URL (http://ichart.yahoo.com/table.csv?s=ANA.L&a=2&b=16&c=2016&d=2&e=19&f=2016&g=d&q=q&y=0&z=ANA.L&x=.csv) gets the CSV file with no problem, so I'm confused as to why I'm getting a 404 response code.
EDIT:
I have added a cURL request prior to using fopen. This is to check for a 404 response code. I am not getting any 404 response codes from the cURL but still getting the same error as mentioned above.
$curlhandle = curl_init($file);
curl_setopt($curlhandle, CURLOPT_RETURNTRANSFER, TRUE);
/* Get the HTML or whatever is linked in $url. */
$response = curl_exec($curlhandle);
/* Check for 404 (file not found). */
$httpCode = curl_getinfo($curlhandle, CURLINFO_HTTP_CODE);
if($httpCode == 404) {
echo "404:".$this->symbol."<br>";
curl_close($curlhandle);
return null;
break;
} else {
curl_close($curlhandle);
$handle = fopen($file, "r");
}
You need to URL encode all of your params otherwise you can have an invalid URL...
$file = 'http://ichart.yahoo.com/table.csv?s='.urlencode($this->symbol).'&a='.urlencode($a).' //etc
Try changing ichart.yahoo.com to ichart.finance.yahoo.com. i.e.
file = 'http://ichart.finance.yahoo.com/table.csv?s='.$this->symbol.'&a='.$a.'&b='.$b.'&c='.$c.'&d='.$d.'&e='.$e.'&f='.$f.'&g='.$g.'&q=q&y=0&z='.$this->symbol.'&x=.csv';
$handle = fopen($file, "r");
//Then use fgetcsv to grab data,
fclose($handle);
Related
I started my PHP script using file_get_contents(), I'm using an online data base and I can get JSON from it using URL, but sometimes (and I can't help it) I get back some 40* or 50* errors responses code, and I wondered if you guys could tell me what's better to use between cURL and file_get_contents, because basically everytime I'll have to check response code and switch case on it to determine what I do next.
200 => get file
403 => print "error"
502 => print 'bad gateway'
...
Hope I was clear, thanks in advance!
How to get the status of an HTTP response?
Using cURL
The function curl_getinfo() get information regarding a specific transfer. The second parameter of this function allows to get a specific information. The constant CURLINFO_HTTP_CODE can be used to get the HTTP status code of the HTTP response.curl_getinfo() should be called after curl_exec() and is relevant if curl_exec()' return is not false. If the response is false, don't forget to use curl_error() in this case to get a readable string of the error.
$url = 'https://stackoverflow.com';
$curlHandle = curl_init($url);
curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, true); // required to get the HTTP header
$response = curl_exec($curlHandle);
$httpCode = $response !== false ? curl_getinfo($curlHandle, CURLINFO_HTTP_CODE) : 0;
curl_close($curlHandle);
var_dump($httpCode); // int(200)
Using streams
The function stream_get_meta_data() retrieves header/meta data from streams/file pointers
$url = 'https://stackoverflow.com';
$httpCode = 0;
if ($fp = fopen($url, 'r')) {
$data = stream_get_meta_data($fp);
[$proto, $httpCode, $msg] = explode(' ', $data['wrapper_data'][0] ?? '- 0 -');
}
fclose($fp);
var_dump($httpCode); // int(200)
1 - I have configure google picker and it is working fine and I select the file from picker and get the file id.
2 - After refresh token etc all process I get the file metadata and get the file export link
$downloadExpLink = $file->getExportLinks();
$downloadUrl = $downloadExpLink['application/vnd.openxmlformats-officedocument.wordprocessingml.document'];
3 - After that I use this
if ($downloadUrl) {
$request = new Google_HttpRequest($downloadUrl, 'GET', null, null);
$httpRequest = Google_Client::$io->authenticatedRequest($request);
if ($httpRequest->getResponseHttpCode() == 200)
{
$content = $httpRequest->getResponseBody();
print_r($content);
} else {
// An error occurred.
return null;
}
and get this response
[responseBody:protected] => PK��DdocProps/app.xml���
�0D���k�I[ѫ��m
��!����A={���}�
2G�Z�g�V��Bľ֧�n�Ҋ�ap!����fb�d����k}Ikc�_`t<+ �(�NJ̽�����#��EU-
�0#���P����........
4 - I use some cURL functions to get file from google drive and save it to server. IN server directory a file created but cropped. I use this code
$downloadExpLink = $file->getExportLinks();
$downloadUrl = $downloadExpLink['application/vnd.openxmlformats-officedocument.wordprocessingml.document'];
//$downloadUrl value is
/*https://docs.google.com/feeds/download/documents/export/Export?id=1CEt1ya5kKLtgK************IJjDEY5BdfaGI&exportFormat=docx*/
When I put this url into browser it will download file successfully but when I use this url to fetch file with cURL or any php code and try to save it on server it saves corrupted file.
$ch = curl_init();
$source = $downloadUrl;
curl_setopt($ch, CURLOPT_URL, $source);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec ($ch);
curl_close ($ch);
$destination = "test/afile5.docx";
$file = fopen($destination, "w+");
fputs($file, $data);
fclose($file);
It result a corrupted file stored on server but whe I use this code to get any file other then google drive I download it successfully on server.
Can any one please help that how to download file from $downloadUrl to my server using php ?
I know that there are at least 10 the same questions with answers but none of them seems to work for me flawlessly. I'm trying to check if internal or external image exists (is image URL valid?).
fopen($url, 'r') fails unless I use #fopen():
Warning: fopen(http://example.com/img.jpg) [function.fopen]: failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in file.php on line 21
getimagesize($img) fails when image doesn't exist (PHP 5.3.8):
Warning: getimagesize() [function.getimagesize]: php_network_getaddresses: getaddrinfo failed
CURL fails because it isn't supported by some servers (although it's present mostly everywhere).
fileExists() fails because it doesn't work with external URLs and
can't possibly check if we're dealing with image.
Four methods that are the most common answers to such question are wrong. What would be the correct way to do that?
getimagesize($img) fails when image doesn't exist: am not sure you understand what you want .....
FROM PHP DOC
The getimagesize() function will determine the size of any given image file and return the dimensions along with the file type and a height/width text string to be used inside a normal HTML IMG tag and the correspondant HTTP content type.
On failure, FALSE is returned.
Example
$img = array("http://i.stack.imgur.com/52Ha1.png","http://example.com/img.jpg");
foreach ( $img as $v ) {
echo $v, getimagesize($v) ? " = OK \n" : " = Not valid \n";
}
Output
http://i.stack.imgur.com/52Ha1.png = OK
http://example.com/img.jpg = Not valid
getimagesize works just fine
PHP 5.3.19
PHP 5.4.9
Edit
#Paul .but your question is essentially saying "How do I handle this so I won't get an error when there's an error condition". And the answer to that is "you can't". Because all these functions will trigger an error when there is an error condition. So (if you don't want the error) you suppress it. None of this should matter in production because you shouldn't be displaying errors anyway ;-) – DaveRandom
This code is actually to check file... But, it does works for images!
$url = "http://www.myfico.com/Images/sample_overlay.gif";
$header_response = get_headers($url, 1);
if ( strpos( $header_response[0], "404" ) !== false )
{
// FILE DOES NOT EXIST
}
else
{
// FILE EXISTS!!
}
function checkExternalFile($url)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_exec($ch);
$retCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
return $retCode;
}
$fileExists = checkExternalFile("http://example.com/your/url/here.jpg");
// $fileExists > 400 = not found
// $fileExists = 200 = found.
If you're using PHP >=5.0.0 you can pass an additional parameter into fopen to specify context options for HTTP, among them whether to ignore failure status codes.
$contextOptions = array( 'http' => array('ignore_errors' => true));
$context = stream_context_create($contextOptions);
$handle = fopen($url, 'r', false, $context);
Use fsockopen, connect to the server, send a HEAD request and see what status you get back.
The only time you need to be aware of problems is if the domain doesn't exist.
Example code:
$file = "http://example.com/img.jpg";
$path = parse_url($file);
$fp = #fsockopen($path['host'],$path['port']?:80);
if( !$fp) echo "Failed to connect... Either server is down or host doesn't exist.";
else {
fputs($fp,"HEAD ".$file." HTTP/1.0\r\n"
."Host: ".$path['host']."\r\n\r\n");
$firstline = fgets($fp);
list(,$status,$statustext) = explode(" ",$firstline,3);
if( $status == 200) echo "OK!";
else "Status ".$status." ".$statustext."...";
}
You can use the PEAR/HTTP_Request2 Package for this. You can find it here
Here comes an example. The Example expects that you have installed or downloaded the HTTP_Request2 package properly. It uses the old style socket adapter, not curl.
<?php
require_once 'HTTP/Request2.php';
require_once 'HTTP/Request2/Adapter/Socket.php';
$request = new HTTP_Request2 (
$your_url,
HTTP_Request2::METHOD_GET,
array('adapter' => new HTTP_Request2_Adapter_Socket())
);
switch($request->send()->getResponseCode()) {
case 404 :
echo 'not found';
break;
case 200 :
echo 'found';
break;
default :
echo 'needs further attention';
}
I found try catch the best solution for this. It is working fine with me.
try{
list($width, $height) = getimagesize($h_image->image_url);
}
catch (Exception $e)
{
}
I know you wrote "without curl" but still, somebody may find this helpfull:
function curl_head($url) {
$ch = curl_init($url);
//curl_setopt($ch, CURLOPT_USERAGENT, 'Your user agent');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1); # get headers
curl_setopt($ch, CURLOPT_NOBODY, 1); # omit body
//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1); # do SSL check
//curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); # verify domain within cert
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); # follow "Location" redirs
//curl_setopt($ch, CURLOPT_TIMEOUT_MS, 700); # dies after 700ms
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
print_r(curl_head('https://www.example.com/image.jpg'));
You will see someting like this HTTP/1.1 200 OK or HTTP/1.1 404 Not Found in returned header array. You can do also multiple parallel requests with curl multi.
There are multiple steps, there is no single solution:
Validate URL
Check whether the file is available (can be done directly with step 3)
Download the image into a tmp file.
Use getimagesize to check the size of the image.
For this kind of work you can catch the exceptions and handle them well to define your answer. In this case you could even suppress errors because it's intended that they trick might fail. So you handle the errors correctly.
Because it's not possible to do a 100% check on it without having the actual image downloaded. So step 1 and 2 are required, 3 and 4 optional for a more definitive answer.
I am using following codes to upload remote files to my server. It works great where direct download link is given but recently I have noticed that few websites are giving mysql links as download link and when we click on that link the files start downloading to my pc. But even in html source of that page it does not show the direct link.
Here is my code:
<form method="post">
<input name="url" size="50" />
<input name="submit" type="submit" />
</form>
<?php
if (!isset($_POST['submit'])) die();
$destination_folder = 'mydownloads/';
$url = $_POST['url'];
$newfname = $destination_folder . basename($url);
$file = fopen ($url, "rb");
if ($file) {
$newf = fopen ($newfname, "wb");
if ($newf)
while(!feof($file)) {
fwrite($newf, fread($file, 1024 * 8 ), 1024 * 8 );
}
}
if ($file) {
fclose($file);
}
if ($newf) {
fclose($newf);
}
?>
It works great for all links where the download link is direct for example if I will give
http://priceinindia.org/muzicpc/48.php?id=415508 link it will upload the music file but the file name will be 48.php?id=415508 but the actual mp3 file is stored at
http://lq.mzc.in/data48-2/37202/Appy_Budday_(Videshi)-Santokh_Singh(www.Mzc.in).mp3
So if I can get the actual destination url the name will be Appy_Budday_(Videshi)-Santokh_Singh(www.Mzc.in).mp3
So I want to get the actual download url.
You should use Curl library for this. http://php.net/manual/en/book.curl.php
An example of how to use curl is provided in tha manual (on that link) befo before you close the connections, call curl_getinfo (http://php.net/manual/en/function.curl-getinfo.php) and specifically get CURLINFO_EFFECTIVE_URL which is what you want.
<?php
// Create a curl handle
$ch = curl_init('http://www.yahoo.com/');
// Execute
$fileData = curl_exec($ch);
// Check if any error occured
if(!curl_errno($ch)) {
$effectiveURL = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
}
// Close handle
curl_close($ch);
?>
(You can also use curl to write directly to a file - use the CURLOPT_FILE options. Also in the manual)
The problem is the original URL is redirecting. You want to catch the URL it is being redirected to, try using the headers and then get the basename($redirect_url) as your file name.
+1 for Robbie using CURL.
If you run (from command line)
[username#localhost ~]$ curl http://priceinindia.org/muzicpc/48.php?id=415508 -I
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.0.10
Date: Wed, 19 Sep 2012 07:31:18 GMT
Content-Type: text/html
Connection: keep-alive
X-Powered-By: PHP/5.3.10
Location: http://lq.mzc.in/data48-2/37202/Appy_Budday_(Videshi)-Santokh_Singh(www.Mzc.in).mp3
You can see the location header here is the new url.
in php try something like
$ch = curl_init('http://priceinindia.org/muzicpc/48.php?id=415508');
curl_setopt($ch, CURLOPT_HEADER, 1); // return header
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false); // dont redirect
$c = curl_exec($ch); //execute
echo curl_getinfo($ch, CURLINFO_HTTP_CODE); // will echo http code. 302 for temp move
echo curl_getinfo($ch, CURLINFO_EFFECTIVE_URL); // url being redirected to
You want to find the location part of the header. not sure the setting im sure though.
EDIT 3..or 4?
Yeah right, I see whats happening. You actually want to follow the location url then echo the effective url without downloading file. try.
$ch = curl_init('http://priceinindia.org/muzicpc/48.php?id=415508');
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$c = curl_exec($ch); //execute
echo curl_getinfo($ch, CURLINFO_EFFECTIVE_URL); // url being redirected to
When I run this my output is
[username#localhost ~]$ php test.php
http://lq.mzc.in/data48-2/37202/Appy_Budday_(Videshi)-Santokh_Singh(www.Mzc.in).mp3
I have script-1 on server A, where user ask for a file.
I have script-2 on server B (the file repository) where I check that user can access it and return the correct file (I'm using Smart File Download http://www.zubrag.com/scripts/download.php).
I've tried cURL and file_get_contents, I've changed Content Header in various ways, but I wasn't still able to download the file.
This is my request:
$request = "http://mysite.com/download.php?f=test.pdf";
and it works fine.
What should I call in script-1 to force the file be downloaded?
Some of my tries
This works, but I don't know how to handle unauthorized or broken downloads
header('Content-type: application/pdf');
$handle = fopen($request, "r");
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
echo $buffer;
}
fclose($handle);
}
This prints the pdf code (not the text) straight in the browser (I think it's a header problem):
$c = curl_init();
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_URL, $request);
$contents = curl_exec($c);
curl_close($c);
if ($contents) return $contents;
else return FALSE;
This generate a white page
file_get_contents($request);
To force download, add
header('Content-disposition: attachment');
But Note, that it's not in HTTP 1.1 spec anymore, see Uses of content-disposition in an HTTP response header first answer
Without your code I don't know what you've tried, but you need to get the contents of the file via cURL and then save it to your server. Something like...
$url = 'http://website.com/file.pdf';
$path = '/tmp/file.pdf';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$contents = curl_exec($ch);
curl_close($ch);
file_put_contents($path, $contents);
If you want to downloads file from the FTP server you can use php File Transfer Protocol (FTP) extension. Please find below code:
<?php
$SERVER_ADDRESS="";
$SERVER_USERNAME="";
$SERVER_PASSWORD="";
$conn_id = ftp_connect($SERVER_ADDRESS);
// login with username and password
$login_result = ftp_login($conn_id, $SERVER_USERNAME, $SERVER_PASSWORD);
$server_file="test.pdf" //FTP server file path
$local_file = "new.pdf"; //Local server file path
##----- DOWNLOAD $SERVER_FILE AND SAVE IT TO $LOCAL_FILE--------##
if (ftp_get($conn_id, $local_file, $server_file, FTP_BINARY)) {
echo "Successfully written to $local_file\n";
} else {
echo "There was a problem\n";
}
ftp_close($conn_id);
?>
Download the file with curl, then check this: http://php.net/function.readfile
It shows how to force download.
SOLVED
I ended by simply redirect the request with:
header("Location: $request");