I am using cURL to retrieve an image, rename it and store it locally.
The images comes up as 0 byte file, no matter, whether I use cURL, like so:
$strImageUrl = curl_init($strImageUrlSource);
$fp = fopen($strTargetImage, 'wb');
curl_setopt($strImageUrl, CURLOPT_FILE, $fp);
curl_setopt($strImageUrl, CURLOPT_HEADER, 0);
curl_exec($strImageUrl);
curl_close($strImageUrl);
fclose($fp);
or file_put/get. like so:
file_put_contents($strImageName, file_get_contents($strImageUrlSource));
The URL I am retrieving is:
<img src='http://i1.au.reastatic.net/150x112/73fa6c02a92d60a76320d0e89dfbc1a36a6e46c818f74772dec65bae6959c62f/main.jpg' width="150" height="112" alt="Wondecla, address available on request" title="Wondecla, address available on request" />
I can save this image properly manually.
When looking at the properties in FireFox it shows three entries:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFgAAABYBAMAAACDuy0HAAAAG1BMVEX+/v4BAQH///8KCgoDAwN/f3/19fWAgID8... etc
http://i1.au.reastatic.net/150x112/73fa6c02a92d60a76320d0e89dfbc1a36a6e46c818f74772dec65bae6959c62f/main.jpg
data:image/png;base64,iVBORw0KG ... etc.
What am I doing wrong here?
This works:
Using file_get_contents
$image = 'http://i1.au.reastatic.net/150x112/73fa6c02a92d60a76320d0e89dfbc1a36a6e46c818f74772dec65bae6959c62f/main.jpg';
$imageName = pathinfo( $image, PATHINFO_BASENAME );
file_put_contents( $imageName, file_get_contents( $image ) );
Using CURL
$image = 'http://i1.au.reastatic.net/150x112/73fa6c02a92d60a76320d0e89dfbc1a36a6e46c818f74772dec65bae6959c62f/main.jpg';
$imageName = pathinfo( $image, PATHINFO_BASENAME );
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $image );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
$source = curl_exec( $ch );
curl_close( $ch );
file_put_contents( $imageName, $source );
Hope this helps.
Use var_dump() to debug. What do you see when you
var_dump(file_get_contents('http://i1.au.reastatic.net/150x112/73fa6c02a92d60a76320d0e89dfbc1a36a6e46c818f74772dec65bae6959c62f/main.jpg'));
Related
I am not sure if hang is the correct term, but once the file is 100% uploaded my code after the upload code is not ran.
set_time_limit( 0 );
$fp = fopen( $zip_file, 'r' );
$curl_handler = curl_init();
curl_setopt( $curl_handler, CURLOPT_URL, 'https://upload-site/' . $zip_name . '.zip' );
curl_setopt( $curl_handler, CURLOPT_HTTPHEADER, [
'requesttoken: ' . $request_token,
'authorization: Basic ' . $basic_authorization_token,
] );
curl_setopt( $curl_handler, CURLOPT_PUT, true );
curl_setopt( $curl_handler, CURLOPT_INFILESIZE, filesize( $zip_file ) );
curl_setopt( $curl_handler, CURLOPT_INFILE, $fp );
curl_setopt( $curl_handler, CURLOPT_NOPROGRESS, false );
curl_setopt( $curl_handler, CURLOPT_PROGRESSFUNCTION, [ $this, 'updateUploadProgress' ] );
curl_setopt( $curl_handler, CURLOPT_TIMEOUT, 0 );
curl_setopt( $curl_handler, CURLINFO_HEADER_OUT, true );
$curl_retval = curl_exec( $curl_handler );
curl_close( $curl_handler );
//This code here is never ran on large file uploads.
Progress function (works)
function updateUploadProgress( $curl_handler, $download_file_size, $downloaded, $upload_file_size, $uploaded ) {
$progress = round( ( $uploaded / $upload_file_size ) * 100 );
echo 'Uploading... ' . $progress . " / 100%\n";
}
When uploading small files around 5mb everything works fine. But uploading a 500mb file after hitting 100/100 on the upload progress none of the code after curl_close() gets ran.
The question is not really a [mre], but I tried to get it to run on my local system.
(My PHP coding skills are low...)
I change the line: $curl_retval = curl_exec( $curl_handler ); to:
//$curl_retval = curl_exec( $curl_handler );
if(curl_exec($curl_handler) === false)
{
echo 'Curl error: ' . curl_error($curl_handler);
}
else
{
echo 'Operation completed without any errors';
}
After updateUploadProgress was called multiple times, I did receive next message:
Curl error: Failed to connect to localhost port 443: Connection refused
This error is correct, because I am not allowed to upload to my local system (😉).
But I do ask meself which message you (#Midger) will be receiving ...
Edit: I contact the support at scrapestack and confirmed that their api doesn't support image files.
I am trying to download a remote image using CURL with php. Below is my code. But whenever I try to open the downloaded image, I always get:
Cannot read this file. This is not a valid bitmap file, or its format is not currently supported.
Anyone know what is wrong with my code? Thank you.
$image ="http://api.scrapestack.com/scrape?access_key=TOKEN-HERE&url=https://i.imgur.com/Cbiu8Ef.png";
$imageName = pathinfo( $image, PATHINFO_BASENAME );
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $image );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTP_CONTENT_DECODING, false);
$source = curl_exec( $ch );
$info = curl_getinfo($ch);
curl_close( $ch );
file_put_contents( $imageName, $source );
I am not able to open the file, when I tried to open it with sublime, it is stuck at Loading Image. When I open it with notepad, I got the following that looks like PNG image, but it is not a valid image. File starts with
�PNG
IHDR � q�I� IDATx�k�]�u�o��(��_�M��m�8:���_r�G
You can see the file here: https://gofile.io/?c=cfsYf2
Looks like the problem is making the curl request through Scrapestack, because if I point the curl to image url directly, the image is downloaded correctly, like below:
$image ="https://i.imgur.com/Cbiu8Ef.png";
Edit: I played around with scrapestack a bit more today, it doesn't seem to support image scraping. It is best if you can reach out to their customer support and find out.
#Towsif is right, you are trying to get the page, not the actual image. I put something together really quick, try and see if this works for you.
$queryString = http_build_query([
'access_key' => 'replace this with your own token',
'url' => 'https://i.imgur.com/Cbiu8Ef.png',
]);
$ch = curl_init(sprintf('%s?%s', 'http://api.scrapestack.com/scrape', $queryString));
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
$image_source = curl_exec( $ch );
curl_close( $ch );
file_put_contents( 'Cbiu8Ef.png' , $image_source );
It looks like the response you get is a corrupt PNG image.
If you are using PHP with version prior of 5.1.3 you need to specify an additional option for binary data transfers, like images:
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
If the above options doesn't solve the issue you may try setting
curl_setopt($ch, CURLOPT_HTTP_CONTENT_DECODING, false);
in case the response has the Content-Type header set wrong letting curl do unwanted decoding on the raw output.
Your problem is with this url.
$image ="http://api.scrapestack.com/scrape?access_key=TOKEN-HERE&url=https://imgur.com/a/E5ehGuv";
If you go to this url
https://imgur.com/a/E5ehGuv
You will see the image page but NOT the image path. pathinfo() function does not work here and raises error.
If you right click on that image and Open image in new tab you will then see the image path, in this case that is
https://i.imgur.com/Cbiu8Ef.png
So you may try with this url
$image ="http://api.scrapestack.com/scrape?access_key=TOKEN-HERE&url=https://i.imgur.com/Cbiu8Ef.png";
I'm using CURL in PHP to download images from URLs from CSV file but if there are more than one images in a single line than some of the images downloaded corrupted and the size of that image is 0 byte.
Example:-
If the CSV file is like this then the second file always corrupted.
Image 1, "https://d2qx4k6xgs9iri.cloudfront.net/ProductImages/ce363947-f23a-46d6-b106-1201cdca37f0.jpg, https://homepages.cae.wisc.edu/~ece533/images/airplane.png"
But If I removed first or second image than the image successfully saved. Example:
Image 2, https://homepages.cae.wisc.edu/~ece533/images/arctichare.png
Here is my code that reads a CSV File
$file = fopen($file, "r");
while (!feof($file)) {
$data = fgetcsv($file);
$images = $data[1];
$images = explode(',', $images); //exploding images by ,
foreach ($images as $image) {
$milliseconds = md5(round(microtime(true) * 1000)) . '.jpg';
$imagename = saveImage($image, $milliseconds);
}
}
Below saveImage function
function saveImage($url,$image_name){
echo $url.'<br/>'; //URL is correct and have image. I have checked it manually
$ch = curl_init ($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
$raw=curl_exec($ch);
curl_close ($ch);
$fp = fopen('assets/products/large/' . $image_name,'x');
fwrite($fp, $raw);
fclose($fp);
}
The Exact Sample for CSV File which I'm Using
Images 1, "https://d2qx4k6xgs9iri.cloudfront.net/ProductImages/ce363947-f23a-46d6-b106-1201cdca37f0.jpg, https://homepages.cae.wisc.edu/~ece533/images/airplane.png"
Images 2, https://homepages.cae.wisc.edu/~ece533/images/arctichare.png
Images 3, "https://homepages.cae.wisc.edu/~ece533/images/fruits.png, https://homepages.cae.wisc.edu/~ece533/images/girl.png"
Images 4, "https://homepages.cae.wisc.edu/~ece533/images/goldhill.bmp, https://homepages.cae.wisc.edu/~ece533/images/tulips.png"
I think the issue may well be that there is space in the url before the protocol - using trim to remove the space would help. Rather than use curl for testing I simply used file_get_contents and it downloaded all the files OK.
$dir = 'c:/temp/downloads/';
$file=__DIR__ . DIRECTORY_SEPARATOR . 'img.csv';
$file=fopen( $file, 'r' );
while( !feof( $file ) ){
$line = fgetcsv( $file );
if( !empty( $line[1] ) ){
$urls = explode( ',', $line[1] );
foreach( $urls as $url ){
$url=trim( $url );
$bytes = file_put_contents( $dir . basename( $url ), file_get_contents( $url ) );
printf('Saved %s - size: %sKb<br />',basename( $url ),$bytes / 1024 );
}
}
}
fclose( $file );
The curl function also needed a little tweak - as the urls are over SSL then you really should add additional parameters to the curl request. I modified the function like so:
function saveImage( $url, $image_path ){
global $cacert;
$fp = fopen( $image_path, 'w+' );
$ch = curl_init( $url );
curl_setopt($ch, CURLOPT_HEADER, 0 );
curl_setopt($ch, CURLOPT_TIMEOUT, 10 );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true );
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2 );
curl_setopt($ch, CURLOPT_CAINFO, $cacert );
curl_setopt($ch, CURLOPT_ENCODING, '' );
curl_setopt($ch, CURLOPT_FILE, $fp );
curl_exec($ch);
curl_close ($ch);
fclose($fp);
}
Where $cacert is defined elsewhere but essentially, on my system is c:\wwwroot\cacert.pem ~ you can download a copy from here - curl.haxx.se
I ran this code rather than the above like so:
while( !feof( $file ) ){
$line = fgetcsv( $file );
if( !empty( $line[1] ) ){
$urls = explode( ',', $line[1] );
foreach( $urls as $url ){
$url=trim( $url );
saveImage( $url, $dir . basename( $url ) );
}
}
}
fclose( $file );
I have an old script that uploads PDF files via an API. I'm getting the error message:
Deprecated: curl_setopt(): The usage of the #filename API for file
uploading is deprecated. Please use the CURLFile class instead.
Here is the relevant code (I think this is all of it). The error points to the line: curl_setopt( $curl_handle, CURLOPT_POSTFIELDS, $postFields ).
//Upload the file
function uploadPdf( $api, $lead_id, $rev, $existing_files = array() ) {
if ( ! file_exists( SERVERPATH . "quotes/quote-". $this->id .".pdf" ) )
$this->createQuotePdf();
$files_array = array( array( 'entityType'=>'files', 'name'=>"quote-". $this->id .".pdf" ) );
// if ( $this->upfile && ! file_exists( SERVERPATH . "uploads/" . $upfile ) )
// $files_array[] = array( array( 'entityType'=>'files', 'name'=> $upfile ) );
foreach ( $existing_files as $file ) {
$files_array[] = (array) $file;
}
//this request gives us the URLs to upload to
$result = $api->editLead( array( 'leadId' => $lead_id, 'rev'=>'REV_IGNORE', 'lead'=> array( 'file' => $files_array ) ) );
//Upload the Quote file
$postFields = array();
$postFields['file'] = "#" . SERVERPATH . "quotes/quote-". $this->id .".pdf";
$postFields['type'] = "application/pdf";
$curl_handle = curl_init();
$file = array_pop( $result->file );
curl_setopt( $curl_handle, CURLOPT_URL, $file->uri );
curl_setopt( $curl_handle, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl_handle, CURLOPT_POST, true );
curl_setopt( $curl_handle, CURLOPT_USERPWD, USERNAME . ":" . API_KEY );
curl_setopt( $curl_handle, CURLOPT_POSTFIELDS, $postFields );
//execute the API Call
$return = curl_exec( $curl_handle ) ;
$this->uploadUpfile($api, $lead_id);
return $return;
}
My knowledge is pretty basic. But I've tried to replace:
$postFields['file'] = "#" . SERVERPATH . "quotes/quote-". $this->id .".pdf";
$postFields['type'] = "application/pdf";
with
$postFields['file'] = curl_file_create(SERVERPATH . "quotes/quote-". $this->id .".pdf", 'application/pdf', SERVERPATH . "quotes/quote-". $this->id .".pdf");
Doing the above has got rid of the error, but the underlying problem where I can't actually open the uploaded file is still happening. So I'm wondering if I've done something wrong?
From PHP 5.5 and above you should use CURLFile to upload file, I have already posted a complete answer describing CURLFile and normal file upload, you can check that answer here.
You can use CURLFile as below, feel free to adjust the code as per your need:
//Upload file using CURLFile
function upload($target, $postFields){
$file = $postFields['file'];
$cFile = new CURLFile($file,$postFields['type'], $file);
$data = array(
'file' => $cFile,
'type' => $postFields['type'],
);
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $target);
curl_setopt($curl, CURLOPT_HEADER , true); //we need header
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); // stop verifying certificate
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_POST, true); // enable posting
curl_setopt($curl, CURLOPT_POSTFIELDS, $data); // post images
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true); // if any redirection after upload
curl_setopt($curl, CURLOPT_SAFE_UPLOAD, true);
$r = curl_exec($curl);
if (curl_errno($curl)) {
$error = curl_error($curl);
print_r($error);
} else {
// check the HTTP status code of the request
$resultStatus = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($resultStatus != 200) {
print_r($resultStatus);
}else{
//successfull
print_r($r);
}
}
curl_close($curl);
}
As your file and filetype are in an array named $postFields, so you can call the above function as below:
upload($target, $postFields);
where $target is the link which you are calling to upload the file.
I am trying to implement methods discussed in this question to write a php function that downloads an audio file for a given string, but I can't seem to get around google's abuse protection. Results are sporadic, sometimes I get an audio file and other times it's an empty 2KB mp3 due to a response with "Our systems have detected unusual traffic from your computer network". Here is what I've got so far ( note the $file has a location in my code but for the purposes of this I've omitted it ) :
function downloadMP3( $url, $file ){
$curl = curl_init();
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl, CURLOPT_REFERER, 'http://translate.google.com/' );
curl_setopt( $curl, CURLOPT_USERAGENT, 'stagefright/1.2 (Linux;Android 5.0)' );
$output = curl_exec( $curl );
curl_close( $curl );
if( $output === false ) {
return false;
}
$fp = fopen( $file, 'wb' );
fwrite( $fp, $output );
fclose( $fp );
return true;
}
$word = "Test";
$file = md5( $word ) . '.mp3';
if ( !file_exists( $file ) ) {
$url = 'http://translate.google.com/translate_tts?q=' . $word . '&tl=en&client=t';
downloadMP3( $url, $file );
}
Try another service, I just found one that works even better than Google Translate; Google Text-To-Speech API