cURL not saving the image correctly - php

$ch = curl_init();
$fp = fopen("$localName",'w');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_URL, $src);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_TIMEOUT, 200);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://google.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
$rawdata=curl_exec ($ch);
curl_close ($ch);
fwrite($fp, $rawdata);
fclose($fp);
... writes the file but invalid (0 bytes). Please tell me what I'm doing wrong.

I ran your code and there were some errors. I have rectified those here:
$src = '<URL to the image>';
$ch = curl_init($src);
//curl_setopt($ch, CURLOPT_FILE, $fp); This option is not required
//curl_setopt($ch, CURLOPT_URL, $host); Since you are setting the source in init skip this
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_TIMEOUT, 200);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://google.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follows redirect responses.
$raw=curl_exec($ch);
if ($raw === false) {
trigger_error(curl_error($ch));
}
curl_close ($ch);
$localName = basename($src); // The file name of the source can be used locally
if(file_exists($localName)){
unlink($localName);
}
$fp = fopen($localName,'wb');
fwrite($fp, $raw);
fclose($fp);

Turned out the problem was some images were loaded but with 302 redirect status message which confused the curl.

Try add a header() for image type.
For example, if it is a PNG image, add in your code:
// ...
header('Content-type: image/png');
file_put_contents($raw);

Had a play, all you seem to need is the following:
<?
$src = 'http://bit.ly/TT5N5M';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $src);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$output = curl_exec($ch);
curl_close($ch);
$fp = fopen("img.jpg",'w');
fwrite($fp, $output);
fclose($fp);

Related

Http 411 php curl due to post request

With this code:
<?php
function request($url, $post, $cook)
{
$ch = curl_init();
$cookie_file_path = "vskcookies.txt";
$urll = 'http://auto.vsk.ru/login.aspx';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_USERAGENT,'"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Trident/7.0; Touch; .NET4.0C; .NET4.0E; Tablet PC 2.0)"');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
$result = curl_exec($ch);
echo "FIELDS:\n\n".$post;
echo "\n\nHEADERS:\n\n";
curl_close($ch);
return $result;
}
$result = request($_POST['url'], $_POST['data'], $_POST['cook']);
if ($result === FALSE)
echo('error');
else
echo($result);
?>
I am getting two cookies as I need but with http 411 error in body:
The same request, the same code, but in the end, right before curl_exec I add
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: application/x-www-form- urlencoded;charset=utf-8"));
As the result, I am getting correct body, but now only one cookie (I need both):
Another variants:
This code
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;charset=utf-8;Content-Length: '.strlen($post)));
Gives 411 error and correct cookies for reason.
This
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length:'.strlen($post)));
Causes nothing: just nothing happens.
Also after both (first)variants, vskcookies.txt contains only one (pool) cookie.
Why that?
It looks like, when I add header it erases request’s body(post fields).
UPDATE
For this code
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;'));
I just sitting and clicking on button that makes ajax request of the script above on my page, getting every time 411 error response (also cookies/time in header updates every time), but after about 10 clicks I got http 200 and the page I need. Then again many times 411 and then again a single 200. By the way cookies file still has only one cookie.
Wtf?
As I mentioned in question’s update, sometimes I get 200 response with two cookies and body I need.
So I wrote this is bad unstable solution, often it can remain about 20 seconds:
<?php
function request($url, $post, $cook)
{
$sta=0;
while($sta != '200')
{
$ch = curl_init();
$cookie_file_path = "vskcookies.txt";
$urll = 'http://auto.vsk.ru/login.aspx';
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_USERAGENT,'"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Trident/7.0; Touch; .NET4.0C; .NET4.0E; Tablet PC 2.0)"');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
//curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;'));
//curl_setopt($ch, CURLOPT_POSTREDIR, 2);
$result = curl_exec($ch);
$sta = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
}
echo "FIELDS:\n\n".$post;
echo "\n\nHEADERS:\n\n";
return $result;
}
$result = request($_POST['url'], $_POST['data'], $_POST['cook']);
if ($result === FALSE)
echo('error');
else
echo($result);
?>
Bet there is a much better correct answer.

Upload the file to the FTP server over HTTPS using curl in php

I am trying to upload the data [can be text or zip] to the ftp site.
As we have proxy in place in the environment so, I decided to upload the data using curl. Before I go head to set proxy server setting, I was testing the script on the environment without proxy.
I followed :-
FTP upload file to distant server with CURL and PHP uploads a blank file
However, I couldn't able to upload a file.
link where, I would like to upload is this format:-
https://fp.emc.com/.....
Do any one know, how to upload a file to ftp server over https using curl function of PHP?
<?php
$sendTo = 'https://ftp.emc.com/....?domain=XX&user=XXX&password=XXX';
$localfile ="23.txt";
$fp = fopen($localfile, 'r');
// Create CURL Connection
$ch = curl_init();
//curl_setopt($ch, CURLOPT_PROTOCOLS, CURLPROTO_HTTPS);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "usr:passwd");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $sendTo);
//curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_UPLOAD, 1);
curl_setopt($ch, CURLOPT_INFILE, $fp);
curl_setopt($ch, CURLOPT_INFILESIZE, filesize($localfile));
echo $ch;
$m=curl_exec ($ch);
echo "m $m<br>";
$error_no = curl_errno($ch);
echo "error_no $error_no<br>";
curl_close ($ch);
if ($error_no == 0) {
$error = 'File uploaded succesfully.';
} else {
$error = 'File upload error.';
}
echo $error;
?>
Try this:
$fp = fopen($filepath, 'r');
$ftp_url = "ftp://user:password#ftpserver:21/" . $filename;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $ftp_url);
curl_setopt($ch, CURLOPT_UPLOAD, 1);
curl_setopt($ch, CURLOPT_INFILE, $fp);
curl_setopt($ch, CURLOPT_INFILESIZE, filesize($filepath));
curl_setopt($ch, CURLOPT_PROXY, $proxy_server_ip);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy_server_port);
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxy_login);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_exec($ch);
curl_close($ch);
It works for me.

facebook image save curl

The size of the save images 0 kb
Working code, except facebook
function imagedownload($url,$saveto){
$ch = curl_init ($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $args);
$result = parse_url($url);
curl_setopt($ch, CURLOPT_REFERER, $result['scheme'].'://'.$result['host']);
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0');
$raw=curl_exec($ch);
curl_close ($ch);
if(file_exists($saveto)){
unlink($saveto);
}
$fp = fopen($saveto,'x');
fwrite($fp, $raw);
fclose($fp);
}
$url="http://scontent-frt3-1.xx.fbcdn.net/v/t1.0-0/cp0/e15/q65/p320x320/13700038_850487491753797_6227258625184891703_n.jpg?oh=793ecde8db1a8e65789534907d08b25e&oe=57F1DDFF";
$konum="images/"
$yolla=imagedownload($url,$konum);
The size of the save images 0 kb
Working code, except facebook
It works if you remove the options to POST and do a regular GET request:
<?php
function imagedownload($url, $saveto)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$result = parse_url($url);
curl_setopt($ch, CURLOPT_REFERER, $result['scheme'] . '://' . $result['host']);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0');
$raw = curl_exec($ch);
curl_close($ch);
if (file_exists($saveto)) {
unlink($saveto);
}
$fp = fopen($saveto, 'x');
fwrite($fp, $raw);
fclose($fp);
}
$url = "http://scontent-frt3-1.xx.fbcdn.net/v/t1.0-0/cp0/e15/q65/p320x320/13700038_850487491753797_6227258625184891703_n.jpg?oh=793ecde8db1a8e65789534907d08b25e&oe=57F1DDFF";
$konum = "test.jpg";
$yolla = imagedownload($url, $konum);
I had try in my localhost this code and it working well.
<?php
ini_set('display_errors', 1);
function imagedownload($url,$saveto){
$ch = curl_init ($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
$raw=curl_exec($ch);
curl_close ($ch);
if(file_exists($saveto)){
unlink($saveto);
}
$fp = fopen($saveto,'x');
fwrite($fp, $raw);
fclose($fp);
}
$url="http://scontent-frt3-1.xx.fbcdn.net/v/t1.0-0/cp0/e15/q65/p320x320/13700038_850487491753797_6227258625184891703_n.jpg?oh=793ecde8db1a8e65789534907d08b25e&oe=57F1DDFF";
$konum="/var/www/html/jsPDF-master/examples/test.jpg";
$yolla=imagedownload($url,$konum);
?>
And result
And one note: Please make sure the directory for save image must have write permission.
Example: chmod -R 777 /var/www/html/jsPDF-master/examples
Or ensure that in php.ini allow_url_fopen is enable

PHP Curl Login https for Esse3

I'm trying to do a login in a Esse3 Platform with Curl PHP.
Here the snippet:
<?
function get_jsessionid($url){
$ch = curl_init($url);
print_r($ch);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$result=curl_exec ($ch);
preg_match_all('/^Set-Cookie:\s*([^;]*)/mi', $result, $matches);
$cookies = array();
foreach($matches[1] as $item) {
parse_str($item, $cookie);
$cookies = array_merge($cookies, $cookie);
}
return $cookies['JSESSIONID'];
}
$username='';
$password='';
$baseurl='https://webstudenti.unica.it/esse3/Home.do';
$loginurl= 'https://webstudenti.unica.it/esse3/auth/Logon.do;jsessionid=' . get_jsessionid($baseurl);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$loginurl);
curl_setopt($ch, CURLOPT_TIMEOUT, 30); //timeout after 30 seconds
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
$status_code = curl_getinfo($ch, CURLINFO_HTTP_CODE); //get status code
$result=curl_exec ($ch);
curl_close ($ch);
print_r($result);
Obviously now I've deleted my credentials, but the code doesn't work. I've tried a lot of time with differents snippet but login doesn't go.
Do you have any solution?
Ps: code works, but the remote webapp give me a login error.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://webstudenti.unica.it/esse3/Home.do');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$response = curl_exec($ch);
preg_match("/Set-cookie: (.*)\n/iU", $response, $matches);
$cookie = trim(substr($matches[1], strpos($matches[1],':')));
curl_setopt($ch, CURLOPT_URL, 'https://webstudenti.unica.it/esse3/auth/Logon.do');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERPWD, "{$username}:{$password}");
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$response = curl_exec($ch);
UPDATE (extra configuration)
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5');

php facebook login + curl + proxy

i am using a curl script to login in for facebook
function cURL($url, $header=NULL, $cookie=NULL, $p=NULL) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, $header);
curl_setopt($ch, CURLOPT_NOBODY, $header);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
if ($p) {
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $p);
}
$result = curl_exec($ch);
if ($result) {
return $result;
} else {
return curl_error($ch);
}
curl_close($ch);
}
// use of that function
$cookie="";
$a = cURL("https://login.facebook.com/login.php?login_attempt=1",true,null,"email=$EMAIL&pass=$PASSWORD");
preg_match('%Set-Cookie: ([^;]+);%',$a,$b);
$c = cURL("https://login.facebook.com/login.php?login_attempt=1",true,$b[1],"email=$EMAIL&pass=$PASSWORD");
preg_match_all('%Set-Cookie: ([^;]+);%',$c,$d);
for($i=0;$i<count($d[0]);$i++) {
$cookie .= $d[1][$i].";
}
$userContent=cURL("http://www.facebook.com/ads/adboard/",null,$cookie,null);
I want it to do it with proxy with curl Please let me know how to do it?
I guess CURLOPT_PROXY is what you want?
curl_setopt( $ch, CURLOPT_PROXY, ... )
http://php.net/curl_setopt
function curl_grab_page($site,$data,$proxy,$proxystatus){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
if ($proxystatus == 'on') {
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
}
use this function it will work :)
use this function
function curl_login($url,$data,$proxy,$proxystatus){
$fp = fopen("cookie.txt", "w");
fclose($fp);
$login = curl_init();
curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($login, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE
5.01; Windows NT 5.0)");
curl_setopt($login, CURLOPT_TIMEOUT, 40);
curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
if ($proxystatus == 'on') {
curl_setopt($login, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($login, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt($login, CURLOPT_PROXY, $proxy);
}
curl_setopt($login, CURLOPT_URL, $url);
curl_setopt($login, CURLOPT_HEADER, TRUE);
curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($login, CURLOPT_POST, TRUE);
curl_setopt($login, CURLOPT_POSTFIELDS, $data);
ob_start(); // prevent any output
return curl_exec ($login); // execute the curl command
ob_end_clean(); // stop preventing output
curl_close ($login);
unset($login);
}
make sure you are using live http header plugins for firefox to get the right data array :)

Categories