I am using curl to connect to http://www.gumtree.com/search?q=bmw&category=all but I am unfortunately getting this non-HTML response :
moÛ6ÞO¹9Ø”‘eÙΚH¶‡¾nÀŠ¢#“íCQ´x’ØP¤FÒVÜ4ÿ}GÉiÔ$[ƒ ñx÷Üû3û‘ëÌmk„ÒUr1+‘ñŬBÇ +™±è想ÓWáñ Ú‘«p>ØljmÜ�2*bkwåœãFd¶‡CJ8Ádh3&qGÆ`
I am using this code :
$cookie_file="C:\Users\Yasha\AppData\Roaming\Microsoft\Windows\Cookies\cto2_gumtree.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.gumtree.com/all/uk/daewoo");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE );
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file); // Cookie management.
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
What am I doing wrong here ?
now if i connect to http://www.gumtree.com/ or http://www.gumtree.com/all it works,
any advice would be appreciated
Related
I'm having trouble understanding how CURL handles headers.
I have a site.com/page1 that I want to access with CURL it does a 308 redirect to site.com/page2/file.zip
What I need is to go through site.com/page1 with CURL but download site.com/page2/file.zip directly from site.com
I'm using this code but it does not work as expected. It accesses site.com/page1 redirects to site.com/page2/file.zip but opens the file in the browser
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_cookie_file);
curl_setopt($ch, CURLOPT_REFERER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_exec($ch);
$error = curl_getinfo($ch);
curl_close($ch);
I guess if I can keep the response headers I'll fix the problem. But how do I do it ?? How do I use the same headers for the CURL visitor that the site I am accessing is sending me.
You want the cURL option RETURNTRANSFER set to true so what is returned comes back to you. Since you are trying to save a ZIP file you'll also need to open a file and use the CURLOPT_FILE option to tell cURL where to save your ZIP file.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt ($ch, CURLOPT_TIMEOUT,'180'); # 3 minute timeout
$FileOut = fopen('MyZIP_File.zip','w') or die('Could not open the output data file');
curl_setopt ($ch, CURLOPT_FILE,$FileOut);
curl_exec ($ch);
fclose($FileOut) or die('We ran into a problem saving data file');
This solved the problem.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_cookie_file);
curl_setopt($ch, CURLOPT_REFERER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
$result = curl_exec($ch);
if (preg_match('~Location: (.*)~i', $result, $match)) {
$location = trim($match[1]);
header('Location:' . $location);
}
Hi I know its a very common topic on StackOverFlow. I have already spent my entire week to search it out.
Such as I have a url : http://bayja.com/forum/index.php
This url I want to get after I post : http://bayja.com/forum/index.php?topic=3454.0
but now I get http://bayja.com/forum/index.php?board=2.0
I use $lastUrl = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
How can I get "http://bayja.com/forum/index.php?topic=3454.0
" ?
Please help me thank you.
here is my code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__).'/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, dirname(__FILE__).'/cookie.txt');
curl_setopt($ch, CURLOPT_USERAGENT, "Automatic SMF poster thing");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "user=".$user."&passwrd=".$password);
curl_setopt($ch, CURLOPT_URL, "$domain?action=login2");
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, "$domain?action=post;board=".$board.".0");
$data = curl_exec($ch);
sleep(3);
preg_match('/name="([^"]+)" value="([^"]{32})"/',$data,$sc);
preg_match ( "<input type=\"hidden\" name=\"seqnum\" value=\"(.*)\">", $data, $seqnum);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "topic=".$topic."&subject=".urlencode($subj)."&icon=xx&sel_face=&sel_size=&sel_color=&message=".urlencode($mess)."&message_mode=0¬ify=0&lock=0&post_vv[q][4]=๔&post_vv[q][3]=๒&post_vv[q][1]=๔&post_vv[q][2]=๒&additional_options=0&post=xxxxx&".$sc[1]."=".$sc[2]."&seqnum=4&seqnum=".$seqnum[1]);
curl_setopt($ch, CURLOPT_URL, "$domain?action=post2;start=0;board=".$board);
curl_exec($ch);
$lastUrl = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
echo $lastUrl; // output "http://bayja.com/forum/index.php?board2"
curl_close($ch);
I want connect to Outlook.com using cURL (php) and get the message from inbox.
<?php
$url = "https://login.live.com/login.srf?wa=wsignin1.0&ct=1369129355&rver=6.1.6206.0&sa=1&ntprob=-1&wp=MBI_SSL_SHARED&wreply=https:%2F%2Fmail.live.com%2F%3Fowa%3D1%26owasuffix%3Dowa%252f&id=64855&snsc=1&cbcxt=mail";
$post_fields ="email=xxxx#outlook.com&pass=xxxx";
$cookie_path = "\hotmail_login\cook";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_path);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$output = curl_exec ($ch);
echo $output;
?>
Any suggestions?
Thanks in advance
To connect with mail sever you should use POP3 or IMAP, there is no point in using curl for this.
http://php.net/manual/en/function.imap-open.php
here you have examples of how to connect with IMAP using PHP and get messages
I'm trying to make an voip call with PHP CURL and MEGAVOIP.
The problem is i can't manage the session to access the page protected by a password.
I looked which variables are posted to the login page to post it with Curl.
But my code doesn't work.
Following Colin Morelli and Waygood's advices, I just added those lines in both commands:
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies_file);
but it's still the same:
Megavoip returns: SESSION EXPIRED
So here is my full code:
<?php
ini_set("display_errors", 1);
$username="***";
$password="***";
$url="https://www.megavoip.com/login";
$url2="https://www.megavoip.com/phone_to_phone/";
$timeout = 10;
$cookies_file = 'cookies.txt';
// HERE I GET THE TOKEN
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies_file);
curl_setopt($ch, CURLOPT_TIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$content = curl_exec($ch);
preg_match_all('/<input[^>]+>/i',$content, $result);
preg_match_all('/(id|value)=("[^"]*")/i',$result[0][5], $img);
$img1=str_replace('"', '', $img[0][0]);
$img2=str_replace('"', '', $img[0][1]);
$img1=substr($img1,3);
$img2=substr($img1,6);
$postdata = "login%5Busername%5D=".$username."&login%5Bpassword%5D=".$password."&page_referrer=login&".$img1."=".$img2;
// HERE I SEND THE VARIABLES
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies_file);
curl_setopt($ch, CURLOPT_TIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$content = curl_exec($ch);
// IF LOGGED HERE I'LL MAKE THE CALL
curl_close($ch);
echo $content;
exit;
?>
Any ideas to help me?
This is a test account so feel free to use my login and password if you want to have a look on this and help me!
Thank you a lot in advance.
Ok you need to forward the cookie/session after login,
you need to first extract the cookie from Header after login like following
// HERE I GET THE TOKEN
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
.... ....
.... ....
$content = curl_exec($ch);
preg_match('/^Set-Cookie:\s*([^;]*)/mi', $content, $m);
parse_str($m[1], $cookies);
$cookie = $cookies['NAMEOFCOOKIEUNEEDHERE'];
After that need to use $cookie variable in curl options like that
// HERE I SEND THE VARIABLES
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIE, 'NAMEOFCOOKIEUNEEDHERE='.$cookie);
I hope your problem will be resolved.
Thanks
Moin
I'm using cUrl(PHP) to post a login request and store response in cookie file.
In my second request I'm passing cookie in header and post data to verify it.
Issue is that cookie file is not created in first succesful request results in failure for second request. Please suggest me where I'm doing wrong.
$cookiefile="/var/www/html/dimdim/cook.txt";
$url_log="http://my.dimdim.com/api/auth/login";
$p_log='request={"account":"bin6k","password":"password","group":"all"}';
$url_ver="http://my.dimdim.com/api/auth/verify";
$p_ver='request={"account":"bin6k","password":"password","group":"all"}';
$ch = curl_init();
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_URL,$url_log);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $p_log);
ob_start(); // prevent any output
$retval=curl_exec ($ch); // execute the curl command
ob_end_clean(); // stop preventing output
curl_close ($ch);
//print_r($retval);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_URL,$url_ver);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $p_log);
$buf2 = curl_exec ($ch);
curl_close ($ch);
echo "<PRE>".htmlentities($buf2);
i had the same problem use this for running in windows localhost server
curl_setopt($ch, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookies.txt");
Try adding a curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile); in there as well.
Also, there's no need to start a fresh curl instance for each request. You can reuse the same instance for multiple requests. Just set the appropriate CURLOPT (url, postfields, get, etc...) each time and curl will sort things out internally.
Try this function the cookijar option is added:
function execute($toLoad) {
if ( !preg_match( '/^http/', $toLoad ) ) {
$toLoad = 'http://'.$toLoad;
}
$cookiefile = APP_PATH.'tmp/cookie.txt';
$data = array();
$ch = curl_init();
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,10);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.6) Gecko/20091216 Fedora/3.5.6-1.fc11 Firefox/3.5.6 GTB6");
curl_setopt($ch, CURLOPT_URL, $toLoad); // The page to download
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, 'http://somesite.com/' );
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
$data['data'] = curl_exec($ch);
$data['status'] = curl_getinfo($ch);
//$this->out(curl_error($ch));
//$this->out(curl_getinfo($ch));
//$this->out('');
//$this->out($data);
//$this->out('');
curl_close($ch);
return $data;
}
you could also try reading the header of the returned data for Set-Cookie: SOMEKEY
and then in the curlopt use
if ($header) {
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cookie:'.$header));
}
Thanks