I've been trying to get the contents of a webpage using cURL, but have trouble getting cURL to accept cookies.
For example, on Target.com, when I cURL it, it still says that I have to enable cookies.
Here is my code:
$url = "http://www.target.com/p/Acer-Gateway-15-6-Laptop-PC-NV57H77u-with-320GB-Hard-Drive-4GB-Memory-Black/-/A-13996190#?lnk=sc_qi_detailbutton";
$ch = curl_init(); // initialize curl handle
curl_setopt($ch, CURLOPT_URL,$url); // set url to post to
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);// allow redirects
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); // return into a variable
curl_setopt($ch, CURLOPT_TIMEOUT, 10); // times out after 4s
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:11.0) Gecko/20100101 Firefox/11.0");
$cookie_file = "cookie1.txt";
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
$result = curl_exec($ch); // run the whole process
curl_close($ch);
echo $result;
What else am I missing?
The cookie1.txt file is 077 permission, by the way.
077 is an malformed permission setting, this means the owner (probably apache) has no access. Try setting it to 644 (owner has read/write) as it's only a file.
Related
ima having a problem with login via curl function......
My problem is that it would like to be able to login without the cookie.txt.......
because if i remove cookie.txt i cant login........ when cookie.txt is there it logins successfully, but i would like to login without using cookies....... i tried unlinking cookie.txt but as i said i cant login then......
PART OF THE CODE
$ret=false;
$useragent = "Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10";
$data = setData($email,$pass);
$ch = curl_init('https://www.website.com/login.php');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_ENCODING , "gzip,deflate");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 40);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_COOKIEFILE, dirname(__FILE__) . '/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . '/cookie.txt');
$source=curl_exec($ch);
$info=curl_getinfo($ch);
if($info["redirect_count"]==1)
{
$ret=true;
}
You can't loging without using cookies, neither via curl, nor via browser (unless the site you are logging to implements a different mechanism to save the session id, for example as part of the urls for example, but this is rarely the case and it doesn't depend on you). The reason is that without the cookie the server can't know that the request comes from you and not from someone else.
Facebook doesn't implement a login system that doesn't use cookies, so you can't.
I'm in this new medium curl but I search the internet for a solution and can not find it. I'm trying to fill a remote form using curl and send data by post. the problem is that the external website has some security measures. One of those is that I need to complete the form to get the value that was generated and keep the cookie. external code page reads:
document.getElementById('sell_session').value = readCookie('classified_session');
My code is this:
$cookie_file = "/home/reelonhe/public_html//temp/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'http://www.olx.com.ar/posting.php?categ_id=857');
curl_setopt($ch , CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2');
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Accept-Language: es-es,en"));
curl_setopt($ch, CURLOPT_SSLVERSION,3);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$result1 = curl_exec($ch);
$error = curl_error($ch);
$contents = curl_exec($ch);
$httpcode = curl_getinfo($ch,CURLINFO_HTTP_CODE);
curl_close($ch);
echo $error;
I tried the absolute path of the cookie with relative path. etc folder and nothing has permission to read and write. Do not know what else to do.
In your cookie file path
$cookie_file = "/home/reelonhe/public_html//temp/cookie.txt";
There seems an extra '/' before temp folder.
It should be
$cookie_file = "/home/reelonhe/public_html/temp/cookie.txt";
I am not sure it will solve your problem.
I am attempting to login to Facebook using curl, but everything I have tried has ended up in Facebook saying, "Cookies are not enabled on your browser. Please enable cookies in your browser preferences to continue."
$login_email = 'email';
$login_pass = 'password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.facebook.com/login.php');
curl_setopt($ch, CURLOPT_POSTFIELDS,'email='.urlencode($login_email).'&pass='.urlencode($login_pass).'&login=Login');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "http://www.facebook.com");
$page = curl_exec($ch) or die(curl_error($ch));
echo $page;
The cookie file 'cookies.txt' exists, and has 644 permissions.
I have also attempted to use multiple of the snippets online, but they all give the same error. I cannot continue with my current project until I get this working and I am able to navigate Facebook using curl as well. Any help is appreciated.
Thanks in advance.
This may help:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
Check this answer:
Post to a Facebook user's wall with cURL PHP
I had the same problem and I fixed it by adding the following:
curl_setopt($s, CURLOPT_COOKIESESSION, false);
This must be used on curl cookie :
curl_setopt($ch, CURLOPT_COOKIEFILE, getcwd () . '/mirazmac_cookie.txt' );
curl_setopt($ch, CURLOPT_COOKIEJAR, getcwd () . '/mirazmac_cookie.txt' );
Better to use use facebook login sdk
Because Facebook constantly makes changes to their source code.
I'm trying to get content of this page: http://www.nytimes.com/2014/01/26/us/politics/rand-pauls-mixed-inheritance.html?hp&_r=0
I tried file_get_contents and curl solution but all gives me a Login page of NYTimes and I have no idea why.
Tried these file_get_contents()/curl getting unexpected page, PHP file_get_contents() behaves differently to browser, file_get_content get the wrong web
Is there any solution? Thanks
EDIT:
//this is the curl code I use
$cookieJar = dirname(__FILE__) . '/cookie.txt';
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieJar);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookieJar);
curl_setopt($ch, CURLOPT_URL, $link);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12');
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
try to test it using saving cookies to same directory where the script resides first
so set the cookies path like that
$cookie = "cookie.txt";
this code works with me and i got the page
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("http://www.nytimes.com/2014/01/26/us/politics/rand-pauls-mixed-inheritance.html?hp&_r=1");
echo $get_page;
?>
I think you need cURL to allow cookies to be saved. Try adding these lines to the cURL setup. For me this worked:
$cookie = dirname(__FILE__) . "\cookie.txt";
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
Use Live HTTP Headers firefox plugin to check what is going on during page access. There can be redirections, cookie set etc. And then try to implement this behaviour with php curl (note: set user-agent as and other client headers the same as browser)
Trying to automate login to a ASP.NET site using PHP & cURL but running into a cookie problem.
When I check in the browser, initial login page stores 5 cookies. Which are ASP.NET_SessionId, __utma, __utmb, __utmc & __utmz
When this page is accessed via cURL the cookie file is storing only one cookie: "ASP.NET_SessionId"
I referred to many posts & tried all kinds of cURL option combinations returning the same result.
I don't know how ASP.NET cookies work or differ from PHP. Any help is appreciated.
Here is my php code:
$cookie_file_path = "tmp/cookie.txt";
$LOGINURL = "https://godaddy.com";
$agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_URL, $LOGINURL);
$content = curl_exec($ch);
curl_close($ch);
echo '<textarea style="width:1000px; height:300px">'.$content.'</textarea>';
__utma, __utmb, __utmc & __utmz are all Google Analytics cookies stored by javascript, thus being created client side.
So no way of processing them through cURL / PHP