I need to fetch some data from external site. To do this I need to load a site that creates some cookie and gives a simple math calculation to generate new link. This part is easy:
$cr = curl_init($url);
curl_setopt($cr, CURLOPT_RETURNTRANSFER, true);
curl_setopt($cr, CURLOPT_COOKIEJAR, 'cookie.txt');
$html = curl_exec($cr);
curl_close($cr);
After those calls I have cookies stored in cookie.txt:
# Netscape HTTP Cookie File
# http://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
domain FALSE / FALSE 0 PHPSESSID o89t753egbp9pq9084n38eg2m1
Now, the question is: how can I load this cookie to use it in my next call to some other site (in the same domain ofcourse)?
You can use
curl_setopt($cr, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($cr, CURLOPT_COOKIEJAR, 'cookie.txt');
in both requests.
A cookie is a header, then you can simply use the command setcookie($name,$value), or if you prefer, using the header php call, using Cookie for header name.
Remember, you don't have to send output to the client before the setcookie command.
You have to read file and split it.
Related
I already have a cookie file saved that I want to reference and update. I also want to specify my own additional cookie values via CURLOPT_COOKIE and save those to my existing cookie file as well.
However, I am unable to get this to work.
My code is:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $website); // Define target site
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return page in string
curl_setopt($ch, CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_COOKIE, "fruit=apple;");
curl_setopt($ch, CURLOPT_COOKIEJAR, "usercookies/cookie_$user.txt"); // Tell cURL where to write cookies
curl_setopt($ch, CURLOPT_COOKIEFILE, "usercookies/cookie_$user.txt"); // Tell cURL which cookies to send
curl_setopt($ch, CURLOPT_TIMEOUT,15);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
$returnx = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
My saved cookie file does not reflect the changes I made via curl_setopt($ch, CURLOPT_COOKIE, "fruit=apple;");. The cookiefile saved should show "fruit=apple" but it's still showing the old values or the values returned by the cURL request.
Do I need to reference the entire domain name in order to get it to save?
The cookie file looks something like this:
# Netscape HTTP Cookie File
# http://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
.go.com TRUE / FALSE 1754020486 one AE4F4981
.go.com TRUE / FALSE 1468965260 two B9A1
Cookies you add manually using CURLOPT_COOKIE won't get saved to the cookie jar at the end of the request.
The only case in which it would is if the server sent back a Set-Cookie header for the cookie you sent in order to update it.
The reason is because cURL requests have a cookie structure that holds cookies which gets written at the end of the request. Data only gets in this structure by a) being read from the cookie file in the first place or b) Set-Cookie headers in the response headers.
With a little care you can append your own cookie to that file with something like this:
$domain = '.go.com';
$expire = time() + 3600;
$name = 'fruit';
$value = 'apple';
file_put_contents($cookieJar, "\n$domain\tTRUE\t/\tFALSE\t$expire\t$name\t$value", FILE_APPEND);
Does anyone knows how I could get a list of cookies for an external URL using php?
I think this could be done with cURL?
When i'm using cookiejar or using get_headers, I only see one cookie (the PHPSESSID) for example. But when you open chrome console (F12) and go to cookie storage, you see a much bigger list. Also the google analytics cookies for example. I want to be able to display that list of cookies. So also 3rd party cookies..
Is there any way to retrieve those cookies? Maybe store them temporary or something?
If I got you right & you mean getting cookies that a page sets when visiting it using cURL,
You can use CURLOPT_COOKIEJAR option of cURL for auto manage cookies with cURL (curl_setopt):
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$cont = curl_exec ($ch);
curl_close ($ch);
Or search response header for Set-Cookie: line:
how to get the cookies from a php curl into a variable
There are 3 files in one folder:
curl.php:
$ch = curl_init('http://localhost/url.php');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
curl_exec($ch);
url.php:
setcookie('test', 'foo', time()+60*60*24, '/');
cookies.txt: (empty)
When I run curl.php, I expect that cookie will be saved in cookies.txt.
But nothing happens.
What is wrong with this, why it does not working?
Does it have permission to write the file in that directory with the user that it executes as?
If no cookies have been received, there will be no file saved and you should note that the cookies are saved to the file when the handle is closed.
It might be necessary to give the absolute path of the text file where the cookies should be saved to be sure you know where they end up.
According to PHP documentation:
CURLOPT_COOKIEJAR - The name of a file to save all internal cookies to when the handle is closed, e.g. after a call to curl_close.
So you need to call curl_close($ch).
Just define path.
Change curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt'); to curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . 'cookies.txt'); and should work.
And remember. COOKIEJAR is to save cookies, and COOKIEFILE is to load again to work with them.
Happy cURL ;P
I would like to perform an http request and pass all cookies received by the current script (in particular session identifying cookies) to this request. Then I would like to save the result in a string for further manipulation. What is the best way to do this in PHP ?
cURL ? - it is simple and supprot cookies .
Edit 19.1 - Here is example
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
CURLOPT_COOKIEJAR is file where cURL put cookies sent from server and CURLOPT_COOKIEFILE is file with cookies for sending by cURL ( setting it to same one will make it cookies file ).
Another option is manually read cookies from result ( set CURLOPT_HEADER to '1' - it will put result header into $output ) and send cookies via CURLOPT_COOKIE ( set it to list of cookies in format 'foo=bar;bar=foo;' )
Note - libcurl must be enabled in php.ini
I'm using a web-service from a provider who is being a little too helpful in anticipating my needs. They have given me a HTML snippet to paste on my website, for users to click on to trigger their services. I'd prefer to script this process, so I've got a php script which posts a cURL request to the same url, as appropriate. However, this provider is keeping tabs on my session, and interprets each new request as an update of the first one, rather than each being a unique request.
I've contacted the provider regarding my issue, and they've gone so far as to inform me that their system is working as intended, and that it's impossible for me to avoid using the same ASP.NET session for each subsequent cURL request. While my favored option would be to switch to a different vendor, that doesn't appear to be an option right now. Is there a reliable way to get a new ASP.NET session with each cURL request?
I've tried the following set of CURLOPT's, to no avail:
//initialize curl
$ch = curl_init($url);
//build a string out of the post_vars
$post_str = http_build_query($post_vars);
//set the necessary curl options
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_str);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_FORBID_REUSE, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "UZ_".uniqid());
curl_setopt($ch, CURLOPT_REFERER, CURRENT_SITE_URL."index.php?newsession=".uniqid());
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Pragma: no-cache", "Cache-Control: no-cache"));
//execute the call to the backend script, retrieve the results
$xmlstr = curl_exec($ch);
If cURL isn't helping much, why not try other methods to call the services from your script, like php's file() function, or file_get_contents().
If you see do not see any difference at all, then the service provider might be using your ip to track your requests. Try using some proxy for a test.
Normal Asp.net session is tracked by a cookie called ASP.NET_SessionId. This cookie is sent within the response to your first request. So as long as your curl requests don't send back this asp.net cookie, each of your requests will have no connection to each other. Use the curl -c option to see what cookies are flying in-between you and them. Overriding this cookie with a cookie file should work if you confirm that it is normal asp.net session being used here.
It is quite poor for a service to use session (http has much cleaner ways of maintaining state which ReST exploits) so I wouldn't completely rule out the vendor switch option.
Well given the options you are using, it seems you have covered your basics. Can you find out how their sessions are setup?
If you know how they setup a session, IE what they use (if it is IP or what not) and then you can figure out a work around. Another option is trying to set the cookies in a different cookie file:
CURLOPT_COOKIEFILE - The name of the file containing the cookie data. The cookie file can be in Netscape format, or just plain HTTP-style headers dumped into a file.
But if all they do is check cookies your current code should work. If you can figure out what the cookie's name is, you can pass a custom cookie that is blank with the request to see if that works. But if you can get information out of them on how their session's work, that would be best.
use these two line to handle the session:
curl_setopt($ch, CURLOPT_COOKIEJAR, "path/to/cookies.txt"); // cookies.txt should be writable
curl_setopt($ch, CURLOPT_COOKIEFILE, "path/to/cookies.txt");