While using cURL with some site, I noticed that some files that I requested actually used several variables from the cookie I set up with curl.
Here's a snapshot:
But when I check my cookie file all it reads is just one value for ASP.NET_SessionId:
www.*******.*** FALSE / FALSE 0 ASP.NET_SessionId ddj24l55lfu11nb1lhuflw55
Of course, the values from the snapshot are taken from my browser (Internet Explorer F12), and that cookie contains Three variables (not one).
Internet Explorer F12 cookie variables Name/Values:
NAME ASPSESSIONIDSACRDADD
VALUE LOONCEMDHCGEJOANEGHHFAFH
NAME ASPSESSIONIDSCBRABDC
VALUE CMONJEMDNICPNPNFICLAPMFM
NAME ASPSESSIONIDQACSBADC
VALUE MCBOGLCCKNIDDBOADNMPCLCD
this is my CURL settings for cookies:
$cookiefile = "d:/cookie.txt";
curl_setopt($curl, CURLOPT_COOKIESESSION, 1);
curl_setopt($curl, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($curl, CURLOPT_COOKIEJAR, $cookiefile);
What is it that I may be missingin cURL ?
Thanks!
By the nature of the div collection, it looks to be like variables are from different browser sessions.
And Even if its not, its up to developer on how to create your application to set and read the data.
Related
I'm using curl succefuly to simulate user on several website but I have a problem on a new one.
I'm handling cookie with this code:
curl_setopt($cs, CURLOPT_COOKIEFILE, "cookies");
curl_setopt($cs, CURLOPT_COOKIEJAR, "cookies");
Usualy that creates a cookies file as you know, but for this website it doesn't, I don't even have a empty file.
However, by putting CURLOPT_HEADER to TRUE I see I receive some "Set-Cookie" values.
I really don't know how it is possible, if someone have an idea.
Thanks
I've got a cURL PHP script which works. It gets my schedule from my school site. Though there is one strange thing: On my webhost it creates the cookie.txt and on my localhost it doesn't.
Why doesn't it create a cookie on my localhost? Any suggestions? Something with relative paths and wampserver?
And the questions that follows the latter:
Is there any (speed) advantage of already being logged in on the school site (storing the cookie and thus saving an cURL request)?
I could for example check after the first cURL request if there is evidence in the response that I am already logged in.
If the answer to the above question is: 'no, this doesn't make the script faster' I've got another question:
Is it than best to specify only the CURLOPT_COOKIEFILE option? With an empty value? So no cookie jar?
I can't give you my login information, though here is the script:
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL,
'http://www.groenewoud.nl/infoweb/infoweb/index.php');
curl_setopt($curl, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($curl, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HEADER, false);
$tokenSource = curl_exec($curl);
print_r (curl_getinfo($curl));
if (!$tokenSource) echo 'token problem';
// Get the token from within the source codes of infoweb.
preg_match('/name="csrf" value="(.*?)"/', $tokenSource, $token);
$postFields = array(
'user' => $userNum,
'paswoord' => $userPass,
'login' => 'loginform',
'csrf' => $token[1]);
$postData = http_build_query($postFields);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, false);
curl_setopt($curl, CURLOPT_POSTFIELDS, $postData);
$tableSource = curl_exec($curl);
print_r( curl_getinfo($curl));
if (!$tableSource) echo 'post problem';
curl_close($curl);
1) /cookie/cookie.txt means you'd need to have your cookie directory in the ROOT directory of your entire server. cookie/cookie.txt (note: NO leading slash) means the cookie directory would be a sub-directory of your script's CURRENT directory. E.g. your script is running in /a/b/c/, then you'd have /a/b/c/cookie/cookie.txt.
2) For speed advantages, there's no change in HTTP speeds - you're still stuck with the same pipes and transfer rates. But having the cookie initially MIGHT save you a few extra hits on the server to simulate the login-sequence, so would effectively be SLIGHTLY faster.
3) As for creating the cookies, that's entirely up to curl's settings. If you don't specify a cookie file or cookie jar, it won't create or look for the cookie file. Check the configuration/compile options between the two servers to see if one specifies some curl defaults that the other doesn't have.
4) str_pos WOULD be faster than a curl request. Think of it as the difference between looking in your fridge for some food versus driving to the grocery store. Fridge is local and therefore faster.
5) curlopt_cookiefile tells curl where to store new cookies. curlopt_cookiejar tells curl where to load cookies from when it first starts up. They CAN be different files, but don't have to be. If you'd like to keep some "clean" baseline cookies, then you use cookiejar = newstuff.txt, and cookiejar=baseline.txt. Once you've got an appropriate cookie environment set up, you reset cookiejar to newstuff.txt for subsequent curl runs.
I have a situation whereby when a page loads, I send some authentication data (in this case the associative array $data) which is verified by a script on another domain. Code below:
$cookie_path = 'cookies.txt';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.mysite.com/verify');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_path);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
$result = curl_exec($ch);
the site then sets a session (in this case I am using the codeigniter framework and sessions are set like: $this->session->set_userdata('logged_in', true); )
however when I load the external site in an iframe it does not seem to be able to detect that the session is set and redirects to the login page.
How do I ensure that my session cookie is being sent properly and can be accessed by an iframe?
Your curl script is running server side and storing the cookie for the second site there, but your browser is loading the second site in the client. You can share cookies across domains.
If you control the site you are attempting to create the session on, you may be able to pass the session ID to the PHP script, then generate the iframe URL dynamically, including the session ID as a query string, eg:
http://www.brainbell.com/tutors/php/php_mysql/Encoding_the_session_ID_as_a_GET_variable.html
Edit
To clarify, if you control the script on the second site, you can modify it to provide the SESSIONID of the authenticated session to your CURL script, which your PHP script making the cURL request can then incorporate into the dynamically generated iFrame src URL.
You can set cookies via:
http://php.net/manual/en/function.setcookie.php
However, you can't set cookies for domains outside of your script's domain.
I'm using a web-service from a provider who is being a little too helpful in anticipating my needs. They have given me a HTML snippet to paste on my website, for users to click on to trigger their services. I'd prefer to script this process, so I've got a php script which posts a cURL request to the same url, as appropriate. However, this provider is keeping tabs on my session, and interprets each new request as an update of the first one, rather than each being a unique request.
I've contacted the provider regarding my issue, and they've gone so far as to inform me that their system is working as intended, and that it's impossible for me to avoid using the same ASP.NET session for each subsequent cURL request. While my favored option would be to switch to a different vendor, that doesn't appear to be an option right now. Is there a reliable way to get a new ASP.NET session with each cURL request?
I've tried the following set of CURLOPT's, to no avail:
//initialize curl
$ch = curl_init($url);
//build a string out of the post_vars
$post_str = http_build_query($post_vars);
//set the necessary curl options
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_str);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_FORBID_REUSE, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "UZ_".uniqid());
curl_setopt($ch, CURLOPT_REFERER, CURRENT_SITE_URL."index.php?newsession=".uniqid());
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Pragma: no-cache", "Cache-Control: no-cache"));
//execute the call to the backend script, retrieve the results
$xmlstr = curl_exec($ch);
If cURL isn't helping much, why not try other methods to call the services from your script, like php's file() function, or file_get_contents().
If you see do not see any difference at all, then the service provider might be using your ip to track your requests. Try using some proxy for a test.
Normal Asp.net session is tracked by a cookie called ASP.NET_SessionId. This cookie is sent within the response to your first request. So as long as your curl requests don't send back this asp.net cookie, each of your requests will have no connection to each other. Use the curl -c option to see what cookies are flying in-between you and them. Overriding this cookie with a cookie file should work if you confirm that it is normal asp.net session being used here.
It is quite poor for a service to use session (http has much cleaner ways of maintaining state which ReST exploits) so I wouldn't completely rule out the vendor switch option.
Well given the options you are using, it seems you have covered your basics. Can you find out how their sessions are setup?
If you know how they setup a session, IE what they use (if it is IP or what not) and then you can figure out a work around. Another option is trying to set the cookies in a different cookie file:
CURLOPT_COOKIEFILE - The name of the file containing the cookie data. The cookie file can be in Netscape format, or just plain HTTP-style headers dumped into a file.
But if all they do is check cookies your current code should work. If you can figure out what the cookie's name is, you can pass a custom cookie that is blank with the request to see if that works. But if you can get information out of them on how their session's work, that would be best.
use these two line to handle the session:
curl_setopt($ch, CURLOPT_COOKIEJAR, "path/to/cookies.txt"); // cookies.txt should be writable
curl_setopt($ch, CURLOPT_COOKIEFILE, "path/to/cookies.txt");
I am tring to read a cookie value which I got after login by sending a POST request.
Then I want to sent that cookie value with another POST request using Curl to another action. But after sending this when I am trying to see all posted header it does display that I have send any cookie value. This value is not available to my posted URL so not able to access the information due to authentication. Please tell me where I have done something wrong:
$URL1 = "http://www.getinf.com/iconf/user?action=buGroup";
$postfields1 = "device=mapp&type=ajax&name1=ra&cc1=91&min1=90name2=imm&cc2=91&min2=97";
// sends a post request
$ch1 = curl_init();
curl_setopt($ch1, CURLOPT_URL,$URL1);
curl_setopt($ch1, CURLOPT_POST, 1);
curl_setopt($ch1, CURLOPT_COOKIE,'JSESSIONID=199FFF6355DEA87F3D72E692E7514AD2');
curl_setopt($ch1, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch1, CURLOPT_HEADER,true);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch1, CURLOPT_POSTFIELDS, $postfields1);
$result = curl_exec($ch1);
print_r(headers_list());// displays all post request data
$ret = ReturnVal($result);
print_r(get_headers($URL1, 1)); //
curl_close ($ch1);
So what is wrong in this code that is preventing JSESSIONID value accessible as a cookie value?
Check the comments (or search "cookie") on this page in the php docs:
Whats not mentioned in the
documentation is that you have to set
CURLOPT_COOKIEJAR to a file for the
CURL handle to actually use cookies,
if it is not set then cookies will not
be parsed.
Try changing CURLOPT_cookie to CURLOPT_COOKIE
CURLOPT_COOKIESESSION is used to start a new Cookie session and ignore all cookies stored.
From PHP.net: Use CURLOPT_COOKIESESSION = TRUE to mark this as a new cookie "session". It will force libcurl to ignore all cookies it is about to load that are "session cookies" from the previous session. By default, libcurl always stores and loads all cookies, independent if they are session cookies or not. Session cookies are cookies without expiry date and they are meant to be alive and existing for this "session" only.
use: curl_setopt($ch, CURLOPT_COOKIESESSION, false);