html page not getting cookies through libcurl - php

i modified my previous code. you can see my previous post if your intersted setting cookie through curl
But here is a fresh beginning my new code looks linke this
i have a php file using curl like this
<?php
$ch=curl_init();
$url="http://localhost/javascript%20cookies/test_cookies.php";
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_COOKIEFILE,dirname(__FILE__) . "/cookie.txt");
curl_setopt($ch,CURLOPT_FOLLOWLOCATION,1);
curl_exec($ch);
curl_close($ch);
?>
the file test_cookies.php looks like this
<?php
if($_COOKIE['user']==1)
{
header("Set-Cookie:user=1; color=blue");
header("Location:http://localhost/javascript%20cookies/test_cookies.html");
}
?>
the file test_cookies.html has some javascript that checks for cookies and if it finds those cookies then it displays the text accordingly.
the php file with curl code is sending the cookies and the test_cookies.php is setting the cookie and redirecting to the page test_cookies.html but this page is not receiving the cookies and thus it is not not showing the content accordingly.
can somebody tell me whats the problem now?
here are the headers i get displayed in firefox on setting CURLOPT_HEADER to true
HTTP/1.1 302 Found Date: Mon, 16 May 2011 15:03:59 GMT Server: Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1 X-Powered-By: PHP/5.3.1 Set-Cookie: user=1; color=blue Location: http://localhost/javascript%20cookies/test_cookies.html Content-Length: 0 Content-Type: text/html HTTP/1.1 200 OK Date: Mon, 16 May 2011 15:03:59 GMT Server: Apache/2.2.14 (Win32) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l mod_autoindex_color PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1 Last-Modified: Mon, 16 May 2011 12:13:24 GMT ETag: "11000000013d0c-493-4a363950a70f3" Accept-Ranges: bytes Content-Length: 1171 Content-Type: text/html
you can see that there are two set of headers displayed.is this because i am making 2 calls to header?

It is not clear what you are trying to achieve with the code. You'd get better help if you explain why you are doing this. It is probable that your approach is wrong and you could get the ultimate result with a different/simpler approach.
And now to answer your question, this is what happens:
From your browser, you send a GET request to curl.php (the name I use for your first file),
curl in that file checks cookie.txt and finds user=1, so it sends a GET request to test_cookies.php and sends the cookie with the request
test_cookies.php sees that user=1 is true, so it sends a header to curl.php and asks it to set two cookies: user=1 and color=blue (for the first cookie this is pointless, user=1 is already set, but let's ignore this as no harm is done)
You have not set CURLOPT_COOKIEJAR option, so when curl.php receives the set cookie header it does not give a damn (nothing happens)
Next, test_cookies.php sends a redirect header to curl.php, since you have set CURLOPT_FOLLOWLOCATION,1, curl.php sends another GET request, this time to get test_cookies.html
Content of test_cookies.html is returned to curl.php,
curl_exec($ch); causes the returned content (source of test_cookies.html) be echoed back to your browser,
Your browser parses what it received and the javascript is executed. It checks for a cookie named user and does not find one, so it displays content for when there is no cookie (because there isn't).
Now, you may wonder what happens if you add:
curl_setopt($ch,CURLOPT_COOKIEJAR,dirname(__FILE__) . "/cookie.txt");
What happens is that your cookie.txt will be updated in step 4 and will have two cookies; user=1 and color=blue. But this does NOT give you the result you expect. Because header("Set-Cookie:user=1; color=blue"); is instructing curl to set the cookie, and curl does this by storing the cookies in the file you specified in CURLOPT_COOKIEJAR option. So, even though you added that option, when you reach step 8 javascript will not find that cookie, because the browser you use does not know or care about your cookie.txt, it looks somewhere else to check if a cookie exists. In Google Chrome for instance the format is SQLite and for XP the cookies are stored in %USERPROFILE%\Local Settings\Application Data\Google\Chrome\User Data\Default\Cookies.

i will now explain what i was trying to do.
i had an html page that asked for some verification code and then it sent that verification code to a php script which on verifying the code set a cookie and redirected to the same html page.
the html page had some externally linked javascript which on checking the cookie value displayed the content of the page with some tweaking.
i am now writing the code for this
the html file with content and form
test_cookies.html
//some css,javascript and html and then a form
<form method="post" action="http://localhost/javascript%20cookies/test_cookies.php">
the php file which verifies the code
test_cookies.php
if($_POST['value']=="code")
setcookie("user",1);
if($_POST['value']!="code")
setcookie("user",1,time()-1);
header("Location:http://localhost/javascript%20cookies/test_cookies.html");
and now the php file with curl code
curl_cookies.php
<?php
$ch=curl_init();
$url="http://localhost/javascript%20cookies/test_cookies.php";
$post="value=code"; //here i have hard-coded the pst value for a demo but i could have got this from the user
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,1);
curl_setopt($ch,CURLOPT_POSTFIELDS,$post);
curl_setopt($ch,CURLOPT_HEADER,1);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
$res=curl_exec($ch);
curl_close($ch);
preg_match('/Location: (.*)\s/',$res,$location);
preg_match('/Set-Cookie: (.*)\s/',$res,$cookie);
$cookie=rtrim($cookie[0])."; path=/ "; //path needed to be changed because curl_cookies.php and test_cookies.html are in different directories.
header($cookie);
header($location[0]);
?>
this finally worked and the browser is displaying the tweaked content. this thing taught me a lot about http.
Thanks to all who helped me when i was in dark.

Related

How to set cookie in Laravel Blade File?

I want to set cookies in laravel blade.php file, not in the controller. How can I set it?
Disclaimer: I will focus my answer on PHP and laravel.
Why not set in controller?
It would really help to know why you cannot / or do not want to set cookies using laravel's cookie Facade in the controller - eg. Cookie::queue, as it's very easy to do!
Here are two ways, from this source.
Via response:
return response(view('welcome'))->cookie('name','value',$min);
Via Queue: Cookie::queue(Cookie::make('name','value',$min)); return view('welcome');
Set-Cookie is a response header, not the body!
Assuming you would set these cookies in PHP , they need to come as part of a response header, and not part of the body (view). This is why you would need to set these in the controller, where you are sending a response!
If you try to use PHP functions to set cookies, you will be met with errors "headers have already been sent"
Per the docs: https://www.php.net/setcookie
setcookie() defines a cookie to be sent along with the rest of the HTTP headers. Like other headers, cookies must be sent before any output from your script (this is a protocol restriction). This requires that you place calls to this function prior to any output, including and tags as well as any whitespace.
To understand what this means, it's helpful to understand the structure of requests and responses:
Requests and Responses are made up of headers and possibly a body.
Note: You can see these in the network tab of your browser's dev tools.
The request headers are like meta data about the request that can tell the server what kind of content is being requested, and who is requesting.
The response headers are like meta data about the response returned that can tell the server what kind of content is being delivered, how long to cache it for, associated cookies that got set.
Example Request Headers:
Content-Type: 'application/json'
Content-Type: 'application/pdf'
Content-Type: 'text/html'
Content-Type: 'text/css'
User-Agent: 'Mozilla/5.0 (<system-information>) <platform> (<platform-details>) <extensions>'
Authorization: 'Bearer <token>'
Example Response Headers:
Content-Type as it may differ from what was requested
Expires: 'Wed, 07 Sep 2022 19:26:49 GMT'
Cross-Origin-Resource-Policy: 'cross-origin'
Date: 'Wed, 07 Sep 2022 19:26:49 GMT'
Content-Length: 0,
Set-Cookie: test_cookie=CheckForPermission; expires=Wed, 07-Sep-2022 19:41:49 GMT; path=/; domain=.doubleclick.net; Secure; HttpOnly; SameSite=none
Notably: - Set-Cookie - tells the browser to add these cookies to application storage (you can view these in application / storage tabs in dev tools)
The response header can have Set-Cookie, not the request header. This makes sense, as usually the cookie information is going to come from the "answer" (response) to the "question" (request) by way of performing some logic, eg - this user is authenticated, here's a cookie to keep their session in place.
Also: Secure & HTTP only Cookies
Cookies can get set with a few options - secure only, and http only. These mean that the cookie must be Set on secure connections (https) and the http only can come from a response and cannot be overridden by JavaScript adjusting (client side)
Example of options for Laravel's Cookie::queue facade:
// $name, $value, $minutes = 0, $path = null, $domain = null, $secure = false, $httpOnly = true
Cookie::queue($name, $value, $ttl, $path, $domain, $secure, $httpOnly);
ttl = "time to live" or how long until it expires eg. 2 minutes

Curl, submitting form with __multiselect parameter

I'm trying to submit a (java servlet) form using CURL in PHP, but it seems like there is a problem with the parameters. I cant really understand why its happening since I'm testing the CURL with a identical string parameters that is being used by the browser.
After some research in diverse forums I wasn't able to find a solution to my particular problem.
this is the POSTFIELDS string generated by the browser (and working):
submissionType=pd&__multiselect_PostCodeList=&selectedPostCode=01&selectedPostCode=02&selectedPostCode=03&__multiselect_selectedPostCodes=
and I'm using and identical (for testing) string in the PHP script but it im getting a HTML file as a answers telling "Missing parameters in search query".
I believe that the form
__multiselect_PostCodeList=
&selectedPostCode=01
&selectedPostCode=02
&selectedPostCode=03
&__multiselect_selectedPostCodes=
is quite wired (never see before this) and I'm wondering that it can be the reason of why the post is not working from CURL.
The form seems to be successfully submitted since I'm getting this header
HTTP/1.1 200 OK
Date: Wed, 07 Aug 2013 08:02:56 GMT
Content-length: 1791
Content-type: text/html;charset=UTF-8
X-Powered-By: Servlet/2.4 JSP/2.0
Vary: Accept-Encoding
Content-Encoding: gzip
Connection: Keep-Alive
Note: I tried submitting the same form from Lynx and I'm also getting the same result ("Missing parameters in search query"). So it seems like its only working with browsers like Mozilla or Chrome.
Please some help will be really appreciated, I don't have any more ideas at this point.
Thanks!
Oscar

sessions not working

So I had developed a basic site, using $_SESSION superglobal variable for the logging in.
so the code basically after checking the login details are valid i store the users details into the session like so:
note I am starting the session before storing these values.
$_SESSION['myusername'] = $myusername;
$_SESSION['myuserid'] = $userid;
$_SESSION['logged_in'] = true;
$_SESSION['mystatus'] = $res['user_status'];
it all worked fine, throughout the time i made the site and tested etc.
now all of a sudden, the sessions are not working, so obviously the users cannot get access after logging in because the site is checking data which isnt in the session.
on the page I store the data like above, straight after i can use this:
echo "username".$_SESSION['myusername'];
echo "status".$_SESSION['mystatus'];
and its there. But when the user is directed to another page and i try:
<?php
session_start();
include ('functions.php');
echo "username".$_SESSION['myusername'];
echo "status".$_SESSION['mystatus']; ....
the values aren't in the session. I have checked that the session id is the same, which it is.
This has always worked, so I am really puzzled.
somebody please help.
EDIT
request header & response header from firebug (page where session appears to be empty)
Response Headersview source
Date Sat, 11 Jun 2011 15:18:48 GMT
Server Apache/2.2.3 (Red Hat)
X-Powered-By PHP/5.1.6
Expires Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma no-cache
Content-Length 3772
Connection close
Content-Type text/html; charset=UTF-8
Request Headersview source
Host students.ee.port.ac.uk
User-Agent Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-gb,en;q=0.5
Accept-Encoding gzip,deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 115
Connection keep-alive
Cookie PHPSESSID=1jqqa2oeivq76h2vhtk4uflkv1
Authorization Basic ZWNlNzAxNDE6cGllczRtZTIy
So it seems you have a problem with keeping your session on a second request.
Session tracking is done via cookies, you should check (with Live HTTP Headers or firebug) the real cookie content sent by the server. In this cookie check the path setting and the server name given, check as well time validity settings, if something is wrong there the browser won't send back the cookie and you'll get a new session on each request.
The web developper Toolbar contains some nice cookie tools as well, where you can display a page containing all cookies details for a given page. If the cookie receveid is not there then the browser assume this cookie is not related to this page. Most of the time a php setting is enforcing the cookie.domain setting to something other than the used DNS.
Given the fact that you haven't changed a thing in the last few weeks and that it used to work, you should check that your server didn't run out of disk space. If it did, it may create a reference to a session but might not be able to serialize the data to disk once the page has been rendered.
This could explain why outputting the $_SESSION[...] works on the same page and why the cookie is set in the response.
Check whether the session id on the second request is the same as the one on the first request.

CURL response different than response to request sent from browser

Attempting to submit a form with CURL, both via PHP and the command line. The response from the server consists of null content (the headers posted below).
When the same URL is submitted via a browser, the response consists of a proper webapge.
Have tried submitting the CURL request parameters via POST and GET via each of the following command line curl flags "-d" "-F" and "-G".
If the query string parameters are posted with "-d" flag, resulting header is:
HTTP/1.1 302 Moved Temporarily
Date: Thu, 02 Jun 2011 21:41:54 GMT
Server: Apache
Set-Cookie: JSESSIONID=DC5F435A96A353289F58593D54B89570; Path=/XXXXXXX
P3P: CP="CAO PSA OUR"
Location: http://www.XXXXXXXX.com/
Content-Length: 0
Connection: close
Content-Type: text/html;charset=UTF-8
Set-Cookie: XXXXXXXXXXXXXXXX=1318103232.20480.0000; path=/
If the query string parameters are posted with "-F" flag, the resulting header is:
HTTP/1.1 100 Continue
HTTP/1.1 500 Internal Server Error
Date: Thu, 02 Jun 2011 21:52:54 GMT
Server: Apache
Content-Length: 1677
Connection: close
Content-Type: text/html;charset=utf-8
Set-Cookie: XXXXXXXXXXXXXX=1318103232.20480.0000; path=/
Vary: Accept-Encoding
<html><head><title>Apache Tomcat/5.5.26 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The server encountered an internal error () that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>javax.servlet.ServletException: Servlet execution threw an exception<br>
</pre></p><p><b>root cause</b> <pre>java.lang.NoClassDefFoundError: com/oreilly/servlet/multipart/MultipartParser<br>
com.corsis.tuesday.servlet.mp.MPRequest.<init>(MPRequest.java:27)<br>
com.corsis.tuesday.servlet.mp.MPRequest.<init>(MPRequest.java:21)<br>
com.corsis.tuesday.servlet.TuesdayServlet.doPost(TuesdayServlet.java:494)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)<br>
</pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/5.5.26 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/5.5.26</h3></body></html>
Questions:
What might cause a server to respond different depending on the nature of the CURL request.
How to successfully submit request via CURL?
HTTP/1.1 100 Continue
I had problems associated with this header before. Some servers simply do not understand it. Try this option to override Expect header.
curl_setopt( $curl_handle, CURLOPT_HTTPHEADER, array( 'Expect:' ) );
To add to what Richard said, I have seen cases where servers check the User-Agent string and behave differently based on its value.
I have just had an experience with this and what fixed it was surprising. In my situation I was logging into a server so I could upload a file, have the server do work on it, and then download the new file. I did this in Chrome first and used the dev tools to capture over 100 HTTP requests in this simple transaction. Most are simply grabbing resources I don't need if I am trying to do all of this from the command line, so I filtered out only the ones I knew at a minimum I should need.
Initially this boiled down to a GET to set the cookie and log in with a username and password, a POST to upload the file, a POST to execute the work on the file, and a GET to retrieve the new file. I could not get the first POST to actually work though. The response from that POST is supposed to be information containing the upload ID, time uploaded, etc, but instead I was getting empty JSON lists even though the status was 200 OK.
I used CURL to spoof the requests from the browser exactly (copying the User-Agent, overriding Expect, etc) and was still getting nothing. Then I started arbitrarily adding in some of the requests that I captured from Chrome between the first GET and POST, and low and behold after adding in a GET request for the JSON history before the POST the POST actually returned what it was supposed to.
TL;DR Some websites require more requests after the initial log in before you can POST. I would try to capture a successful exchange between the server and browser and look at all of the requests. Some requests might not be as superfluous as the seem.

Using cookies with CURL

I'm writing an "API" for a website which doesn't have it.
Basically, my PHP code logs into the website and grabs the data I need (two different transfers).
At login time, I'm getting a bit of a problem. The website sets a couple of cookies through HTTP, which I'm capturing using CURL's cookie mechanism.
This seems to work out nicely, except that they are also trying to set a cookie via javascript in that same response.
I don't need to parse the javascript since the cookie they set is entirely predictable.
What I need is to somehow tell CURL that this cookie exists, WHILE it stills maintains the other cookies.
Help? :)
After submitting the login details via curl POST, I get to these headers:
HTTP/1.1 200 OKDate: Fri, 20 Aug 2010 09:39:14 GMT
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=ISO-8859-1
Content-Length: 492
Set-Cookie: JSESSIONID=5DE1F32B3668DABB408BBEA10C28DBD5.testmmf1; Path=/merchantlogin
Set-Cookie: loginType=M
Connection: close
And this is the page content:
<script type="text/javascript">
var nextyear = new Date();
nextyear.setFullYear(nextyear.getFullYear() + 1);
document.cookie = 'login=' + document.referrer + '; expires=' + nextyear.toGMTString();
</script>
Notice the Set-Cookie and document.cookie parts.
Generate cookie file via code, and before making request to location witch requires that cookie add it simply through setopt with option CURLOPT_COOKIEFILE
You could set the cookie using curl_setopt and the CURLOPT_COOKIE option first. Of course doing this will erase your other cookies, but they'll be gotten back, right?
If you could get a hold of the current value of CURLOPT_COOKIE, you could append your cookie with a semicolon. But PHP doesn't seem to have a curl_getopt function.

Categories