I am writing a cURL script to access the current days interest rate from the Fannie Mae website which is https. I havent been able to get past the CURLOPT_SSL_VERIFYPEER, true); option.
No username or password is required, however I need SSL verification turned on.
Testing on XAMPP dev server.
I have downloaded the .crt and .pem certs from the website using FF and saved them in the same source dir and pointed to both using CURLOPT_CAINFO, no luck
I downloaded the latest cacert.pem file from http://curl.haxx.se/ca/cacert.pem and pointed to that as well using CURLOPT_CAINFO, no luck.
If I turn CURLOPT_SSL_VERIFYPEER, to false I can retrieve the header (see below), however when I set it to true there is no header.
Tried about 7-8 solutions found by searching on here along with reading the php documention on cURL and trying several workarounds listed there, no luck.
I need to be able to retrieve the header and eventually the body using CURLOPT_SSL_VERIFYPEER, true
Any help is appreciated.
<?php
// script is designed to access an https site and retrieve the last table showing the most recent 90 day commitment for the Fannie Mae 30 year fixed rate mortgage. Site is designed to work with cookies and has a valid SSL cert.
//turn error reporting on
error_reporting(E_ALL); ini_set("display_errors", 1);
// cookie file name/location
$cookie_file_path = "cookies.txt";
// verify if cookie file is accessible and writable
if (! file_exists($cookie_file_path) || ! is_writable($cookie_file_path))
{
echo 'Cookie file missing or not writable.';
exit;
}
// url connection
$url = "https://www.fanniemae.com/content/datagrid/hist_net_yields/cur30.html";
// Initiate connection
$ch = curl_init();
// Set cURL and other options
curl_setopt($ch, CURLOPT_URL, $url); // set url
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); // set browser/user agent
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // automatically follow Location: headers (ie redirects)
curl_setopt($ch, CURLOPT_AUTOREFERER, 1); // auto set the referer in the event of a redirect
curl_setopt($ch, CURLOPT_MAXREDIRS, 5); // make sure we dont get stuck in a loop
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10); // 10s timeout time for cURL connection
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // allow https verification if true
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); // check common name and verify with host name
curl_setopt($ch, CURLOPT_SSLVERSION,3); // verify ssl version 2 or 3
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . "VeriSignClass3PublicPrimaryCertificationAuthority-G5.pem"); // allow ssl cert direct comparison
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'read_header'); // get header
curl_setopt($ch, CURLOPT_NOBODY, true); // exclude body
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE); // set new cookie session
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path); // file to save cookies in
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path); // file to read cookies in
// grab URL and pass it to the browser
curl_exec($ch);
// close cURL connection, save cookie file, free up system resources
curl_close($ch);
// show header
function read_header($ch, $string) {
print "Received header: $string";
return strlen($string);
}
?>
This is the header that is received if CURLOPT_SSL_VERIFYPEER is set to false, blank if true
Received header: HTTP/1.1 200 OK
Received header: Date: Thu, 19 Sep 2013 00:40:16 GMT
Received header: Server: Apache
Received header: Set-Cookie: JSESSIONID=4297C1E1760A836F691FE821FBF8B805.cportal-cl01; Path=/; Secure; HttpOnly
Received header: Cache-Control: no-store
Received header: Expires: Wed, 31 Dec 1969 23:59:59 GMT
Received header: Pragma: no-cache
Received header: X-FRAME-OPTIONS: SAMEORIGIN
Received header: Content-Language: en-US
Received header: Content-Length: 9344
Received header: Content-Type: text/html;charset=ISO-8859-1
Received header:
You're excluding the body by using curl_setopt($ch, CURLOPT_NOBODY, true);. And I don't think you need to install certificate on your machine. The following few lines will give you everything.
$url = 'https://www.fanniemae.com/content/datagrid/hist_net_yields/cur30.html';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url); // set url
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); // set browser/user agent
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'read_header'); // get header
curl_exec($ch);
function read_header($ch, $string) {
print "Received header: $string";
return strlen($string);
}
Related
I'm trying to connect to an API, authenticate a user and then view the user details. This is accomplished by first accessing the login endpoint at
http://api.example.com/login/<username>/<password>
to log in and then the following to view user details:
http://api.example.com/user/
This all works in a web browser. However, once I try to use Curl, the login works fine, but when attempting to view user details, I get back a 401, unauthorized error. I believe this is because Curl isn't saving the session cookies properly? Can someone point out why it isn't working and how to fix it? I've tried searching stack exchange, however, none of the solutions I've tried have worked for my situation. The code I'm using to curl the endpoints is shown below. Thanks!
define("COOKIE_FILE", "cookie.txt");
// Login the user
$ch = curl_init('http://api.example.com/login/joe/smith');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
// Read the session saved in the cookie file
echo "<br/><br/>";
$file = fopen("cookie.txt", 'r');
echo fread($file, 100000000);
echo "<br/><br/>";
// Get the users details
$ch = curl_init('http://api.example.com/user');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
This code will output:
HTTP/1.1 200 OK Date: Mon, 22 Oct 2012 21:23:57 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.14 Set-Cookie: cfapi=f481129c9616b8f69cc36afe16466545; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: application/json X-Powered-By: CFWAPI 0.1a Content-Length: 46 {"status":200,"msg":"Successfully Logged In."}
# Netscape HTTP Cookie File # http://curl.haxx.se/rfc/cookie_spec.html # This file was generated by libcurl! Edit at your own risk. api.example.com FALSE / FALSE 0 cfapi 94f63b07ccf7e34358c1c922341c020f
HTTP/1.1 401 Unauthorized Date: Mon, 22 Oct 2012 21:23:57 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.14 Set-Cookie: cfapi=a8eb015a7c423dde95aa01579c4729a4; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: application/json X-Powered-By: CFWAPI 0.1a Content-Length: 49 {"status":401, "msg":"You need to login first!"}
You also need to set the option CURLOPT_COOKIEFILE.
The manual describes this as
The name of the file containing the cookie data. The cookie file can
be in Netscape format, or just plain HTTP-style headers dumped into a
file. If the name is an empty string, no cookies are loaded, but
cookie handling is still enabled.
Since you are using the cookie jar you end up saving the cookies when the requests finish, but since the CURLOPT_COOKIEFILE is not given, cURL isn't sending any of the saved cookies on subsequent requests.
You have correctly used "CURLOPT_COOKIEJAR" (writing) but you also need to set "CURLOPT_COOKIEFILE" (reading)
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
This is how you do CURL with sessions
//initial request with login data
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/login.php');
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/32.0.1700.107 Chrome/32.0.1700.107 Safari/537.36');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, "username=XXXXX&password=XXXXX");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie-name'); //could be empty, but cause problems on some hosts
curl_setopt($ch, CURLOPT_COOKIEFILE, '/var/www/ip4.x/file/tmp'); //could be empty, but cause problems on some hosts
$answer = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
//another request preserving the session
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/profile');
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_POSTFIELDS, "");
$answer = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
I've seen this on ImpressPages
Yup, often called a 'cookie jar' Google should provide many examples:
http://devzone.zend.com/16/php-101-part-10-a-session-in-the-cookie-jar/
http://curl.haxx.se/libcurl/php/examples/cookiejar.html <- good example IMHO
Copying that last one here so it does not go away...
Login to on one page and then get another page passing all cookies from the first page along Written by Mitchell
<?php
/*
This script is an example of using curl in php to log into on one page and
then get another page passing all cookies from the first page along with you.
If this script was a bit more advanced it might trick the server into
thinking its netscape and even pass a fake referer, yo look like it surfed
from a local page.
*/
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/checkpwd.asp");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "UserID=username&password=passwd");
ob_start(); // prevent any output
curl_exec ($ch); // execute the curl command
ob_end_clean(); // stop preventing output
curl_close ($ch);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/list.asp");
$buf2 = curl_exec ($ch);
curl_close ($ch);
echo "<PRE>".htmlentities($buf2);
?>
I'm trying to connect to an API, authenticate a user and then view the user details. This is accomplished by first accessing the login endpoint at
http://api.example.com/login/<username>/<password>
to log in and then the following to view user details:
http://api.example.com/user/
This all works in a web browser. However, once I try to use Curl, the login works fine, but when attempting to view user details, I get back a 401, unauthorized error. I believe this is because Curl isn't saving the session cookies properly? Can someone point out why it isn't working and how to fix it? I've tried searching stack exchange, however, none of the solutions I've tried have worked for my situation. The code I'm using to curl the endpoints is shown below. Thanks!
define("COOKIE_FILE", "cookie.txt");
// Login the user
$ch = curl_init('http://api.example.com/login/joe/smith');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
// Read the session saved in the cookie file
echo "<br/><br/>";
$file = fopen("cookie.txt", 'r');
echo fread($file, 100000000);
echo "<br/><br/>";
// Get the users details
$ch = curl_init('http://api.example.com/user');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
This code will output:
HTTP/1.1 200 OK Date: Mon, 22 Oct 2012 21:23:57 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.14 Set-Cookie: cfapi=f481129c9616b8f69cc36afe16466545; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: application/json X-Powered-By: CFWAPI 0.1a Content-Length: 46 {"status":200,"msg":"Successfully Logged In."}
# Netscape HTTP Cookie File # http://curl.haxx.se/rfc/cookie_spec.html # This file was generated by libcurl! Edit at your own risk. api.example.com FALSE / FALSE 0 cfapi 94f63b07ccf7e34358c1c922341c020f
HTTP/1.1 401 Unauthorized Date: Mon, 22 Oct 2012 21:23:57 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.14 Set-Cookie: cfapi=a8eb015a7c423dde95aa01579c4729a4; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: application/json X-Powered-By: CFWAPI 0.1a Content-Length: 49 {"status":401, "msg":"You need to login first!"}
You also need to set the option CURLOPT_COOKIEFILE.
The manual describes this as
The name of the file containing the cookie data. The cookie file can
be in Netscape format, or just plain HTTP-style headers dumped into a
file. If the name is an empty string, no cookies are loaded, but
cookie handling is still enabled.
Since you are using the cookie jar you end up saving the cookies when the requests finish, but since the CURLOPT_COOKIEFILE is not given, cURL isn't sending any of the saved cookies on subsequent requests.
You have correctly used "CURLOPT_COOKIEJAR" (writing) but you also need to set "CURLOPT_COOKIEFILE" (reading)
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
This is how you do CURL with sessions
//initial request with login data
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/login.php');
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/32.0.1700.107 Chrome/32.0.1700.107 Safari/537.36');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, "username=XXXXX&password=XXXXX");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie-name'); //could be empty, but cause problems on some hosts
curl_setopt($ch, CURLOPT_COOKIEFILE, '/var/www/ip4.x/file/tmp'); //could be empty, but cause problems on some hosts
$answer = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
//another request preserving the session
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/profile');
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_POSTFIELDS, "");
$answer = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
I've seen this on ImpressPages
Yup, often called a 'cookie jar' Google should provide many examples:
http://devzone.zend.com/16/php-101-part-10-a-session-in-the-cookie-jar/
http://curl.haxx.se/libcurl/php/examples/cookiejar.html <- good example IMHO
Copying that last one here so it does not go away...
Login to on one page and then get another page passing all cookies from the first page along Written by Mitchell
<?php
/*
This script is an example of using curl in php to log into on one page and
then get another page passing all cookies from the first page along with you.
If this script was a bit more advanced it might trick the server into
thinking its netscape and even pass a fake referer, yo look like it surfed
from a local page.
*/
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/checkpwd.asp");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "UserID=username&password=passwd");
ob_start(); // prevent any output
curl_exec ($ch); // execute the curl command
ob_end_clean(); // stop preventing output
curl_close ($ch);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/list.asp");
$buf2 = curl_exec ($ch);
curl_close ($ch);
echo "<PRE>".htmlentities($buf2);
?>
I am opening a HTTPS page using cURL. The page I request issues a redirect request. I have set cURL to follow the redirect, but I cannot seem to be able to get it to request the correct page. I have tracked the same request in a browser and I see my browser making a different request to what cURL makes. What can I do to correct this? The correct URL is shown in the output of a verbose cURL dump. It follows the "* Issue another request to this URL"
Here is a snippet of the output from cURL's verbose output:
< HTTP/1.1 302 Moved Temporarily
< Location: /XXX
< Content-Type: text/html; charset=UTF-8
< Date: Tue, 31 Dec 2013 15:51:46 GMT
< Expires: Tue, 31 Dec 2013 15:51:46 GMT
< Cache-Control: private, max-age=0
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< Server: GSE
< Alternate-Protocol: 443:quic
< Transfer-Encoding: chunked
<
* Ignoring the response-body
* Connection #0 to host 127.0.0.1 left intact
* Issue another request to this URL: 'XYYYZ'
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET /??? HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0
The PHP code I use follows:
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:8888');
$target = ADDR;
curl_setopt($ch, CURLOPT_URL, $target);
$page = curl_exec($ch);
cURL follows the Location: Header, but be sure to send the exact headers (content-language, referer) browser does using CURLOPT_HTTPHEADER option because some servers refuse connectios to prevent automated requests. In Firefox you have live http headers to see what browser does.
Also make sure the Location: header contains the absolute url and not a relative path according to http 1.1.
If that dosen't work you can use the option CURLOPT_HEADER with curl_info to catch the 302 and redirect it manually.
Here i post an example to do it manually so you check if would produce an infinite loop.
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:8888');
$target = ADDR;
curl_setopt($ch, CURLOPT_URL, $target);
$page = curl_exec($ch);
$curl_info = curl_getinfo($ch);
if ($curl_info['http_code'] == 302 || $curl_info['http_code'] == 301)
{
$response_headers = substr($page, 0, $curl_info['header_size']);
if (preg_match('#Location: (.*)#', $response_headers, $location_header))
{
// Call again curl to follow location; Better to wrap the curl process in a function called follow_location
// echo $location_header return an Array
// echo $location_header[0] return "Location: http//blablabla"
// echo $location_header[1] return URL only "http://blablbalba.com" and you can process with cURL :D
echo $location_header[1];
}
}
I'm trying to connect to an API, authenticate a user and then view the user details. This is accomplished by first accessing the login endpoint at
http://api.example.com/login/<username>/<password>
to log in and then the following to view user details:
http://api.example.com/user/
This all works in a web browser. However, once I try to use Curl, the login works fine, but when attempting to view user details, I get back a 401, unauthorized error. I believe this is because Curl isn't saving the session cookies properly? Can someone point out why it isn't working and how to fix it? I've tried searching stack exchange, however, none of the solutions I've tried have worked for my situation. The code I'm using to curl the endpoints is shown below. Thanks!
define("COOKIE_FILE", "cookie.txt");
// Login the user
$ch = curl_init('http://api.example.com/login/joe/smith');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
// Read the session saved in the cookie file
echo "<br/><br/>";
$file = fopen("cookie.txt", 'r');
echo fread($file, 100000000);
echo "<br/><br/>";
// Get the users details
$ch = curl_init('http://api.example.com/user');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
This code will output:
HTTP/1.1 200 OK Date: Mon, 22 Oct 2012 21:23:57 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.14 Set-Cookie: cfapi=f481129c9616b8f69cc36afe16466545; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: application/json X-Powered-By: CFWAPI 0.1a Content-Length: 46 {"status":200,"msg":"Successfully Logged In."}
# Netscape HTTP Cookie File # http://curl.haxx.se/rfc/cookie_spec.html # This file was generated by libcurl! Edit at your own risk. api.example.com FALSE / FALSE 0 cfapi 94f63b07ccf7e34358c1c922341c020f
HTTP/1.1 401 Unauthorized Date: Mon, 22 Oct 2012 21:23:57 GMT Server: LiteSpeed Connection: close X-Powered-By: PHP/5.3.14 Set-Cookie: cfapi=a8eb015a7c423dde95aa01579c4729a4; path=/ Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: application/json X-Powered-By: CFWAPI 0.1a Content-Length: 49 {"status":401, "msg":"You need to login first!"}
You also need to set the option CURLOPT_COOKIEFILE.
The manual describes this as
The name of the file containing the cookie data. The cookie file can
be in Netscape format, or just plain HTTP-style headers dumped into a
file. If the name is an empty string, no cookies are loaded, but
cookie handling is still enabled.
Since you are using the cookie jar you end up saving the cookies when the requests finish, but since the CURLOPT_COOKIEFILE is not given, cURL isn't sending any of the saved cookies on subsequent requests.
You have correctly used "CURLOPT_COOKIEJAR" (writing) but you also need to set "CURLOPT_COOKIEFILE" (reading)
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
This is how you do CURL with sessions
//initial request with login data
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/login.php');
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/32.0.1700.107 Chrome/32.0.1700.107 Safari/537.36');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, "username=XXXXX&password=XXXXX");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie-name'); //could be empty, but cause problems on some hosts
curl_setopt($ch, CURLOPT_COOKIEFILE, '/var/www/ip4.x/file/tmp'); //could be empty, but cause problems on some hosts
$answer = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
//another request preserving the session
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/profile');
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_POSTFIELDS, "");
$answer = curl_exec($ch);
if (curl_error($ch)) {
echo curl_error($ch);
}
I've seen this on ImpressPages
Yup, often called a 'cookie jar' Google should provide many examples:
http://devzone.zend.com/16/php-101-part-10-a-session-in-the-cookie-jar/
http://curl.haxx.se/libcurl/php/examples/cookiejar.html <- good example IMHO
Copying that last one here so it does not go away...
Login to on one page and then get another page passing all cookies from the first page along Written by Mitchell
<?php
/*
This script is an example of using curl in php to log into on one page and
then get another page passing all cookies from the first page along with you.
If this script was a bit more advanced it might trick the server into
thinking its netscape and even pass a fake referer, yo look like it surfed
from a local page.
*/
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/checkpwd.asp");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "UserID=username&password=passwd");
ob_start(); // prevent any output
curl_exec ($ch); // execute the curl command
ob_end_clean(); // stop preventing output
curl_close ($ch);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/list.asp");
$buf2 = curl_exec ($ch);
curl_close ($ch);
echo "<PRE>".htmlentities($buf2);
?>
I am getting an error on the browser saying:
HTTP/1.1 500 Internal Server Error Date: Fri, 06 May 2011 20:25:28 GMT Server: IBM_HTTP_Server/6.0.2.43 Apache/2.0.47 (Unix) $WSEP: Set-Cookie: JSESSIONID=0000HpGRXpuwrdY_u0k-ecHKAFK:14ekdcv70; Path=/ Connection: close Transfer-Encoding: chunked Content-Type: text/html;charset=ISO-8859-1 Content-Language: en Error 500: Browser must support session cookies.
How to solve this problem?
here what I did:
session_start();
$postData = http_build_query($_GET);
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . "\BuiltinObjectToken-VerisignClass3PublicPrimaryCertificationAuthority.crt");
curl_setopt($ch, CURLOPT_URL, "https://zzzzzz.zzzzz.co.uk/zzz/zzzz/" . $form_link );
curl_setopt($ch, CURLOPT_POST, 1 );
curl_setopt($ch, CURLOPT_POSTFIELDS, $postDataCapcha);
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookiefile");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookiefile");
curl_setopt($ch, CURLOPT_COOKIE, session_name() . '=' . session_id());
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$PagaeCapcha = curl_exec($ch);
exit($PagaeCapcha);
Set-Cookie: JSESSIONID=0000HpGRXpuwrdY_u0k-ecHKAFK:14ekdcv70; Path=/
This is the response header which initially sets a session cookie. This one will not get stored in your cookiefile jar. It's a temporary cookie, and you are throwing it away.
You will have to first issue a requesting request that points to the e.g. homepage. And only afterwards send the actual data request to the desired endpoint /zzz/zzzz/.
Problem is that you're sending the name of the session that YOUR copy of PHP has created. This is almost certainly not the name of the session that the .co.uk server has created. So it's seeing that as your "browser" not supporting cookies - it tries to set a session cookie named 'JSESSIONSID', but you send back a cookie named 'PHP_SESSID' (or whatever).