My code
<?php
$url='Search.jsp';
// disguises the curl using fake headers and a fake user agent.
function disguise_curl($url)
{
$curl = curl_init();
// Setup headers - I used the same headers from Firefox version 2.0.0.6
// below was split up because php.net said the line was too long. :/
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'https://lalpacweb.blackpool.gov.uk/protected/wca/publicRegisterVehicleSearch.jsp');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_COOKIESESSION, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_COOKIEJAR, "cookies.txt");
curl_setopt($curl, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt( $curl, CURLOPT_POST, 1);
curl_setopt ($curl, CURLOPT_POSTFIELDS, 'search.licenceTypeID=34&search.licenceLinkFileID=2&search.vehicleRegNumber=5&publicRegisterVehicle=Search');
$html = curl_exec($curl); // execute the curl command
echo curl_getinfo($curl, CURLINFO_HTTP_CODE);
curl_close($curl); // close the connection
return $html; // and finally, return $html
}
// uses the function and displays the text off the website
$text = disguise_curl($url);
echo $text;
?>
It returns the page, with the form filled, but it does not post it. The curl_getinfo response I get is..
200HTTP/1.1 200 OK Pragma: no-cache Cache-Control:
no-cache,no-store,must-revalidate Expires: Thu, 01 Jan 1970 00:00:00
GMT Content-Type: text/html;charset=ISO-8859-1 Content-Language: en-GB
Content-Length: 5901 Date: Sun, 19 Feb 2012 12:24:08 GMT Server:
Apache
Any ideas ?
Thanks for your help
There's a few things you'll probably want to do, firstly I believe it works better across different operating systems if you supply an absolute path to the cookiejar:
curl_setopt($curl, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookies.txt");
curl_setopt($curl, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookies.txt");
Also, you can have the script visit the homepage first to grab the session cookie:
disguise_curl("https://lalpacweb.blackpool.gov.uk");
Then you can post the form to https://lalpacweb.blackpool.gov.uk/protected/actions/PublicRegister.action (make sure cookies.txt exists):
<?php
// disguises the curl using fake headers and a fake user agent.
function disguise_curl($url, $post = false)
{
$curl = curl_init();
// Setup headers - I used the same headers from Firefox version 2.0.0.6
// below was split up because php.net said the line was too long. :/
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'https://lalpacweb.blackpool.gov.uk/protected/wca/publicRegisterVehicleSearch.jsp');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_COOKIESESSION, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookies.txt");
curl_setopt($curl, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookies.txt");
curl_setopt($curl, CURLOPT_HEADER, 1);
if ($post)
{
curl_setopt( $curl, CURLOPT_POST, 1);
curl_setopt ($curl, CURLOPT_POSTFIELDS, 'search.licenceTypeID=34&search.licenceLinkFileID=2&search.vehicleRegNumber=5&publicRegisterVehicle=Search');
}
$html = curl_exec($curl); // execute the curl command
//echo curl_getinfo($curl, CURLINFO_HTTP_CODE);
curl_close($curl); // close the connection
return $html; // and finally, return $html
}
// Visit the home-page first to get the session cookie
disguise_curl("https://lalpacweb.blackpool.gov.uk");
// uses the function and displays the text off the website
$url = 'https://lalpacweb.blackpool.gov.uk/protected/actions/PublicRegister.action';
$text = disguise_curl($url, true);
echo $text;
?>
When opening https://lalpacweb.blackpool.gov.uk/protected/wca/publicRegisterVehicleSearch.jsp with my browser, I'm redirected to https://lalpacweb.blackpool.gov.uk/sessiontimeout.jsp and presented with a "Session Timeout" error. Maybe you must make two requests. One to login (and possibly obtain the session cookie) and one to actually perform the search. curl should automatically send cookies it has received in previous requests within the same session. Otherwise set it curl_setopt($curl, CURLOPT_COOKIE, 'CookieName=CookieValue');.
$post = urlencode('search.licenceTypeID=34&search.licenceLinkFileID=2&search.vehicleRegNumber=5&publicRegisterVehicle=Search');
or
$post = array(
'search.licenceTypeID' => 34,
'search.licenceLinkFileID' => 2,
'search.vehicleRegNumber' => 5,
'publicRegisterVehicle' => 'Search'
)
curl_setopt ($init, CURLOPT_POSTFIELDS, $post);
Related
To be clear, I want the page containing my form (page1.php) to check the data entered by the user from my other server page (https://anotherserver.com/checker) and send it to page1.php as true or false. This is my first time using curl. I can't see anything with this code..
here is my form page code page1.php
function dataFn($url, $data = array()) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($data));
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_TIMEOUT, 3000);
$data = curl_exec($curl);
curl_close($curl);
return json_decode($data, true);
}
$data = dataFn(base64_encode('my base64 key'), ['value1' => $value2, 'value2' => $value2]);
var_dump($data);
die();
if (!$data) {
// some err
} else {
if ($data['status'] == 0) {
// some code
}
}
here is my checker server page
<?php
// validation page
print_r($_POST['value1']);
?>
You didn't make too many errors. It definitely will not with your code.
Two ways to make your post data.
$post = 'key1=value1&key2=value2&key3=value3';
$post = array('key1'=>value1,'key2'=>value2,'key3'=>'value3');
depending on the data you may need to use urlencode()
$post = urlencode($post);
I do a lot of curl these are my standard post options
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
curl_setopt($ch, CURLOPT_ENCODING,"");
these are my troubleshooting options
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FAILONERROR,true);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
You NEED to see the request (out) and response headers (in), so you NEED these these two options.
These two option must come out after fixing the problem.
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_HEADER, true);
If it's HTTPS you need this one:
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
I always make my own request headers. You can remove that option, it's not mandatory.
$request = array();
$request[] = "Host: www.example.com";
$request[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$request[] = "User-Agent: MOT-V9mm/00.62 UP.Browser/6.2.3.4.c.1.123 (GUI) MMP/2.0";
$request[] = "Accept-Language: en-US,en;q=0.5";
$request[] = "Connection: keep-alive";
$request[] = "Cache-Control: no-cache";
$request[] = "Pragma: no-cache";
If you are having trouble making the POST you look at the header you send and receive.
$data = curl_exec($ch);
if (curl_errno($ch)){echo 'Error: ' . curl_error($ch);}
Sometimes I will save the all response curl info as text and sometime echo
$info = rawurldecode(var_export(curl_getinfo($ch),true));
echo rawurldecode(var_export(curl_getinfo($ch),true));
You want the CURLINFO_HEADER_OUT from the curl_getinfo()
Then you can see what you really sent.
echo curl_getinfo($ch,CURLINFO_HEADER_OUT);
The http response id very important to know.
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
There is a lot of info in the curl_getinfo which may give you other hints.
If I had the URL I'm sure I could get it working.
I find no way to PHP cURL this URL :
http://www.bvger.ch/publiws/pub/cache.jsf?displayName=A-1695/2006&decisionDate=2007-02-27
Can any of you help me ? I tried many ways without any success. For example :
FUNCTION get_data2($url)
{
$curl = curl_init();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'http://www.google.com/bot.html');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 10);
$html = curl_exec($curl); // execute the curl command
curl_close($curl); // close the connection
return $html; // and finally, return $html
}
OR
FUNCTION get_data1($url)
{
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1" );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Both return nothing echoed.
This worked fine for me...
function get_data2($url)
{
$curl = curl_init();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'http://www.google.com/bot.html');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 10);
$html = curl_exec($curl); // execute the curl command
curl_close($curl); // close the connection
return $html; // and finally, return $html
}
$url = 'http://www.bvger.ch/publiws/pub/cache.jsf?displayName=A-1695/2006&decisionDate=2007-02-27';
echo get_data2($url);
Don't capitalize "function". Also you must have been cut-n-pasting and didn't correct one of the lines to match the others using $curl instead of $ch.
I have some problem with the curl function. I am fetching Instagram posts with curl, but suddenly the function stopped working. I tried various changes but I can't make it work.
This is the function:
function cek($url) {
$curl = curl_init();
/**
* Setup headers - I used the same headers from Firefox version 2.0.0.6
* below was split up because php.net said the line was too long.
*/
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)');
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'http://www.google.com');
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($curl, CURLOPT_MAXREDIRS, 10); /* Max redirection to follow */
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 30);
$html = curl_exec($curl); // execute the curl command
curl_close($curl); // close the connection
return $html; // and finally, return $html
}
function get_curl($url) {
if(function_exists('curl_init')) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$output = curl_exec($ch);
echo curl_error($ch);
curl_close($ch);
return $output;
} else {
return file_get_contents($url);
}
}
and this is where I call the function:
#$var = cek('https://api.instagram.com/v1/tags/'.instagram_tag.'/media/recent?client_id='.intagram_clientid.'&count=2');
$var = get_curl('https://api.instagram.com/v1/tags/'.instagram_tag.'/media/recent?max_tag_id=11&min_tag_id=1&access_token='.intagram_token.'&callback=?');
$cek = json_decode($var);
I'm trying to download a file that is generated when a POST button is pressed on an HTTPS page. I'm having to pass the contents of a cookie to show that I'm logged in but I've tested that elsewhere on the same site and that's all working fine.
I'm successfully generating the POST request to request the data that I need and I'm sending it but I'm getting nothing back from the web server at all.
Here's my code:
// Build header
$header = array();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
$header[] = "Content-Type: application/x-www-form-urlencoded";
// Now do transfer
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://domain/pathtoscript");
// Send referer to make this look real
curl_setopt($ch, CURLOPT_REFERER, $referer);
// And disguise ourselves as a browser
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
// Return results as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// Follow redirects
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
// Now send post data
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
// Define where we can store cookies
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookieFile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieFile);
// Sender header and permitted encodings
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_ENCODING, '');
// Actually do it !
$result = curl_exec ($ch);
curl_close ($ch);
// Show result in browser (should be CSV text file)
echo $result;
What am I doing wrong ? I don't want a dialogue box to open to prompt me to save the file, I'd just like the content of the file in $result so that I can parse it later in my application.
Thanks in advance !
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Php - Debugging Curl
I am using curl to fetch the user data from youtube API
The code is
$url_userInfo = 'https://gdata.youtube.com/feeds/api/users/default?access_token=ya29.AHES6ZS7GMdZf91LbMtoOdhFSFOpTuHHT-t7pSggAp-tS0A;
print_r($url_userInfo);
$ch = curl_init($url_userInfo);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, '3');
$content = curl_exec($ch);
curl_close($ch);
print_r( $content);
If I manually visit this url it displays the data in xml form. but there is nothing to print in $content.
Is there any problem with code??
That's actually a HTTPS link, i.e. it uses SSL, and you need to get cacert.pem, and set up cURL for SSL to make that work.
You can get the certificate here!
and you would set it up like so:
$curl = curl_init();
$browser = $_SERVER['HTTP_USER_AGENT'];
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
curl_setopt($curl, CURLOPT_FAILONERROR,true);
curl_setopt($curl, CURLOPT_USERAGENT, $browser);
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_REFERER, 'http://www.google.com');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_AUTOREFERER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl, CURLOPT_TIMEOUT, 20);
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY); //needed for SSL
curl_setopt($curl, CURLOPT_CAINFO, "/scripts/cacert.pem"); //path to file
curl_setopt($curl, CURLOPT_URL, $url_userInfo);
$content = curl_exec($curl);
echo curl_error($curl); //display errors under development
curl_close($ch);
print_r( $content );
Using a proper user agent, and authenticating with SSL and a certificate, just like the browser would.