I'm using this code for to fake the referrer of a user when he clicks on my link, to make it look like he's coming from Facebook:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://bit.ly/randomurl');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, 'https://www.facebook.com/')
$html = curl_exec($ch);
?>
But it doesn't seem to be working, as the referrer I see is the url of the code above.
How can I fix it? And I really could appreciate some help with the coding as I'm not a coder.
I'm using Live HTTP Headers from Mozilla
You are examining the headers sent by Firefox, but the referer header you are setting manually is being sent by PHP/cURL. That is a different HTTP client and a different set of HTTP requests.
Firefox will request your PHP program (and send normal referer headers to it).
Your PHP program will request http://bit.ly/randomurl (and send the referer header you manually specify to it).
http://bit.ly/randomurl will respond to your PHP program.
Your PHP program will respond to Firefox.
Related
I`m using cuRL to get some data from remote server... The response is in JSON format..
This is my code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER , 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_URL, 'http://www.myaddress.com/mypage.php');
curl_setopt($ch, CURLOPT_POSTFIELDS, array("id" => $id));
$return = curl_exec($ch);
curl_close($ch);
If I access the link in the browser the page load OK, but if I access through the cuRL return a 404 error...
I can guess a few things that it can be checked from the server side, to show the error.
1) As it is stated in other answers, be sure to set all the necessary headers, you can check them e.g. by firebug, as it is shown in here,
or you can get the headers by php get_headers function.
to set it use
curl_setopt($ch, CURLOPT_HTTPHEADER, array("HeaderName: HeaderValue"));
2) When you open a page in the browser(excluding form submit with post method) it makes a get request, instead of post, so if in the server side it is checked $_GET, then your post request will not be considered.
3) If you sure that it should be a post request(say, it is a form submit), then the following can be a problem: some forms can have hidden fields, that again are being checked in the server, and if they are not set, error can be returned. So, you should look at the source code of the form and add them(if there are any) to your post parameters.
4) if you are submitting a form, be sure to set the submit button with its name and value as well, because similar to hidden fields, this can be checked as well.
5) Cookies can be a problem as well, because by default browser has it , and curl does not. To to able to set and read cookies use this code
// set cookie
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
// use cookie
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
here, $cookie_file path to the cookies file. Do not know in linux or mac, but in windows be sure to use absolute path to the cookie file.
6) Also, you can set the referer by
curl_setopt($ch, CURLOPT_REFERER, 'http://www.myaddress.com/mypage.php');
EDIT: In case of ajax request you might want to add a header X-Requested-With with value as XMLHttpRequest
It's possible the server check the HTTP Header, it's the case in the majority of case.
So add the same HTTP Header of your browser, verify with Firebug :
curl_setopt($ch, CURLOPT_HTTPHEADER, array('SomeName: SomeValue'));
Probably there is something else the browser is sending your cURL code is not. You can use any of the tools other folks have suggested, Firebug, Wireshark, Fiddler, etc, etc.
What you need to do is add missing pieces to your request to match the browser as closely as possible in the cURL request until the remote page responds with a 200.
I notice you're doing a POST. In many cases what happens with your browser is you visit a page with a GET request. A session is initialized on the remote site and a cookie is saved in your browser with the session id.
This cookie then needs to be supplied along with subsequent POST requests. PHP cURL has many options to support cookies. There may be other requirements such as CSRF tokens and so forth.
Again, reverse-engineering is the key.
I have, of course, read several questions with exactly this asked, but I have to say it didn't work for me at all. What I am about to accomplish is
sending 'X-Requested-With: XMLHttpRequest' header via PHP and curl
sending other http request headers via PHP and curl
provided solutions didn't work for me.
How do I know I'm not sending right http request headers?
Simply by
(1)comparing real headers generated by XMLHttpRequest(triggering JQuery click) and those simulated by PHP and curl in Firefox add-on Live HTTP headers
(2)Print_r() -ing $_SERVER variable in target script
What do I get that is incorrect/below my expectations?
First and most important:
Firefox Live HTTP headers does not capture my headers (just like they don't exists).
Second, by print_r($_SERVER):
if I get anything of simulated headers at all, I get [HTTP_X_REQUESTED_WITH] => XMLHttpRequest - not the: [X_REQUESTED_WITH] => XMLHttpRequest.
That problem persists almost for any header I send via curl_setopt($ch, CURLOPT_HTTPHEADER, $curl_header) - any of these is being prefixed with 'HTTP' ('Header1: value1' - I get 'HTTP_HEADER1').
I'm using XAMPP with PHP version 5.4.7, CURL 7.24.0 .
Before I ask if what I'm trying to accomplish is possible or maybe not and say thanks in advance for responses, it's not bad idea to provide my code - one of many code solutions that I've tried.
$curl_header = array('X-Requested-With: XMLHttpRequest');
$data = "name=miloshio"; // just to be sure I'm doing the POST request
$ch = curl_init('http://example.com/test.php');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $curl_header);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_VERBOSE, true);
$result = curl_exec($ch);
echo $result;
Sum of my questions:
Is it possible to send exactly 'X-Requested-With: XMLHttpRequest'
header via PHP and curl?
Is it possible to avoid attaching 'HTTP_' prefix to custom headers
send by PHP and curl?
Are there well-known limitations in matter of using PHP and curl?
Firefox Live HTTP headers won't show your headers as they're sent by the server to another server and not to the client(browser).
Curl send the headers correctly, using CURLOPT_PROXY You can try to put curl traffic through a debuging proxy like Fiddler if You're using windows for development, I'm sure there are linux alternatives
If you try to get the headers from $SERVER variable, they will be prefixed with HTTP, you can use apache_request_headers to get the headers without HTTP_ prefix.
So how can I check using codeigniter if the client is curl, and then return something different for it?
You can fake the user-agent when using cURL, so it's pointless depending on the user-agent sent when you KNOW it's a cURL request.
For example: I recently wrote an app which gets the pagerank of a url from google. Now Google doesn't like this, so it allows only a certain user agent to access its pagerank servers. Solution? Spoof the user-agent using cURL and Google will be none the wiser.
Moral of the story: cURL user agents are JUST NOT reliable.
If you still want to do this, then you should be able to get the passed user agent just like normal
$userAgent=$_SERVER['HTTP_USER_AGENT'];
EDIT A quick test proved this:
dumpx.php:
<?php
$url="http://localhost/dump.php";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
if($_GET['u']==y) {
curl_setopt($ch, CURLOPT_USERAGENT, "booyah!");
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
//curl_setopt($ch, CURLOPT_CUSTOMREQUEST,'GET');
curl_setopt ($ch, CURLOPT_HEADER, 0);
$exec=curl_exec ($ch);
?>
dump.php:
<?php
var_dump($_SERVER);
?>
Case 1: http://localhost/dumpx.php?u=y
'HTTP_USER_AGENT' => string 'booyah!' (length=7)
Case 2: http://localhost/dumpx.php?u=n
No $_SERVER['HTTP_USER_AGENT']
This proves that there is no default user agent for curl: it will just not pass it in the request header
If you want to detect bots you can not rely on user agent. Best practices are:
Check, that your visitor runs js (not all human users also do).
Check, that your visitor loads additional files linked to webpage (css, images, etc.)
Check visitor timeouts. Humans usualy don't load 10 pages per second.
cURL stands for - Client URL Library and the whole point of it is to be able to make requests that are identical to what a client would make.
The only thing you can do is detect the information that is part of the request, such as the IP address, HTTP Request Headers, cookies/session id cookie, URL (path/page), and any post/get data. If the person using curl to make the request is doing it from an expected IP address and is supplying any expected header/cookie/token/URL/post/get values, then you would not be able to distinguish a curl request from a browser making the request.
You can spoof or set a custom user agent header when using cURL, so it wouldn't be reliable.
Otherwise, you can do this:
if(strtolower($this->input->server('HTTP_USER_AGENT', true)) == 'curl')
{
// Is using cURL
}
This would only occur if the cURL request contained curl in the user agent header.
As far as I know, there is no default user agent set when doing a curl request.
I want to set a request header for a url xyz.com
is it the right way to set it in php?
header('Authorization: AuthSub token="xxxxxx"');
header('location:https://www.google.com/accounts/AuthSubRevokeToken');
I am trying to set the header for this URL for a call.But the Authorization: AuthSub header doesnt shows up in the request headers section of the FireFox NET panel.Which is used to show the requests.
Any idea about it?
Thanx.
I was using curl previously,But it didnt seemed to issue any request as i cant see it in the NET panel of FireFox.
Code is as follows:
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL,"https://www.google.com/accounts/AuthSubRevokeToken");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Authorization: AuthSub token="1/xxx"'
));
$result = curl_exec($curl);
curl_close($curl);
echo 'hererer'.$result;exit;
header sets response headers, not request headers. (If you were trying to send a HTTP request elsewhere, it would have no effect.)
Please also note what the manual says about Remember that header() must be called before any actual output is sent, ....
And turn on error_reporting(E_ALL); before using header() to see if that is the issue for you.
Header names and values need to be separated by one colon plus a space, so the location "header" is just wrong, it should be:
header('Location: https://www.google.com/accounts/AuthSubRevokeToken');
(It's common to write the case this way, too, but not a need)
Next to that the header function is setting response headers, not request headers. So you're basically using the wrong tool.
In PHP you can not set request headers, that's part of the client (e.g. browser), not the server. So header just looks wrong here. Which HTTP client are you using?
A call, as in using CURL to request another page? The header() function applies only for web-browser<->server communications. It cannot affect any requests your server-side script does to other webservers. For that, you need to modify the particular method you're using, e.g. curl or streams.
For curl, see CURLOPT_HTTPHEADER here: http://php.net/curl_setopt
I've been banging my head against a wall for a few hours now - and it's probably something really obvious I've missed!
I'm trying to connect to a payment service provider (PSP) using CURL, post data and follow the post so the user actually ends up on the PSP's site.
Using the following:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://psp.com/theirpage');
curl_setopt($ch, CURLOPT_REFERER, "http://mysite.com/mypage");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$params);
curl_setopt($ch, CURLOPT_POST, 1);
$result=curl_exec($ch);
curl_close($ch);
This successfully connects, verifies the data I've passed, but instead of redirecting the user to the PSP, it just loads the HTML on my site. Safe mode is off, and open_basedir is blank.
What am I doing wrong?
CURL would do an internal redirect and it wont have any effect on the user viewing your curl script. Keep in mind that the payment was made by your server NOT the users computer, hence expecting the session to work for the user is incorrect. cURL 'is the browser'.
If you just want a redirect after payment is made via cURL, you will have to do it via header() or by using some JS like window.location.
The curl request is being made from your server, and as such your server is receiving the response page. There's no way to initiate the request from the server and have the client receive the response. Either return the HTML to the user from your site (as you're doing), or make the request from the client's browser using Javascript. Hope that helps