I have a PHP curl request which works - However, if i set CURLOPT_FOLLOWLOCATION as 1, the Curl post is submitted with the postdata, the redirected page HTML is captured and posted on my localserver.
The entire redirect is contained within my local server and it doesnt actually transfer me across to the redirection itself with the data (which is what i want). After a few seconds of the html being downloaded on my post.php, it redirects to a page not found.
However, if i set the CURLOPT_FOLLOWLOCATION as 0 and then fetch the correct URL and then redirect it through header("Location: $redirect"); - it transfers fine but there no more data being transferred.
What would be the best way of transferring that data to the new header location.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'postdata=' . urlencode(xmlGrab()));
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
// Download the given URL, and return output
$output = curl_exec($ch);
$headers = substr($output, 0, $curl_info["header_size"]); //split out header
$redirect = curl_getinfo($ch)['redirect_url'];
header('HTTP/1.1 307 Temporary Redirect');
header("Location: $redirect");
// Close the cURL resource, and free system resources
curl_close($ch);
Example Scenario:
User Submits a post http://localhost:8888/post.php
Post.php contains a connection to google.co.uk
Post.php connects and makes a post to google.co.uk
The request downloads google.co.uk?q=blahblah to post.php
post.php looks exactly like google.co.uk?q=blahblah
3 seconds later, it redirects to http;//localhost:8888/?q=blahblah
Check the value of $redirect.
If it's a relative url you have to prepend the protocol+hostname+etc to the url, otherwise you'll end up on your local server.
Related
I'm trying to pass an authorization token between two pages of my application and would like to do so using HTTP headers, however my code is not working as expected.
I've tried retrieving the header using getallheaders() from the second page however the header that I pass it from the first page is not present in the list.
This is the flow of what I'm doing
Placing token into the header array
Executing the POST call
Redirecting to second page using header("Location:")
(on second page) Checking the headers
//placing token into header/////////////////////////////////////////
$token_header['Authorization'] = $accessTokenResult['access_token'];
$header = array();
foreach($token_header as $key => $parsed_urlvalue)
{
$header[] = "$key: $parsed_urlvalue";
}
//sending the request//////////////////////////////////////////////
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_URL, "./custom_request.php");
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
$result = curl_exec($ch);
curl_close($ch);
//redirecting to the second page////////////////////////////////////
header("Location: custom_request.php");
die("Redirected to custom request page");
Other page
$headers_recieved = getallheaders();
echo $headers_recieved['Authorization'];
However I'm getting an invalid index error since 'Authorization' is not present.
I have seen some of the questions that others have asked regarding this same issue but I don't seem to be having much luck in my situation and would appreciate some clarification.
I need to pass a session variable from domain1.com to domain2.com. I have full access to both domains. I am using cURL to have domain1.com/caller.php access domain2.com/receiver.php. Once cURL finishes executing I get a status returned. If that status is good I use header("Location: auto_submit.html") to load a page that automatically submits a form/redirects to domain2.com/test.php. However when the page loads the session variable is not set. Here is what I have in domain1.com/caller.php
...
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "domain2.com/receiver.php");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $result[0]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER , true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
$curl_result = json_decode(curl_exec($ch));
curl_close($ch);
if ($curl_result->status === 1)
{
header("Location: ./auto_submit.html");
exit;
} else {
foo();
}
...
auto_submit.html just redirects to domain2.com/test.php.
What, if anything, do I need on domain2.com/test.php so that it will use the information in the cookie file and will be accessible in $_SESSION?
Maybe I just need a pair of fresh eyes....
I need to POST to a page behind .htaccess Basic Authentication. I successfully log in and get past the .htBA, then POST to the target page. I know that the script is getting to that page as I'm logging the access. However $_POST is empty -- evident from both checking the var as well as the target script not working the way it should. (I control all pages).
I've tried many combos of the various curl opts below to no avail. I'm not getting any errors from the second hit.
Thanks.
$post_array = array(
'username'=>$u,
'password'=>$p
);
// Login here
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://example.com/admin/login.php');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:43.0) Gecko/20100101 Firefox/43.0');
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath('temp/cookies.txt') );
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath('temp/cookies.txt'));
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_REFERER, 'http://example.com/index.php');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array));
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'method' => 'POST',
"Authorization: Basic ".base64_encode("$username:$password"),
));
$logInFirst = curl_exec ($ch);
/* Don't close handle as need the auth for next page
* load up a new page */
$post_array_2 = array(
'localfile'=>'my_data.csv',
'theater_mode'=>'normal'
);
//curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath('temp/cookies.txt') );
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath('temp/cookies.txt'));
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_REFERER, 'http://example.com/admin/post_here.php');
curl_setopt($ch, CURLOPT_URL, 'http://example.com/admin/post_here.php');
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array_2));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: multipart/form-data;',
"Authorization: Basic ".base64_encode("$username:$password"),
));
$runAi = curl_exec($ch);
$run_error = curl_error($ch); echo '<hr>'.$run_error.'<hr>';
curl_close($ch);
Here's the code on the target page (post_here.php), which results in a zero count. So I know that the target script is being hit, and based on the output, there are no POSTs.
$pa = ' There are this many keys in POST: '.count($_POST);
foreach ($_POST as $key => $value) {
$pa .= ' '.$key.':'.$value.' ---- ';
}
The error is on the second request:
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array_2));
// ...
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: multipart/form-data;',
// ...
You send the header Content-Type: multipart/form-data but the data is encoded as application/x-www-form-urlencoded (by http_build_query()).
The data you want to post on the second request contains 'localfile'=>'my_data.csv'. If you want to upload a file on the second request then the content type is correct (but you don't need to set it manually). Don't use http_build_query() but pass an array to CURLOPT_POSTFIELDS, as is explained in the documentation.
Also, for file uploads you have to put a # in front of the file name and make sure curl is able to find the file. The best way to do this is to use the complete file path:
$post_array_2 = array(
'localfile' => '#'.__DIR__'/my_data.csv',
'theater_mode' => 'normal'
);
The code above assumes my_data.csv is located in the same directory as the PHP script (which is not recommended). You should use dirname() to navigate from the script's directory to the directory where the CSV file is stored, to compose the correct path.
As the documentation also states, since PHP 5.5 the # prefix is deprecated and you should use the CURLFile class for file uploads:
$post_array_2 = array(
'localfile' => new CURLFile(__DIR__'/my_data.csv'),
'theater_mode' => 'normal'
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array_2);
As a side note, when you call curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY); it means curl is allowed to negotiate the authentication method with the server. But you also send the header "Authorization: Basic ".base64_encode("$username:$password") and this removes any negotiation because it forces Authorization: Basic.
Also, in order to negociate, curl needs to know the (user, password) combination. You should always use curl_setopt(CURLOPT_USERPWD, "$username:$password") to tell it the user and password. Manual crafting the Authorization header is not recommended.
If you are sure Authorization: Basic is the method you need then you can
use curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC).
You do not see anything inside post because you are using 'Content-Type: multipart/form-data;',. Just remove that and you should be fine.
If you want to upload a file (i.e. my_data.csv) that case you need to follow this way:
## change your file name as following in your param
'localfile'=> '#'.'./my_data.csv',
## after that remove http_build_query() from post
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array_2);
This will automatically add the header multipart with your post.
You may look your uploaded file using $_FILES variable.
Finally, You can observe what curl is enabling verbose mode.
curl_setopt($ch, CURLOPT_VERBOSE, true);
Tips: While using cookie, always close curl after each and every curl_exec() you do. Otherwise it will not probably write things into cookie file after every requests you make!
I am making a CURL get request via php to another domain to get json value, but as i know curl uses temporary session, but how can i maintain all the browser session in curl request ? here is my code
// create curl resource
$ch = curl_init();
// set url
curl_setopt($ch, CURLOPT_URL, "http://api.json"); //api.json is displaying value from session
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
how can i maintain the browser session ...
You can use the CURLOPT_COOKIEJAR option (see documentation) to save all cookies to a file. You can import this file at a later time using the CURLOPT_COOKIEFILE option, this will send all cookies stored in the specified jar.
Example based on your code for keeping a persistent session between script executions:
// create curl resource
$ch = curl_init();
// set url
curl_setopt($ch, CURLOPT_URL, "http://api.json"); //api.json is displaying value from session
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// Set the cookie jar for both importing and exporting
curl_setopt($ch, CURLOPT_COOKIEFILE, "curl-cookie-session.tmp");
curl_setopt($ch, CURLOPT_COOKIEJAR, "curl-cookie-session.tmp");
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$cookie_value = $cookie_name.'='.$_SESSION[$cookie_name];
curl_setopt($ch, CURLOPT_COOKIE, $cookie_value);
$xml_contents = curl_exec ($ch);
curl_close ($ch);
return $xml_contents;
For this you need to store the cookie and in next request attach into the header that works .
I am trying to parse a page which contains some links. These links, if followed, will redirect to some files to download.
For example, Download which redirects to <a href="http://example.com/1.pdf".
I don't want to download the file, I just want to get the file link (int this case http://example.com/1.pdf).
I am trying this:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE); // Return in string
curl_setopt($ch, CURLOPT_URL, $url);
curl_exec($ch);
var_dump(curl_getinfo($ch));
But, it gives me the file contents.
Does anyone have any idea how to this?
==EDIT==
Thank you guys. I solved it like this:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_exec($ch);
$info = curl_getinfo($ch);
Now, $info contains the header and I can the link from it.
The reason the output is being sent to the screen is because you're telling cURL to do so. If you want to store the response in a variable the following line:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE);
should read:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
Then, actually retrieve the returned output from curl_exec like so:
$output = curl_exec($ch);
Once you have the returned HTML content from the remote page in the $output variable you can use DOMdocs or regex (but preferably DOM) to parse out any information you want.
UPDATE
I can't tell because the question is vaguely worded: is there actually a Location header redirect happening? If so, you'll want to do as #heiko suggests to prevent cURL from following the redirect and retrieve the headers. Then you can easily parse the contents of the location header:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
curl_setopt($ch, CURLINFO_HEADER, TRUE); // add header output
# make sure to not follow Location: Header
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
# add Response Header to Output, so that you can find the Location-Header in there!
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
Use RETURN TRANSFER as 1, also use htmlentities() if you want to display HTML source on your page , else just echo the variable ( to display the page [redirects to google] ).
<?php
$url = "http://www.google.co.in";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Return in string
curl_setopt($ch, CURLOPT_URL, $url);
$varx = curl_exec($ch);
echo htmlentities($varx);
?>
With the $varx variable , use Regular Expressions to match which data you want.