I am sending raw JSON POST data with cURL via PHP, and the server receives it correctly when the JSON is not too long.
However, it seems to receive an empty body when it's bigger.
This is how I get the body from the server:
$dataReceived = file_get_contents("php://input");
When I log $dataReceived on the server side, the expected content is in the log when not big, e.g. this:
{"products":[],"allProductIds":["898","920","937","947","979","1007","1044","1064","1124","1146","1162","1178","1194","1215","1239","1291","1346","1395","1444","1629","1673","1695","1757","1868","1929","1992","2018","2050","2096","2112","2136","2178","2225","2280","2303","2318","2340","2394","2447","2459","2476","2486","2510","2520","2558","2591","2626","2655","2689","2727","2975","3175","3224","3283","3311","3333","3342","3353","3376","3389","3400","3419","3469","3506","3535","3610","3722","3765","3782","3802","4002","4020","4039","4060","4094","4130","4171","4213","4246","4281","4317","4348","4426","4473","4502","4545","4558","4563","4663","4688","4713","4733","4757","4875","4924","4984","5028","5057","5088","5102","5117","5145","5174","5197","5216","5236","5255","5275","5310","5336","5369","5422","5433","5464","5477","5505","5516","5541","5563","5587","5616","5627","5673","5703","5744","5773","5822","5839","5903","5999","6075","6196","6326","6483","6794","8360","8429","8599","8724","8798","8953","9150","9198","9247","9295","9347","9415","9463","9636","9991","10105","10219","10337","10451","10566","10681","10796","10912","11134","11244","11354","11465","11576","11687","11798","11874","11924","11973","12025","12074","12125","12175","12640","12699","12838","12844","12848","12852","12862","12866","12870","12874","12880","12884","12888","12894","12935","12943","12947","12952","12958","12962","12966","12970","12974","12979","12988","12992","12997","13001","13010","13012","13016","13020","13026","13030","13034","13039","13049","13059","13093","13128","13144","13148","13154","13158","13162","13172","13182","13217","13223","13368","2851","3828","4804"],"brandlyProductIds":["229082","226046","223246","216800","216750","215456","213613","213157","143436","143406","143362","143312","134066","134022","221020","242298","144021","143921","242348","333387","333424","333443","333517","333624","333681","334021","334043","334086","334135","334159","334184","334220","334261","334318","334351","334376","334398","334448","334505","334530","334552","334581","334606","334639","334672","334701","334734","334763","334792","334825","334962","335099","335325","335382","335439","335464","335482","335491","335500","335521","335532","335553","335575","335680","335770","335795","335877","335991","336032","336049","336066","336088","336276","336293","336310","336329","336370","336411","336452","336493","336530","336561","336598","336631","336705","336751","336782","336819","336850","336859","336977","337002","337033","337058","337083","337133","337207","337256","337366","337403","337428","337456","337473","337490","337515","337540","337565","337582","337599","337616","337637","337682","337710","337738","337791","337800","337831","337881","337906","337928","337953","337981","338006","338031","338056","338099","338135","338175","338201","338244","338269","338340","338422","187730","338815","338968","339121","339427","340235","340308","340461","340614","340687","340840","340987","341060","341133","341206","341279","341352","341425","341675","341981","342072","342254","342350","342441","342532","342714","342805","342896","343042","343115","343188","343261","343407","343480","343553","343626","343690","343754","343818","343882","343946","344010","225836","225770","344210","344214","344216","344218","303935","303610","303601","344223","344234","344236","344238","344241","204360","344254","344256","344262","344264","344268","344270","344272","344274","344277","344279","344281","344283","344285","344287","344288","344290","344292","344295","344299","344301","344303","344308","344313","344330","344333","344335","344337","344340","344342","344344","344349","344354","344371","150836","344434"]}
But i get an empty $dataReceived for example when the sent JSON is the one in this pastebin https://pastebin.com/p5ZLrxig
This is the code i have to send the request, with verbose log
protected static function _apiCall(string $url, array $additionalHeaders = [], string $method="GET", string $data=''){
if($method=="POST"){
$curlLog = __DIR__ . '/curl-log.txt';
$f = fopen($curlLog, 'w');
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, $f);
curl_setopt($ch, CURLOPT_STDOUT, $f);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$hdr = [
'Content-Type: application/json',
'Content-Length: ' . strlen($data)
];
foreach($additionalHeaders as $h) $hdr[] = $h;
curl_setopt($ch, CURLOPT_HTTPHEADER, $hdr);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data );
$info = curl_getinfo($ch);
$result = curl_exec($ch);
curl_close($ch);
fclose($f);
mail("myemail#email.com","$url curl info",print_r($info,true) . ", result:$result, curl log: " . file_get_contents("file://$curlLog") . "\r\n, data:$data");
} else {
// ...
}
}
This is the log i get in the email for the big JSON: https://pastebin.com/0pYHTk4h
Now, can it be an issue with POST size limit, either in PHP or Apache settings?
But i doubt so, since the data is just 33Kb
Content-Length: 34581
Also, if it was the case, shouldn't we expect the server (Apache + PHP 7.4.x) to return an HTTP error code like HTTP 500?
Instead, we have HTTP 100 and HTTP 200.
Seems not related to JSON formatting too, again because we'd expect the server to return an error code.
One aspect that may be relevant: the called URL (from which i am logging php://input on server side) actually is a custom Wordpress REST API endpoint (that i have created by calling register_rest_route inside a rest_api_init action handler).
I am not currently aware of POST size limits or related issues with WP API, i have no idea if the issue can be with WP.
Any hints are appreciated!
I have a PHP script that does an HTTP request on behalf of the browser and the outputs the response to the browser. Problem is when I click the links from the browser on this page it complains about cookie variables. I'm assuming it needs the browsers cookie(s) for the site.
how can I intercept and forward it to the remote site?
This is how I forward all browser cookies to curl and also return all cookies for the curl request back to the browser. For this I needed to solve some problems like getting cookies from curl, parsing http header, sending multiple cookies and session locking:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// get http header for cookies
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// forward current cookies to curl
$cookies = array();
foreach ($_COOKIE as $key => $value)
{
if ($key != 'Array')
{
$cookies[] = $key . '=' . $value;
}
}
curl_setopt( $ch, CURLOPT_COOKIE, implode(';', $cookies) );
// Stop session so curl can use the same session without conflicts
session_write_close();
$response = curl_exec($ch);
curl_close($ch);
// Session restart
session_start();
// Seperate header and body
list($header, $body) = explode("\r\n\r\n", $response, 2);
// extract cookies form curl and forward them to browser
preg_match_all('/^(Set-Cookie:\s*[^\n]*)$/mi', $header, $cookies);
foreach($cookies[0] AS $cookie)
{
header($cookie, false);
}
echo $body;
In fact, it is possible. You just have to take the cookie ofthe browser and pass it as a parameter to curl to mimik the browser.
It's like a session jacking...
Here is a sample code:
// Init curl connection
$curl = curl_init('http://otherserver.com/');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
// You can add your GET or POST param
// Retrieving session ID
$strCookie = 'PHPSESSID=' . $_COOKIE['PHPSESSID'] . '; path=/';
// We pass the sessionid of the browser within the curl request
curl_setopt( $curl, CURLOPT_COOKIE, $strCookie );
// We receive the answer as if we were the browser
$curl_response = curl_exec($curl);
It works very well if your purpose is to call another website, but this will fail if you call your web server (the same that is launching the curl command). It's because your session file is still open/locked by this script so the URL you are calling can't access it.
If you want to bypass that restriction (call a page on the same server), you have to close the session file with this code before you execute the curl :
$curl = curl_init('http://sameserver.com/');
//...
session_write_close();
$curl_response = curl_exec($curl);
Hope this will help someone :)
From curl_setopt:
By default, libcurl always stores and loads all cookies, independent if they are session cookies or not.
However you may need to set cookies directly, which can be done using:
curl_setopt($ch, CURLOPT_COOKIE, 'foo=bar');
Which is the same as the Set-Cookie HTTP header. Check you're not using curl_setopt($ch, CURLOPT_COOKIESESSION, true) as this will make libcurl ignore some cookies.
You can't.
If you curl the request, you will need to parse the output, and replace all links so they go thru your server.
www.yourdomain.com/f?=www.someotherdomain.com/realpage
The only way this would work is if you use persistent cookies in your curl request. CURL can keep cookies itself. Assign a session ID to the cookie file (in curl) so subsequent requests get the same cookies. When a user clicks a link, you will need to curl the request again.
It is a security issue to allow site1 to set cookies for site2. Imagine if you could set cookies in the browser for paypal and trick the user into thinking they had logged int or some other malicious action.
The Cookie is usually sent with the HTTP request header like
Host stackoverflow.com
User-Agent ...
Accept-Language en-us,en;q=0.5
Referer http://stackoverflow.com/unanswered
Cookie bla=blabla;blubb=blu
So I guess that just have to modify the cookie part in your header.
PiTheNumber's answer was great but I ran into some issues with it that caused it to still print the headers to the page. So I adjusted it to use the more reliable curl_getinfo function. This version also follows redirects.
public function get_page_content( $url ) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_HEADER, 1);
// Forward current cookies to curl
$cookies = array();
foreach ($_COOKIE as $key => $value) {
if ($key != 'Array') {
$cookies[] = $key . '=' . $value;
}
}
curl_setopt( $ch, CURLOPT_COOKIE, implode(';', $cookies) );
$destination = $url;
while ($destination) {
session_write_close();
curl_setopt($ch, CURLOPT_URL, $destination);
$response = curl_exec($ch);
$curl_info = curl_getinfo($ch);
$destination = $curl_info["redirect_url"];
session_start();
}
curl_close($ch);
$headers = substr($response, 0, $curl_info["header_size"]);
$body = substr($response, $curl_info["header_size"]);
// Extract cookies from curl and forward them to browser
preg_match_all('/^(Set-Cookie:\s*[^\n]*)$/mi', $headers, $cookies);
foreach($cookies[0] AS $cookie) {
header($cookie, false);
}
return $body;
}