php://input empty when sending long JSON via raw POST - php
I am sending raw JSON POST data with cURL via PHP, and the server receives it correctly when the JSON is not too long.
However, it seems to receive an empty body when it's bigger.
This is how I get the body from the server:
$dataReceived = file_get_contents("php://input");
When I log $dataReceived on the server side, the expected content is in the log when not big, e.g. this:
{"products":[],"allProductIds":["898","920","937","947","979","1007","1044","1064","1124","1146","1162","1178","1194","1215","1239","1291","1346","1395","1444","1629","1673","1695","1757","1868","1929","1992","2018","2050","2096","2112","2136","2178","2225","2280","2303","2318","2340","2394","2447","2459","2476","2486","2510","2520","2558","2591","2626","2655","2689","2727","2975","3175","3224","3283","3311","3333","3342","3353","3376","3389","3400","3419","3469","3506","3535","3610","3722","3765","3782","3802","4002","4020","4039","4060","4094","4130","4171","4213","4246","4281","4317","4348","4426","4473","4502","4545","4558","4563","4663","4688","4713","4733","4757","4875","4924","4984","5028","5057","5088","5102","5117","5145","5174","5197","5216","5236","5255","5275","5310","5336","5369","5422","5433","5464","5477","5505","5516","5541","5563","5587","5616","5627","5673","5703","5744","5773","5822","5839","5903","5999","6075","6196","6326","6483","6794","8360","8429","8599","8724","8798","8953","9150","9198","9247","9295","9347","9415","9463","9636","9991","10105","10219","10337","10451","10566","10681","10796","10912","11134","11244","11354","11465","11576","11687","11798","11874","11924","11973","12025","12074","12125","12175","12640","12699","12838","12844","12848","12852","12862","12866","12870","12874","12880","12884","12888","12894","12935","12943","12947","12952","12958","12962","12966","12970","12974","12979","12988","12992","12997","13001","13010","13012","13016","13020","13026","13030","13034","13039","13049","13059","13093","13128","13144","13148","13154","13158","13162","13172","13182","13217","13223","13368","2851","3828","4804"],"brandlyProductIds":["229082","226046","223246","216800","216750","215456","213613","213157","143436","143406","143362","143312","134066","134022","221020","242298","144021","143921","242348","333387","333424","333443","333517","333624","333681","334021","334043","334086","334135","334159","334184","334220","334261","334318","334351","334376","334398","334448","334505","334530","334552","334581","334606","334639","334672","334701","334734","334763","334792","334825","334962","335099","335325","335382","335439","335464","335482","335491","335500","335521","335532","335553","335575","335680","335770","335795","335877","335991","336032","336049","336066","336088","336276","336293","336310","336329","336370","336411","336452","336493","336530","336561","336598","336631","336705","336751","336782","336819","336850","336859","336977","337002","337033","337058","337083","337133","337207","337256","337366","337403","337428","337456","337473","337490","337515","337540","337565","337582","337599","337616","337637","337682","337710","337738","337791","337800","337831","337881","337906","337928","337953","337981","338006","338031","338056","338099","338135","338175","338201","338244","338269","338340","338422","187730","338815","338968","339121","339427","340235","340308","340461","340614","340687","340840","340987","341060","341133","341206","341279","341352","341425","341675","341981","342072","342254","342350","342441","342532","342714","342805","342896","343042","343115","343188","343261","343407","343480","343553","343626","343690","343754","343818","343882","343946","344010","225836","225770","344210","344214","344216","344218","303935","303610","303601","344223","344234","344236","344238","344241","204360","344254","344256","344262","344264","344268","344270","344272","344274","344277","344279","344281","344283","344285","344287","344288","344290","344292","344295","344299","344301","344303","344308","344313","344330","344333","344335","344337","344340","344342","344344","344349","344354","344371","150836","344434"]}
But i get an empty $dataReceived for example when the sent JSON is the one in this pastebin https://pastebin.com/p5ZLrxig
This is the code i have to send the request, with verbose log
protected static function _apiCall(string $url, array $additionalHeaders = [], string $method="GET", string $data=''){
if($method=="POST"){
$curlLog = __DIR__ . '/curl-log.txt';
$f = fopen($curlLog, 'w');
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, $f);
curl_setopt($ch, CURLOPT_STDOUT, $f);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$hdr = [
'Content-Type: application/json',
'Content-Length: ' . strlen($data)
];
foreach($additionalHeaders as $h) $hdr[] = $h;
curl_setopt($ch, CURLOPT_HTTPHEADER, $hdr);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data );
$info = curl_getinfo($ch);
$result = curl_exec($ch);
curl_close($ch);
fclose($f);
mail("myemail#email.com","$url curl info",print_r($info,true) . ", result:$result, curl log: " . file_get_contents("file://$curlLog") . "\r\n, data:$data");
} else {
// ...
}
}
This is the log i get in the email for the big JSON: https://pastebin.com/0pYHTk4h
Now, can it be an issue with POST size limit, either in PHP or Apache settings?
But i doubt so, since the data is just 33Kb
Content-Length: 34581
Also, if it was the case, shouldn't we expect the server (Apache + PHP 7.4.x) to return an HTTP error code like HTTP 500?
Instead, we have HTTP 100 and HTTP 200.
Seems not related to JSON formatting too, again because we'd expect the server to return an error code.
One aspect that may be relevant: the called URL (from which i am logging php://input on server side) actually is a custom Wordpress REST API endpoint (that i have created by calling register_rest_route inside a rest_api_init action handler).
I am not currently aware of POST size limits or related issues with WP API, i have no idea if the issue can be with WP.
Any hints are appreciated!
Related
Does Marketo API block curl on a per account basis?
I am trying to connect to the Marketo.com REST API using curl.
I can't get a response from the identity service. I only get an error message
"[curl] 6: Couldn't resolve host 'MY_CLIENT_ENDPOINT.mktorest.com'
,
but I can print the constructed url and paste it into a browser address bar and this will provide the expected response with the access_token element.
I can use curl in php and in a terminal to access my gmail account so curl is able to access an https service.
I have tried sending the parameters in the curl url as a get request and also by declaring them with curl's -F option as a post request
My application uses dchesterton/marketo-rest-api available on github, but I have also tried a simple php curl request just to get the access token.
private function getToken() {
$url = "$this->client_url/identity/oauth/token?grant_type=client_credentials&client_id=$this->client_id&client_secret=$this->client_secret";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
$errors = curl_error($ch);
curl_close($ch);
file_put_contents($this->logDir . 'access_token_response' . date('Y-m-d') . '.txt', $url . "\n" . $response . "\n", FILE_APPEND);
if ($errors) {
file_put_contents($this->logDir . 'access_token_errors' . date('Y-m-d') . '.txt', $errors . "\n", FILE_APPEND);
}
return $response['access_token'];
}
Again, this fails with the same error but produces a perfectly formed url that I can paste into the browser and get a valid response.
I have also tried this using post instead of get as I have for every other test mentioned, and these have been tried on my localhost and on a test server.
Can anyone explain to me why this would fail?
Does Marketo block curl on a per account basis?
I was trying to implement something similar but my code wasn't working. I'm not sure exactly what is failing but I tried your code and it seems to work perfectly after some slight modifications:
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, json_encode($request_data));
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
$response = curl_exec($curl);
$errors = curl_error($curl);
curl_close($curl);
I hope this helps.
PHP Curl results into invalid json
I want to send json data via PUT to a REST service using the following php code:
$data = '{"api_key":"my-api-key","post":{"exception":["2015-04-10T11:09:51+00:00 ERR (3):\\nexception 'Exception' with message 'a simple exception' in \/private\/var\/www\/index.php:1\\nStack trace:\\n#0 {main}"],"access":["::1 - - [10\/Apr\/2015:13:08:17 +0200] \"GET \/index.php HTTP\/1.1\" 200 19039"]}}';
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data))
);
$response = curl_exec($curl);
As you can see I send valid json (validated by http://jsonlint.com/). On the side of the service I get the following json:
{"api_key":"my-api-key","post":{"exception":["2015-04-10T11:09:51+00:00 ERR (3):\\\\nexception \'Exception\' with message \'a simple exception\' in \\/private\\/var\\/www\\/index.php:1\\\\nStack trace:\\\\n#0 {main}"],"access":["::1 - - [10\\/Apr\\/2015:13:08:17 +0200] \\"GET \\/index.php HTTP\\/1.1\\" 200 19039"]}}
Validating this says I got a parse error. And this seems correct as I can't understand why further escaping is done like \\\\n. What am I doing wrong here?
I doubt that this is a valid use of a PUT and should be a POST but that is just a technicality. If you can, make it a POST. The PUT is likely the root of the issue.
The CURLOPT_PUT uses CURLOPT_INFILE and CURLOPT_INFILESIZE
The URLOPT_CUSTOMREQUEST should work fine but it appears you have some RFC 3986 escaping going on for some unexplainable reason.
You may want to use rawurldecode ( ) on the Service Side.
This will give you (appears correct but I'm not the guy to verify):
{"api_key":"my-api-key","post":{"exception":["2015-04-10T11:09:51+00:00 ERR (3):\\nexception 'Exception' with message 'a simple exception' in \/private\/var\/www\/index.php:1\\nStack trace:\\n#0 {main}"],"access":["::1 - - [10\/Apr\/2015:13:08:17 +0200] \"GET \/index.php HTTP\/1.1\" 200 19039"]}}
Disclaimer: I have never used curl to POST Content-Type: application/json data.
You need to look at your Request Header and maybe Response. It would be interesting to see if you get the proper PUT Response.
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
Check your Request Header Content-Type
To get the headers:
The response Header will be in the returned transfer.
The Request Header will be in curl_getinfo()
The curl info will also include the HTTP Response Status. Which for a PUT should be 201 or 301 but not important.
$data = curl_exec($ch);
$skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE));
$responseHeader = substr($data,0,$skip);
$data= substr($data,$skip);
$info = var_export(curl_getinfo($ch),true);
echo $responseHeader . $info . $data;
If you still want to send the json without curl escaping it I have some thoughts on that. but there is probably a post on that subject here somewhere.
PHP HTTP POST Request
I need to send an XML string via HTTP POST to another server using the settings below...
POST /xmlreceive.asmx/CaseApplicationZipped HTTP/1.1
Host: www.dummyurl.co.uk
Content-Type: application/x-www-form-urlencoded
Content-Length: length
XMLApplication=XMLstring&byArray=base64string
I'm guessing I need to set this up via cURL or maybe fsockopen.
I've tried the following but not having any luck at getting it to work.
$url = "http://www.dummyurl.co.uk/XMLReceive.asmx/CaseApplicationZipped";
$headers = array(
"Content-Type: application/x-www-form-urlencoded"//,
);
$post = http_build_query(array('XMLApplication' => $XML, 'byArray' => $base64));
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
echo "response: ".$response;
The remote server gives the following response...
"Object reference not set to an instance of an object."
Not enough rep yet to comment, so I'll post this as an answer.
PHP's cURL automatically send the Host (from $url) and the Content-Length headers, so you don't need to specify those manually.
A handy function exists for building the $post string: http_build_query. It'll handle properly encoding your POST body. It would look something like
$post = http_build_query(array('XMLApplication' => $XML, 'byArray' => $base64));
If you want to log out the headers you received, check the curl_getopt function.
As for the error you received, it seems like you're passing the remote site things it doesn't expect. I can't speak for what you're passing or what the site's expecting, but Input string was not in a correct format seems to imply that your $XML is not formatted correctly, or is being passed as an incorrect parameter.
How to POST an XML file using cURL on php?
I am using a local server on my computer and i am trying to make 2 php scripts to send an xml file and receive it.
To send the xml file i use this code :
<?php
/*
* XML Sender/Client.
*/
// Get our XML. You can declare it here or even load a file.
$file = 'http://localhost/iPM/books.xml';
if(!$xml_builder = simplexml_load_file($file))
exit('Failed to open '.$file);
// We send XML via CURL using POST with a http header of text/xml.
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://localhost/iPM/receiver.php");
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_builder);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_REFERER, 'http://localhost/iPM/receiver.php');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$ch_result = curl_exec($ch);
curl_close($ch);
// Print CURL result.
echo $ch_result;
?>
To receive the xml file i use this code :
<?php
/*
* XML Server.
*/
// We use php://input to get the raw $_POST results.
$xml_post = file_get_contents('php://input');
// If we receive data, save it.
if ($xml_post) {
$xml_file = 'received_xml_' . date('Y_m_d-H-i-s') . '.xml';
$fh = fopen($xml_file, 'w') or die();
fwrite($fh, $xml_post);
fclose($fh);
// Return, as we don't want to cause a loop by processing the code below.
return;
}
?>
When i run the post script i get this error :
Notice: Array to string conversion in C:\xampp\htdocs\iPM\main.php on line 17
which refers to line :
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_builder);
which i dont know what exactly does. The xml file i receive is created but when i open it i get this :
XML Parsing Error: syntax error
Location: file:///C:/xampp/htdocs/iPM/received_xml_2013_01_14-01-06-09.xml
Line Number 1, Column 1:
I tried to comment this specific line as i thought the problem lies there but then when i run my post script i get this error :
Request entity too large!
The POST method does not allow the data transmitted, or the data volume exceeds the capacity limit.
If you think this is a server error, please contact the webmaster.
Error 413
but the xml file is only 5kbs so this is not the problem.
Does anyone have any idea what i should do here? All i am trying to do is make a script to send an xml file and a script to receive it and save it as an xml.
curl_setopt($ch, CURLOPT_POSTFIELDS, $foo) sets your request's body, the data to be posted. It expects $foo to be a set of key-value pairs provided either as an array: $foo = array( 'foo' => 'some value', 'bar' => 2 ); or as a percent-encoded string: $foo = 'foo=some%20value&bar=2' Instead, you're providing $xml_builder variable which is a SimpleXMLElement object returned by simplexml_load_file($file). Try this: $postfields = array( 'xml' => $your_xml_as_string; // get it with file_get_contents() for example ); curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields); Then on the receiving end: $received_xml = $_POST['xml'];
How do I log the raw HTTP headers with a PHP script?
I'm using a cURL script to send POST data through a proxy to a script and I want to see what raw HTTP headers the cURL script is sending. List of things I've tried:
echo curl_getinfo($ch, CURLINFO_HEADER_OUT) gives no output.
file_get_contents('php://input') gets some HTTP headers but not all.
print_r($_SERVER) also gets some HTTP headers but not all (I know this because there should be a X-Forwarded-For header and there isn't)
Printing all superglobals ($_POST, $_GET, $_REQUEST, $_FILES etc) still doesn't show the raw HTTP headers.
http_get_request_headers(), apache_request_headers(), $http_response_header, $HTTP_RAW_POST_DATA aren't outputting everything.
Help?
Turn on CURLOPT_HEADER, not CURLINFO_HEADER_OUT, then split on \r\n\r\n (which is where the header ends) with a maximum split count of 2 :
<?php
$ch = curl_init('http://www.yahoo.com/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$result = curl_exec($ch);
if ($result !== false) {
$split_result = split("\r\n\r\n", $result, 2);
$header = $split_result[0];
$body = $split_result[1];
/** Process here **/
} else {
/** Error handling here **/
}
You need to also set the CURLINFO_HEADER_OUT option:
CURLINFO_HEADER_OUT
TRUE to track the
handle's request string.
Available
since PHP 5.1.3. The CURLINFO_ prefix
is intentional.
http://www.php.net/manual/en/function.curl-setopt.php
The following works:
<?php
$ch = curl_init('http://www.google.com');
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
echo curl_getinfo($ch, CURLINFO_HEADER_OUT);
If you running as a module on Apache then apache_request_headers() does what you need. For just about any other architecture you'll only be able to pick out the stuff which is recorded in $_SERVER or you'll need to find some way to record the information using the web server config.