I am sending raw JSON POST data with cURL via PHP, and the server receives it correctly when the JSON is not too long.
However, it seems to receive an empty body when it's bigger.
This is how I get the body from the server:
$dataReceived = file_get_contents("php://input");
When I log $dataReceived on the server side, the expected content is in the log when not big, e.g. this:
{"products":[],"allProductIds":["898","920","937","947","979","1007","1044","1064","1124","1146","1162","1178","1194","1215","1239","1291","1346","1395","1444","1629","1673","1695","1757","1868","1929","1992","2018","2050","2096","2112","2136","2178","2225","2280","2303","2318","2340","2394","2447","2459","2476","2486","2510","2520","2558","2591","2626","2655","2689","2727","2975","3175","3224","3283","3311","3333","3342","3353","3376","3389","3400","3419","3469","3506","3535","3610","3722","3765","3782","3802","4002","4020","4039","4060","4094","4130","4171","4213","4246","4281","4317","4348","4426","4473","4502","4545","4558","4563","4663","4688","4713","4733","4757","4875","4924","4984","5028","5057","5088","5102","5117","5145","5174","5197","5216","5236","5255","5275","5310","5336","5369","5422","5433","5464","5477","5505","5516","5541","5563","5587","5616","5627","5673","5703","5744","5773","5822","5839","5903","5999","6075","6196","6326","6483","6794","8360","8429","8599","8724","8798","8953","9150","9198","9247","9295","9347","9415","9463","9636","9991","10105","10219","10337","10451","10566","10681","10796","10912","11134","11244","11354","11465","11576","11687","11798","11874","11924","11973","12025","12074","12125","12175","12640","12699","12838","12844","12848","12852","12862","12866","12870","12874","12880","12884","12888","12894","12935","12943","12947","12952","12958","12962","12966","12970","12974","12979","12988","12992","12997","13001","13010","13012","13016","13020","13026","13030","13034","13039","13049","13059","13093","13128","13144","13148","13154","13158","13162","13172","13182","13217","13223","13368","2851","3828","4804"],"brandlyProductIds":["229082","226046","223246","216800","216750","215456","213613","213157","143436","143406","143362","143312","134066","134022","221020","242298","144021","143921","242348","333387","333424","333443","333517","333624","333681","334021","334043","334086","334135","334159","334184","334220","334261","334318","334351","334376","334398","334448","334505","334530","334552","334581","334606","334639","334672","334701","334734","334763","334792","334825","334962","335099","335325","335382","335439","335464","335482","335491","335500","335521","335532","335553","335575","335680","335770","335795","335877","335991","336032","336049","336066","336088","336276","336293","336310","336329","336370","336411","336452","336493","336530","336561","336598","336631","336705","336751","336782","336819","336850","336859","336977","337002","337033","337058","337083","337133","337207","337256","337366","337403","337428","337456","337473","337490","337515","337540","337565","337582","337599","337616","337637","337682","337710","337738","337791","337800","337831","337881","337906","337928","337953","337981","338006","338031","338056","338099","338135","338175","338201","338244","338269","338340","338422","187730","338815","338968","339121","339427","340235","340308","340461","340614","340687","340840","340987","341060","341133","341206","341279","341352","341425","341675","341981","342072","342254","342350","342441","342532","342714","342805","342896","343042","343115","343188","343261","343407","343480","343553","343626","343690","343754","343818","343882","343946","344010","225836","225770","344210","344214","344216","344218","303935","303610","303601","344223","344234","344236","344238","344241","204360","344254","344256","344262","344264","344268","344270","344272","344274","344277","344279","344281","344283","344285","344287","344288","344290","344292","344295","344299","344301","344303","344308","344313","344330","344333","344335","344337","344340","344342","344344","344349","344354","344371","150836","344434"]}
But i get an empty $dataReceived for example when the sent JSON is the one in this pastebin https://pastebin.com/p5ZLrxig
This is the code i have to send the request, with verbose log
protected static function _apiCall(string $url, array $additionalHeaders = [], string $method="GET", string $data=''){
if($method=="POST"){
$curlLog = __DIR__ . '/curl-log.txt';
$f = fopen($curlLog, 'w');
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, $f);
curl_setopt($ch, CURLOPT_STDOUT, $f);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$hdr = [
'Content-Type: application/json',
'Content-Length: ' . strlen($data)
];
foreach($additionalHeaders as $h) $hdr[] = $h;
curl_setopt($ch, CURLOPT_HTTPHEADER, $hdr);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data );
$info = curl_getinfo($ch);
$result = curl_exec($ch);
curl_close($ch);
fclose($f);
mail("myemail#email.com","$url curl info",print_r($info,true) . ", result:$result, curl log: " . file_get_contents("file://$curlLog") . "\r\n, data:$data");
} else {
// ...
}
}
This is the log i get in the email for the big JSON: https://pastebin.com/0pYHTk4h
Now, can it be an issue with POST size limit, either in PHP or Apache settings?
But i doubt so, since the data is just 33Kb
Content-Length: 34581
Also, if it was the case, shouldn't we expect the server (Apache + PHP 7.4.x) to return an HTTP error code like HTTP 500?
Instead, we have HTTP 100 and HTTP 200.
Seems not related to JSON formatting too, again because we'd expect the server to return an error code.
One aspect that may be relevant: the called URL (from which i am logging php://input on server side) actually is a custom Wordpress REST API endpoint (that i have created by calling register_rest_route inside a rest_api_init action handler).
I am not currently aware of POST size limits or related issues with WP API, i have no idea if the issue can be with WP.
Any hints are appreciated!
for the url "http://hq.sinajs.cn/list=sh600123"
i can get the response through any of the browser, the result as below,
var hq_str_sh600123="兰花科创,13.53,13.63,13.45,13.61,13.43,13.45,13.46,1113110,15047856,3200,13.45,1500,13.44,11590,13.43,36900,13.42,68900,13.41,9800,13.46,6400,13.47,26496,13.48,14453,13.49,3400,13.50,2013-08-30,09:44:08,00";
also i can get the same response from the linux curl command
curl http://hq.sinajs.cn/list=sh600123
var hq_str_sh600123="兰花科创,13.53,13.63,13.44,13.61,13.43,13.43,13.44,1144910,15475169,39090,13.43,37100,13.42,91000,13.41,235500,13.40,5800,13.39,800,13.44,41300,13.45,9800,13.46,6400,13.47,25496,13.48,2013-08-30,09:44:43,00";
but the big problem is i can't get the right response from the PHP cURL function
my trunk code is like this, have deleted the business logic
$url = "http://hq.sinajs.cn/";//the url
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // the result could be got by the return value
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Expect:' )); // this line is added according to the advice from the internet
curl_setopt($ch, CURLOPT_URL,$url);
$post_data = "list=".urldecode($code); // here the code can be "sh600485"
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
$result = curl_exec($ch);
The $result is false, and i add verbose code like this :
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, $verbose = fopen('php://temp', 'rw+'));
echo "Verbose information:\n<pre>", !rewind($verbose), htmlspecialchars(stream_get_contents($verbose)), "</pre>\n";
The output is
Verbose information:
* About to connect() to hq.sinajs.cn port 80 (#0)
* Trying 202.108.37.102...
* connected
* Connected to hq.sinajs.cn (202.108.37.102) port 80 (#0)
POST / HTTP/1.1 Host: hq.sinajs.cn Accept: / Content-Length: 13 Content-Type: application/x-www-form-urlencoded
upload completely sent off: 13 out of 13 bytes
Empty reply from server
Connection #0 to host hq.sinajs.cn left intact
Verbose information:
* Connection #0 seems to be dead!
* Closing connection #0
* About to connect() to hq.sinajs.cn port 80 (#0)
* Trying 202.108.37.102...
* connected
* Connected to hq.sinajs.cn (202.108.37.102) port 80 (#0)
POST / HTTP/1.1 Host: hq.sinajs.cn Accept: / Content-Length: 13 Content-Type: application/x-www-form-urlencoded
upload completely sent off: 13 out of 13 bytes
Empty reply from server
Connection #0 to host hq.sinajs.cn left intact
my question is why the difference from the result is so big?
what is the root cause the website server ? does it forbid the PHP cURL access?
how to solve the problem ?
Thanks in advance!
Your first two examples you are making HTTP GET requests. In your failing php/curl example, you are using HTTP POST.
I suspect you
Try changing the CURLOPT_POST option:
curl_setopt($ch, CURLOPT_POST, 0);
The following example works for me. Notice I've added the query parameter to the url and changed the HTTP method to GET.
<?php
$url = "http://<INSERT URL HERE>?list=sh600123";
// ^ use query params instead of post values...
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
$result = curl_exec($ch);
var_dump($result);
I am trying to debug an http post the I am trying to send from list application. I have been able to send the correct post from php CURL which corectly interfaces with my drupal 7 website and uploads an image.
In order to get this to work in my lisp application I really need to see the content body of my http post I have been able to see the headers using a call like this:
curl_setopt($curl, CURLOPT_STDERR, $fp);
curl_setopt($curl, CURLOPT_VERBOSE, 1);
and the headers look the same in my lisp application but I have been unable to examine the body of the post. I have searched online and other people have asked this question but no one posted a response.
The content type of my http post is:
application/x-www-form-urlencoded
I have also tried many http proxy debuging tools but they only ever the http GET to get my php page but never capture the get sent from server once the php code is executed.
EDIT: I have added a code snipet showing where I actually upload the image file.
// file
$file = array(
'filesize' => filesize($filename),
'filename' => basename($filename),
'file' => base64_encode(file_get_contents($filename)),
'uid' => $logged_user->user->uid,
);
$file = http_build_query($file);
// REST Server URL for file upload
$request_url = $services_url . '/file';
// cURL
$curl = curl_init($request_url);
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: application/x-www-form-urlencoded'));
curl_setopt($curl, CURLOPT_STDERR, $fp);
curl_setopt($curl, CURLOPT_VERBOSE, 1);
curl_setopt($curl, CURLOPT_POST, 1); // Do a regular HTTP POST
curl_setopt($curl, CURLOPT_POSTFIELDS, $file); // Set POST data
curl_setopt($curl, CURLOPT_HEADER, FALSE); // Ask to not return Header
curl_setopt($curl, CURLOPT_COOKIE, "$cookie_session"); // use the previously saved session
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_FAILONERROR, TRUE);
curl_setopt_array($curl, array(CURLINFO_HEADER_OUT => true) );
$response = curl_exec($curl);
CURLOPT_VERBOSE should actually show the details. If you're looking for the response body content, you can also use CURLOPT_RETURNTRANSFER, curl_exec() will then return the response body.
If you need to inspect the request body, CURLOPT_VERBOSE should give that to you but I'm not totally sure.
In any case, a good network sniffer should give you all the details transparently.
Example:
$curlOptions = array(
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_FOLLOWLOCATION => TRUE,
CURLOPT_VERBOSE => TRUE,
CURLOPT_STDERR => $verbose = fopen('php://temp', 'rw+'),
CURLOPT_FILETIME => TRUE,
);
$url = "http://stackoverflow.com/questions/tagged/java";
$handle = curl_init($url);
curl_setopt_array($handle, $curlOptions);
$content = curl_exec($handle);
echo "Verbose information:\n", !rewind($verbose), stream_get_contents($verbose), "\n";
curl_close($handle);
echo $content;
Output:
Verbose information:
* About to connect() to stackoverflow.com port 80 (#0)
* Trying 64.34.119.12...
* connected
* Connected to stackoverflow.com (64.34.119.12) port 80 (#0)
> GET /questions/tagged/java HTTP/1.1
Host: stackoverflow.com
Accept: */*
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Date: Wed, 14 Mar 2012 19:27:53 GMT
< Content-Length: 59110
<
* Connection #0 to host stackoverflow.com left intact
<!DOCTYPE html>
<html>
<head>
<title>Newest 'java' Questions - Stack Overflow</title>
<link rel="shortcut icon" href="http://cdn.sstatic.net/stackoverflow/img/favicon.ico">
<link rel="apple-touch-icon" href="http://cdn.sstatic.net/stackoverflow/img/apple-touch-icon.png">
<link rel="search" type="application/opensearchdescription+xml" title="Stack Overflow" href="/opensearch.xml">
...
Just send it to a random local port and listen on it.
# terminal 1
nc -l localhost 12345
# terminal 2
php -e
<?php
$curl = curl_init('http://localhost:12345');
// etc
If you're talking about viewing the response, if you add curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );, then the document returned by the request should be returned from your call to curl_exec.
If you're talking about viewing the postdata you are sending, well, you should be able to view that anyway since you're setting that in your PHP.
EDIT: Posting a file, eh? What is the content of $file? I'm guessing probably a call to file_get_contents()?
Try something like this:
$postdata = array( 'upload' => '#/path/to/upload/file.ext' );
curl_setopt( $curl, CURLOPT_POSTFIELDS, $postdata );
You can't just send the file, you still need a postdata array that assigns a key to that file (so you can access in PHP as $_FILES['upload']). Also, the # tells cURL to load the contents of the specified file and send that instead of the string.
You were close:
The PHP manual instructs that you must call the constant CURLINFO_HEADER_OUT in both curl_setopt and curl_getinfo.
$ch = curl_init($url);
... other curl options ...
curl_setopt($ch,CURLINFO_HEADER_OUT,true);
curl_exec(ch);
//Call curl_getinfo(*args) after curl_exec(*args) otherwise the output will be NULL.
$header_info = curl_getinfo($ch,CURLINFO_HEADER_OUT); //Where $header_info contains the HTTP Request information
Synopsis
Set curl_setopt
Set curl_getinfo
Call curl_getinfo after curl_exec
I think you're better off doing this with a proxy than in the PHP. I don't think it's possible to pull the raw POST data from the PHP CURL library.
A proxy should show you the request and response contents
To get the header the CURLINFO_HEADER_OUT flag needs to be set before curl_exec is called.
Then use curl_getinfo with the same flag to get the header after curl_exec.
If you want to see the post data, grab the value you set at CURLOPT_POSTFIELDS
For example:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://example.com/webservice");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($payload));
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_exec($ch);
$header = curl_getinfo($ch, CURLINFO_HEADER_OUT);
curl_close($ch);
echo "Request-Header:\r\n" . $header . "\r\n";
echo "Request-Body(URL Encoded):\r\n" . http_build_query($payload) . "\r\n";
echo "Request-Body(Json Encoded):\r\n" . json_encode($payload) . "\r\n";