Slack PHP API - Avoid Timeout error - php

I am trying to use Slack Custom command and not pretty sure how to use delayed messages since the Yoda Speak External API takes more than 3 seconds to respond.
I have done the following:
Sent the slack command /Yoda in my case and received the reponse_url.
Used the following to post the following to the response URL.
$data_string = '{"response_type": "in_channel", "text":"Checking,please wait..."}' ;
$chs = curl_init();
curl_setopt($chs, CURLOPT_URL, $response_url);
curl_setopt($chs, CURLOPT_POST, true);
curl_setopt($chs, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($chs, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($chs, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($chs, CURLOPT_RETURNTRANSFER, true);
curl_setopt($chs, CURLOPT_POST, 1);
curl_setopt($chs, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
$results = curl_exec($chs);
Now, when I call the Yoda API, it gives the following error "Timeout was reached". I read about delayed responses but not sure how should I proceed from here.
$chsres = curl_init();
curl_setopt($chsres, CURLOPT_URL, "https://yoda.p.mashape.com/yoda?sentence=welcome+to+stack");
curl_setopt($chsres, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($chsres, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($chsres, CURLOPT_VERBOSE, true);
curl_setopt($chsres, CURLOPT_TIMEOUT, 45);
curl_setopt($chsres, CURLOPT_RETURNTRANSFER, true);
curl_setopt($chsres, CURLOPT_HTTPHEADER, array('Content-Type:application/json', "X-Mashape-Key:> deMeGoBfMvmshQSemozTqJEY9z0jp1eIhuAjsnx9cQAQsHUifD"));
$resultchsres = curl_exec($chsres);
echo $resultchsres;
Can someone please let me know how to get rid of the timeout error using delayed responses?
UPDATED CODE:
$response_url = $_POST['response_url'];
$text = $_POST['text'];
$term = str_replace(' ', '+', $text);
//https://paypal.slack.com/services/B0VQMHX8W#service_setup
//initial respond with 200OK for timeout
ignore_user_abort(true);
set_time_limit(0);
ob_start();
echo('{"response_type": "in_channel", "text": "Checking, please wait..."}');
header($_SERVER["SERVER_PROTOCOL"] . " 200 OK");
header("Content-Type: application/json");
header('Content-Length: '.ob_get_length());
ob_end_flush();
ob_flush();
flush();
$chsres = curl_init();
curl_setopt_array($chsres, array(
CURLOPT_URL => "https://yoda.p.mashape.com/yoda?sentence=$term",
CURLOPT_SSL_VERIFYPEER => FALSE,
CURLOPT_SSL_VERIFYHOST => FALSE,
CURLOPT_VERBOSE => true,
CURLOPT_RETURNTRANSFER => FALSE,
CURLOPT_HTTPHEADER => array('Content-Type:application/json', "X-Mashape-Key: deMeGoBfMvmshQSemozTqJEY9z0jp1eIhuAjsnx9cQAQsHUifD"),
CURLOPT_RETURNTRANSFER => true
));
$yodaresponse = curl_exec($chsres);
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => $response_url,
CURLOPT_POST => 1,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POSTFIELDS => $yodaresponse
));
$resp = curl_exec($curl);
var_dump($resp);
curl_close($curl);
I still get the same error "Darn – that slash command didn't work (error message: Timeout was reached). Manage the command at slash-command"

You're doing all the right things, just need to change the order.
Respond to the original request with a 200 OK response immediately. See this answer for details, but essentially:
ignore_user_abort(true);
ob_start();
echo('{"response_type": "in_channel", "text": "Checking, please wait..."}');
header($_SERVER["SERVER_PROTOCOL"] . " 200 OK");
header("Content-Type: application/json");
header('Content-Length: '.ob_get_length());
ob_end_flush();
ob_flush();
flush();
Then make the Yoda API request using curl, as you're doing
Once you have the Yoda results, send them to Slack at $response_url using curl, as you're doing.

If you're using FPM then this is what you want - http://php.net/manual/en/function.fastcgi-finish-request.php
Your code would then look like this...
<?php
$response_url = $_POST["response_url"];
$term = rawurlencode($_POST["text"]);
error_log("POST: " . print_r($_POST, 1));
$response = ["response_type"=>"in_channel", "text"=>"Checking, please wait..."];
echo json_encode($response);
header("Content-Type: application/json");
fastcgi_finish_request();
$ch = curl_init();
...

Another approach that will work is to use a curl request with a short timeout to spawn a second PHP script. Since my provider has put some restrictions on my PHP environment (e.g. no process spawning) this has been the only approach that has worked for me.
The first script will terminate shortly after and send an HTTP OK back to Slack. The second script will continue running, handle the time consuming processing (e.g. calling external APIs) and finally send the result as delayed response to the response_url.
1st script
This is the curl request in your first script:
<?php>
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "second.php?redirect_url=$redirect_url");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT_MS, 400); //just some very short timeout
curl_setopt($ch, CURLOPT_NOSIGNAL, 1);
curl_exec($ch);
curl_close($ch);
/* send short response back to user, e.g. "Processing your request..." */
?>
The length of the timeouts is arbitrary, however in my tests a very short timeout (e.g. 10ms) did not work.
You will also need to implement a way to transfer input data between the two scripts as illustrated with passing the request_url as URL parameter.
Finally for slash commands Slack requires you to send a short response back to the user.
2nd script
This is how your 2nd script looks like:
<?php
ignore_user_abort(true); //very important!
usleep (500000); //to ensure 2nd script responds after 1st
/* call external API */
/* send response back to Slack using response_url */
?>
The statement ignore_user_abort(true); is mandatory to ensure your 2nd script keeps running after the curl timeout.
The usleep with 0.5 secs is to ensure that the 2nd script responds after the first, but not mandatory for this solution to work.
The example is based on one answer of the "Continue PHP execution after sending HTTP response" question.

From what I can see in the documentation, you're doing things mostly correctly. Just by echoing anything out, you're already passing a 200 OK message, so no need to do it explicitly. You should check to make sure this isn't a server problem though; is the URL being posted to valid? Not getting mangled by a rewrite rule along the way?
I've made some changes to your code below, including some debugging that will go to your error log (i.e. Apache's error log, by default.) Give it a try, and at the very least you'll have some more debugging details.
<?php
$response_url = $_POST["response_url"];
$term = rawurlencode($_POST["text"]);
error_log("POST: " . print_r($_POST, 1));
ob_end_clean();
ob_start();
$response = ["response_type"=>"in_channel", "text"=>"Checking, please wait..."];
echo json_encode($response);
header("Content-Type: application/json");
header("Content-Length: " . ob_get_size());
ob_end_flush();
flush();
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => "https://yoda.p.mashape.com/yoda?sentence=$term",
CURLOPT_HTTPHEADER => ["X-Mashape-Key: deMeGoBfMvmshQSemozTqJEY9z0jp1eIhuAjsnx9cQAQsHUifD"],
CURLOPT_RETURNTRANSFER => true
]);
$yodaresponse = curl_exec($ch);
curl_close($ch);
error_log("Yoda response: $yodaresponse");
$yodajson = json_encode([
"response_type"=>"in_channel",
"text"=>$yodaresponse
]);
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => $response_url,
CURLOPT_POST => 1,
CURLOPT_HTTPHEADER => ["Content-Type: application/json"],
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POSTFIELDS => $yodajson
]);
$resp = curl_exec($ch);
curl_close($ch);
error_log("API response: $resp");

Posting an answer as I don't have enough reputation to post comments...
I've had the same problem, and then I realized that Slack treats requests and responses differently. Specifically, HTTP request and response differ in their first line.
HTTP request example:
GET /hello.htm HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: www.tutorialspoint.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
HTTP response example:
HTTP/1.1 200 OK
Date: Mon, 27 Jul 2009 12:28:53 GMT
Server: Apache/2.2.14 (Win32)
Last-Modified: Wed, 22 Jul 2009 19:15:56 GMT
Content-Length: 11
Content-Type: text/xml
Connection: Closed
hello there
If you can access raw bytes to be sent in PHP (never used PHP, so not familiar), just make it look like a response, rather than a request. Otherwise, send a response immediately, then do the work that you need, and send a request with the new message. This can be done in number of ways, one of which was outlined by #miken32, I opted out for invoking a background process in python.

Related

Why is the curl code not executed in PHP script? (WAMP Server)

I am trying to get a token to use the Microsoft Graph API (https://learn.microsoft.com/en-us/graph/auth-v2-user?context=graph%2Fapi%2F1.0&view=graph-rest-1.0) via Curl. I have set up a simple Php file with this function:
function getToken() {
echo "start gettoken";
var_dump(extension_loaded('curl'));
$jsonStr = http_build_query(Array(
"client_id" => "***",
"scope" => "https://graph.microsoft.com/.default",
"client_secret" => "***",
"grant_type" => "client_credentials"
));
$headers = Array("Content-Type: application/x-www-form-urlencoded", "Content-Length: " . strlen($jsonStr));
$ch = curl_init("https://login.microsoftonline.com/***.onmicrosoft.com/oauth2/v2.0/token");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonStr);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$token = curl_exec($ch);
echo "test after curl";
return $token;
curl_error($ch);
}
However, what I want to know is why the curl request is not working. Also the echo after the curl codeblock is not being executed, while 'start gettoken' is. PHP_curl is enabled in my WAMP. Why is this?
Are you sure CURL is enabled because that code you have posted is ok and giving echo response before and after curl execution.
you're sending the token request in a JSON-format, and then you're lying to the server saying it's application/x-www-form-urlencoded-encoded when it's actually application/json-encoded! since these 2 formats are completely incompatible, the server fails to parse it, and... ideally it should have responded HTTP 400 bad request (because your request can't be parsed as x-www-form-urlencoded)
anyhow, to actually send it in the application/x-www-form-urlencoded-format, replace json_encode() with http_build_query()
also get rid of the "Content-Length:"-header, it's easy to mess up (aka error-prone) if you're doing it manually (and indeed, you messed it up! there's supposed to be a space between the : and the number, you didn't add the space, but the usual error is supplying the wrong length), but if you don't do it manually, then curl will create the header for you automatically, which is not error-prone.

Php curl request returns '405 Not Allowed', Is there any other method for scraping?

I am using php curl, my code is working for other websites but when i request for this url https://i.local.ch/#q?q=manager&origin=&rid=ac8EV&sort=relevance it returns:
405 Not Allowed
nginx
Note: SSl is active on my domain
include "simple_html_dom.php";
$url="https://i.local.ch/#q?q=manager&origin=&rid=ac8EV&sort=relevance";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
$info = curl_getinfo($ch);
print_r( $info );
echo var_dump($server_output);
HTTP 405 errors are caused when an HTTP method is not allowed by a web server for a requested URL.
Make same request from browser and check request headers. Make sure you are using correct method get, post or put while making request. Also check if any additional headers are sent with request as some urls only allowed from ajax request and you can achieve this by adding proper headers with request.
You can send additional headers using below method
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
"Accept-Charset" => "ISO-8859-1,utf-8;q=0.7,*;q=0.7",
"Keep-Alive" => "115",
"Connection" => "keep-alive",
"X-Requested-With" => "XMLHttpRequest"
));

file_get_contents fails via php, works via browser

What I'm trying to achieve:
Get request to an API Endpoint, retrieving an XML and subsequently parse the results.
I am sending a file_get_contents request to achieve this.
Issues:
`file_get_Contents` fails, error:
Warning: file_get_contents(https://api.twitter.com/1.1/statuses/mentions_timeline.json):
failed to open stream:
A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond.
Update 17/08
To consolidate my current understanding:
1. PHP FAILS:
1.a it fails via php (timeout)
1.b it fails via command line (curl -G http://api.eve-central.com/api/quicklook?typeid=34)
1.c file_get_contents
1.d file_get_contents w/ create_stream_context
2. What WORKS:
2.a Pasting the url in a chrome tab
2.b via postman
What has been attempted:
- Check Headers in Postman ,and try to replicate them via php
Postman Headers sent back by eve-central:
Access-Control-Allow-Origin → *
Connection → Keep-Alive
Content-Encoding → gzip
Content-Type → text/xml; charset=UTF-8
Date → Wed, 17 Aug 2016 10:40:24 GMT
Proxy-Connection → Keep-Alive
Server → nginx
Transfer-Encoding → chunked
Vary → Accept-Encoding
Via → HTTP/1.1 proxy10014
Corresponding Code:
$headers = array(
'method' => 'GET',
'header' => 'Connection: Keep-Alive',
'header' => 'Content-Encoding: gzip',
'header' => 'Content-Type: text/xml',
'header' => 'Proxy-Connection: Keep-Alive',
'header' => 'Server: nginx',
'header' => 'Transfer-Encoding: chunked',
'header' => 'Vary: Accept-Encoding',
'header' => 'Via: HTTP/1.1 proxy10014');
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt($curl, CURLOPT_PORT , 8080); // Attempt at changing port in the event it was blocked.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_POST, false );
curl_setopt($curl, CURLOPT_URL, $url );
$resp = curl_exec($curl);
if(curl_error($curl))
{
echo 'error:' . curl_error($curl);
}
Use Wireshark to capture the GET request to see if changing the port helped
Run cUrl via command line
I'm out of ideas and option.
So the questions are:
If it works in a browser, and in Postman, why does it not work via PHP ?
How can I modify my code so that it mimics what Postman does? ?
Previous Attempts
What I have tried:
Various cURL options from other threads, such as
function curl_get_contents($url) {
$ch = curl_init();
if (!$ch)
{
die("Couldn't initialize a cURL handle");
} else
echo "Curl Handle initialized ";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
$data = curl_exec($ch);
// Check if any error occurred
if (!curl_errno($ch))
{
$info = curl_getinfo($ch);
echo 'Took ', $info['total_time'], ' seconds to send a request to ', $info['url'], "";
displayData($info);
} else
echo "Failed Curl, reason: ".curl_error($ch)." ";
curl_close($ch);
return $data;
}
result: nothing, no data returned.
- Checked php.ini options:
- allow_fopen is On
- allow_url_include = on
- relevant ssl extensions are enabled
- Raised the timeout window
- both via php.ini
- also via explicit declaration within the php file.
- Tried with a different url
- same error, so it doesn't really depends on my particular endpoint
- for example, both twitter/wikipedia/google return the specific error
- tried with:
- file_get_contents on a local xml file (https://msdn.microsoft.com/en-us/library/ms762271(v=vs.85).aspx) --> works
- file_get_contents on a remote xml file (http://www.xmlfiles.com/examples/note.xml) --> fails same error
- Overall, the following is true, so far:
- Curl fails, timeout
- file_get_Contents fails, timeout
- Open XML file url in a browser works
- Make a GET request via Postman, works
Obviously, in all cases where the file_get_contents fails via php, I can easily access the file via any browser.
Tried to work around the issue.
Attempt 1:
Use nitrous.io, create a LAMP stack, perform the deed via the platform
results: file_get_contents works, however, due to the large number of xml files to be retrieved, the operation times-out.
Tentative solution:
- Download XML files from source
- Zip them
- Download xml_file
- Locally parse said xml files
Later on, write a small php scripts that, when invoked, performs the bits above, sends the data to the local directory, which then unpacks it and performs additional work on it.
Another attempt would be to use Google Sheets, with a user function that pulls the data into the sheet, and just dump the excel file / values into mysql.
For my purposes, while an awfully ignorant solution, it does the trick.
Code used for avoiding timeout issue on shared host:
function downloadUrlToFile2($url, $outFileName)
{
//file_put_contents($xmlFileName, fopen($link, 'r'));
//copy($link, $xmlFileName); // download xml file
;
echo "Passing $url into $outFileName ";
// $outFileName = touch();
$fp = fopen($outFileName, "w");
if(is_file($url))
{
copy($url, $outFileName); // download xml file
} else
{
$ch = curl_init();
$options = array(
CURLOPT_TIMEOUT => 28800, // set this to 8 hours so we dont timeout on big files
CURLOPT_URL => $url
);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt_array($ch, $options);
$contents = curl_exec($ch);
fwrite($fp, $contents);
curl_close($ch);
}
}
I have also added this on top of the ini script:
ignore_user_abort(true);
set_time_limit(0);
ini_set('memory_limit', '2048M');
I see some issue with HTTPS url request, for fix issue you have to add below lines in your CURL request
function curl_get_contents($url) {
$ch = curl_init();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
curl_setopt( $ch, CURLOPT_HTTPHEADER, $header );
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
// I have added below two lines
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

PHP curl for the sc2ranks.com API

This is the first question ever here, so I'll do my best.
Background
I'm working on a small Starcraft II website that uses an API from a well known website. After looking at the documentation I got the example working rather quickly, returning the JSON code:
curl -X POST 'http://api.sc2ranks.com/v2/characters/search' -d 'name=Wodan&bracket=1v1&league=gold&expansion=hots&rank_region=global&api_key='<api_key>'
Hoping for an easy ride I created a small script that performs a HTTP-GET request on the API returning the base statistics for every in-game user. This URL can ofcourse be directly formed in the browser:
http://api.sc2ranks.com/v2/characters/eu/1616021?api_key=<api_key>
This is handled in my code in the following way (note: $url is the above URL and $this->session is the initialization of curl):
curl_setopt($this->session,CURLOPT_URL, $url);
curl_setopt($this->session,CURLOPT_RETURNTRANSFER, true);
curl_setopt($this->session,CURLOPT_HEADER, true);
if( !$result = curl_exec($this->session) )
exit("cURL Error: " . curl_error($this->session));
else
return $result;
The problem
Well on my way I decided to use the more advanced functions of the API to get more detailed information on the member. This is however where the confusion starts. Looking back at the above original curl post in the terminal, it shows two options. The -X (request) and the -d (post data). Figuring the first would be the URL make the HTTP-POST request to and the later being the data, I came up with the following example:
$url = "http://api.sc2ranks.com/v2/characters/search/";
$data = "league=all&rank_region=global&expansion=hots&bracket=1v1&name=Wodan?api_key=<my_key>";
curl_setopt($this->session, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($this->session, CURLOPT_POSTFIELDS, $data);
curl_setopt($this->session, CURLOPT_RETURNTRANSFER, true);
curl_setopt($this->session, CURLOPT_HTTPHEADER, array(
'Content-Type: text/html',
'Content-Length: ' .strlen($data))
);
if( !$result = curl_exec($this->session) )
exit("cURL Error: " . curl_error($this->session));
else
return $result;
I've also tried to pass the above through as JSON format.
The output As a result I only get the following as a server response:String '{' was not found in 'HTTP/1.1 404 Not Found Server: nginx Date: Sun, 17 Nov 2013 22:14:03 GMT Content-Type: text/html; charset=utf-8 Content-Length: 0 Connection: close Status: 404 Not Found X-Request-Id: 700726570451c891d48862edfc8545fb X-Runtime: 0.009002 X-Rack-Cache: invalidate, pass '
What I think is going wrong Most likely I'm not forming the request right to the server or I need to target a specific file, other then they have documented it. I have found two similar questions here concerning POST's with Curl. But they were of little help to me and I feel that this might help more people out.
Hopefully someone in this community has some more experience with curl or the actual API.
use the following code
$apiParams = array ("league" => "all",
"rank_region" => "global",
"expansion" => "hots",
"bracket" => "1v1",
"name" => "Wodan",
"api_key" => "<my_key>");
$Url = "http://api.sc2ranks.com/v2/characters/search";
$curlHandler = curl_init($Url);
curl_setopt($curlHandler, CURLOPT_POST, true);
curl_setopt($curlHandler, CURLOPT_POSTFIELDS, $apiParams);
curl_setopt($curlHandler, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curlHandler, CURLOPT_HTTPHEADER, array(
'Content-Type: text/html',
'Content-Length: ' .strlen($data))
);
$statusCode = curl_exec($curlHandler);
curl_close($curlHandler);
Hello people and thank you all for your help! I have solved the problem on my own now, but I had to scroll through the PHP manual for setopt itself. This is the following code that works:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"http://api.sc2ranks.com/v2/characters/search");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS,"name=Wodan&bracket=1v1&league=gold&expansion=hots&rank_region=global&api_key=<my_key>");
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_exec ($ch);
curl_close ($ch);
Note the commented out line above. The manual say's the following about CURLOPT_RETURNTRANSFER:
TRUE to return the transfer as a string of the return value of curl_exec() instead of outputting it out directly.
Since this example is not a HTTP GET-request but a HTTP POST-request there will be no direct string collected from the URL. When set to false it will collect any output that gets returned.
My interpretation of the manual might be wrong, but this has solved the issue at least. I hope this will help other people. Thanks everyone for your help.

Getting content body from http post using php CURL

I am trying to debug an http post the I am trying to send from list application. I have been able to send the correct post from php CURL which corectly interfaces with my drupal 7 website and uploads an image.
In order to get this to work in my lisp application I really need to see the content body of my http post I have been able to see the headers using a call like this:
curl_setopt($curl, CURLOPT_STDERR, $fp);
curl_setopt($curl, CURLOPT_VERBOSE, 1);
and the headers look the same in my lisp application but I have been unable to examine the body of the post. I have searched online and other people have asked this question but no one posted a response.
The content type of my http post is:
application/x-www-form-urlencoded
I have also tried many http proxy debuging tools but they only ever the http GET to get my php page but never capture the get sent from server once the php code is executed.
EDIT: I have added a code snipet showing where I actually upload the image file.
// file
$file = array(
'filesize' => filesize($filename),
'filename' => basename($filename),
'file' => base64_encode(file_get_contents($filename)),
'uid' => $logged_user->user->uid,
);
$file = http_build_query($file);
// REST Server URL for file upload
$request_url = $services_url . '/file';
// cURL
$curl = curl_init($request_url);
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: application/x-www-form-urlencoded'));
curl_setopt($curl, CURLOPT_STDERR, $fp);
curl_setopt($curl, CURLOPT_VERBOSE, 1);
curl_setopt($curl, CURLOPT_POST, 1); // Do a regular HTTP POST
curl_setopt($curl, CURLOPT_POSTFIELDS, $file); // Set POST data
curl_setopt($curl, CURLOPT_HEADER, FALSE); // Ask to not return Header
curl_setopt($curl, CURLOPT_COOKIE, "$cookie_session"); // use the previously saved session
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_FAILONERROR, TRUE);
curl_setopt_array($curl, array(CURLINFO_HEADER_OUT => true) );
$response = curl_exec($curl);
CURLOPT_VERBOSE should actually show the details. If you're looking for the response body content, you can also use CURLOPT_RETURNTRANSFER, curl_exec() will then return the response body.
If you need to inspect the request body, CURLOPT_VERBOSE should give that to you but I'm not totally sure.
In any case, a good network sniffer should give you all the details transparently.
Example:
$curlOptions = array(
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_FOLLOWLOCATION => TRUE,
CURLOPT_VERBOSE => TRUE,
CURLOPT_STDERR => $verbose = fopen('php://temp', 'rw+'),
CURLOPT_FILETIME => TRUE,
);
$url = "http://stackoverflow.com/questions/tagged/java";
$handle = curl_init($url);
curl_setopt_array($handle, $curlOptions);
$content = curl_exec($handle);
echo "Verbose information:\n", !rewind($verbose), stream_get_contents($verbose), "\n";
curl_close($handle);
echo $content;
Output:
Verbose information:
* About to connect() to stackoverflow.com port 80 (#0)
* Trying 64.34.119.12...
* connected
* Connected to stackoverflow.com (64.34.119.12) port 80 (#0)
> GET /questions/tagged/java HTTP/1.1
Host: stackoverflow.com
Accept: */*
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Date: Wed, 14 Mar 2012 19:27:53 GMT
< Content-Length: 59110
<
* Connection #0 to host stackoverflow.com left intact
<!DOCTYPE html>
<html>
<head>
<title>Newest 'java' Questions - Stack Overflow</title>
<link rel="shortcut icon" href="http://cdn.sstatic.net/stackoverflow/img/favicon.ico">
<link rel="apple-touch-icon" href="http://cdn.sstatic.net/stackoverflow/img/apple-touch-icon.png">
<link rel="search" type="application/opensearchdescription+xml" title="Stack Overflow" href="/opensearch.xml">
...
Just send it to a random local port and listen on it.
# terminal 1
nc -l localhost 12345
# terminal 2
php -e
<?php
$curl = curl_init('http://localhost:12345');
// etc
If you're talking about viewing the response, if you add curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );, then the document returned by the request should be returned from your call to curl_exec.
If you're talking about viewing the postdata you are sending, well, you should be able to view that anyway since you're setting that in your PHP.
EDIT: Posting a file, eh? What is the content of $file? I'm guessing probably a call to file_get_contents()?
Try something like this:
$postdata = array( 'upload' => '#/path/to/upload/file.ext' );
curl_setopt( $curl, CURLOPT_POSTFIELDS, $postdata );
You can't just send the file, you still need a postdata array that assigns a key to that file (so you can access in PHP as $_FILES['upload']). Also, the # tells cURL to load the contents of the specified file and send that instead of the string.
You were close:
The PHP manual instructs that you must call the constant CURLINFO_HEADER_OUT in both curl_setopt and curl_getinfo.
$ch = curl_init($url);
... other curl options ...
curl_setopt($ch,CURLINFO_HEADER_OUT,true);
curl_exec(ch);
//Call curl_getinfo(*args) after curl_exec(*args) otherwise the output will be NULL.
$header_info = curl_getinfo($ch,CURLINFO_HEADER_OUT); //Where $header_info contains the HTTP Request information
Synopsis
Set curl_setopt
Set curl_getinfo
Call curl_getinfo after curl_exec
I think you're better off doing this with a proxy than in the PHP. I don't think it's possible to pull the raw POST data from the PHP CURL library.
A proxy should show you the request and response contents
To get the header the CURLINFO_HEADER_OUT flag needs to be set before curl_exec is called.
Then use curl_getinfo with the same flag to get the header after curl_exec.
If you want to see the post data, grab the value you set at CURLOPT_POSTFIELDS
For example:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://example.com/webservice");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($payload));
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_exec($ch);
$header = curl_getinfo($ch, CURLINFO_HEADER_OUT);
curl_close($ch);
echo "Request-Header:\r\n" . $header . "\r\n";
echo "Request-Body(URL Encoded):\r\n" . http_build_query($payload) . "\r\n";
echo "Request-Body(Json Encoded):\r\n" . json_encode($payload) . "\r\n";

Categories