Paypal IPN issue - Not handling some payments - php

I am currently developing a modification for an open source forum software. This modification allows an user to donate through that forum software.
However, recently an user reported an issue which may be caused by my code. I use another open source library to handle the IPN connection - An IPN Listener PHP class.
The user who reported this issue is receiving the following e-mail:
Hello <My Name>,
Please check your server that handles PayPal Instant Payment
Notifications (IPN). Instant Payment Notifications sent to the
following URL(s) are failing:
http://www.MySite.com/donate/handler.php
If you do not recognize this URL, you may be using a service provider
that is using IPN on your behalf. Please contact your service provider
with the above information. If this problem continues, IPNs may be
disabled for your account.
Thank you for your prompt attention to this issue.
Sincerely, PayPal
I am fearing that the issue comes from my side, therefore I have to look into this and make sure.
I lightly modified the IPN Listener script, which leads me to think that my modification is causing this issue. Paypal also had some changes recently which might have provoked this problem.
This is how the class looks like momentarily:
/**
* PayPal IPN Listener
*
* A class to listen for and handle Instant Payment Notifications (IPN) from
* the PayPal server.
*
* https://github.com/Quixotix/PHP-PayPal-IPN
*
* #package PHP-PayPal-IPN
* #author Micah Carrick
* #copyright (c) 2011 - Micah Carrick
* #version 2.0.5
* #license http://opensource.org/licenses/gpl-license.php
*
* This library is originally licensed under GPL v3, but I received
* permission from the author to use it under GPL v2.
*/
class ipn_handler
{
/**
* If true, the recommended cURL PHP library is used to send the post back
* to PayPal. If flase then fsockopen() is used. Default true.
*
* #var boolean
*/
public $use_curl = true;
/**
* If true, explicitly sets cURL to use SSL version 3. Use this if cURL
* is compiled with GnuTLS SSL.
*
* #var boolean
*/
public $force_ssl_v3 = true;
/**
* If true, cURL will use the CURLOPT_FOLLOWLOCATION to follow any
* "Location: ..." headers in the response.
*
* #var boolean
*/
public $follow_location = false;
/**
* If true, an SSL secure connection (port 443) is used for the post back
* as recommended by PayPal. If false, a standard HTTP (port 80) connection
* is used. Default true.
*
* #var boolean
*/
public $use_ssl = true;
/**
* If true, the paypal sandbox URI www.sandbox.paypal.com is used for the
* post back. If false, the live URI www.paypal.com is used. Default false.
*
* #var boolean
*/
public $use_sandbox = false;
/**
* The amount of time, in seconds, to wait for the PayPal server to respond
* before timing out. Default 30 seconds.
*
* #var int
*/
public $timeout = 60;
private $post_data = array();
private $post_uri = '';
private $response_status = '';
private $response = '';
const PAYPAL_HOST = 'www.paypal.com';
const SANDBOX_HOST = 'www.sandbox.paypal.com';
/**
* Post Back Using cURL
*
* Sends the post back to PayPal using the cURL library. Called by
* the processIpn() method if the use_curl property is true. Throws an
* exception if the post fails. Populates the response, response_status,
* and post_uri properties on success.
*
* #param string The post data as a URL encoded string
*/
protected function curlPost($encoded_data)
{
global $user;
if ($this->use_ssl)
{
$uri = 'https://' . $this->getPaypalHost() . '/cgi-bin/webscr';
$this->post_uri = $uri;
}
else
{
$uri = 'http://' . $this->getPaypalHost() . '/cgi-bin/webscr';
$this->post_uri = $uri;
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $uri);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $encoded_data);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, $this->follow_location);
curl_setopt($ch, CURLOPT_TIMEOUT, $this->timeout);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
if ($this->force_ssl_v3)
{
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
}
$this->response = curl_exec($ch);
$this->response_status = strval(curl_getinfo($ch, CURLINFO_HTTP_CODE));
if ($this->response === false || $this->response_status == '0')
{
$errno = curl_errno($ch);
$errstr = curl_error($ch);
throw new Exception($user->lang['CURL_ERROR'] . "[$errno] $errstr");
}
}
/**
* Post Back Using fsockopen()
*
* Sends the post back to PayPal using the fsockopen() function. Called by
* the processIpn() method if the use_curl property is false. Throws an
* exception if the post fails. Populates the response, response_status,
* and post_uri properties on success.
*
* #param string The post data as a URL encoded string
*/
protected function fsockPost($encoded_data)
{
global $user;
if ($this->use_ssl)
{
$uri = 'ssl://' . $this->getPaypalHost();
$port = '443';
$this->post_uri = $uri . '/cgi-bin/webscr';
}
else
{
$uri = $this->getPaypalHost(); // no "http://" in call to fsockopen()
$port = '80';
$this->post_uri = 'http://' . $uri . '/cgi-bin/webscr';
}
$fp = fsockopen($uri, $port, $errno, $errstr, $this->timeout);
if (!$fp)
{
// fsockopen error
throw new Exception($user->lang['FSOCKOPEN_ERROR'] . "[$errno] $errstr");
}
$header = "POST /cgi-bin/webscr HTTP/1.1\r\n";
$header .= "Content-Length: " . strlen($encoded_data) . "\r\n";
$header .= "Content-Type: application/x-www-form-urlencoded\r\n";
$header .= "Host: " . $this->getPaypalHost() . "\r\n";
$header .= "Connection: close\r\n\r\n";
fputs($fp, $header . $encoded_data . "\r\n\r\n");
while(!feof($fp))
{
if (empty($this->response))
{
// extract HTTP status from first line
$this->response .= $status = fgets($fp, 1024);
$this->response_status = trim(substr($status, 9, 4));
}
else
{
$this->response .= fgets($fp, 1024);
}
}
fclose($fp);
}
private function getPaypalHost()
{
if ($this->use_sandbox)
{
return ipn_handler::SANDBOX_HOST;
}
else
{
return ipn_handler::PAYPAL_HOST;
}
}
/**
* Get POST URI
*
* Returns the URI that was used to send the post back to PayPal. This can
* be useful for troubleshooting connection problems. The default URI
* would be "ssl://www.sandbox.paypal.com:443/cgi-bin/webscr"
*
* #return string
*/
public function getPostUri()
{
return $this->post_uri;
}
/**
* Get Response
*
* Returns the entire response from PayPal as a string including all the
* HTTP headers.
*
* #return string
*/
public function getResponse()
{
return $this->response;
}
/**
* Get Response Status
*
* Returns the HTTP response status code from PayPal. This should be "200"
* if the post back was successful.
*
* #return string
*/
public function getResponseStatus()
{
return $this->response_status;
}
/**
* Get Text Report
*
* Returns a report of the IPN transaction in plain text format. This is
* useful in emails to order processors and system administrators. Override
* this method in your own class to customize the report.
*
* #return string
*/
public function getTextReport()
{
$r = '';
// date and POST url
for ($i = 0; $i < 80; $i++)
{
$r .= '-';
}
$r .= "\n[" . date('m/d/Y g:i A') . '] - ' . $this->getPostUri();
if ($this->use_curl)
{
$r .= " (curl)\n";
}
else
{
$r .= " (fsockopen)\n";
}
// HTTP Response
for ($i = 0; $i < 80; $i++)
{
$r .= '-';
}
$r .= "\n{$this->getResponse()}\n";
// POST vars
for ($i = 0; $i < 80; $i++)
{
$r .= '-';
}
$r .= "\n";
foreach ($this->post_data as $key => $value)
{
$r .= str_pad($key, 25) . "$value\n";
}
$r .= "\n\n";
return $r;
}
/**
* Process IPN
*
* Handles the IPN post back to PayPal and parsing the response. Call this
* method from your IPN listener script. Returns true if the response came
* back as "VERIFIED", false if the response came back "INVALID", and
* throws an exception if there is an error.
*
* #param array
*
* #return boolean
*/
public function processIpn($post_data = null)
{
global $user;
$encoded_data = 'cmd=_notify-validate';
if ($post_data === null)
{
// use raw POST data
if (!empty($_POST))
{
$this->post_data = $_POST;
$encoded_data .= '&' . file_get_contents('php://input');
}
else
{
throw new Exception($user->lang['NO_POST_DATA']);
}
}
else
{
// use provided data array
$this->post_data = $post_data;
foreach ($this->post_data as $key => $value)
{
$encoded_data .= "&$key=" . urlencode($value);
}
}
if ($this->use_curl)
{
$this->curlPost($encoded_data);
}
else
{
$this->fsockPost($encoded_data);
}
if (strpos($this->response_status, '200') === false)
{
throw new Exception($user->lang['INVALID_RESPONSE'] . $this->response_status);
}
if (strpos(trim($this->response), "VERIFIED") !== false)
{
return true;
}
elseif (trim(strpos($this->response), "INVALID") !== false)
{
return false;
}
else
{
throw new Exception($user->lang['UNEXPECTED_ERROR']);
}
}
/**
* Require Post Method
*
* Throws an exception and sets a HTTP 405 response header if the request
* method was not POST.
*/
public function requirePostMethod()
{
global $user;
// require POST requests
if ($_SERVER['REQUEST_METHOD'] && $_SERVER['REQUEST_METHOD'] != 'POST')
{
header('Allow: POST', true, 405);
throw new Exception($user->lang['INVALID_REQUEST_METHOD']);
}
}
}
Is there any issue with this script which is causing this problem?
P.S: The URL donate/handler.php is indeed the IPN handler/listener file, so it's a recognized URL.

For the debug part
You can also check on Paypal the state of your IPN.
My Account > History > IPN History.
It will list all IPN that were sent to your server. You will see a status for each of them. It might help. But as Andrew Angell says, take a look at your log.
For the PHP part
Paypal provide a lots a goodness stuff on their Github. You should definitively take a closer look.
They have a dead simple IPNLister sample that you should use (instead of a custom one - even if it seems good). It use built-in function from Paypal itself. And I personally use it too. You shouldn't re-invent the wheel :)
<?php
require_once('../PPBootStrap.php');
// first param takes ipn data to be validated. if null, raw POST data is read from input stream
$ipnMessage = new PPIPNMessage(null, Configuration::getConfig());
foreach($ipnMessage->getRawData() as $key => $value) {
error_log("IPN: $key => $value");
}
if($ipnMessage->validate()) {
error_log("Success: Got valid IPN data");
} else {
error_log("Error: Got invalid IPN data");
}
As you can see, it's simple.
I use it in a slightly different way:
$rawData = file_get_contents('php://input');
$ipnMessage = new PPIPNMessage($rawData);
$this->forward404If(!$ipnMessage->validate(), 'IPN not valid.');
$ipnListener = new IPNListener($rawData);
$ipnListener->process();
The IPNListener class is custom to me: it does handle what to do with the IPN. It parse the response and do action depending on the state:
function __construct($rawData)
{
$rawPostArray = explode('&', $rawData);
foreach ($rawPostArray as $keyValue)
{
$keyValue = explode ('=', $keyValue);
if (count($keyValue) == 2)
{
$this->ipnData[$keyValue[0]] = urldecode($keyValue[1]);
}
}
// log a new IPN and save in case of error in the next process
$this->ipn = new LogIpn();
$this->ipn->setContent($rawData);
$this->ipn->setType(isset($this->ipnData['txn_type']) ? $this->ipnData['txn_type'] : 'Not defined');
$this->ipn->save();
}
/**
* Process a new valid IPN
*
*/
public function process()
{
if (null === $this->ipnData)
{
throw new Exception('ipnData is empty !');
}
if (!isset($this->ipnData['txn_type']))
{
$this->ipn->setSeemsWrong('No txn_type.');
$this->ipn->save();
return;
}
switch ($this->ipnData['txn_type'])
{
// handle statues
}
}

Check your web server logs. That will show you what result is coming up when the IPN script is hit, and since it's failing you must be getting some sort of 500 internal server error. The logs will give you the error info that you would normally see on screen, like a syntax error, line number, etc.
What I like to do for troubleshooting, too, is create a simulator of my own by building a basic HTML form with the action set to the URL of my IPN listener. Add hidden fields with the names/values you'd expect to get from an IPN and then you can load that in a browser and submit it directly so that you can see the result on screen. You'll probably find that you have an error in your code somewhere causing the script to be unable to complete.
Keep in mind that when testing this way the data isn't coming from PayPal so it will not be verified. You'll need to make sure your code logic is setup to handle that accordingly.
Once you're able to get everything running smoothly testing that way, I'd use the PayPal IPN listener as another confirmation, and then you can rest assured that you've fixed the issue.

I highly recommend using the IPN simulator in the Paypal developer site. It can build an IPN request for you and send it to your server and report back what it got.

Check your logs under your Paypal account, it will show a list of IPN requests sent and also the results. If you have some major issues you can use the GET string provided to test out use cases.

There was indeed an issue with my code. I did some casting mistake and it caused the IPN to fail. This issue has been resolved.
Thanks for the help everyone.

Related

What would cause this different outcome in this POST request in Postman vs. Async request?

I have been testing out an app that I created with a PHP backend and I have ran into something that was a bit confusing to me.
I kept running into errors with POSTMAN when I would try and run this POST request to my server.
Could this be because I am not setting the correct content-type header in POSTMAN like I do in my apps http request?
Here is a look at my POST code from my backend:
/**
* Get POST parameter
*
* #param String $key
* #return string
*/
public function post(String $key = '') {
$postdata = file_get_contents("php://input");
$request = json_decode($postdata, true);
if ($key != '') {
return isset($request[$key]) ? $this->clean($request[$key]) : null;
}
return ($request);
}
If I change the code to this:
/**
* Get $_POST parameter
*
* #param String $key
* #return string
*/
public function post(String $key = '') {
if ($key != '')
return isset($_POST[$key]) ? $this->clean($_POST[$key]) : null;
return $this->clean($_POST);
}
Then it works perfectly in POSTMAN, but does not work in my app's http request.
Any thoughts on what I have set-up incorrectly that is causing this?

How to construct 3rd parameter of edai.Search method within CURL request to XPLAN API?

I am trying to run a search on an API which requires my query data to be styled as XML nested in an XML request. I'll post my entire class and method calls (which I sent to iress tech support) so that it can be fully reviewed and in the off-chance that anyone has access to the same API, they can instantly reproduce the issue for themselves.
class XMLCurler
{
private $username = '[redacted]';
private $password = '[redacted]';
private $url = 'https://[redacted].xplan.iress.com.au/RPC2/';
public $ch; // the curl handle
public $token;
public $results;
public function __construct() {
if ($this->connect()) {
if ($this->login()) {
echo "<div class=\"success\">Successful Connection & Login. Token: {$this->token}</div>";
}
}
}
public function __destruct() {
if ($this->ch) {
$this->disconnect();
}
}
public function connect() {
if (!$this->ch = curl_init($this->url)) { // generate curl handle
echo "<div class=\"error\">CURL Error While Connecting (check url)";
return false;
}
return true;
}
public function disconnect() {
curl_close($this->ch);
}
public function processResponse($response) {
if (!$response) {
echo "<div class=\"error\">CURL Error While Attempting to Login - No XML token string<br><b>" , curl_error($this->ch) , "</b></div>";
return false;
}
$decoded = xmlrpc_decode($response);
if (is_array($decoded) && xmlrpc_is_fault($decoded)) {
echo "<div class=\"error\">Error Response: {$decoded['faultString']} ({$decoded['faultCode']})</div>";
return false;
}
return $decoded;
}
public function login() {
$postfields = xmlrpc_encode_request('edai.Login', array($this->username, $this->password)); // package as xml
curl_setopt($this->ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($this->ch, CURLOPT_POSTFIELDS, $postfields);
curl_setopt($this->ch, CURLOPT_SSL_VERIFYHOST, 0); // not advised, I need to find out how to avoid this
curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, 0); // not advised, I need to find out how to avoid this
if (!$token = $this->processResponse(curl_exec($this->ch))) {
return false;
}
if (!preg_match("~^[\w+]{20}$~", $token)) {
echo "<div class=\"error\">Invalid/Unexpected Token Generated<br><b>$token</b>";
return false;
}
$this->token = $token; // cache the valid token
return true;
}
public function listChildren($path) {
$method = "edai.ListChildren";
$request = xmlrpc_encode_request($method, array($this->token, $path));
echo "<div class=\"notice\">XMLRPC Encoded Request (for $method): <pre>" , htmlentities($request) , "</pre></div>";
curl_setopt($this->ch, CURLOPT_POSTFIELDS, $request);
if (!$results = $this->processResponse(curl_exec($this->ch))) {
return false;
}
$this->results = $results; // cache the valid results
return true;
}
public function search($basepath, $queryxml) {
$method = "edai.Search";
/** Desperate/Manual xml construction ...
* $xml = new DOMDocument("1.0", "utf-8");
* $xml->appendChild($methodCall = $xml->createElement("methodCall"));
* $methodCall->appendChild($methodName = $xml->createElement("methodName"));
* $methodCall->appendChild($params = $xml->createElement("params"));
* $params->appendChild($param1 = $xml->createElement("param"));
* $param1->appendChild($value1 = $xml->createElement("value"));
* $value1->appendChild($string1 = $xml->createElement("string"));
* $params->appendChild($param2 = $xml->createElement("param"));
* $param2->appendChild($value2 = $xml->createElement("value"));
* $value2->appendChild($string2 = $xml->createElement("string"));
* $params->appendChild($param3 = $xml->createElement("param"));
* $param3->appendChild($value3 = $xml->createElement("value"));
* $value3->appendChild($string3 = $xml->createElement("string"));
* $string3->appendChild($EntitySearch = $xml->createElement("EntitySearch"));
* $EntitySearch->appendChild($SearchResult1 = $xml->createElement("SearchResult"));
* $SearchResult1->setAttribute("field", "first_name");
* $EntitySearch->appendChild($SearchResult2 = $xml->createElement("SearchResult"));
* $SearchResult2->setAttribute('field', "last_name");
* $EntitySearch->appendChild($SearchQuick = $xml->createElement("SearchQuick"));
* $SearchQuick->appendChild($s = $xml->createElement("s"));
* $xpath = new DOMXPath($xml);
* $result1 = $xpath->query("//methodName");
* $result1->item(0)->nodeValue = $method;
* $result2 = $xpath->query("//params/param[1]/value/string");
* $result2->item(0)->nodeValue = $this->token;
* $result3 = $xpath->query("//params/param[2]/value/string");
* $result3->item(0)->nodeValue = "entitymgr/client";
* $result4 = $xpath->query("//SearchQuick/s");
* $result4->item(0)->nodeValue = "last_name:Smith";
* $xml->formatOutput = true;
* $request = $xml->saveXML();
*/
/** Desperately attempted passing array ...
* $queryarray = array(
* "EntitySearch" => array(
* array(
* "SearchResult" => array(
* "#attr" => array(
* "field" => "first_name"
* )
* )
* ),
* array(
* "SearchResult" => array(
* "#attr" => array(
* "field" => "last_name"
* )
* )
* ),
* array(
* "SearchQuick" => array(
* "s" => "last_name:Smith"
* )
* )
* )
* );
*/
$request = xmlrpc_encode_request($method, array($this->token, $basepath, $queryxml)); // this mutates the nested $queryxml string
// Causes:
//Error Response: UNKNOWN(CORBA.UNKNOWN(omniORB.UNKNOWN_PythonException, CORBA.COMPLETED_MAYBE)) (-32505)
//$request = html_entity_decode($request); // repair encoded entities
//$request = preg_replace('~(?:>\K\s+)|(?:\s+(?=<))~', '', $request); // strip every whitespace character between tags (hack)
// Causes:
// Error Response: ExpatError(syntax error: line 1, column 0 (byte 0)) (-32505)
echo "<div class=\"notice\">XMLRPC Encoded Request (for $method): <pre>" , htmlentities($request) , "</pre></div>";
curl_setopt($this->ch, CURLOPT_POSTFIELDS, $request);
if (!$results = $this->processResponse(curl_exec($this->ch))) {
return false;
}
$this->results = $results; // cache the valid results
return true;
}
}
Below is how I make the calls. edai.ListChildren works because I don't have to send any XML data. edai.Search does not work because I am failing to properly prepare the XML query within the XML request.
$XC = new XMLCurler();
/* edai.ListChildren works as desired/expected */
$path = "/entitymgr/client";
if ($XC->listChildren($path)) {
echo "<div>List of Children Successful.<pre>";
var_export($XC->results);
echo "</pre></div>";
}
/* edai.Search does not work */
$basepath = "entitymgr/client";
$queryxml = <<<XML
<EntitySearch>
<SearchResult field="first_name"/>
<SearchResult field="last_name"/>
<SearchQuick><s>last_name:Smith</s></SearchQuick>
</EntitySearch>
XML;
if ($XC->search($basepath, $queryxml)) {
echo "<div>Search Successful.<pre>";
var_export($XC->results);
echo "</pre></div>";
}
This is the attempted request and error message.
This is the relevant portion of the manual I was provided (XPLAN XML-RPC EXTERNAL DATA ACCESS INTERFACE 7 May 2013):
I have contacted iress.com a couple of weeks ago, they called me to loosely confirm that I was authorized to access the API, and told me that they'd be in touch -- that follow up call hasn't happened and I would like to get back to work on this project.
I do know for a fact that there is a last name of Smith to match my query.
I have no Python experience so the error responses are no help to me. I have made more hail mary attempts than I have posted, but I am tired of wasting my time. I don't know if the third parameter is meant to be nested inside of a <value>, <param>, <struct>, <string>, <array>, <xml>, or something else entirely.
If anyone has any suggestions regarding how to prepare my XML query for the request, I'll run them and supply feedback.
I am also happy to receive advice on the class design, security concerns, and completely different php approaches for getting the edai.Search to return some useful data.
As requested by #ThW, here is a collection of xml attempts and their respective error responses: https://pastebin.com/dYtwXWxz
A shot in the dark, since I can't test the API directly...
Perhaps the xmlrpc_encode_request call could do with named params:
$params = [
'context' => $this->token,
'basepath' => $basepath, // try with a leading slash, too, in spite of the docs listing it w/o one
'queryxml' => $queryxml,
];
$request = xmlrpc_encode_request($method, $params); // this mutates the nested $queryxml string
If that doesn't work, stop messing around with the code and install SoapUI or Postman or Insomnia or similar, and manually construct your requests.
I suspect you'll have a working request within a half-hour, and can work backwards from that to debug your code / rewrite your code. I'd do it for you if I had access to the API.
Things to check:
does the encoding make a difference (should it be utf8 instead of )?
the XML query needs to be treated as a string, so make sure it winds up encoded / wrapped in CDATA tags when your GUI client makes its requests. Depending on the client you pick it may be done for you, just make sure its done
After personally speaking to IRESS support and continued investigation, unfortunately (and despite a manual being written which expresses how to integrate with the API), the only licensed usage of the API is for "uploading documents with a Toshiba scanner".
To gain access, new license paperwork will need to be drawn up by the legal teams from IRESS and the company in the above redacted url. This is not likely to be a speedy endeavour.
It turns out that the error was not in my php code (which I have since developed further to better handle the response data), but rather the xml query that I was sending.
Instead of using what the documentation suggested:
<SearchQuick><s>last_name:Smith</s></SearchQuick>
Use the following equivalent valid expression:
<SearchByField field="last_name" op="equal"><s>Smith</s></SearchByField>

ReCaptcha not working properly on iPhone

I have a website with a simple contact form. The validation is somewhat minimal because it doesn't go into a database; just an email. The form works as such:
There are 5 fields - 4 of which are required. The submit is disabled until the 4 fields are valid, and then you can submit it. Then everything is validated on the server again, including the recaptcha (which is not validated by me client side). The whole process is done with ajax, and there are multiple tests that must pass on the server side or 4** headers are returned, and the fail callback handler is called.
Everything works like gangbusters on Chrome on the desktop (I haven't tried other browsers, but I can't imagine why they'd be different), but on the iPhone the reCaptcha always validates even if I don't check the box for the test.
In other words: I still have to fill out the four values correctly in order to submit, but if I don't check the box for the reCaptcha, the request still succeeds.
I can post some code if anyone thinks that would be helpful, but it appears that the problem is with the device and not with the code. Does anyone have any insight into this?
Note: The server side is PHP/Apache if this is helpful.
Update: 5/28/2015:
I'm still debugging this, but it seems like Mobile Safari is ignoring my response headers on my iPhone. When I output the response to the page what I get on Desktop for (data,status,xhr) is:
data: my response which at this point just says error or success -> error
status: error
xhr: {'error',400,'error'}
On Mobile safari:
data: error
status: success
xhr: {'error',200,'success'}
So - it seems like it's just ignoring my response headers. I tried explicitly setting {"headers":{"cache-control":"no-cache"}} but to no avail.
Update: 6/3/2015
Per Request, here is the code. This is almost certainly more than you need. It has also become more obtuse because of the changes I've made to try and fix it. Also note that, while it may appear that there are variables that haven't been defined, they (should) have been defined in other files.
The client side
$('#submit').on('click', function(e) {
$(this).parents('form').find('input').each(function() {
$(this).trigger('blur');
})
var $btn = $(this);
$btn = $btn.button('loading');
var dfr = $.Deferred();
if ($(this).attr('disabled') || $(this).hasClass('disabled')) {
e.preventDefault();
e.stopImmediatePropagation();
dfr.reject();
return false;
} else {
var input = $('form').serializeArray();
var obj = {},
j;
$.each(input, function(i, a) {
if (a.name === 'person-name') {
obj.name = a.value;
} else if (a.name === 'company-name') {
obj.company_name = a.value;
} else {
j = a.name.replace(/(g-)(.*)(-response)/g, '$2');
obj[j] = a.value;
}
});
obj.action = 'recaptcha-js';
obj.remoteIp = rc.remoteiP;
rc.data = obj;
var request = $.ajax({
url: rc.ajaxurl,
type: 'post',
data: obj,
headers: {
'cache-control': 'no-cache'
}
});
var success = function(data) {
$btn.data('loadingText', 'Success');
$btn.button('reset');
$('#submit').addClass('btn-success').removeClass('btn-default');
$btn.button('loading');
dfr.resolve(data);
};
var fail = function(data) {
var reason = JSON.parse(data.responseText).reason;
$btn.delay(1000).button('reset');
switch (reason) {
case 'Recaptcha Failed':
case 'Recaptcha Not Checked':
case 'One Or more validator fields not valid or not filled out':
case 'One Or more validator fields is invalid':
// reset recaptcha
if ($('#submit').data('tries')) {
$('#submit').remove();
$('.g-recaptcha').parent().addBack().remove();
myPopover('Your request is invalid. Please reload the page to try again.');
} else {
$('#submit').data('tries', 1);
grecaptcha.reset();
myPopover('One or more of your entries are invalid. Please make corrections and try again.');
}
break;
default:
// reset page
$('#submit').remove();
$('.g-recaptcha').remove();
myPopover('There was a problem with your request. Please reload the page and try again.');
break;
}
dfr.reject(data);
};
request.done(success);
request.fail(fail);
}
The Server:
function _send_email(){
$recaptcha=false;
/* * */
if(isset($_POST['recaptcha'])):
$gRecaptchaResponse=$_POST['recaptcha'];
$remoteIp=isset($_POST['remoteIp']) ? $_POST['remoteIp'] : false;
/* ** */
if(!$remoteIp):
$response=array('status_code'=>'409','reason'=>'remoteIP not set');
echo json_encode($response);
http_response_code(409);
exit();
endif;
/* ** */
/* ** */
if($gRecaptchaResponse==''):
$response=array('status_code'=>'400','reason'=>'Recaptcha Failed');
echo json_encode($response);
http_response_code(400);
exit();
endif;
/* ** */
if($recaptcha=recaptcha_test($gRecaptchaResponse,$remoteIp)):
$recaptcha=true;
/* ** */
else:
$response=array('status_code'=>'400','reason'=>'Recaptcha Failed');
echo json_encode($response);
http_response_code(400);
exit();
endif;
/* ** */
/* * */
else:
$response=array('status_code'=>'400','reason'=>'Recaptcha Not Checked');
echo json_encode($response);
http_response_code(400);
exit();
endif;
/* * */
/* * */
if($recaptcha==1):
$name=isset($_POST['name']) ? $_POST['name'] : false;
$company_name=isset($_POST['company_name']) ? $_POST['company_name'] : false;
$phone=isset($_POST['phone']) ? $_POST['phone'] : false;
$email=isset($_POST['email']) ? $_POST['email'] : false;
/* ** */
if(isset($_POST['questions'])):
$questions=$_POST['questions']=='' ? 1 : $_POST['questions'];
/* *** */
if(!$questions=filter_var($questions,FILTER_SANITIZE_SPECIAL_CHARS)):
$response=array('status_code'=>'400','reason'=>'$questions could not be sanitized');
echo json_encode($response);
http_response_code(400);
exit();
endif;
/* *** */
/* ** */
else:
$questions=true;
endif;
/* ** */
/* ** */
if( count( array_filter( array( $name,$company_name,$phone,$email ),"filter_false" ) ) !=4 ):
$response=array('status_code'=>'400','reason'=>'One Or more validator fields not valid or not filled out');
echo json_encode($response);
http_response_code(400);
exit();
endif;
/* ** */
$company_name=filter_var($company_name,FILTER_SANITIZE_SPECIAL_CHARS);
$name=filter_var($name,FILTER_SANITIZE_SPECIAL_CHARS);
$phone=preg_replace('/[^0-9+-]/', '', $phone);
$email=filter_var($email,FILTER_VALIDATE_EMAIL);
/* ** */
if($company_name && $recaptcha && $name && $phone && $email && $questions):
$phone_str='Phone: ' . $phone;
$company_str='Company: ' . $company_name;
$email_str='Email String: ' . $email;
$name_str='Name: '.$name;
$questions=$questions==1 ? '' : $questions;
$body="$name_str\r\n\r\n$company_str\r\n\r\n$email_str\r\n\r\n$phone_str\r\n\r\n____________________\r\n\r\n$questions";
$mymail='fake#fake.com';
$headers = array();
$headers[] = "MIME-Version: 1.0";
$headers[] = "Content-type: text/plain; charset=\"utf-8\"";
$headers[] = "From: $email";
$headers[] = "X-Mailer: PHP/" . phpversion();
/* *** */
if(mail('$mymail', 'Information Request from: ' . $name,$body,implode("\r\n",$headers))):
$response=array('status_code'=>'200','reason'=>'Sent !');
echo json_encode($response);
http_response_code(200);
exit();
/* *** */
else:
$response=array('status_code'=>'400','reason'=>'One Or more validator fields is invalid');
echo json_encode($response);
http_response_code(400);
exit();
endif;
/* *** */
endif;
/* ** */
endif;
/* * */
$response=array('status_code'=>'412','reason'=>'There was an unknown error');
echo json_encode($response);
http_response_code(412);
exit();
}
function recaptcha_test($gRecaptchaResponse,$remoteIp){
$secret=$itsasecret; //removed for security;
require TEMPLATE_DIR . '/includes/lib/recaptcha/src/autoload.php';
$recaptcha = new \ReCaptcha\ReCaptcha($secret);
$resp = $recaptcha->verify($gRecaptchaResponse, $remoteIp);
if ($resp->isSuccess()) {
return true;
// verified!
} else {
$errors = $resp->getErrorCodes();
return false;
}
}
Like that question iOS: Authentication using XMLHttpRequest - Handling 401 reponse the easiest way to solve that is disregard natural headers validation and, on the callback sucess, validate with some flag.
I've saw some cases like that and never smell good.
Is your "remoteIP" variable correctly set in the client side?
Even if your Ajax request sends an empty or false value, the isset() function in your php script will return true, and thus populates the $remoteIp wrongly.
Try doing:
$remoteIp = $_SERVER['REMOTE_ADDR'];
Ajax just makes the browser do the request, thus PHP can perfectly grab the ip of our user.
I'm sure that if you're passing the wrong value, ReCaptcha will mess up in one way or another.
It's also safer to never trust any Javascript variables over Ajax, as those should be treated as user input as well.
The captcha is designed to prevent malicious clients (robots), so theoretically if a client bypasses the captcha, it is a server side problem. (However, if a client fails to complete the captcha, it may be a server side problem or a client side problem.)
So the problem must be at the server. Even from security considerations, you should be using $_SERVER['REMOTE_ADDR'] rather than $_POST['remoteIp'] because $_POST['remoteIp'] may be faked (by a malicious client). In fact, $_SERVER['REMOTE_ADDR'] is much more reliable than the client-side $_POST['remoteIp'].
I made a script 2 or 3 months ago that still works perfectly, try this:
<?php
$siteKey = ''; // Public Key
$secret = ''; // Private Key
/**
* This is a PHP library that handles calling reCAPTCHA.
* - Documentation and latest version
* https://developers.google.com/recaptcha/docs/php
* - Get a reCAPTCHA API Key
* https://www.google.com/recaptcha/admin/create
* - Discussion group
* http://groups.google.com/group/recaptcha
*
* #copyright Copyright (c) 2014, Google Inc.
* #link http://www.google.com/recaptcha
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
/**
* A ReCaptchaResponse is returned from checkAnswer().
*/
class ReCaptchaResponse
{
public $success;
public $errorCodes;
}
class ReCaptcha
{
private static $_signupUrl = "https://www.google.com/recaptcha/admin";
private static $_siteVerifyUrl =
"https://www.google.com/recaptcha/api/siteverify?";
private $_secret;
private static $_version = "php_1.0";
/**
* Constructor.
*
* #param string $secret shared secret between site and ReCAPTCHA server.
*/
function ReCaptcha($secret)
{
if ($secret == null || $secret == "") {
die("To use reCAPTCHA you must get an API key from <a href='"
. self::$_signupUrl . "'>" . self::$_signupUrl . "</a>");
}
$this->_secret=$secret;
}
/**
* Encodes the given data into a query string format.
*
* #param array $data array of string elements to be encoded.
*
* #return string - encoded request.
*/
private function _encodeQS($data)
{
$req = "";
foreach ($data as $key => $value) {
$req .= $key . '=' . urlencode(stripslashes($value)) . '&';
}
// Cut the last '&'
$req=substr($req, 0, strlen($req)-1);
return $req;
}
/**
* Submits an HTTP GET to a reCAPTCHA server.
*
* #param string $path url path to recaptcha server.
* #param array $data array of parameters to be sent.
*
* #return array response
*/
private function _submitHTTPGet($path, $data)
{
$req = $this->_encodeQS($data);
$response = file_get_contents($path . $req);
return $response;
}
/**
* Calls the reCAPTCHA siteverify API to verify whether the user passes
* CAPTCHA test.
*
* #param string $remoteIp IP address of end user.
* #param string $response response string from recaptcha verification.
*
* #return ReCaptchaResponse
*/
public function verifyResponse($remoteIp, $response)
{
// Discard empty solution submissions
if ($response == null || strlen($response) == 0) {
$recaptchaResponse = new ReCaptchaResponse();
$recaptchaResponse->success = false;
$recaptchaResponse->errorCodes = 'missing-input';
return $recaptchaResponse;
}
$getResponse = $this->_submitHttpGet(
self::$_siteVerifyUrl,
array (
'secret' => $this->_secret,
'remoteip' => $remoteIp,
'v' => self::$_version,
'response' => $response
)
);
$answers = json_decode($getResponse, true);
$recaptchaResponse = new ReCaptchaResponse();
if (trim($answers ['success']) == true) {
$recaptchaResponse->success = true;
} else {
$recaptchaResponse->success = false;
$recaptchaResponse->errorCodes = $answers [error-codes];
}
return $recaptchaResponse;
}
}
$reCaptcha = new ReCaptcha($secret);
if(isset($_POST["g-recaptcha-response"])) {
$resp = $reCaptcha->verifyResponse(
$_SERVER["REMOTE_ADDR"],
$_POST["g-recaptcha-response"]
);
if ($resp != null && $resp->success) {echo "OK";}
else {echo "CAPTCHA incorrect";}
}
?>
<html>
<head>
<title>Google reCAPTCHA</title>
<script src="https://www.google.com/recaptcha/api.js"></script>
</head>
<body>
<form action="reCAPTCHA.php" method="POST">
<input type="submit" value="Submit">
<div class="g-recaptcha" data-sitekey="<?php echo $siteKey; ?>"></div>
</form>
</body>
</html>
Normally, it should work (just add your private key and your public key), I tested on my iPhone SE, 2 seconds ago, and it worked perfectly.

error 500 setting header to return 400 error PHP

I'm developing a RESTful web service and, honestly, it is my first ws. I decided to use PHP because I think that I know that language.
This is my RestHandler object, but when I debug the request, accessing to a method not implemented, with Charles, it returns the right response, but 500 error instead of 400. Why?
class RestHandler {
private $method;
private $actionName;
/**
* #param $method
* #param $action
*/
public function __construct($method, $action)
{
$this->method = $method;
$this->actionName = $action;
if (isset($this->method) && isset($this->actionName))
{
if (! method_exists($this, $this->actionName))
{
// Action is not implemented in the object.
$this->handleErrorReturning("Not implemented method.", 400);
return;
}
// OK, proceed with actions
$this->handleProceedRequest();
}
else
{
// Return error 406 Missing parameter
$this->handleErrorReturning("Missing parameter", 406);
}
}
private function handleProceedRequest()
{
if (strcasecmp($this->method, "get") == 0)
{
// No JSON to read
}
}
/**
* #param $errorDescription
* #param $errorCode
*/
private function handleErrorReturning($errorDescription, $errorCode)
{
header($_SERVER["SERVER_PROTOCOL"]." ".$errorDescription." ".$errorCode);
header('Content-Type: application/json; charset=utf-8');
$errorResponse = new ResponseError($errorCode, $errorDescription);
echo $errorResponse;
}
}
This is the Charles snapshot
SOLVED
I inverted errorDescription with errorCode and now it works. It was a stupid error. Thanks
Try setting your header like this:
header($_SERVER['SERVER_PROTOCOL'] . ' Not implemented method', true, 400);

How to make HTTP requests in PHP and not wait on the response

Is there a way in PHP to make HTTP calls and not wait for a response? I don't care about the response, I just want to do something like file_get_contents(), but not wait for the request to finish before executing the rest of my code. This would be super useful for setting off "events" of a sort in my application, or triggering long processes.
Any ideas?
The answer I'd previously accepted didn't work. It still waited for responses. This does work though, taken from How do I make an asynchronous GET request in PHP?
function post_without_wait($url, $params)
{
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;
fwrite($fp, $out);
fclose($fp);
}
If you control the target that you want to call asynchronously (e.g. your own "longtask.php"), you can close the connection from that end, and both scripts will run in parallel. It works like this:
quick.php opens longtask.php via cURL (no magic here)
longtask.php closes the connection and continues (magic!)
cURL returns to quick.php when the connection is closed
Both tasks continue in parallel
I have tried this, and it works just fine. But quick.php won't know anything about how longtask.php is doing, unless you create some means of communication between the processes.
Try this code in longtask.php, before you do anything else. It will close the connection, but still continue to run (and suppress any output):
while(ob_get_level()) ob_end_clean();
header('Connection: close');
ignore_user_abort();
ob_start();
echo('Connection Closed');
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush();
flush();
The code is copied from the PHP manual's user contributed notes and somewhat improved.
You can do trickery by using exec() to invoke something that can do HTTP requests, like wget, but you must direct all output from the program to somewhere, like a file or /dev/null, otherwise the PHP process will wait for that output.
If you want to separate the process from the apache thread entirely, try something like (I'm not sure about this, but I hope you get the idea):
exec('bash -c "wget -O (url goes here) > /dev/null 2>&1 &"');
It's not a nice business, and you'll probably want something like a cron job invoking a heartbeat script which polls an actual database event queue to do real asynchronous events.
You can use this library: https://github.com/stil/curl-easy
It's pretty straightforward then:
<?php
$request = new cURL\Request('http://yahoo.com/');
$request->getOptions()->set(CURLOPT_RETURNTRANSFER, true);
// Specify function to be called when your request is complete
$request->addListener('complete', function (cURL\Event $event) {
$response = $event->response;
$httpCode = $response->getInfo(CURLINFO_HTTP_CODE);
$html = $response->getContent();
echo "\nDone.\n";
});
// Loop below will run as long as request is processed
$timeStart = microtime(true);
while ($request->socketPerform()) {
printf("Running time: %dms \r", (microtime(true) - $timeStart)*1000);
// Here you can do anything else, while your request is in progress
}
Below you can see console output of above example.
It will display simple live clock indicating how much time request is running:
As of 2018, Guzzle has become the defacto standard library for HTTP requests, used in several modern frameworks. It's written in pure PHP and does not require installing any custom extensions.
It can do asynchronous HTTP calls very nicely, and even pool them such as when you need to make 100 HTTP calls, but don't want to run more than 5 at a time.
Concurrent request example
use GuzzleHttp\Client;
use GuzzleHttp\Promise;
$client = new Client(['base_uri' => 'http://httpbin.org/']);
// Initiate each request but do not block
$promises = [
'image' => $client->getAsync('/image'),
'png' => $client->getAsync('/image/png'),
'jpeg' => $client->getAsync('/image/jpeg'),
'webp' => $client->getAsync('/image/webp')
];
// Wait on all of the requests to complete. Throws a ConnectException
// if any of the requests fail
$results = Promise\unwrap($promises);
// Wait for the requests to complete, even if some of them fail
$results = Promise\settle($promises)->wait();
// You can access each result using the key provided to the unwrap
// function.
echo $results['image']['value']->getHeader('Content-Length')[0]
echo $results['png']['value']->getHeader('Content-Length')[0]
See http://docs.guzzlephp.org/en/stable/quickstart.html#concurrent-requests
/**
* Asynchronously execute/include a PHP file. Does not record the output of the file anywhere.
*
* #param string $filename file to execute, relative to calling script
* #param string $options (optional) arguments to pass to file via the command line
*/
function asyncInclude($filename, $options = '') {
exec("/path/to/php -f {$filename} {$options} >> /dev/null &");
}
Fake a request abortion using CURL setting a low CURLOPT_TIMEOUT_MS
set ignore_user_abort(true) to keep processing after the connection closed.
With this method no need to implement connection handling via headers and buffer too dependent on OS, Browser and PHP version
Master process
function async_curl($background_process=''){
//-------------get curl contents----------------
$ch = curl_init($background_process);
curl_setopt_array($ch, array(
CURLOPT_HEADER => 0,
CURLOPT_RETURNTRANSFER =>true,
CURLOPT_NOSIGNAL => 1, //to timeout immediately if the value is < 1000 ms
CURLOPT_TIMEOUT_MS => 50, //The maximum number of mseconds to allow cURL functions to execute
CURLOPT_VERBOSE => 1,
CURLOPT_HEADER => 1
));
$out = curl_exec($ch);
//-------------parse curl contents----------------
//$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
//$header = substr($out, 0, $header_size);
//$body = substr($out, $header_size);
curl_close($ch);
return true;
}
async_curl('http://example.com/background_process_1.php');
Background process
ignore_user_abort(true);
//do something...
NB
If you want cURL to timeout in less than one second, you can use
CURLOPT_TIMEOUT_MS, although there is a bug/"feature" on "Unix-like
systems" that causes libcurl to timeout immediately if the value is <
1000 ms with the error "cURL Error (28): Timeout was reached". The
explanation for this behavior is:
[...]
The solution is to disable signals using CURLOPT_NOSIGNAL
Resources
curl timeout less than 1000ms always fails?
http://www.php.net/manual/en/function.curl-setopt.php#104597
http://php.net/manual/en/features.connection-handling.php
The swoole extension. https://github.com/matyhtf/swoole
Asynchronous & concurrent networking framework for PHP.
$client = new swoole_client(SWOOLE_SOCK_TCP, SWOOLE_SOCK_ASYNC);
$client->on("connect", function($cli) {
$cli->send("hello world\n");
});
$client->on("receive", function($cli, $data){
echo "Receive: $data\n";
});
$client->on("error", function($cli){
echo "connect fail\n";
});
$client->on("close", function($cli){
echo "close\n";
});
$client->connect('127.0.0.1', 9501, 0.5);
let me show you my way :)
needs nodejs installed on the server
(my server sends 1000 https get request takes only 2 seconds)
url.php :
<?
$urls = array_fill(0, 100, 'http://google.com/blank.html');
function execinbackground($cmd) {
if (substr(php_uname(), 0, 7) == "Windows"){
pclose(popen("start /B ". $cmd, "r"));
}
else {
exec($cmd . " > /dev/null &");
}
}
fwite(fopen("urls.txt","w"),implode("\n",$urls);
execinbackground("nodejs urlscript.js urls.txt");
// { do your work while get requests being executed.. }
?>
urlscript.js >
var https = require('https');
var url = require('url');
var http = require('http');
var fs = require('fs');
var dosya = process.argv[2];
var logdosya = 'log.txt';
var count=0;
http.globalAgent.maxSockets = 300;
https.globalAgent.maxSockets = 300;
setTimeout(timeout,100000); // maximum execution time (in ms)
function trim(string) {
return string.replace(/^\s*|\s*$/g, '')
}
fs.readFile(process.argv[2], 'utf8', function (err, data) {
if (err) {
throw err;
}
parcala(data);
});
function parcala(data) {
var data = data.split("\n");
count=''+data.length+'-'+data[1];
data.forEach(function (d) {
req(trim(d));
});
/*
fs.unlink(dosya, function d() {
console.log('<%s> file deleted', dosya);
});
*/
}
function req(link) {
var linkinfo = url.parse(link);
if (linkinfo.protocol == 'https:') {
var options = {
host: linkinfo.host,
port: 443,
path: linkinfo.path,
method: 'GET'
};
https.get(options, function(res) {res.on('data', function(d) {});}).on('error', function(e) {console.error(e);});
} else {
var options = {
host: linkinfo.host,
port: 80,
path: linkinfo.path,
method: 'GET'
};
http.get(options, function(res) {res.on('data', function(d) {});}).on('error', function(e) {console.error(e);});
}
}
process.on('exit', onExit);
function onExit() {
log();
}
function timeout()
{
console.log("i am too far gone");process.exit();
}
function log()
{
var fd = fs.openSync(logdosya, 'a+');
fs.writeSync(fd, dosya + '-'+count+'\n');
fs.closeSync(fd);
}
You can use non-blocking sockets and one of pecl extensions for PHP:
http://php.net/event
http://php.net/libevent
http://php.net/ev
https://github.com/m4rw3r/php-libev
You can use library which gives you an abstraction layer between your code and a pecl extension: https://github.com/reactphp/event-loop
You can also use async http-client, based on the previous library: https://github.com/reactphp/http-client
See others libraries of ReactPHP: http://reactphp.org
Be careful with an asynchronous model.
I recommend to see this video on youtube: http://www.youtube.com/watch?v=MWNcItWuKpI
class async_file_get_contents extends Thread{
public $ret;
public $url;
public $finished;
public function __construct($url) {
$this->finished=false;
$this->url=$url;
}
public function run() {
$this->ret=file_get_contents($this->url);
$this->finished=true;
}
}
$afgc=new async_file_get_contents("http://example.org/file.ext");
Event Extension
Event extension is very appropriate. It is a port of Libevent library which is designed for event-driven I/O, mainly for networking.
I have written a sample HTTP client that allows to schedule a number of
HTTP requests and run them asynchronously.
This is a sample HTTP client class based on Event extension.
The class allows to schedule a number of HTTP requests, then run them asynchronously.
http-client.php
<?php
class MyHttpClient {
/// #var EventBase
protected $base;
/// #var array Instances of EventHttpConnection
protected $connections = [];
public function __construct() {
$this->base = new EventBase();
}
/**
* Dispatches all pending requests (events)
*
* #return void
*/
public function run() {
$this->base->dispatch();
}
public function __destruct() {
// Destroy connection objects explicitly, don't wait for GC.
// Otherwise, EventBase may be free'd earlier.
$this->connections = null;
}
/**
* #brief Adds a pending HTTP request
*
* #param string $address Hostname, or IP
* #param int $port Port number
* #param array $headers Extra HTTP headers
* #param int $cmd A EventHttpRequest::CMD_* constant
* #param string $resource HTTP request resource, e.g. '/page?a=b&c=d'
*
* #return EventHttpRequest|false
*/
public function addRequest($address, $port, array $headers,
$cmd = EventHttpRequest::CMD_GET, $resource = '/')
{
$conn = new EventHttpConnection($this->base, null, $address, $port);
$conn->setTimeout(5);
$req = new EventHttpRequest([$this, '_requestHandler'], $this->base);
foreach ($headers as $k => $v) {
$req->addHeader($k, $v, EventHttpRequest::OUTPUT_HEADER);
}
$req->addHeader('Host', $address, EventHttpRequest::OUTPUT_HEADER);
$req->addHeader('Connection', 'close', EventHttpRequest::OUTPUT_HEADER);
if ($conn->makeRequest($req, $cmd, $resource)) {
$this->connections []= $conn;
return $req;
}
return false;
}
/**
* #brief Handles an HTTP request
*
* #param EventHttpRequest $req
* #param mixed $unused
*
* #return void
*/
public function _requestHandler($req, $unused) {
if (is_null($req)) {
echo "Timed out\n";
} else {
$response_code = $req->getResponseCode();
if ($response_code == 0) {
echo "Connection refused\n";
} elseif ($response_code != 200) {
echo "Unexpected response: $response_code\n";
} else {
echo "Success: $response_code\n";
$buf = $req->getInputBuffer();
echo "Body:\n";
while ($s = $buf->readLine(EventBuffer::EOL_ANY)) {
echo $s, PHP_EOL;
}
}
}
}
}
$address = "my-host.local";
$port = 80;
$headers = [ 'User-Agent' => 'My-User-Agent/1.0', ];
$client = new MyHttpClient();
// Add pending requests
for ($i = 0; $i < 10; $i++) {
$client->addRequest($address, $port, $headers,
EventHttpRequest::CMD_GET, '/test.php?a=' . $i);
}
// Dispatch pending requests
$client->run();
test.php
This is a sample script on the server side.
<?php
echo 'GET: ', var_export($_GET, true), PHP_EOL;
echo 'User-Agent: ', $_SERVER['HTTP_USER_AGENT'] ?? '(none)', PHP_EOL;
Usage
php http-client.php
Sample Output
Success: 200
Body:
GET: array (
'a' => '1',
)
User-Agent: My-User-Agent/1.0
Success: 200
Body:
GET: array (
'a' => '0',
)
User-Agent: My-User-Agent/1.0
Success: 200
Body:
GET: array (
'a' => '3',
)
...
(Trimmed.)
Note, the code is designed for long-term processing in the CLI SAPI.
For custom protocols, consider using low-level API, i.e. buffer events, buffers. For SSL/TLS communications, I would recommend the low-level API in conjunction with Event's ssl context. Examples:
SSL echo server
SSL client
Although Libevent's HTTP API is simple, it is not as flexible as buffer events. For example, the HTTP API currently doesn't support custom HTTP methods. But it is possible to implement virtually any protocol using the low-level API.
Ev Extension
I have also written a sample of another HTTP client using Ev extension with sockets in non-blocking mode. The code is slightly more verbose than the sample based on Event, because Ev is a general purpose event loop. It doesn't provide network-specific functions, but its EvIo watcher is capable of listening to a file descriptor encapsulated into the socket resource, in particular.
This is a sample HTTP client based on Ev extension.
Ev extension implements a simple yet powerful general purpose event loop. It doesn't provide network-specific watchers, but its I/O watcher can be used for asynchronous processing of sockets.
The following code shows how HTTP requests can be scheduled for parallel processing.
http-client.php
<?php
class MyHttpRequest {
/// #var MyHttpClient
private $http_client;
/// #var string
private $address;
/// #var string HTTP resource such as /page?get=param
private $resource;
/// #var string HTTP method such as GET, POST etc.
private $method;
/// #var int
private $service_port;
/// #var resource Socket
private $socket;
/// #var double Connection timeout in seconds.
private $timeout = 10.;
/// #var int Chunk size in bytes for socket_recv()
private $chunk_size = 20;
/// #var EvTimer
private $timeout_watcher;
/// #var EvIo
private $write_watcher;
/// #var EvIo
private $read_watcher;
/// #var EvTimer
private $conn_watcher;
/// #var string buffer for incoming data
private $buffer;
/// #var array errors reported by sockets extension in non-blocking mode.
private static $e_nonblocking = [
11, // EAGAIN or EWOULDBLOCK
115, // EINPROGRESS
];
/**
* #param MyHttpClient $client
* #param string $host Hostname, e.g. google.co.uk
* #param string $resource HTTP resource, e.g. /page?a=b&c=d
* #param string $method HTTP method: GET, HEAD, POST, PUT etc.
* #throws RuntimeException
*/
public function __construct(MyHttpClient $client, $host, $resource, $method) {
$this->http_client = $client;
$this->host = $host;
$this->resource = $resource;
$this->method = $method;
// Get the port for the WWW service
$this->service_port = getservbyname('www', 'tcp');
// Get the IP address for the target host
$this->address = gethostbyname($this->host);
// Create a TCP/IP socket
$this->socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
if (!$this->socket) {
throw new RuntimeException("socket_create() failed: reason: " .
socket_strerror(socket_last_error()));
}
// Set O_NONBLOCK flag
socket_set_nonblock($this->socket);
$this->conn_watcher = $this->http_client->getLoop()
->timer(0, 0., [$this, 'connect']);
}
public function __destruct() {
$this->close();
}
private function freeWatcher(&$w) {
if ($w) {
$w->stop();
$w = null;
}
}
/**
* Deallocates all resources of the request
*/
private function close() {
if ($this->socket) {
socket_close($this->socket);
$this->socket = null;
}
$this->freeWatcher($this->timeout_watcher);
$this->freeWatcher($this->read_watcher);
$this->freeWatcher($this->write_watcher);
$this->freeWatcher($this->conn_watcher);
}
/**
* Initializes a connection on socket
* #return bool
*/
public function connect() {
$loop = $this->http_client->getLoop();
$this->timeout_watcher = $loop->timer($this->timeout, 0., [$this, '_onTimeout']);
$this->write_watcher = $loop->io($this->socket, Ev::WRITE, [$this, '_onWritable']);
return socket_connect($this->socket, $this->address, $this->service_port);
}
/**
* Callback for timeout (EvTimer) watcher
*/
public function _onTimeout(EvTimer $w) {
$w->stop();
$this->close();
}
/**
* Callback which is called when the socket becomes wriable
*/
public function _onWritable(EvIo $w) {
$this->timeout_watcher->stop();
$w->stop();
$in = implode("\r\n", [
"{$this->method} {$this->resource} HTTP/1.1",
"Host: {$this->host}",
'Connection: Close',
]) . "\r\n\r\n";
if (!socket_write($this->socket, $in, strlen($in))) {
trigger_error("Failed writing $in to socket", E_USER_ERROR);
return;
}
$loop = $this->http_client->getLoop();
$this->read_watcher = $loop->io($this->socket,
Ev::READ, [$this, '_onReadable']);
// Continue running the loop
$loop->run();
}
/**
* Callback which is called when the socket becomes readable
*/
public function _onReadable(EvIo $w) {
// recv() 20 bytes in non-blocking mode
$ret = socket_recv($this->socket, $out, 20, MSG_DONTWAIT);
if ($ret) {
// Still have data to read. Append the read chunk to the buffer.
$this->buffer .= $out;
} elseif ($ret === 0) {
// All is read
printf("\n<<<<\n%s\n>>>>", rtrim($this->buffer));
fflush(STDOUT);
$w->stop();
$this->close();
return;
}
// Caught EINPROGRESS, EAGAIN, or EWOULDBLOCK
if (in_array(socket_last_error(), static::$e_nonblocking)) {
return;
}
$w->stop();
$this->close();
}
}
/////////////////////////////////////
class MyHttpClient {
/// #var array Instances of MyHttpRequest
private $requests = [];
/// #var EvLoop
private $loop;
public function __construct() {
// Each HTTP client runs its own event loop
$this->loop = new EvLoop();
}
public function __destruct() {
$this->loop->stop();
}
/**
* #return EvLoop
*/
public function getLoop() {
return $this->loop;
}
/**
* Adds a pending request
*/
public function addRequest(MyHttpRequest $r) {
$this->requests []= $r;
}
/**
* Dispatches all pending requests
*/
public function run() {
$this->loop->run();
}
}
/////////////////////////////////////
// Usage
$client = new MyHttpClient();
foreach (range(1, 10) as $i) {
$client->addRequest(new MyHttpRequest($client, 'my-host.local', '/test.php?a=' . $i, 'GET'));
}
$client->run();
Testing
Suppose http://my-host.local/test.php script is printing the dump of $_GET:
<?php
echo 'GET: ', var_export($_GET, true), PHP_EOL;
Then the output of php http-client.php command will be similar to the following:
<<<<
HTTP/1.1 200 OK
Server: nginx/1.10.1
Date: Fri, 02 Dec 2016 12:39:54 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
X-Powered-By: PHP/7.0.13-pl0-gentoo
1d
GET: array (
'a' => '3',
)
0
>>>>
<<<<
HTTP/1.1 200 OK
Server: nginx/1.10.1
Date: Fri, 02 Dec 2016 12:39:54 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
X-Powered-By: PHP/7.0.13-pl0-gentoo
1d
GET: array (
'a' => '2',
)
0
>>>>
...
(trimmed)
Note, in PHP 5 the sockets extension may log warnings for EINPROGRESS, EAGAIN, and EWOULDBLOCK errno values. It is possible to turn off the logs with
error_reporting(E_ERROR);
Concerning "the Rest" of the Code
I just want to do something like file_get_contents(), but not wait for the request to finish before executing the rest of my code.
The code that is supposed to run in parallel with the network requests can be executed within a the callback of an Event timer, or Ev's idle watcher, for instance. You can easily figure it out by watching the samples mentioned above. Otherwise, I'll add another example :)
I find this package quite useful and very simple: https://github.com/amphp/parallel-functions
<?php
use function Amp\ParallelFunctions\parallelMap;
use function Amp\Promise\wait;
$responses = wait(parallelMap([
'https://google.com/',
'https://github.com/',
'https://stackoverflow.com/',
], function ($url) {
return file_get_contents($url);
}));
It will load all 3 urls in parallel.
You can also use class instance methods in the closure.
For example I use Laravel extension based on this package https://github.com/spatie/laravel-collection-macros#parallelmap
Here is my code:
/**
* Get domains with all needed data
*/
protected function getDomainsWithdata(): Collection
{
return $this->opensrs->getDomains()->parallelMap(function ($domain) {
$contact = $this->opensrs->getDomainContact($domain);
$contact['domain'] = $domain;
return $contact;
}, 10);
}
It loads all needed data in 10 parallel threads and instead of 50 secs without async it finished in just 8 secs.
Here is a working example, just run it and open storage.txt afterwards, to check the magical result
<?php
function curlGet($target){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $target);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec ($ch);
curl_close ($ch);
return $result;
}
// Its the next 3 lines that do the magic
ignore_user_abort(true);
header("Connection: close"); header("Content-Length: 0");
echo str_repeat("s", 100000); flush();
$i = $_GET['i'];
if(!is_numeric($i)) $i = 1;
if($i > 4) exit;
if($i == 1) file_put_contents('storage.txt', '');
file_put_contents('storage.txt', file_get_contents('storage.txt') . time() . "\n");
sleep(5);
curlGet($_SERVER['HTTP_HOST'] . $_SERVER['SCRIPT_NAME'] . '?i=' . ($i + 1));
curlGet($_SERVER['HTTP_HOST'] . $_SERVER['SCRIPT_NAME'] . '?i=' . ($i + 1));
Here is my own PHP function when I do POST to a specific URL of any page....
Sample: *** usage of my Function...
<?php
parse_str("email=myemail#ehehehahaha.com&subject=this is just a test");
$_POST['email']=$email;
$_POST['subject']=$subject;
echo HTTP_POST("http://example.com/mail.php",$_POST);***
exit;
?>
<?php
/*********HTTP POST using FSOCKOPEN **************/
// by ArbZ
function HTTP_Post($URL,$data, $referrer="") {
// parsing the given URL
$URL_Info=parse_url($URL);
// Building referrer
if($referrer=="") // if not given use this script as referrer
$referrer=$_SERVER["SCRIPT_URI"];
// making string from $data
foreach($data as $key=>$value)
$values[]="$key=".urlencode($value);
$data_string=implode("&",$values);
// Find out which port is needed - if not given use standard (=80)
if(!isset($URL_Info["port"]))
$URL_Info["port"]=80;
// building POST-request: HTTP_HEADERs
$request.="POST ".$URL_Info["path"]." HTTP/1.1\n";
$request.="Host: ".$URL_Info["host"]."\n";
$request.="Referer: $referer\n";
$request.="Content-type: application/x-www-form-urlencoded\n";
$request.="Content-length: ".strlen($data_string)."\n";
$request.="Connection: close\n";
$request.="\n";
$request.=$data_string."\n";
$fp = fsockopen($URL_Info["host"],$URL_Info["port"]);
fputs($fp, $request);
while(!feof($fp)) {
$result .= fgets($fp, 128);
}
fclose($fp); //$eco = nl2br();
function getTextBetweenTags($string, $tagname) {
$pattern = "/<$tagname ?.*>(.*)<\/$tagname>/";
preg_match($pattern, $string, $matches);
return $matches[1];
}
//STORE THE FETCHED CONTENTS to a VARIABLE, because its way better and fast...
$str = $result;
$txt = getTextBetweenTags($str, "span"); $eco = $txt; $result = explode("&",$result);
return $result[1];
<span style=background-color:LightYellow;color:blue>".trim($_GET['em'])."</span>
</pre> ";
}
</pre>
ReactPHP async http client
https://github.com/shuchkin/react-http-client
Install via Composer
$ composer require shuchkin/react-http-client
Async HTTP GET
// get.php
$loop = \React\EventLoop\Factory::create();
$http = new \Shuchkin\ReactHTTP\Client( $loop );
$http->get( 'https://tools.ietf.org/rfc/rfc2068.txt' )->then(
function( $content ) {
echo $content;
},
function ( \Exception $ex ) {
echo 'HTTP error '.$ex->getCode().' '.$ex->getMessage();
}
);
$loop->run();
Run php in CLI-mode
$ php get.php
Symfony HttpClient is asynchronous https://symfony.com/doc/current/components/http_client.html.
For example you can
use Symfony\Component\HttpClient\HttpClient;
$client = HttpClient::create();
$response1 = $client->request('GET', 'https://website1');
$response2 = $client->request('GET', 'https://website1');
$response3 = $client->request('GET', 'https://website1');
//these 3 calls with return immediately
//but the requests will fire to the website1 webserver
$response1->getContent(); //this will block until content is fetched
$response2->getContent(); //same
$response3->getContent(); //same
Well, the timeout can be set in milliseconds,
see "CURLOPT_CONNECTTIMEOUT_MS" in http://www.php.net/manual/en/function.curl-setopt

Categories