I'm developing a system consisting in frontend built in CakePHP framework and Java based backend. The communication between this two ecosystems is carried out by sending JSON messages from CakePHP controller to RabbitMQ broker. When message is consumed, the result is being send back to the frontend.
Now, I need to consume the message and push the result from the controller to user browser. For the PHP part I'm using a phpamqplib, but it needs to have an infinite loop when listening for new messages:
$channel->basic_consume('AMQP.COMMAND.OUTPUT.QUEUE',
'consumer',
false,
false,
false,
false,
array($this, 'processMessage'));
function shutdown($ch, $conn){
$ch->close();
$conn->close();
}
register_shutdown_function('shutdown', $channel, $conn);
while (count($channel->callbacks)) {
$read = array($conn->getSocket()); // add here other sockets that you need to attend
$write = null;
$except = null;
if (false === ($num_changed_streams = stream_select($read, $write, $except, 60))) {
/* Error handling */
} elseif ($num_changed_streams > 0) {
$channel->wait();
}
}
In my controller this is provoking Apache Server to throw an error because maximum execution of 30 seconds is exceeded.
I really need help here. What's the best solution to listen for new messages and then pushing the result to the view?
Thanks
Cheers.
I highly recommend converting this to an AJAX-based infrastructure, and re-factor your code to do this:
CakePHP makes an AJAX call to load the page every x seconds
The AJAX URL gets the remaining elements from the queue, and outputs them
Your code doesn't look complete, so I can't refactor it completely, but you could change the AJAX URL to do something like this:
if (count($channel->callbacks)) {
$read = array($conn->getSocket()); // add here other sockets that you need to attend
$write = null;
$except = null;
if (false === ($num_changed_streams = stream_select($read, $write, $except, 60))) {
/* Error handling */
}
}
and close the channel when done.
Your other option, if you really want to use push, is to use web sockets. Do a search, or this tutorial might help you get started.
Related
Is there a way of overwriting retries for an individual call in AWS SDK for PHP?
The following code explains the question:
// Create client with a default of 2 retries
$sqsClient = new sqsClient('2012-11-05', ['retries' => 2]);
// This will retry twice to get the queue attributes (perfect)
try {
$sqsClient->getQueueAttributes();
} catch(Exception $e) {
}
// I want the following to NEVER retry
try {
$sqsClient->turnOffRetryLogic(???);
$sqsClient->receiveMessages(['WaitTimeSeconds' => 5]);
} catch(Exception $e) {
}
// Now set the retries back to as before.
Retries are handled by Middleware - but as the Middleware class is marked "final" I need to pass in a "decider"? This means we need to hook into one of the handlers but none appear to be connected to retries.
Edit:
I have managed to prove the concept of a new "decider" by directly editing the AWS SDK as follows:
final class Middleware
{
public static function retry(
callable $decider = null,
callable $delay = null,
$stats = false
) {
....
$decider = function() {
echo 'retries cancelled';
return false;
};
....
So the question is how to do this without editing the SDK. Have tried various middleware hooks as follows, without success.
$decider = function() {
echo 'No retries';
return false;
};
$SqsClient->getHandlerList()->appendSign(\AWS\Middleware::retry($decider, null), 'retry');
$result = $SqsClient->receiveMessage($aParams);
(Code samples snipped to only show relevant parts)
Next code removes retry handler
$sqsClient->getHandlerList()->remove('retry');
Sqs client isn't going to retry after that. To restore default behavior you can attach default handler back
$decider = RetryMiddleware::createDefaultDecider(3);
$sqsClient->getHandlerList()->appendSign(
Middleware::retry($decider, null, false),
'retry'
);
Though, two separate clients with retries enabled and disabled sound more transparent for me.
I was trying to use Server Side Events mechanics in my project. (This is like Long Polling on steroids)
Example from "Sending events from the server" subtitle works beautifully. After few seconds, from disconnection, the apache process is killed. This method works fine.
BUT! If I try to use RabbitMQ, Apache does't get the process killed after browser disconnects from server (es.close()). And process leaves as is and gets killed only after the docker container restarts.
connection_aborted and connection_status don't work at all. connection_aborted returns only 0 and connection_status returns CONNECTION_NORMAL even after disconnect. It happens only when I use RabbitMQ. Without RMQ this functions works well.
ignore_user_abort(false) doesn't work either.
Code example:
<?php
use PhpAmqpLib\Channel\AMQPChannel;
use PhpAmqpLib\Connection\AbstractConnection;
use PhpAmqpLib\Exception\AMQPTimeoutException;
use PhpAmqpLib\Message\AMQPMessage;
class RequestsRabbit
{
protected $rabbit;
/** #var AMQPChannel */
protected $channel;
public $exchange = 'requests.events';
public function __construct(AbstractConnection $rabbit)
{
$this->rabbit = $rabbit;
}
public function getChannel()
{
if ($this->channel === null) {
$channel = $this->rabbit->channel();
$channel->exchange_declare($this->exchange, 'fanout', false, false, false);
$this->channel = $channel;
}
return $this->channel;
}
public function send($message)
{
$channel = $this->getChannel();
$message = json_encode($message);
$channel->basic_publish(new AMQPMessage($message), $this->exchange);
}
public function subscribe(callable $callable)
{
$channel = $this->getChannel();
list($queue_name) = $channel->queue_declare('', false, false, true, false);
$channel->queue_bind($queue_name, $this->exchange);
$callback = function (AMQPMessage $msg) use ($callable) {
call_user_func($callable, json_decode($msg->body));
};
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);
while (count($channel->callbacks)) {
if (connection_aborted()) {
break;
}
try {
$channel->wait(null, true, 5);
} catch (AMQPTimeoutException $exception) {
}
}
$channel->close();
$this->rabbit->close();
}
}
What happens:
Browser establishes SSE connection to the server. var es = new EventSource(url);
Apache2 spawns new process to handle this request.
PHP generates a new Queue and connects to it.
Browser closes connection es.close()
Apache2 doesn't kill process and it stays as is. Queue of RabbitMQ will not be deleted. If I do some reconnections, it spawns a bunch of processes and a bunch of queues (1 reconnection = 1 process = 1 queue).
I close all tabs -- processes alive. I close browser -- the same situation.
Looks line some kind of PHP bug. Or of Apach2?
What I use:
Last Docker and docker-compose
php:7.1.12-apache or php:5.6-apache image (this happens on both versions of PHP)
Some screenshots:
Please, help me to figure out what's going on...
P.S. Sorry for my English. If you can find a mistake or typo, point to it in the comments. I'll be very grateful :)
You don't say if you're using send() or subscribe() (or both) during your server-side events. Assuming you're using subscribe() there is no bug. This loop:
while (count($channel->callbacks)) {
if (connection_aborted()) {
break;
}
try {
$channel->wait(null, true, 5);
} catch (AMQPTimeoutException $exception) {
}
}
Will run until the process is killed or the connection is remotely closed from RabbitMQ. This is normal when listening for queued messages. If you need to stop the loop at some point you can set a variable to check in the loop or throw an exception when the SSE is ended (although I find this awkward).
I have created a soft phone for use with Twilio, using Twilio.js (1.4) and the Twilio REST API.
On the connection callback, I have a need to fetch the childSid for a call. To accommodate this I created a route in my Laravel app to use the Calls list resource and get it into the browser using jQuery.get() on the connection callback.
For some reason the API does not respond at all if I don't first wait about 12 seconds after the initial connection. After using sleep(12) in my PHP function I can successfully read the calls and filter for the ParentSid with no issues.
Is there a reason the API will not respond if invoked too soon after a connection is made via Twilio.js? It seems to only do this when I'm using $client->calls>read(). I have no problem retrieving a parentCallSid from a call immediately using $client->calls($callSid)->fetch().
Here is the original code:
public function showChildCallSid(Request $request, Client $client) {
$callSid = $request->input('CallSid');
sleep(12); // only works after waiting about 12 seconds
$call = $client->calls->read(['ParentCallSid' => $callSid])[0];
return $call->sid;
}
I believe the issue was ultimately a problem with syntax. I revised to the code as shown below and it works very well now (only needing a 1-second pause usually):
public function showChildCallSid(Request $request, Client $client) {
$callSid = $request->input('CallSid');
$attempt = 1;
$maxAttempts = 15;
do {
$calls = $client->calls->read(['ParentCallSid' => $callSid]);
if (sizeof($calls) > 0) {
break;
}
sleep(1);
$attempt++;
} while ($attempt < $maxAttempts);
$childSid = $calls[0]->sid;
return $childSid;
}
I can see in the SWF Management Console a Workflow has around 18 events, with the 16th event being my ActivityTaskCompleted event however whenever i poll for decisions i only get up to the 15th event so i never get to call RespondDecisionTaskCompleted with the decision type CompleteWorkflowExecution as such my workflows are always sitting in the Active state until they timeout.
The flow i'm using is from a PHP SWF git i found a while ago, i unfortunately do not have the link to it anymore though.
$response = $this->swf->PollForDecisionTask($opts);
$decision_list = self::_decide(new HistoryEventIterator($this->swf, $opts, $response), $this->swf);
if(count($decision_list) > 0)
{
//Some decisions
}
else
{
//No decisions
}
Where the HistoryEventIterator looks like so
public function __construct(Aws\Swf\SwfClient $swf_client, $original_request, $original_response) {
$this->swf = $swf_client;
$this->events = array();
$this->event_index = 0;
$this->next_page_token = null;
$this->original_poll_request = $original_request;
$this->_process_poll_response($original_response);
$this->workflow_id = $original_response->get('workflowExecution')['workflowId'];
$this->run_id = $original_response->get('workflowExecution')['runId'];
}
protected function _process_poll_response($response) {
if ($response->hasKey("nextPageToken")) {
$this->next_page_token = (string) $response->get("nextPageToken");
} else {
$this->next_page_token = null;
}
$next_events = $response->get("events");
$next_events_object = new ArrayObject($next_events);
$next_events_copy = $next_events_object->getArrayCopy();
$this->events = array_merge($this->events, $next_events);
}
I have omitted error checking and functions of HistoryEventIterator that would not be called in this scenario.
I have output the next_page_token of HistoryEventIterator and found it was always NULL.
Should the RespondDecisionTaskCompleted called from an Activity reach the decider? If so, what could be the cause for mine not? Surely it wouldn't be paging after 15 events, and simply not paging correctly.
I can verify that the Domain, Activity Task List, and Decider Task List are accurate as the Workflow shows up in the SWF Management Console, as does the decisions and the Activity (The Activity even has the status Completed) There is appropriate error checking and Try/Catch blocks and in no cases are there any exceptions.
my web app requires making 7 different soap wsdl api requests to complete one task (I need the users to wait for the result of all the requests). The avg response time is 500 ms to 1.7 second for each request. I need to run all these request in parallel to speed up the process.
What's the best way to do that:
pthreads or
Gearman workers
fork process
curl multi (i have to build the xml soap body)
Well the first thing to say is, it's never really a good idea to create threads in direct response to a web request, think about how far that will actually scale.
If you create 7 threads for everyone that comes along and 100 people turn up, you'll be asking your hardware to execute 700 threads concurrently, which is quite a lot to ask of anything really...
However, scalability is not something I can usefully help you with, so I'll just answer the question.
<?php
/* the first service I could find that worked without authorization */
define("WSDL", "http://www.webservicex.net/uklocation.asmx?WSDL");
class CountyData {
/* this works around simplexmlelements being unsafe (and shit) */
public function __construct(SimpleXMLElement $element) {
$this->town = (string)$element->Town;
$this->code = (string)$element->PostCode;
}
public function run(){}
protected $town;
protected $code;
}
class GetCountyData extends Thread {
public function __construct($county) {
$this->county = $county;
}
public function run() {
$soap = new SoapClient(WSDL);
$result = $soap->getUkLocationByCounty(array(
"County" => $this->county
));
foreach (simplexml_load_string(
$result->GetUKLocationByCountyResult) as $element) {
$this[] = new CountyData($element);
}
}
protected $county;
}
$threads = [];
$thread = 0;
$threaded = true; # change to false to test without threading
$counties = [ # will create as many threads as there are counties
"Buckinghamshire",
"Berkshire",
"Yorkshire",
"London",
"Kent",
"Sussex",
"Essex"
];
while ($thread < count($counties)) {
$threads[$thread] =
new GetCountyData($counties[$thread]);
if ($threaded) {
$threads[$thread]->start();
} else $threads[$thread]->run();
$thread++;
}
if ($threaded)
foreach ($threads as $thread)
$thread->join();
foreach ($threads as $county => $data) {
printf(
"Data for %s %d\n", $counties[$county], count($data));
}
?>
Note that, the SoapClient instance is not, and can not be shared, this may well slow you down, you might want to enable caching of wsdl's ...