I have a consumer worker in PHP script.
But sometimes the RabbitMQ server stops from running,
I get this error:
PHP Fatal error: Uncaught exception 'ErrorException' with message 'fwrite(): send of 19 bytes failed with errno=32 Broken pipe' in /home/user/pusher/rabbitmq-worker/vendor/php-amqplib/php-amqplib/PhpAmqpLib/Wire/IO/StreamIO.php:281
I would like to be able to handle this error with a try/catch block to throw a nice error to my console or try to reconnect after a while.
This is what I have code so far:
while (true) {
try {
$connection = new AMQPStreamConnection(RABBITMQ_HOST, RABBITMQ_PORT, RABBITMQ_USER, RABBITMQ_PASS);
$channel = $connection->channel();
$channel->queue_declare(RABBITMQ_DT_QUEUE, false, true, false, false);
$channel->basic_qos(null, 11, null);
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$callback = function($req) {
sleep(3);//Some task that takes 3 sec
};
$channel->basic_consume(RABBITMQ_QUEUE, '', false, false, false, false, $callback);
while (count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
} catch (Exception $e) {
sleep(1);
}
What's missing?
Define next parameter on connection
keepalive: true
heartbeat: 15 // An example
It is two latest arguments of AMQPStreamConnection::__constructor()
NOTE:
if you connect to some distant server, I recommend set high up timeouts
connection_timeout: 10
read_write_timeout: 30
UPD:
You can catch ErrorException and try connect to server again
Related
We are using RabbitMQ v3.7.5, AMQP 0.9.1, Erlang 20.1.7.1, PHP 7, centos 7
Our consumers are written using php-amqplib v2.9.1
We start our consumers by calling a URL using curl. The consumers work as expected for sometime, but after sometime all consumers gets doubled i.e it shows two connection running for same consumer. Once this happens, none of the consumers respond when there is data on the queue but all the data received on the queue is lost.
Has anyone faced the same issue?
Consumer Code (I am using the same code given on the official site: https://www.rabbitmq.com/tutorials/tutorial-two-php.html)
require_once __DIR__ . '/vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('task_queue', false, true, false, false);
echo " [*] Waiting for messages. To exit press CTRL+C\n";
$callback = function ($msg) {
echo ' [x] Received ', $msg->body, "\n";
sleep(substr_count($msg->body, '.'));
echo " [x] Done\n";
$msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
};
$channel->basic_qos(null, 1, null);
$channel->basic_consume('task_queue', '', false, false, false, false, $callback);
while (count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
Expected Result: The consumer should not be duplicated.
I'm trying to send some data from php application to the user's browser using websockets. Therefore I've decided to use Swoole in combination with RabbitMQ.
It's the first time I'm working with websockets and after reading some posts about Socket.IO, Ratchet, etc. I've decided to halt on Swoole because it's written in C and handy to use with php.
This is how I understood the idea of enabling data transfer using websockets:
1) Start RabbitMQ worker and Swoole server in CLI
2) php application sends data to RabbitMQ
3) RabbitMQ sends message with data to worker
4) Worker receives message with data + establishes socket connection with Swoole socket server.
5) Swoole server broadcasts data to all connections
The question is how to bind Swoole socket server with RabbitMQ? Or how to make RabbitMQ to establish connection with Swoole and send data to it?
Here is the code:
Swoole server (swoole_sever.php)
$server = new \swoole_websocket_server("0.0.0.0", 2345, SWOOLE_BASE);
$server->on('open', function(\Swoole\Websocket\Server $server, $req)
{
echo "connection open: {$req->fd}\n";
});
$server->on('message', function($server, \Swoole\Websocket\Frame $frame)
{
echo "received message: {$frame->data}\n";
$server->push($frame->fd, json_encode(["hello", "world"]));
});
$server->on('close', function($server, $fd)
{
echo "connection close: {$fd}\n";
});
$server->start();
Worker which receives message from RabbitMQ, then makes connection to Swoole and broadcasts the message via socket connection (worker.php)
$connection = new AMQPStreamConnection('0.0.0.0', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('task_queue', false, true, false, false);
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$callback = function($msg){
echo " [x] Received ", $msg->body, "\n";
sleep(substr_count($msg->body, '.'));
echo " [x] Done", "\n";
$msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
// Here I'm trying to make connection to Swoole server and sernd data
$cli = new \swoole_http_client('0.0.0.0', 2345);
$cli->on('message', function ($_cli, $frame) {
var_dump($frame);
});
$cli->upgrade('/', function($cli)
{
$cli->push('This is the message to send to Swoole server');
$cli->close();
});
};
$channel->basic_qos(null, 1, null);
$channel->basic_consume('task_queue', '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
New task where the message will be send to RabbitMQ (new_task.php):
$connection = new AMQPStreamConnection('0.0.0.0', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('task_queue', false, true, false, false);
$data = implode(' ', array_slice($argv, 1));
if(empty($data)) $data = "Hello World!";
$msg = new AMQPMessage($data,
array('delivery_mode' => AMQPMessage::DELIVERY_MODE_PERSISTENT)
);
$channel->basic_publish($msg, '', 'task_queue');
echo " [x] Sent ", $data, "\n";
$channel->close();
$connection->close();
After starting both swoole server and worker I'm triggering new_task.php from command line:
php new_task.php
In command line prompt where a RabbitMQ Worker is running (worker.php) I can see that a message is delivered to the worker ("[x] Received Hello World!" message is appearing).
However in command line prompt where Swoole server is running happens nothing.
So the questions are:
1) Is the idea of this approach right?
2) What am I doing wrong?
In the callback(in worker.php) that fires when a message is received you're using swoole_http_client which is async only. This seems to results in the code never being fully executed as the callback function returns before the async code is triggered.
A synchronous method of doing the same thing will solve the problem. Here is a simple example:
$client = new WebSocketClient('0.0.0.0', 2345);
$client->connect();
$client->send('This is the message to send to Swoole server');
$recv = $client->recv();
print_r($recv);
$client->close();
Check out the WebSocketClient class and example usage at github.
You can also wrap it in a coroutine, like this:
go(function () {
$client = new WebSocketClient('0.0.0.0', 2345);
$client->connect();
$client->send('This is the message to send to Swoole server');
$recv = $client->recv();
print_r($recv);
$client->close();
});
How can I return message back to the queue if processing result did not suit me. Found only information about message acknowledgments but I think that it does not suit me. I need that if as a result of processing I get the parameter RETRY message is added back to the queue. And then this worker or another one picks it up again and tries to process it.
For example:
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($msg) {
$condition = json_decode($msg->body);
if (!$condition) {
# return to the queue
}
};
$channel->basic_consume('test', '', false, true, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
?>
set auto no_ack flag to false
queue: Queue from where to get the messages
consumer_tag: Consumer identifier
no_local: Don't receive messages published by this consumer.
no_ack: Tells the server if the consumer will acknowledge the messages.
exclusive: Request exclusive consumer access, meaning only this consumer can access the queue
nowait:
callback: A PHP Callback
$channel->basic_consume('test', '', false, false, false, false, $callback);
you must use acknowledgments , if your proccess not work you can ignore ack
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($message) {
$condition = json_decode($message->body);
if (!$condition) {
// return to the queue
$message->delivery_info['channel']->basic_nack($message->delivery_info['delivery_tag']);
}else{
// send ack , remove from queue
$message->delivery_info['channel']->basic_ack($message->delivery_info['delivery_tag']);
}
};
$channel->basic_consume('test', '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
Of course with this approach you will face with the message always in the head of the queue, there is also another possibility,
if you really want to have a track of retry you can follow the below approach
defining a queue for retry, preferably your queue-name -retry and define a dead-letter queue preferably: -dlq
Then you can do something like below:
How to set up -retry queue:
this is the most important part of it. you need to declare queue with the following features:
x-dead-letter-exchange: should be same as your main queue routing key
x-dead-letter-routing-key: should be same as your main queue routing key
x-message-ttl: the delay between retries
the codes are sudo code, please do not copy-paste, this is just a hint to give you the idea about it
$maximumRetry = 5;
$callback = function($message) {
$body = json_decode($message->body);
try {
// process result is your condition
$message->delivery_info['channel']->basic_ack($message->delivery_info['delivery_tag']);
} catch(Exception $e) {
// return to the queue
$body['try_attempt'] = !empty($body['try_attempt'])? int($body['try_attempt']) + 1: 1
if ($body['try_attempt'] >= $maximumRetry ){
$message->delivery_info['channel']->basic_nack($message->delivery_info['delivery_tag']);
return
}
$msg = new AMQPMessage(json_encode($message));
$channel->basic_publish($msg, '', 'test-retry');
}
};
We gonna need 3 queues for retying.
queue.example
bindings:
exchange: queue.exchange
routing: queue.example
features:
x-dead-letter-exchange: queue.exchange
x-dead-letter-routing-key: queue.example-dlq
queue.example-dlq
bindings:
exchange: queue.exchange
routing: queue.example-dlq
queue.example-retry
bindings:
exchange: queue.exchange
routing: queue.example-retry
features:
x-dead-letter-exchange: queue.exchange
x-dead-letter-routing-key: queue.example-added
x-message-ttl: 10000
------------- Update -------------
Quorum queues are providing ability out of the box so in the consumer, you can understand how many times each message was retried and you can also define a dead-letter queue for it easily, for more information you can read more about quorom queues and poison message handling
The solution turned out to be easier than I thought, it turns out the task was not specifically about RabbitMQ, but about the scope of variables. If anyone is interested in a solution, here:
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($msg) {
global $channel;
$condition = json_decode($msg->body);
if (!$condition) {
$msg = new AMQPMessage(json_encode(array(
'condition' => false
)));
$channel->basic_publish($msg, '', 'test');
}
};
$channel->basic_consume('test', '', false, true, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
?>
This may be a bit late, but this is how you should do it with this version of php-amqplib "php-amqplib/php-amqplib": "^3.1" You need to set the no_ack parameter of basic consume method to false(which is the default) then explicitly specify it in the callback using the method nack on the AMQPMessage object passed to the callback
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($msg) {
$condition = json_decode($msg->body);
if (!$condition) {
// message will be added back to the queue
$msg->nack(true);
}
};
$channel->basic_consume('test', '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
?>
I create RabbitMQ listener:
$connection = new AMQPConnection(
$AMQP_config['server'],
$AMQP_config['port'],
$AMQP_config['user'],
$AMQP_config['password'],
$AMQP_config['virtual_host']
);
$channel = $connection->channel();
$channel->basic_qos(
null,
1,
null
);
$channel->basic_consume(
$AMQP_config['queue'],
'',
false,
false,
false,
false,
array($this, 'CallbackResponse')
);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
public function CallbackResponse(AMQPMessage $msg)
{
$response = json_decode($msg->body)->acopMessage;
if ($response->reqMRef == $this->_request_reference) {
$msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
}
}
I have 5 messages on my RabbitMQ server. But I receive only one callback, only one entering into CallbackResponse().
I want to check all messages from the queue, find the one I've sent, read it, so there will be 4 messages left.
What I doing wrong, why I receive only first message?
Using this: https://github.com/videlalvaro/php-amqplib
Your QoS is set to 1. So RabbitMQ will only send one message at a time.
As you only ack the message you are expecting for, the first message you received which does not match you condition remains unack. Thus, RabbitMQ will not send new message.
So messages that do not match must be unacked and requeued
I am attempting to write a background function that communicates with an external API to retrieve & update information on about 500,000 records on our server. This takes time to run. The problem I'm running into is this PHP fatal error from a function called inside a foreach loop, which shuts down the function entirely:
SoapClient::__doRequest(): connect() failed: Connection timed out in /path/to/function on line ABC
Here is a mock version of the function, with an indication of the line where the fatal error is being thrown, shutting down the function:
private function APIUpdateFunction($table,$record,$fields,$data) {
$soap = new SoapClient("https://www.endpoint.com/API/stuff.whatever");
$result = $soap->UpdateRecord(array(
"credentials" => API_CREDS,
"table" => $table,
"record" => $record,
"fields" => $fields,
"data" => $data
)); // THIS IS WHERE THE FATAL ERROR IS BEING THROWN
}
What I would like to do is create a workaround where IF the function shuts down as a result of this connection timeout, I would like to either:
A) Perform a new set of actions, such as restarting the function itself
or
B) Bypass the error entirely, and check in the next step for a response value. For example, if $result = NULL, the remaining items in this function would be bypassed and return a FALSE value.
What is the solution?
You can control the time your script takes to connect to the remote host/service and the time for how long the socket connection should wait for a response from the server.
The first option can be configured by specifying the timeout when you create the Soap Client:
$client = new SoapClient($wsdl, array("connection_timeout"=>15));
The second option can be configured using:
ini_set('default_socket_timeout', 180);
Now if you want to catch a timeout error, either during the host connection or the socket connection, you should catch a SoapFault exception, for example:
private function APIUpdateFunction($table,$record,$fields,$data)
{
try {
$soap = new SoapClient("https://www.endpoint.com/API/stuff.whatever");
$result = $soap->UpdateRecord(array(
"credentials" => API_CREDS,
"table" => $table,
"record" => $record,
"fields" => $fields,
"data" => $data
)); // THIS IS WHERE THE FATAL ERROR IS BEING THROWN
} catch (SoapFault $e) {
// There was an error with the Soap Client
// Do something here
}
}