I create RabbitMQ listener:
$connection = new AMQPConnection(
$AMQP_config['server'],
$AMQP_config['port'],
$AMQP_config['user'],
$AMQP_config['password'],
$AMQP_config['virtual_host']
);
$channel = $connection->channel();
$channel->basic_qos(
null,
1,
null
);
$channel->basic_consume(
$AMQP_config['queue'],
'',
false,
false,
false,
false,
array($this, 'CallbackResponse')
);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
public function CallbackResponse(AMQPMessage $msg)
{
$response = json_decode($msg->body)->acopMessage;
if ($response->reqMRef == $this->_request_reference) {
$msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
}
}
I have 5 messages on my RabbitMQ server. But I receive only one callback, only one entering into CallbackResponse().
I want to check all messages from the queue, find the one I've sent, read it, so there will be 4 messages left.
What I doing wrong, why I receive only first message?
Using this: https://github.com/videlalvaro/php-amqplib
Your QoS is set to 1. So RabbitMQ will only send one message at a time.
As you only ack the message you are expecting for, the first message you received which does not match you condition remains unack. Thus, RabbitMQ will not send new message.
So messages that do not match must be unacked and requeued
Related
Is there a way to make this work..
I'm trying to publish a 'PING' request to a device, and then subscribe to another channel and wait for the 'PONG' response. Here's the code:
public function ping(Request $request)
{
$device = Device::find($request->id);
if ($device) {
//Listen on laravel_device_DEVICEID
$listen_topic = 'laravel_device_' . $device->device_id;
$result = [];
$mqtt = MQTT::connection();
$mqtt->subscribe($listen_topic, function (string $topic, string $message) use ($mqtt, &$result) {
$result['topic'] = $topic;
$result['message'] = $message;
$mqtt->interrupt();
}, 0);
$mqtt->loop(true, true);
//Submit PING message
$topic = $device->device_id;
$message = json_encode([
'type' => 'ping',
'device_id' => $device->device_id
]);
$mqtt = MQTT::connection();
$mqtt->publish($topic, $message);
$device->last_ping_response = 2;
$device->save();
}
}
I'm so free to copy my response from the original question in the php-mqtt/laravel-client repository.
Yes, it is possible and a fairly common thing to do as well. The pattern is called RPC (Remote Procedure Call).
Your code already looks quite good, the only issue is the order of subscribe(), publish() and loop(). Subscribing first is cruicial, since you do not want to miss a response before your subscription goes through. This part is correct already. But you may start the loop only after publishing the request, otherwise the publish() is never reached (because the loop does not exit by itself, only when it receives a message).
Here is the updated piece of code of yours, with some best practices added in. I assumed the response message is what you want to save as $device->last_ping_response:
use PhpMqtt\Client\MqttClient;
use PhpMqtt\Client\Exceptions\MqttClientException;
public function ping(Request $request)
{
$device = Device::find($request->id);
// Exit early if we have no device to ping. Doing it this way round,
// we save one level of identation for the rest of the code.
if ($device === null) {
return;
}
// Prepare all data before connecting to the broker. This way, we do not waste
// time preparing data while already connected.
$subscribeTopic = "laravel_device_{$device->device_id}";
$publishTopic = $device->device_id;
$request = json_encode([
'type' => 'ping',
'device_id' => $device->device_id,
]);
try {
$mqtt = MQTT::connection();
// Ensure we are ready to receive a response to the request
// we are going to send.
$mqtt->subscribe(
$subscribeTopic,
function (string $topic, string $message) use ($mqtt, $device) {
// Update the device based on the response.
$device->last_ping_response = $message;
$device->save();
$mqtt->interrupt();
},
MqttClient::QOS_AT_LEAST_ONCE
);
// Here we register a timeout using a loop event handler.
// The event handler is passed the elapsed time
// the loop runs already (in seconds).
// We do this because in case the receiver of our request is offline,
// we would be stuck in a loop forever.
$mqtt->registerLoopEventHandler(
function function (MqttClient $client, float $elapsedTime) {
// After 10 seconds, we quit the loop.
if ($elapsedTime > 10) {
$client->interrupt();
}
}
);
// Send the request. We use QoS 1 to have guaranteed delivery.
$mqtt->publish($publishTopic, $request, MqttClient::QOS_AT_LEAST_ONCE);
// Wait for a response. This will either return when we receive
// a response on the subscribed topic, or after 10 seconds
// due to our registered loop event handler.
$mqtt->loop();
// Always disconnect gracefully.
$mqtt->disconnect();
} catch (MqttClientException $e) {
\Log::error("Pinging device failed.", [
'device' => $device->id,
'exception' => $e->getMessage(),
]);
}
}
I'm trying to send some data from php application to the user's browser using websockets. Therefore I've decided to use Swoole in combination with RabbitMQ.
It's the first time I'm working with websockets and after reading some posts about Socket.IO, Ratchet, etc. I've decided to halt on Swoole because it's written in C and handy to use with php.
This is how I understood the idea of enabling data transfer using websockets:
1) Start RabbitMQ worker and Swoole server in CLI
2) php application sends data to RabbitMQ
3) RabbitMQ sends message with data to worker
4) Worker receives message with data + establishes socket connection with Swoole socket server.
5) Swoole server broadcasts data to all connections
The question is how to bind Swoole socket server with RabbitMQ? Or how to make RabbitMQ to establish connection with Swoole and send data to it?
Here is the code:
Swoole server (swoole_sever.php)
$server = new \swoole_websocket_server("0.0.0.0", 2345, SWOOLE_BASE);
$server->on('open', function(\Swoole\Websocket\Server $server, $req)
{
echo "connection open: {$req->fd}\n";
});
$server->on('message', function($server, \Swoole\Websocket\Frame $frame)
{
echo "received message: {$frame->data}\n";
$server->push($frame->fd, json_encode(["hello", "world"]));
});
$server->on('close', function($server, $fd)
{
echo "connection close: {$fd}\n";
});
$server->start();
Worker which receives message from RabbitMQ, then makes connection to Swoole and broadcasts the message via socket connection (worker.php)
$connection = new AMQPStreamConnection('0.0.0.0', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('task_queue', false, true, false, false);
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$callback = function($msg){
echo " [x] Received ", $msg->body, "\n";
sleep(substr_count($msg->body, '.'));
echo " [x] Done", "\n";
$msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
// Here I'm trying to make connection to Swoole server and sernd data
$cli = new \swoole_http_client('0.0.0.0', 2345);
$cli->on('message', function ($_cli, $frame) {
var_dump($frame);
});
$cli->upgrade('/', function($cli)
{
$cli->push('This is the message to send to Swoole server');
$cli->close();
});
};
$channel->basic_qos(null, 1, null);
$channel->basic_consume('task_queue', '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
New task where the message will be send to RabbitMQ (new_task.php):
$connection = new AMQPStreamConnection('0.0.0.0', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('task_queue', false, true, false, false);
$data = implode(' ', array_slice($argv, 1));
if(empty($data)) $data = "Hello World!";
$msg = new AMQPMessage($data,
array('delivery_mode' => AMQPMessage::DELIVERY_MODE_PERSISTENT)
);
$channel->basic_publish($msg, '', 'task_queue');
echo " [x] Sent ", $data, "\n";
$channel->close();
$connection->close();
After starting both swoole server and worker I'm triggering new_task.php from command line:
php new_task.php
In command line prompt where a RabbitMQ Worker is running (worker.php) I can see that a message is delivered to the worker ("[x] Received Hello World!" message is appearing).
However in command line prompt where Swoole server is running happens nothing.
So the questions are:
1) Is the idea of this approach right?
2) What am I doing wrong?
In the callback(in worker.php) that fires when a message is received you're using swoole_http_client which is async only. This seems to results in the code never being fully executed as the callback function returns before the async code is triggered.
A synchronous method of doing the same thing will solve the problem. Here is a simple example:
$client = new WebSocketClient('0.0.0.0', 2345);
$client->connect();
$client->send('This is the message to send to Swoole server');
$recv = $client->recv();
print_r($recv);
$client->close();
Check out the WebSocketClient class and example usage at github.
You can also wrap it in a coroutine, like this:
go(function () {
$client = new WebSocketClient('0.0.0.0', 2345);
$client->connect();
$client->send('This is the message to send to Swoole server');
$recv = $client->recv();
print_r($recv);
$client->close();
});
How can I return message back to the queue if processing result did not suit me. Found only information about message acknowledgments but I think that it does not suit me. I need that if as a result of processing I get the parameter RETRY message is added back to the queue. And then this worker or another one picks it up again and tries to process it.
For example:
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($msg) {
$condition = json_decode($msg->body);
if (!$condition) {
# return to the queue
}
};
$channel->basic_consume('test', '', false, true, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
?>
set auto no_ack flag to false
queue: Queue from where to get the messages
consumer_tag: Consumer identifier
no_local: Don't receive messages published by this consumer.
no_ack: Tells the server if the consumer will acknowledge the messages.
exclusive: Request exclusive consumer access, meaning only this consumer can access the queue
nowait:
callback: A PHP Callback
$channel->basic_consume('test', '', false, false, false, false, $callback);
you must use acknowledgments , if your proccess not work you can ignore ack
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($message) {
$condition = json_decode($message->body);
if (!$condition) {
// return to the queue
$message->delivery_info['channel']->basic_nack($message->delivery_info['delivery_tag']);
}else{
// send ack , remove from queue
$message->delivery_info['channel']->basic_ack($message->delivery_info['delivery_tag']);
}
};
$channel->basic_consume('test', '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
Of course with this approach you will face with the message always in the head of the queue, there is also another possibility,
if you really want to have a track of retry you can follow the below approach
defining a queue for retry, preferably your queue-name -retry and define a dead-letter queue preferably: -dlq
Then you can do something like below:
How to set up -retry queue:
this is the most important part of it. you need to declare queue with the following features:
x-dead-letter-exchange: should be same as your main queue routing key
x-dead-letter-routing-key: should be same as your main queue routing key
x-message-ttl: the delay between retries
the codes are sudo code, please do not copy-paste, this is just a hint to give you the idea about it
$maximumRetry = 5;
$callback = function($message) {
$body = json_decode($message->body);
try {
// process result is your condition
$message->delivery_info['channel']->basic_ack($message->delivery_info['delivery_tag']);
} catch(Exception $e) {
// return to the queue
$body['try_attempt'] = !empty($body['try_attempt'])? int($body['try_attempt']) + 1: 1
if ($body['try_attempt'] >= $maximumRetry ){
$message->delivery_info['channel']->basic_nack($message->delivery_info['delivery_tag']);
return
}
$msg = new AMQPMessage(json_encode($message));
$channel->basic_publish($msg, '', 'test-retry');
}
};
We gonna need 3 queues for retying.
queue.example
bindings:
exchange: queue.exchange
routing: queue.example
features:
x-dead-letter-exchange: queue.exchange
x-dead-letter-routing-key: queue.example-dlq
queue.example-dlq
bindings:
exchange: queue.exchange
routing: queue.example-dlq
queue.example-retry
bindings:
exchange: queue.exchange
routing: queue.example-retry
features:
x-dead-letter-exchange: queue.exchange
x-dead-letter-routing-key: queue.example-added
x-message-ttl: 10000
------------- Update -------------
Quorum queues are providing ability out of the box so in the consumer, you can understand how many times each message was retried and you can also define a dead-letter queue for it easily, for more information you can read more about quorom queues and poison message handling
The solution turned out to be easier than I thought, it turns out the task was not specifically about RabbitMQ, but about the scope of variables. If anyone is interested in a solution, here:
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($msg) {
global $channel;
$condition = json_decode($msg->body);
if (!$condition) {
$msg = new AMQPMessage(json_encode(array(
'condition' => false
)));
$channel->basic_publish($msg, '', 'test');
}
};
$channel->basic_consume('test', '', false, true, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
?>
This may be a bit late, but this is how you should do it with this version of php-amqplib "php-amqplib/php-amqplib": "^3.1" You need to set the no_ack parameter of basic consume method to false(which is the default) then explicitly specify it in the callback using the method nack on the AMQPMessage object passed to the callback
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$connection = new AMQPStreamConnection($AMQP);
$channel = $connection->channel();
$channel->queue_declare('test', false, false, false, false);
$callback = function($msg) {
$condition = json_decode($msg->body);
if (!$condition) {
// message will be added back to the queue
$msg->nack(true);
}
};
$channel->basic_consume('test', '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
?>
Currently, I'm using https://github.com/php-amqplib/php-amqplib and I've read a lot of examples in this repository but I still don't understand how to get all the messages from the queue?
I just need to receive some messages, group them by value and perform an action.
Is it possible to do with RabbitMQ at all?
How can I implement this in php?
<?php
use PhpAmqpLib\Connection\AMQPStreamConnection;
$connection = new AMQPStreamConnection('rabbitmq', 5672, 'guest', 'guest');
$channel = $connection->channel();
$queueName = 'task_queue';
$channel->queue_declare($queueName, false, true, false, false);
$result = $channel->basic_get($queueName);
var_dump($result);
$channel->close();
$connection->close();
You cannot get all messages currently available by calling single method.
Closest possible solution is by using basic_consume method.
For example:
function process_message($message)
{
echo "Received message '" . $message->body . "'\n";
/** Do your grouping here **/
}
$channel->basic_consume($queue, '', false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while (count($channel->callbacks)) {
$channel->wait();
}
You can check official RabbitMQ PHP tutorial or demo from php-amqplib.
I have a consumer worker in PHP script.
But sometimes the RabbitMQ server stops from running,
I get this error:
PHP Fatal error: Uncaught exception 'ErrorException' with message 'fwrite(): send of 19 bytes failed with errno=32 Broken pipe' in /home/user/pusher/rabbitmq-worker/vendor/php-amqplib/php-amqplib/PhpAmqpLib/Wire/IO/StreamIO.php:281
I would like to be able to handle this error with a try/catch block to throw a nice error to my console or try to reconnect after a while.
This is what I have code so far:
while (true) {
try {
$connection = new AMQPStreamConnection(RABBITMQ_HOST, RABBITMQ_PORT, RABBITMQ_USER, RABBITMQ_PASS);
$channel = $connection->channel();
$channel->queue_declare(RABBITMQ_DT_QUEUE, false, true, false, false);
$channel->basic_qos(null, 11, null);
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$callback = function($req) {
sleep(3);//Some task that takes 3 sec
};
$channel->basic_consume(RABBITMQ_QUEUE, '', false, false, false, false, $callback);
while (count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
} catch (Exception $e) {
sleep(1);
}
What's missing?
Define next parameter on connection
keepalive: true
heartbeat: 15 // An example
It is two latest arguments of AMQPStreamConnection::__constructor()
NOTE:
if you connect to some distant server, I recommend set high up timeouts
connection_timeout: 10
read_write_timeout: 30
UPD:
You can catch ErrorException and try connect to server again