I want to log working status from workers' callbacks and include a number of messages in the queue left.
The only solution I found so far is getting the second member of queue_declare result array, but this should be called once per worker launch, and I need info to be updated every new message.
UPD:
Solution based on IMSoP's answer:
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare('test1');
echo "[*] Waiting for messages. To exit press CTRL+C\n";
$callback = function ($msg) use ($channel) {
list (, $cn) = $channel->queue_declare('test1', true);
echo ' [x] Received ', $msg->body, " $cn left";
for ($i = 0; $i < $msg->body; ++$i) {
sleep(1);
echo '.';
}
echo "\n";
};
$channel->basic_qos(null, 1, null);
$channel->basic_consume('test1', '', false, true, false, false, $callback);
while (count($channel->callbacks)) {
$channel->wait();
}
For some reason always gives 0 as message count.
The queue_declare method has a parameter called "passive" which can be used for this purpose: it checks if the queue exists, by name only, and ignores any other parameters.
According to the AMQP documentation:
If set, the server will reply with Declare-Ok if the queue already exists with the same name, and raise an error if not. The client can use this to check whether a queue exists without modifying the server state. When set, all other method fields except name and no-wait are ignored. A declare with both passive and no-wait has no effect. Arguments are compared for semantic equivalence.
Note that Declare-Ok is not just a status, but the name of the full response structure, with fields queue, message-count, and consumer-count.
In PHP-AMQPLib, you can use this to log the status of a set of queues something like this:
foreach ( $this->registeredQueues as $queueName ) {
// The second parameter to queue_declare is $passive
// When set to true, everything else is ignored, so need not be passed
list($queueName, $messageCount, $consumerCount)
= $this->rabbitChannel->queue_declare($queueName, true);
$this->logger->info(
"Queue $queueName has $messageCount messages and $consumerCount active consumers."
);
}
Related
I need to notify the client(s) whether some of the clients made changes on the database. I consider it to be a single-directional connection (it's only the server who sends events), so I do not need WebSockets.
The logic is as follows:
Client fetches the common API (let's say /api.php?action=add&payload=...), and with this fetch the change on DB is being made.
After mysqli_result returned true, the server-sent event should be triggered to notify other clients (or their service-workers) about DB changed.
The caveats are:
My SSE logic is placed on a separate file (/sse.php) to not make a mess in API logic and to listen to separate endpoint clientside. So I need to trigger SSE from api.php and push the result from there to all clients.
In sse.php I defined a function for pushing which takes the message from api.php as a param. Whatever I've tried, this function was never called.
I strongly want to avoid making daemons and any kind of infinite loops. But without them, the connection for the clientside listener (eventSource) closes and reopens every 3 seconds.
The current state of this all:
api.php
<?php
header("Access-Control-Allow-Origin: *");
include 'connection.php'; // mysqli credentials and connection object
include_once 'functions.php'; // logics
$action = isset($_GET['a']) ? $_GET['a'] : 'fail';
switch ($action) {
case "add":
$result = api__handle_add(); // adds data to DB and returns stringified success/failure result
require "./sse.php"; // file with SSE logic with ev__ namespace
ev__send_data($result); // this defined function should send message to clients
break;
// ...
case "testquery":
require "./sse.php";
ev__send_data("Test event!!! Ololo");
break;
case "fail": default:
header('HTTP/1.1 403 Forbidden', true, 403);
}
sse.php
<?php
header("Access-Control-Allow-Origin: *");
header("Cache-Control: no-cache");
header("Content-Type: text/event-stream");
header("Connection: keep-alive");
$_fired = false; // flag checking for is anything happening right now
while (1) { // the infinite loop I strongly want to get rid of
if (!$_fired) {
usleep(5000); // nothing to send if nothing is fired, but I should keep the connection alive somehow
} else {
// here I should call 'ev__send_data($message_from_api_php)'
// thus I require this file to `api.php` and try to call 'ev__send_data' from there.
}
ob_flush();
flush();
}
function ev__send_data(string $msg) {
global $_fired;
$_fired = true;
echo "data: $msg". PHP_EOL;
echo PHP_EOL;
ob_flush();
flush();
usleep(1000);
$_fired = false;
}
The clientside implementation is typical, with onopen, onerror and onmessage handlers of eventSource. I think I don't need to take it here as it has no differences from thousands of examples I've seen here on SO and at other sources.
To conclude: I want to trigger SSE's from outside SSE-file and pass pieces of data from outside, and I want to get rid of while (1) {} and emit events only when something really happens. I've come across thousands of tutorials and SO topics, and found nothing but typical "tutorial cases" about how to send server time to the client every n seconds and stuff. And I think I don't need WebSockets (considering that my local dev environment is Win10 + XAMPP, my prod environment will be something Linux-like and I don't think I can implement my own WebSocket without third-party dependencies which I don't want to install).
Do I have any chance?
EDIT
I found a little different approach. Now my server event generation depends on boolean flags that were settled in MySQL database in separate flags table. When I need an event to be emitted, API changes definite flag (for the very this example, when one client had submitted some changes in DB, API also makes the query to flags table and sets flag DB_CHANGED to 1). And sse.php makes query to flags table each iteration of the infinite loop (every 5 seconds) and emits events depending on the flags' state, and after emitting, sets the corresponding flag to 0 via one more query. This kinda works.
Example of current sse.php:
<?php
header("Cache-Control: no-cache");
header("Content-Type: text/event-stream");
require_once "connection.php"; // mysqli connection object
set_time_limit(600);
stream();
function stream() {
global $connection;
while (1) { // oh that infinite loop
$flags = []; // there will be fetched flags from DB
$flags_sqli = $connection->query("SELECT * FROM `flags`");
while ($flag = $flags_sqli->fetch_assoc()) {
$flags[$flag['key_']] = ev__h__to_bool_or_string($flag['value_']);
}
if ($flags['ON_TEST']) {
ev__send_data("Test event!!! Ololo");
ev__update_flag("ON_TEST", "0");
} elseif ($flags['ON_DB_ADD']) {
ev__handle_add();
} else {
echo PHP_EOL; // keep connection alive - push single "end of line"
}
ob_flush();
flush();
sleep(5); // interval per iteration - 5 secs
}
}
function ev__send_data(string $msg) : void {
echo "data: ".date("d-m-Y")." ".date("H:i:s").PHP_EOL;
echo "data: $msg".PHP_EOL;
echo PHP_EOL;
}
function ev__handle_add() : void {
global $connection;
$table = $connection->query('SELECT `value_` FROM `flags` WHERE `key_` = "LAST_ADDED_TABLE"')->fetch_row()[0];
$query = "SELECT * FROM `banners_".$table."` WHERE `id` = (SELECT MAX(`id`) FROM `banners_".$table."`)";
$row = $connection->query($query)->fetch_assoc();
echo "event: db_add_row".PHP_EOL;
echo 'data: { "manager": '.$row['manager_id'].', "banner_name": "'.$row['name'].'", "time": "'.date("d-m-Y").' '.date("H:i:s").'" }'.PHP_EOL;
echo PHP_EOL;
ev__update_flag("ON_DB_ADD", "0");
}
function ev__update_flag(string $key, string $value) {
global $connection;
$query = "UPDATE `flags` SET `value_` = '$value' WHERE `flags`.`key_` = '$key';";
$connection->query($query);
}
function ev__h__to_bool_or_string($value) {
return $value === "0" || $value === "1"
? boolval(intval($value))
: $value;
}
Therefore, I didn't get rid of the infinite loop. Considering that there will be a very few concurrent connections (max 3-5 sessions per one time) (when in prod), this very case should not make performance troubles. But what if it would be some kind of hard-loaded service? As an answer, I want to see optimization tips for the case of Server-Sent Events.
Is there a way of overwriting retries for an individual call in AWS SDK for PHP?
The following code explains the question:
// Create client with a default of 2 retries
$sqsClient = new sqsClient('2012-11-05', ['retries' => 2]);
// This will retry twice to get the queue attributes (perfect)
try {
$sqsClient->getQueueAttributes();
} catch(Exception $e) {
}
// I want the following to NEVER retry
try {
$sqsClient->turnOffRetryLogic(???);
$sqsClient->receiveMessages(['WaitTimeSeconds' => 5]);
} catch(Exception $e) {
}
// Now set the retries back to as before.
Retries are handled by Middleware - but as the Middleware class is marked "final" I need to pass in a "decider"? This means we need to hook into one of the handlers but none appear to be connected to retries.
Edit:
I have managed to prove the concept of a new "decider" by directly editing the AWS SDK as follows:
final class Middleware
{
public static function retry(
callable $decider = null,
callable $delay = null,
$stats = false
) {
....
$decider = function() {
echo 'retries cancelled';
return false;
};
....
So the question is how to do this without editing the SDK. Have tried various middleware hooks as follows, without success.
$decider = function() {
echo 'No retries';
return false;
};
$SqsClient->getHandlerList()->appendSign(\AWS\Middleware::retry($decider, null), 'retry');
$result = $SqsClient->receiveMessage($aParams);
(Code samples snipped to only show relevant parts)
Next code removes retry handler
$sqsClient->getHandlerList()->remove('retry');
Sqs client isn't going to retry after that. To restore default behavior you can attach default handler back
$decider = RetryMiddleware::createDefaultDecider(3);
$sqsClient->getHandlerList()->appendSign(
Middleware::retry($decider, null, false),
'retry'
);
Though, two separate clients with retries enabled and disabled sound more transparent for me.
I am quite new in RabbitMQ. I'm working with php-amqplib library with codeigniter, and still wondering about some knowledge which I am lacking.
Why $channel->wait() is used?
Why it is always reside inside an endless while loop?
How to/Can I bypass the Infinite while loop.
Like In a situation when one user of my project wants to broadcast new campaign to 100k leads, the second user gets effected if he has some 100 mails to be sent, The second has to wait for 100k mails to get delivered first then the last user gets his turn.
I need a solution for Concurrent Consumers, who works smoothly without affecting the other
Here is my code snippet:
public function campaign2(){
$this->load->library('mylibrary');
for( $i=1;$i<=5;$i++ ) {
$url = "http://localhost/myproject/rabbit/waiting";
$param = array('index' => $i);
$this->waiting($i);
}
}
public function waiting($i)
{
ini_set('memory_limit','400M');
ini_set('max_execution_time', 0);
ini_set('display_errors', 1);
${'conn_'.$i} = connectRabbit();
${'channel_'.$i} = ${'conn_'.$i}->channel();
${'channel_'.$i}->exchange_declare('ha-local-campaign-'.$i.'-exchange', 'fanout', false, true, false);
$q = populateQueueName('campaign-'.$i);
${'channel_'.$i}->queue_declare($q, false, true, false, false);
${'channel_'.$i}->queue_bind($q, 'ha-local-campaign-'.$i.'-exchange', 'priority.'.$i);
$consumer_tag = 'campaign_consumer' ;
function process_message($msg) {
echo 'Mail Sent';
$msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
}
function shutdown($channel, $conn){
echo '['.date('H:i:s').'] Campaign consumer - Shutdown!!';
}
${'channel_'.$i}->basic_consume($q, $consumer_tag, false, false, true, false,'process_message');
while(1) {
${'channel_'.$i}->wait();
}
register_shutdown_function('shutdown', ${'channel_'.$i}, ${'conn_'.$i});
}
If anyone kindly guide me through the process I will be grateful.
When you call $channel->wait() you are:
Inspecting the channel's queues to see if there are pending messages.
For each message you are going to call the registered callback for the corresponding channel's callback.
From the "hello world example", step by step::
// First, you define `$callback` as a function receiving
// one parameter (the _message_).
$callback = function($msg) {
echo " [x] Received ", $msg->body, "\n";
};
// Then, you assign `$callback` the the "hello" queue.
$channel->basic_consume('hello', '', false, true, false, false, $callback);
// Finally: While I have any callbacks defined for the channel,
while(count($channel->callbacks)) {
// inspect the queue and call the corresponding callbacks
//passing the message as a parameter
$channel->wait();
}
// This is an infinite loop: if there are any callbacks,
// it'll run forever unless you interrupt script's execution.
Have your second user send use a different queue. You can have as many queues as you want.
I need to publish messages from php script, I can publish a single message fine. But now I need to publish different messages in loop, can't find proper way how to do it, here is what I tried:
$counter = 0;
$closure = function (\Thruway\ClientSession $session) use ($connection, &$counter) {
//$counter will be always 5
$session->publish('com.example.hello', ['Hello, world from PHP!!! '.$counter], [], ["acknowledge" => true])->then(
function () use ($connection) {
$connection->close(); //You must close the connection or this will hang
echo "Publish Acknowledged!\n";
},
function ($error) {
// publish failed
echo "Publish Error {$error}\n";
}
);
};
while($counter<5){
$connection->on('open', $closure);
$counter++;
}
$connection->open();
Here I want to publish $counter value to subscribers but the value is always 5, 1.Is there a way that I open connection before loop and then in loop I publish messages
2.How to access to $session->publish() from loop ?
Thanks!
There are a couple different ways to accomplish this. Most simply:
$client = new \Thruway\Peer\Client('realm1');
$client->setAttemptRetry(false);
$client->addTransportProvider(new \Thruway\Transport\PawlTransportProvider('ws://127.0.0.1:9090'));
$client->on('open', function (\Thruway\ClientSession $clientSession) {
for ($i = 0; $i < 5; $i++) {
$clientSession->publish('com.example.hello', ['Hello #' . $i]);
}
$clientSession->close();
});
$client->start();
There is nothing wrong with making many short connections to the router. If you are running in a daemon process though, it would probably make more sense to setup something that just uses the same client connection and then use the react loop to manage the loop instead of while(1):
$loop = \React\EventLoop\Factory::create();
$client = new \Thruway\Peer\Client('realm1', $loop);
$client->addTransportProvider(new \Thruway\Transport\PawlTransportProvider('ws://127.0.0.1:9090'));
$loop->addPeriodicTimer(0.5, function () use ($client) {
// The other stuff you want to do every half second goes here
$session = $client->getSession();
if ($session && ($session->getState() == \Thruway\ClientSession::STATE_UP)) {
$session->publish('com.example.hello', ['Hello again']);
}
});
$client->start();
Notice that the $loop is now being passed into the client constructor and also that I got rid of the line disabling automatic reconnect (so if there are network issues, your script will reconnect).
When I run my program through command line, then my PHP script works fine and gives output properly.
But when I run it through my browser, then it processes fine but the output I printed - is not showing.
( Note : My script is running in a while loop and it will run forever )
Code - I am writing some part of my class file
public function call($n) {
global $argv;
$this->response = null;
$this->corr_id = uniqid();
$msgBody = 'test msg';
//Create queue
$this->createQueue($queueName);
$msg = new AMQPMessage(
(string) $msg_body,
array('correlation_id' => $this->corr_id,
'reply_to' => $this->callback_queue,
'priority' => 2)
);
$this->channel->basic_publish($msg, '', $queueName);
while(!$this->response) {
$this->channel->wait();
}
return $this->response;
}
$response = call();
createQueue() function is in another class
function createQueue($queueName='')
{
$exchange = 'router';
$queue = 'msgs';
$consumer_tag = 'consumer';
$connection = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $connection->channel();
$channel->queue_declare($queueName, false, false, false, false);
echo " [x] Awaiting for message\n";
$callback = function($req) {
$n = $req->body;
echo "$n\n";
$msg = new AMQPMessage(
'msg',
array('correlation_id' => $req->get('correlation_id'))
);
$req->delivery_info['channel']->basic_publish(
$msg, '', $req->get('reply_to'));
$req->delivery_info['channel']->basic_ack(
$req->delivery_info['delivery_tag']);
};
$channel->basic_qos(null, 1, null);
$channel->basic_consume($queueName, '', false, false, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
}
Yes we can :)
Have a look at http://www.php.net/manual/en/function.flush.php - this is the way to go for permanently running scripts when you need to output before the response is complete.
Also have a look at the first user comment there - it is about sending in 4KB chunks with str_pad() as browsers tend to not display anything at all until a specific amount of data is reached.
That is bound to happen when you have a script running in a while loop and run forever.
Running the script in browser will timeout and will not output stuff since the script is not completed.
while you run a script on Command Line, the script will be executed line by line and output will be shown
Why does the loop need to run forever and ever? If it never finishes, it will never actually return a response to the client.
As the php runs, it builds an HTML document in a buffer as you add to it, and then all at once it sends the page.
If the loop generates content for the page on every iteration, you may want to stop it after a certain number of cycles through the loop, so that you are not sending the user an infinite stream of data.
If you are doing something such as a calculation, you should look into using JavaScript, and making AJAX calls to the php scripts.