Amphp : Run many async loops with same connection (eventstore client) - php

I'm using eventstore client which uses amphp. I need inside my application to reuse the connection in many parts.
So I created a connection provider:
public function getConnection(): EventStoreConnection
{
if ($this->connection) {
return $this->connection;
}
$this->connection = $this->createConnection();
wait($this->connection->connectAsync());
return $this->connection;
}
And then I use this connection at many places:
\Amp\Loop::run(function () use ($eventStoreEvents, $streamName) {
$connection = $this->connectionProvider->getConnection();
// Creation of an event stream
yield $connection->appendToStreamAsync($streamName, ExpectedVersion::ANY, $eventStoreEvents);
// sleep(10); // This sleep does not work, code continue like nothing happend
});
\Amp\Loop::run(function () use ($streamName, $aggregateFqcn, &$aggregateRoot) {
$start = 0;
$count = \Prooph\EventStore\Internal\Consts::MAX_READ_SIZE;
$connection = $this->connectionProvider->getConnection();
do {
$events = [];
/** #var StreamEventsSlice $streamEventsSlice */
$streamEventsSlice = yield $connection
->readStreamEventsForwardAsync(
$streamName,
$start,
$count,
true
);
if (!$streamEventsSlice->status()->equals(SliceReadStatus::success())) {
dump($streamEventsSlice); // Event stream does not exist
// Error here: the event stream doesn't exist at this point.
throw new RuntimeGangxception('Impossible to generate the aggregate');
}
} while (! $streamEventsSlice->isEndOfStream());
});
The problem: it seems that the first request is not over but the second loop starts already. The sleep uncommented doesn't have any effect!
But the event stream is finally created with the related events inside, so the first request worked.
If I start a connection then close then start a new one, it works. But it's slow, due to handshake overhead on each new connection.
I tried a similar example with the WebSocket library of Amphp and it worked. Do you see anything wrong?
Here is my test with websocket that worked:
$connection = \Amp\Promise\wait(connect('ws://localhost:8080'));
Amp\Loop::run(function () use ($connection) {
/** #var Connection $connection */
yield $connection->send("Hello...");
sleep(10); // This sleep works!
});
Amp\Loop::run(function () use ($connection) {
/** #var Connection $connection */
yield $connection->send("... World !");
});
$connection->close();

What you are trying to do makes no sense. You should read amphp's documenation.
Amp uses a global accessor for the event loop as there’s only one event loop for each application. It doesn’t make sense to have two loops running at the same time, as they would just have to schedule each other in a busy waiting manner to operate correctly.
That said, there is literally NO SECOND LOOP.

Prooph eventstore library is based on amphp but doesn't follow all principles: you can't wait for the connection to be ready. It will be even worse if you try to use it at scale, so don't try to wait for the promise is complete.
As an alternative, you can set a promise for later and check if the connection is null. That's what actually does the library internally to process further steps.
On my side, I decided to stop using this library. But as an alternative you can use the library that uses the HTTP client, it's also from the prooph team.

Related

Unable to do non blocking I/O in React PHP

I am trying to insert a record inside database coming to a react socket server. I am lost on how to do my operation in a non blocking way
$loop = Factory::create();
$server = new Server('127.0.0.1:4040', $loop);
$database = new Database();
$server->on('connection', function(ConnectionInterface $conn) use ($database) {
$conn->write('Welcome, you can start writing your notes now...');
$conn->on('data', function($data) use ($conn, $database) {
$database->write($data);
$conn->write('I am supposed to execute before database write');
});
});
$loop->run();
The write method in database has a sleep(10) seconds before executing the sql statement. So I am expecting the next message I am supposed to.. should be printed immediately.
My expectation was that when ever there is a I/O operation, the operation will be moved to Event Table and don't block the call stack. As per the definition of event loop and non blocking.
How can I perform the same operation in non blocking way.
Thanks
Hey ReactPHP core team member here. The loop expects everything to be asynchronous so putting a sleep in your $database->write($data); will block the loop. Your database connection has to utilise the event loop for it to be non-blocking. My suggestion would be to look at https://github.com/friends-of-reactphp/mysql or https://github.com/voryx/PgAsync or check the list here https://github.com/reactphp/react/wiki/Users#databases depending on your database. ReactPHP won't magically make everything non-blocking, you have to use packages that take care of that for you.

DB Connection in PHP Socket Server

I'm running a Yii2 console application which starts a websocket chat service. That all is working fine and as it's supposed to, but after some time of inactivity I get SQLSTATE[HY000]: General error: 2006 MySQL server has gone away. I tried to increase timeouts, set the user abort to false and to set PDO::ATTR_PERSISTENT => true in the PDO's constructor, but this is still happening.
Is there a way to check if a database connection is still active or if not how to reconnect to the db. Either in pure php or better with the Yii2 framework.
I had a similar problem and solved it by creating my own class for DB connection, which ensures that connection is active before actual query.
class DbConnection extends \yii\db\Connection {
private $stamp;
/**
* {#inheritdoc}
*/
public function createCommand($sql = null, $params = []) {
try {
// send ping on every 10 seconds
if ($this->stamp < time()) {
$this->stamp = time() + 10;
parent::createCommand('SELECT 1')->execute();
}
} catch (\yii\db\Exception $e) {
// if ping fail, reconnect
$this->close();
$this->open();
}
return parent::createCommand($query);
}
}
Once every 10 seconds it sends "ping" query before creating a command. If ping fails (connection was interrupted), it tries to reconnect.
This will not prevent from disconnecting, but it will automatically reconnect in case if connection was interrupted. This may be tricky if you're using transactions - if connection is interrupted in the middle of transaction, transaction will be implicitly rollback by DB, and above code will implicitly reconnect, so you don't even notice that your transaction was rollback at some point.
Also I didn't test it in master-slave configuration. But it worked perfectly fine in my case (read only connection to single server), so you may use it as a base and adjust for your needs with additional checks for transactions or master/slave connections.
Not sure about where exactly you should go for this code exactly, but the concerned code to check if the connection is active following methods in yii\db\Connection class can be used
getIsActive() : Returns a value indicating whether the DB connection
is established.
and to reconnect you can use
open() : Establishes a DB connection. It does nothing if a DB
connection has already been established.
if(Yii::$app->db->isActive === FALSE){
Yii::$app->db->open();
}

Using reactive PHP in a blocking application

I'm currently working on a PHP application that will be using some websocket connections to talk to another service.
To talk to this websocket service, we are using Ratchet - which is a PHP library based on react PHP.
This piece of code needs to send and respond to a couple of requests, and after that, should return the information to the "main thread".
Example flow:
HTTP request -> controller -> Starts a service which opens a websocket client -> websocket client is talking to server -> once its done it should return the outcome to the controller code -> controller outputs to user
The issue I'm having is that I'm not familiar with Reactive PHP and am not sure how to handle this.
I've tried;
$service = new WebsocketService();
$startTimer = time();
$service->getList(44);
while($service->getResponse() == null) {
usleep(500);
if (time() > $startTimer + 10) {
continue; //Timeout on 10 seconds
}
}
var_dump($service->getResponse());
The service code would set its "response" variable to something other than null once its done. This obviously fails, because the sleep method is blocking the thread. Also without, it seems like the while loop is blocking I/O and the reactive code fails.
A solution would be to open up a new thread and run the websocket code there, but I wouldn't be happy with that.
I feel like I need to implement some sort of "watcher" around the websocket process, but I'm not sure how to do that.
Our Websocket service client code looks like this;
private $response = null;
/**
* #return null|object
*/
public function getResponse() {
return $this->response;
}
public function getList($accountId) {
$this->response = null;
\Ratchet\Client\connect('ws://192.168.56.1:8080')->then(function(\Ratchet\Client\WebSocket $conn) use ($accountId) {
$login = new \stdClass();
$login->action = 'login';
$conn->on('message', function($msg) use ($conn, $login, $accountId) {
try {
$response = json_decode($msg);
if ($response->result_id == 100) {
//Succesfully logged in to websocket server
//Do our request now.
$message = new \stdClass();
$message->target = 'test';
$conn->send(json_encode($message));
}
if (isset($response->reply) && $response->reply == 'list') {
$this->response = $response; //This is the content I need returned in the controller
$conn->close(); //Dont need it anymore
}
} catch (\Exception $e) {
echo 'response exception!';
//Do nothing for now
}
});
$conn->send(json_encode($login));
}, function ($e) {
echo "Could not connect: {$e->getMessage()}\n";
});
}
Running the code like this also does not work;
$service = new WebsocketService();
$service->getList(44);
echo 'Test';
var_dump($service->getResponse());
because the "test" echo comes before I even get a response from the websocket server.
Please, enlighten me! I'm not sure what to search for.
PHP and websockets still seem to be a bit experimental. Nevertheless I have found a great tutorial on medium.com, written by Adam Winnipass which should be really helpful for solving your problem: https://medium.com/#winni4eva/php-websockets-with-ratchet-5e76bacd7548
The only difference is that they are implementing their websocket client with JavaScript instead of PHP. But in the end there should not be much of a difference, because as soon as we have opened the Websocket connection of each end both applications have to send and also wait to receive notifications - this is how they illustrate it:
Seems like one possibility to create a successful Websocket connection is to extend the MessageComponentInterface
use Ratchet\MessageComponentInterface;
which also requires
use Ratchet\ConnectionInterface;
The message component interface defines the following methods:
onOpen
onMessage
onClose
onError
And I think this is how the Ratchet library is implementing it. This is how they are finally starting their server:
use Ratchet\Server\IoServer;
use MyApp\MyCustomMessageComponentInterface;
use Ratchet\Http\HttpServer;
use Ratchet\WebSocket\WsServer;
require dirname(__DIR__) . '/vendor/autoload.php';
$server = IoServer::factory(
new HttpServer(
new WsServer(
new MyCustomMessageComponentInterface()
)
),
8080
);
$server->run();
With this architecture you already can receive (onMessage) and sending is also possible with the send() method.
I can not solve the exact problem with your existing code. But I guess if you are using the pre-built classes and interfaces of the library as intended (and demonstrated here) you should be able to achieve what you want by adding your code to the corresponding methods.
More information and examples can be found in the docs:
http://socketo.me/docs/server
http://socketo.me/api/namespace-Ratchet.html
Are you extending class with WsServer, This might be issue, if you are getting fatal errors. I am not sure whether you are getting fatal errors or warnings. Also i notice the public function onOpen() opens a connection. Please try referring this document http://socketo.me/api/class-Ratchet.WebSocket.WsServer.html might be useful.

How to pass some data to a running thread in PHP?

So let's say I create a thread and detach it from the main process, and start it.
So, after the thread is detached, how is it possible to pass some chunks of data like strings, or ints to the already running thread?
Edit
What I am basically doing is trying to implement the WS protocol:
<?php
// Pseudo-Code
class LongRunningThread extends \Thread {
private $handshakeReq;
public function __construct(Request $handshakeRequest) {
$this->handshakeReq = $handshakeRequest;
}
public function run() {
// Do handshake
// But do not exit, because after the handshake is done the socket connection needs to be maintained.
// Probably some trigger which notifies that a new message is here and the message arrives <automagically>
if(trigger) {
$message = $message;
$this->onNewWsMessage($message);
}
}
public function onNewWsMessage(string $rawMessage) {
// Process the message...
}
}
$stream = stream_socket_server(sprintf("tcp://%s:%d",
"localhost",
1337
), $errno, $errmsg);
// Boiler plate, and connection acceptance (blah blah blah)
// $client is the the accepted connection
$message = fread($client, 4096);
// Cannot pass the $client in here because the instability of resources with threads
// as passing them here, apparently converts them to <bool> false
$longRunningThread = new \LongRunningThread($message);
$longRunningThread->start() && $longRunningThread->join();
I found various answers related to passing data to a running thread, but I couldn't find any specifically for PHP.
I am using pthreads
The actual question is quite vague.
What you want to do falls to my understanding under the IPC (interprocess communication) and can be implemented with a couple of ways (to my knowledge the most common one is :http://php.net/manual/en/function.stream-socket-pair.php).
I would suggest though that you could use some kind of queueing and polling system like rabbitmq to pass around messages.It will provide some overhead but its a well known and highly used solution

PHP parallel/asynchronous SSH connections

I'm trying to open multiple connections (various devices) to run a command and get the output.
The problem is that i have to run them "all at once"/parallel.
If i wait for one result and then to run the other one it takes way too long
and with a large number of devices that can go very bad.
I'm also using curl which I know that there is curl_multi and I was wondering if there was something similar with SSH for php.
I'm using Net_SSH2 for now.
You'll need to use two PHP libraries: https://robo.li/tasks/Remote/#ssh and https://github.com/cheprasov/php-parallel. Your class method might be something similar to the example below:
function runParallelSSH() {
$parallel = new Parallel(new ApcuStorage());
foreach ($credentials as $user => $host) {
$connection = sprintf('%s#%s', $user, $host);
$connections[] = $connection;
$Parallel->run($connection, function() {
$gitTask = $this->taskGitStack()
->checkout('master')
->pull();
});
}
$results = $parallel->wait($connections);
}
Without using thirdparties like curl_multi you have to use PHP multithreading, for this you need an extension pthreads.
Look in the docs for PHP threading
The most interesting feature is using it like this (code modified from PHP.net)
class My extends Thread {
public function run() {
//curl_exec whatever
}
}
$my = new My();
//start as many as you need
$my->start();
//wait for the threads to finnish and join one thread at a time with main-process-thread:
var_dump($my->join());:
Good Luck!

Categories