Summary
I am trying to set up heartbeats with my Consumer so it can detect a dropped connection - if the broker is rebooted for example so it may reconnect to the failover.
I tested the code with the Consumer/Dispatcher running locally and the queue running in AWS and everything worked fine. However, when moving the code to AWS the Consumer sets up heartbeats with the server/broker, but the heartbeat is either never sent by the server/broker or never received by the client/Consumer. As a result the HeartbeatException is thrown as soon as the requested server heartbeat interval has passed.
I based my code on the examples in the stomp-php-examples github
Repo: https://github.com/stomp-php/stomp-php-examples
Heartbeat example: https://github.com/stomp-php/stomp-php-examples/blob/support/version-4/src/heartbeats_server.php
My next best guess to as why this isn't working is something related to the queue config as I'm using the default config provided by AWS (I think). I have googled and searched all over about config settings for heartbeats but haven't got very far as this is a new topic. Any help would be appreciated!
Setup
Amazon MQ (ActiveMQ 5.15.14)
stomp-php 5.0 (latest version as of today)
I'm happy to provide any more details of my setup.
Code
Consumer (stripped down version)
abstract class AmqConsumerAbstract
{
/** #var StatefulStomp */
protected $consumer;
const HEARTBEAT = 5000;
public function listen(): void
{
$observer = new ServerAliveObserver();
$this->client->getConnection()->getObservers()->addObserver($observer);
$this->client->setHeartbeat(0, self::HEARTBEAT); // heartbeats setup here
// Note: the heartbeat above is longer than the read-timeout below
$this->client->getConnection()->setReadTimeout(2, 0);
$this->client->connect();
$this->consumer = new StatefulStomp($this->client);
$this->consumer->subscribe(
$this->getQueueName(),
null,
'client'
);
if (!$observer->isEnabled()) {
// we never get here so I assume everything is working OK
echo 'The Server is not supporting heartbeats.');
exit(1);
} else {
echo sprintf('The Server should send us signals every %d ms.', $observer->getInterval() * 1000);
}
try {
while (true) {
$frame = $this->consumer->read(); // I assumed this line would read the heartbeat?
// there is then some logic that deals with payload and does
// $this->consumer->begin();
// $this->consumer->ack($frame);
// $this->consumer->commit();
if ($observer->isDelayed()) {
$this->echoLog('ServerAliveObserver: Server has been delayed.');
}
}
} catch (HeartbeatException $e) {
echo 'AMQ (STOMP) Error, the server failed to send us heartbeats within the defined interval: ' . $e->getMessage()
$this->reconnect();
} catch (ConnectionException $e) {
echo $e->getMessage();
$this->reconnect();
} catch (Exception $e) {
echo 'AMQ (STOMP) Queue error: ' . $e->getMessage();
exit(1);
}
}
}
I have reproduced on a single-instance broker with version 5.15.14.
I have tried switching on STOMP debugging for the broker, but it appears the uri attribute under <transportConnector> is not allowed. The config will save but pulls out the uri attribute.
<transportConnector> Elements and Their Attributes Permitted in Amazon MQ Configurations
Working with Spring XML configuration files (xsd files)
Related
I am using SOAP in a php script to call a web service from a Linux Centos 6 server. In this week I have been getting could not connect to host error from soapCall method. Of course this exception does not happen all the time but I see this exception most of the time, for example if I call soapCall 5 times, this error happens 3 times. My code is as below and I have not changed it at all for some months but recently it gets this error most of the time.
$wsdl="http://x.x.x.x:x/gw/services/Service?wsdl";
//Set key as HTTP Header
$aHTTP['http']['header'] = "key:" .$key ."\r\n";
$context = stream_context_create($aHTTP);
try
{
$client = new SoapClient($wsdl,array("soap_version" => SOAP_1_2,'trace' => 1,"stream_context" => $context));
}
catch(Exception $e)
{
return "something";
}
//I make $parametrs
try
{
$res = $client->__soapCall("send",array($parametrs));
}
catch(Exception $e)
{
print_r($e->getMessage()); //Most of the time it prints could not connect to host
}
I have read most answers to related questions here but my problem has not been solved. Also I added
ini_set("soap.wsdl_cache_enabled", "0");
ini_set("soap.wsdl_cache_ttl","0");
but nothing changed. I searched a lot about the reason and I found something about SSL but I have not found exact solution.
It must be mentioned that I can see the wsdl URL in browser and I have it's ping.
I want to verify a signature for a soap request on a soap server implemented in php.
The server code:
$Server = new SoapServer();
$d = new DOMDocument();
$d->load('php://input');
$s = new WSSESoapServer($d);
try {
if($s->process()) {
// Valid signature
$Server->handle($s->saveXML());
} else {
throw new Exception('Invalid signature');
}
} catch (Exception $e) {
echo "server exception: " . $e;
}
The error:
exception 'Exception' with message 'Error loading key to handle Signature' in /<path>/wse/src/WSSESoapServer.php:146
I have implemented a Client to sign SOAP requests using this library: https://github.com/robrichards/wse-php. There are no examples of how to implement the server...
How do I load the public key to check the signature?
[Edit]
I have now been able to load the supplied key using
$key = new XMLSecurityKey(XMLSecurityKey::RSA_SHA1, array('type' => 'public'));
$key->loadKey(CERT, true);
I am no longer getting an error message when validating the signature:
$x = new XMLSecurityDSig();
$d = $x->locateSignature($soapDoc);
$valid = $x->verify($key);
However, $valid is always false. I have no idea if it's because the key is loaded wrong or is actually invalid. I can find very little information on implementing a SOAP server with PHP and no information on implementing a SOAP server that relies on checking a signed request.
CLARIFICATION
My client talks to a remote web service and gets an acknowledgement.
Then the remote server takes some time to process my request.
A remote client (that I have no control over) then makes a request to
my service.
The last step is where I have trouble verifying the signature
Well anyways, your first approach looks fine to me, my server has the same structure. Unfortunately, WSSESoapServer does not inherit from SoapServer, thus is not really an SoapServer, but rather a SoapSignatureValidator and should be called like that. It would be easily possible to correct that behaviour, that one would not need a separate SoapServer instance (should be transparent and automatically).
<?php
require 'soap-server-wsse.php';
try {
// get soap message
$xmlSoap = DOMDocument::load('php://input');
// validate signature
$validateSignature = new WSSESoapServer($xmlSoap);
if(!$validateSignature->process())
file_put_contents("log.txt", "soapserver: SIGNATURE VALIDATION ERROR - CONTINUING WITHOUT SIGNATURE\n", FILE_APPEND);
//throw new Exception('Invalid Signature'); # this would cancel the execution and not send an answer
$sServer = new SoapServer($wsdl);
// actually process the soap message and create & send answer
//$sServer->setClass() or setObject() or set setFunction()
$sServer->handle($validateSignature->saveXML());
} catch (Exception $fault) {
file_put_contents("log.txt", "soapserver soapfault: ".print_r($fault, true), FILE_APPEND);
}
?>
I am trying to test synchronized Payout for my PHP application but response from server is always the same and it points me to INTERNAL_ERROR. I am using paypal/rest-api-sdk-php I believe of version v1.0.0 (composer.json says nothing) - should be the newest in my JSON I have version set to dev-master.
try {
/** #var PayoutBatch $payoutBatch */
$payoutBatch = $payout->createSynchronous($apiContext);
} catch (\Exception $ex) {
echo $ex->getMessage();
}
try {
/** #var PayoutBatch $response */
$response = Payout::get($payoutBatch->getBatchHeader()->getPayoutBatchId(), $apiContext);
$status = $response->getBatchHeader()->getBatchStatus();
if ($status === "SUCCESS") {
//do some logic
}
} catch (\Exception $ex) {
echo $ex->getMessage();
}
Have any of you experienced this issue?
Also a sub-question, don't PayPal PHP SDK has defined some constants where I can find message statuses?
Can you please make sure you have Payouts Enabled in your sandbox environment. This allows you to get the proper scope that is required to make payout calls.
However, if you have already done that and still failing, can you please try the samples out that are shipped along with SDK.
All you need to do is run one command, and you will have the samples running, as mentioned here
php -f vendor/paypal/rest-api-sdk-php/sample/index.php
This will start a sample server, and allow you to play around it.
It has a payout samples, that you could just click and run. If those works, we can now make one tiny change to get samples to use your clientId and secret. Find samples/bootstrap.php in sdk code, and change those clientId and secret, and run the samples again. If you get a failure, let me know and I could look further into that issue.
I have next code:
<?php
// callback function for recive the message and canceling consumer
function consumer(\AMQPEnvelope $envelope, \AMQPQueue $queue)
{
$queue->ack($envelope->getDeliveryTag());
$queue->cancel($envelope->getCorrelationId());
echo "Message was recived and consumer will be canceled by consumer tag: {$envelope->getCorrelationId()}\n";
}
// generating uniqie exchange and queue
$correlationId = uniqid(str_replace('.', '', (string)microtime(TRUE)) . '_');
$queueName = "databus_response_{$correlationId}";
$consumerTag = "consumer_tag_{$correlationId}";
// establesh connection
$connection = new \AMQPConnection(array('host'=>'127.0.0.1', 'user'=>'guest', 'password'=>'guest'));
$connection->connect();
$channel = new \AMQPChannel($connection);
// declare exchange
$exchange = new \AMQPExchange($channel);
$exchange->setFlags(AMQP_AUTODELETE);
$exchange->setType(AMQP_EX_TYPE_TOPIC);
$exchange->setName($queueName);
$exchange->declareExchange();
// declare queue
$queue = new \AMQPQueue($channel);
$queue->setFlags(AMQP_EXCLUSIVE);
$queue->setName($queueName);
$queue->declareQueue();
$queue->bind($queueName, '#');
// publish message in exchange
$exchange->publish('Test message', NULL, AMQP_PASSIVE, array('correlation_id' => $consumerTag));
// run consumer for getting this echange and canceling consumer after recive the message
$queue->consume('consumer', AMQP_NOWAIT, $consumerTag);
How you can see, i send one message to queue, and run consumer on this queue. In consumer method, you can see, what i want stop consumer on this queue after reciving first message by "cancel" method, but consumer not stopped. What i do wrong?
AMQP PECL module version 1.2.0 from https://github.com/pdezwart/php-amqp/tree/v1.2.0
PHP 5.4.4-14+deb7u10 (cli)
Linux v270 3.2.0-4-amd64 #1 SMP Debian 3.2.54-2 x86_64 GNU/Linux
just return false from consumer callback when you want to stop consuming.
From AMQPQueue.php stub file:
The AMQPQueue::consume() will not return the processing thread back to
the PHP script until the callback function returns FALSE.
P.S.: sad to say that php-amqp extension still lack of good documentation, but you can always read method annotation in stub files or read a bit outdated documentation on official php site in Polish language here - http://www.php.net/manual/pl/book.amqp.php (don't worry, nobody translated it, so all sensitive doc is in English).
Nowadays in the php-amqplib v2.* world you can also call basic_cancel on your AMQPChannel object.
here is a great blog post about controlling the number of consumers and their ttl using this technique.
I am processing a huge xml document (which contains around a million entries) and subsequently importing a formatted version to the db using rabbitmq. Each time after publishing around 200,000 entries I receive a broken pipe error , and rabbitmq is unable to recover from it.
Notice Error: fwrite(): send of 2651 bytes failed with errno=11
Resource temporarily unavailable in
[/var/www/ribbon/app/Console/Command/lib/php_amqplib/amqp.inc, line
439]
Notice Error: fwrite(): send of 33 bytes failed with errno=104
Connection reset by peer in
[/var/www/ribbon/app/Console/Command/lib/php_amqplib/amqp.inc, line
439]
Notice Error: fwrite(): send of 19 bytes failed with errno=32 Broken
pipe in [/var/www/ribbon/app/Console/Command/lib/php_amqplib/amqp.inc,
line 439]
This subsequently causes a node down error and the process needs to be manually killed to recover from it.
These are my class methods:-
public function publishMessage($message) {
if (!isset($this->conn)) {
$this->_createNewConnectionAndChannel();
}
try {
$this->ch->basic_publish(
new AMQPMessage($message, array('content_type' => 'text/plain')),
$this->defaults['exchange']['name'],
$this->defaults['binding']['routing_key']
);
} catch (Exception $e) {
echo "Caught exception : " . $e->getMessage();
echo "Creating new connection.";
$this->_createNewConnectionAndChannel();
$this->publishMessage($message); // try again
}
}
protected function _createNewConnectionAndChannel() {
if (isset($this->conn)) {
$this->conn->close();
}
if(isset($this->ch)) {
$this->ch->close();
}
$this->conn = new AMQPConnection(
$this->defaults['connection']['host'],
$this->defaults['connection']['port'],
$this->defaults['connection']['user'],
$this->defaults['connection']['pass']
);
$this->ch = $this->conn->channel();
$this->ch->access_request($this->defaults['channel']['vhost'], false, false, true, true);
$this->ch->basic_qos(0 , 20 , 0); // fair dispatching
$this->ch->queue_declare(
$this->defaults['queue']['name'],
$this->defaults['queue']['passive'],
$this->defaults['queue']['durable'],
$this->defaults['queue']['exclusive'],
$this->defaults['queue']['auto_delete']
);
$this->ch->exchange_declare(
$this->defaults['exchange']['name'],
$this->defaults['exchange']['type'],
$this->defaults['exchange']['passive'],
$this->defaults['exchange']['durable'],
$this->defaults['exchange']['auto_delete']
);
$this->ch->queue_bind(
$this->defaults['queue']['name'],
$this->defaults['exchange']['name'],
$this->defaults['binding']['routing_key']
);
}
Any help will be appreciated.
Make sure you have added virtualhost access for your user on Rabbit MQ. I've created new user and forgot set access rights for "/" host which is used by default.
You can do that via management panel yourhost:15672 > Admin > click on user > Look for "Set permission".
P.S. I assume your RabbitMQ service is running, user exists and password is correct.
Actually this problem happens when you have a big content inside your message and your consumer expend too much time to process only one message, that is problem to response "ACK" to rabbit and try to consume another message.
When I have this problem for example I try to "fit" my messages, because its a products worker and each message had some like 1k products id, so I change to 100 products and it works very well.
You can read more about Detecting Dead TCP Connections with Heartbeats here
This problem happened to me when my connection to RabbitMQ was broken (the reason does not matter, in my case I intentionally stopped RabbitMQ service for some failure tests), and I was trying to reconnect to the RabbitMQ again by closing the old connection and initializing a new one, but I received Broken pipe or closed connection error.
The way I solved this problem was to use reconnect() method on my connection:
$channel->reconnect();