I am working on a Webservice and I'm using Predis as a redis library and I want the clients to be able to reuse the same TCP socket.
Currently , after running a load test on my service , I found that the number of socket in TIME_WAIT state increase fast on the server and at some point the connexion to Redis server gets refused.
According Redis documentation using PhpiredisStreamConnection with presistant option fixes the problem , but after adding this to my connect code , I m still facing the same issue. Any ideas?
Im using TCP connection as the redis instance are not on front servers.
<?php
include 'autoload.php';
$parameters = array(
'tcp://some.host01:6379?database=0&alias=master&persistent=1',
'tcp://some.host02:6379?database=0&alias=slave&persistent=1',
);
$options = array( 'replication' => true ,
'connections' => array('tcp' => 'Predis\Connection\PhpiredisStreamConnection','unix' => 'Predis\Connection\PhpiredisStreamConnection') );
$predis = new Predis\Client($parameters, $options);
?>
As per the documents Predis works for persistent connections when php process are configured as persistent process . Look at the following for more details :
https://github.com/joindin/joindin-web2/blob/master/vendor/predis-0.8/FAQ.md
http://php-fpm.org/
Related
We are using linux server CENTOS 7 but virtual. We install to /var/mqm (9.2-IBM-MQC-linux) to our server. we install /mqseries-0.15.0 with /opt/cpanel/ea-php74/root/usr/bin/phpize to our server. we try to connect with IBM MQ with php code
<?php
$mq_host_ip='****(1414)';
$queue_name = '****';
$mq_server = '****';
$mqcno = array(
'Version' => MQSERIES_MQCNO_VERSION_2,
'Options' => MQSERIES_MQCNO_STANDARD_BINDING,
'MQCD' => array(
"ChannelName" => "*****",
'ConnectionName' => $mq_host_ip,
'TransportType' => MQSERIES_MQXPT_TCP
)
);
// Connect to the MQ server
mqseries_connx($mq_server,$mqcno,$conn,$comp_code,$reason);
if ($comp_code !== MQSERIES_MQCC_OK) {
echo 'Cannot open connection to server: ' . $comp_code ."--".$reason."--". mqseries_strerror($reason);
}else{
echo 'Connection good!';
}
?>
we get the error this
Cannot open connection to server: 2--2035--Not authorized for access.
.This is the our folder /home/trialwebsite/public_html.
Best Regards.
Murat ÖZKAN
The queue manager error log will usually tell you what specifically is wrong with the authentication/authorisation for a client program. The userid associated with the channel does not have appropriate rights or cannot be authenticated.
But one problem you might have with PHP is that it looks like the PECL-published PHP MQ library has not been maintained for a number of years and does not recognise the MQCSP structure needed to set the userid/password that the queue manager might be expecting. There is an un-merged PR opened at https://github.com/php/pecl-networking-mqseries/pull/5 which claims to add the MQCSP support so you might be able to do a git clone and use that branch as an alternative source of the interface.
I am hosting a PHP 7.2 website written on CodeIgniter 3 framework on Linux Azure App Service (plan P1v2). The database is MySQL 5.7 running on Azure Database for MySQL (General Purpose, 2vCores, 5GB storage) with SSL disabled.
When I browse my website, it literally takes about 8-9 seconds to load where browser would stay the white page and display everything at once after finished loading. (most of website contents are pulled from database)
I have been monitoring both App Service and MySQL DB and the average %CPU have never reached 50% of their allocated resource, so the specs does not seem to be the problem.
My outsourced developer team has their own testing environment (not on Azure) and they said it loads almost instantly on their environment, so they are blaming Azure.
I then noticed that Azure Advisor was telling me two things about this database where the impact is High
Improve MySQL connection latency
Our internal telemetry indicates that your application connecting to MySQL server may not be managing connections efficiently. This may result in higher application latency. To improve connection latency, we recommend that you enable connection redirection. This can be done by enabling the connection redirection feature of the PHP driver.
Improve MySQL connection management
Our internal telemetry indicates that your application connecting to MySQL server may not be managing connections efficiently. This may result in unnecessary resource consumption and overall higher application latency. To improve connection management, we recommend that you reduce the number of short-lived connections and eliminate unnecessary idle connections. This can be done by configuring a server side connection-pooler, such as ProxySQL.
I am now working on enabling the connection redirection to reduce latency but still not successful as I am not familiar with CodeIgniter so I need to wait for the developers to fix it.
My question is: what is/are the cause(s) of this performance issue? Are those two recommendations really the causes of this problem?
Both App Service and database are in the same region. Do they communicate locally like some kind of LAN connection? I hope the connection from App Service would not go out through the internet then back to Azure DB, causing slow performance.
Thanks.
I have fixed this issue, turns out the Azure Advisor was right! I have enabled connection redirection and now my website finishes loading in 3 seconds. Here are the what I did.
TL;DR Configure your app to use SSL when connecting to database and also put the mysqlnd_azure extension file in your deployed app, set Azure Web App's App Settings to read the ini file that make it loads the extension. Then set Azure MySQL DB server parameter redirect_enabled to ON and enable Enforce SSL with TLS 1.2. Restart the app.
Beware: Make sure that setting Enforce SSL is the last thing you do, otherwise it will reject all non-SSL connections and your app wouldn't work at all.
Update Jan 2021
I have just found out that it seems like Microsoft has already built-in the required mysqlnd_azure extension to Azure App Service. The majority of steps below are not required anymore.
You should first test if the extension has already been loaded by following step 7. If mysqlnd_azure extension is on the list then you can continue to step 8-10. You still need to follow step 1. to configure SSL certificate for your database though.
Afterwards, confirm if the database connection redirection is in use by following the sample code link in the reference at the end of this answer. You should see the text mysqlnd_azure.enableRedirect: preferred, followed by text in this format: [random text and numbers].[random text and numbers again].[your azure region].worker.database.windows.net.
If the text still shows your MySQL hostname you used, e.g. mydatabase.mysql.database.azure.com, then connection redirection is not in use. Check again.
Original answer below this line.
Configure your app to use SSL when connecting to database. In CodeIgniter, it's the database.php file in /application/config/. Make sure all MySQL drivers that you use have SSL enabled. Download the SSL certificate from https://www.digicert.com/CACerts/BaltimoreCyberTrustRoot.crt.pem and store it in your app's directory e.g. /cert/BaltimoreCyberTrustRoot.crt.pem
/*Snippet from database.php*/
$db['production'] = array(
'dsn' => '',
'hostname' => getenv("DB_HOST"),
'username' => getenv("DB_USER"),
'password' => getenv("DB_PWD"),
'database' => getenv("DB_NAME"),
'dbdriver' => 'mysqli',
'dbprefix' => '',
'pconnect' => FALSE,
'db_debug' => FALSE,
'cache_on' => FALSE,
'cachedir' => '',
'char_set' => 'utf8',
'dbcollat' => 'utf8_general_ci',
'swap_pre' => '',
'encrypt' => [
'ssl_key' => NULL,
'ssl_cert' => NULL,
'ssl_ca' => '/home/site/wwwroot/cert/BaltimoreCyberTrustRoot.crt.pem',
'ssl_capath' => NULL,
'ssl_cipher' => NULL,
'ssl_verify' => FALSE
],
'compress' => FALSE,
'stricton' => FALSE,
'failover' => array(),
'save_queries' => TRUE
);
/*Snippet for PHP (PDO)*/
<?php
define('CONN_HOST', getenv("DB_HOST"));
define('CONN_DATABASE', getenv("DB_NAME"));
define('CONN_USER', getenv("DB_USER"));
define('CONN_PASSWORD', getenv("DB_PWD"));
define('CONN_OPTION', array(
PDO::MYSQL_ATTR_SSL_CA => '/home/site/wwwroot/cert/BaltimoreCyberTrustRoot.crt.pem',
PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"
));
Install mysqlnd_azure extension by following this link https://azureossd.github.io/2019/01/29/azure-app-service-linux-adding-php-extensions/ or if you just want the .so extension right away, download it from http://www.mediafire.com/file/g6mzeld0wnqedw0/mysqlnd_azure.so/file otherwise build it yourself from https://github.com/microsoft/mysqlnd_azure
Put mysqlnd_azure.so in your app's directory. E.g. /bin/mysqlnd_azure.so Then deploy your code to Azure
SSH into your App Service, create a directory in /home/site called ini. Create an .ini file with any name e.g. settings.ini
Use vi or nano to edit that ini file and paste these lines
extension=/home/site/wwwroot/bin/mysqlnd_azure.so
mysqlnd_azure.enableRedirect = on
Tell your app to load the mysqlnd_azure extension by adding an application settings called PHP_INI_SCAN_DIR with value /usr/local/etc/php/conf.d:/home/site/ini
Azure will force restart your app after saving the app settings. After that you can check if the extension has been loaded by SSH again and run php -m to see all loaded extensions. If you see mysqlnd_azure in the list, you're on the right path! If not, this command should tell why it couldn't load that extension
On Azure Portal, navigate to your Azure MySQL DB and set server parameter redirect_enabled to ON
Under Connection Security, enable Enforce SSL connection and choose TLS 1.2
Restart the app again.
If you use sslmode=verify-ca or sslmode=verify-full, you need to switch to the new SSL certificate on October 26, 2020. Refer to https://learn.microsoft.com/en-us/azure/mysql/concepts-certificate-rotation Update: Microsoft has extended the deadline for root certificate deprecation till February 15, 2021.
If you have any deployment slots, make sure to do step 4-7 for every slot.
References:
Sample code to confirm if connection redirection is working https://learn.microsoft.com/en-us/azure/mysql/howto-redirection#confirm-redirection
App Service Linux - Update PHP Settings https://azureossd.github.io/2019/01/29/azure-app-service-linux-update-php-settings/
Configure SSL connectivity in your application to securely connect to Azure Database for MySQL https://learn.microsoft.com/en-us/azure/mysql/howto-configure-ssl
I am able to connect to redis cluster in elasticache with ec2 instance (given in aws documentation) and be able to add and get keys, values. But when i try to connect through phpredis on same ec2 instance, I'm getting no error and also no data. Please Help me with this. There's not much info on the internet for this specific problem. I am able to connect to redis running on the same ec2 instance but not to elasticache. If i could get some example on how to except for changing the host (endpoint of redis cluster).
Thanks
Use Predis library.
Connect to Redis ElastiCache Endpoint in Cluster mode using Predis, see below example.
try{
// Put your AWS ElastiCache Configuration Endpoint here.
$servers = ['aliceredis.8xyzwu.clustercfg.euw2.cache.amazonaws.com:6379'];
// Tell client to use 'cluster' mode.
$options = ['cluster' => 'redis'];
// Create your redis client
$redis = new Predis\Client($servers, $options);
// Do something you want:
// Set the expiration for 7 seconds
$redis->set("tm", "I have data for 7s.");
$redis->expire("tm", 7);
$ttl = $redis->ttl("tm"); // will be 7 seconds
// Print out value of the key 'tm'
var_dump(array("msg"=>"Successfully connected to Redis Cluster.", "val"=>$redis->get("tm"))) ;
}
catch(Exception $ex){
echo ('Error: ' . $ex->getMessage() ); // output error message.
}
I am using redis as the driver for caching data. The database configuration of Laravel has the ability to define the Redis connection information.
'redis' => array(
'cluster' => true,
'default' => array(
'host' => '127.0.0.1',
'port' => 6379,
'database' => 0,
),
),
But if I wanted to have multiple connections defined and use a specific connection to use for the cache, how can I do this on Laravel 4. There is no connection configuration on cache.php where I can specify the redis connection name. It currently has a connection config that will be used if the cache driver is database.
EDIT
I just went through the Laravel code and when initializing the Redis Driver, it looks like Laravel is not looking into the connection. Is my understanding correct?
http://laravel.com/api/source-class-Illuminate.Cache.CacheManager.html#63-73
protected function createRedisDriver()
{
$redis = $this->app['redis'];
return $this->repository(new RedisStore($redis, $this->getPrefix()));
}
Laravel can handle multiple connections. See this question/answer on adding/using multiple database connections.
Once you define multiple connections for redis, you'll need to do some leg work to access those somewhere in your code. That might look something like this:
$redisCache = App::make('cache'); // Assumes "redis" set as your cache
$redisCache->setConnection('some-connection'); // Your redis cache connection
$redisCache->put($key, $value');
Edit
I'll add a little here to give you an idea of how to do this so you don't need the connection logic everywhere:
Most simply, you can bind an instance of your redis cache somewhere (perhaps a start.php or other app/start/*.php file) in your app:
App::singleton('rediscache', function($app){
$redisCache = $app['cache'];
$redisCache->setConnection('some-connection'); // Your redis cache connection
return $redisCache;
});
Then, in your code, you can do this to cache:
$cache = App::make('rediscache');
$cache->put($key, $value); // Or whatever you need to do
You can also create a Service Provider if you have your own application library of code. You can register 'rediscache' within there and then use it in the same way in your application.
Hope that helps as a start - there are other code architectures - using Dependency Injection and maybe a repository to help organize your code further.
As the title, I'm looking for a php Redis client that support persistent connection, because my web application receives a lot of requests(each request, it'll put an item in to Redis queue) and I want to avoid create new connection every request.
Not sure if this is supported but you should definitely look at Predis and Rediska, this two (especially Predis AFAIK) are the best PHP Redis clients available.
PhpRedis currently supports persistent connections. Using PHP 7.0 and PhpRedis 3.0, making a persistent connection with pconnect() like this:
for ($i=0;$i<1000;$i++) {
$redis = new Redis();
$result = $redis->pconnect('127.0.0.1');
$redis->set("iterator",$i);
$response=$redis->get("iterator");
$redis->close();
unset($redis);
}
is about 10 times faster (9.6 msec vs 0.83 msec per connection) than connect():
for ($i=0;$i<1000;$i++) {
$redis = new Redis();
$result = $redis->connect('127.0.0.1');
$redis->set("iterator",$i);
$response=$redis->get("iterator");
$redis->close();
unset($redis);
}
Note: "This feature is not available in threaded versions". (I'm running under IIS on Windows, so I run the NTS version.)
Predis supports persistent connection. you just need to add persistent paramater as 1.
you can use the code below
$client = new Predis\Client(array(
'scheme' => 'tcp',
'host' => '127.0.0.1',
'port' => 6379,
'database' => 15,
'persistent'=> 1
));
instead of
$client = new Predis\Client('tcp://127.0.0.1:6379?database=15');
you can find more parameters for the connection here :
https://github.com/nrk/predis/wiki/Connection-Parameters
Predis supports persistent connections using it's PhpiredisStreamConnection with the persistent=1 flag syntax since v0.8.0:
<?php
$client = new Predis\Client('tcp://127.0.0.1?persistent=1', array(
'connections' => array(
'tcp' => 'Predis\Connection\PhpiredisStreamConnection',
'unix' => 'Predis\Connection\PhpiredisStreamConnection',
),
);
PHP-Redis supports persistent connections since it uses a php extension written in C which gives it a mechanism for sharing connections between requests. Look at the documentation on popen and pconnect.
Predis cannot support persistent connections because it is 100% PHP and PHP shares nothing between each request.