I've got a class that is pulling and aggregating data from a Mongo DB. This is all working fine...I have multiple queries being run for a connection under the same connection (in the same class). However, every time I refresh the page, a new connection is opened...I can see this in my mongod output:
[initandlisten] connection accepted from 127.0.0.1:46770 #12 (6 connections now open)
I see this count up and up with every page refresh. This should be fine, I think; however, the connections never seem to close.
After a while, the connections / locks take up too much memory in Mongo, and I can no longer run the queries.
This dev environment is on a 32-bit version of Mongo, so I don't know if this is only happening because of that. (Our prod env is 64-bit, and I cannot change the dev env right now.)
My question is: Should I be closing the connection after all the queries have been made for a particular user? How should I be handling the connection pool?
The service class:
class MongoService{
protected $mongoServer;
public function setSpokenlayerMongoServer($mongoServer)
{ $this->mongoServer = $mongoServer; }
protected $mongoUser;
public function setSpokenlayerMongoUser($mongoUser)
{ $this->mongoUser = $mongoUser; }
protected $mongoPassword;
public function setSpokenlayerMongoPassword($mongoPassword)
{ $this->mongoPassword = $mongoPassword; }
protected $conn;
public function setServiceConnection($conn)
{ $this->conn = $conn; }
public function connect(){
try {
$this->conn = $this->getMongoClient();
} catch(Exception $e) {
/* Can't connect to MongoDB! */
// logException($e);
die("Can't do anything :(");
}
}
public function getDatabase($name){
if(!isset($this->conn)){
$this->connect();
}
return $this->conn->$name;
}
protected function getMongoClient($retry = 3) {
$connectString= "mongodb://".$this->mongoUser.":".$this->mongoPassword."#". $this->mongoServer."";
try {
return new MongoClient($connectString);
} catch(Exception $e) {
/* Log the exception so we can look into why mongod failed later */
// logException($e);
}
if ($retry > 0) {
return $this->getMongoClient(--$retry);
}
throw new Exception("I've tried several times getting MongoClient.. Is mongod really running?");
}
}
And in the service class where the queries are:
protected function collection(){
if(!isset($this->collection)){
$this->collection = $this->db()->selectCollection($this->collectionName);
}
return $this->collection;
}
And then a query is done like so:
$results = $this->collection()->aggregate($ops);
Is this correct behavior?
Known issue if you're using Azure IaaS, possible issue on other platforms.
One option would be to change the Mongo configuration:
MongoDefaults.MaxConnectionIdleTime = TimeSpan.FromMinutes(5);
This would kill all idle connections after 5 minutes.
Related
I'm using Flysystem SFTP in Laravel 8. I've multiple accounts for sftp and I'm looping on every one for making adapter and then reading files from server. This all is working through console command and is registered in Schedule. The issue is when any of the connection fails due to username or password issue, it stops the execution of schedule task and skips the remaining. How can I check if connection is successful or not and continue to my next sftp connection. Thanks in advance.
foreach ($credentials as $cred) {
try {
$driver = Storage::createSFtpDriver($cred);
if($driver->exists('/reports/')) {
//Other code
} else {
continue;
}
} catch (Exception $e) {
continue;
}
}
See SFTP V3, there SftpConnectionProvider reads:
connectivity checker (must be an implementation of League\Flysystem\PhpseclibV2\ConnectivityChecker to check if a connection can be established (optional, omit if you don't need some special handling for setting reliable connections)
So the answer is SftpConnectivityChecker implements ConnectivityChecker
... to be passed into SftpConnectionProvider constructor. That interface only has one single method to override:
public class SftpConnectivityChecker implements ConnectivityChecker {
public function isConnected(SFTP $connection): bool {
$connected = false
// TODO: inspect the $connection status.
return $connected;
}
}
Likely to be configured alike this:
'sftp' => [
'connectivityChecker' => 'SftpConnectivityChecker'
]
And don't use continue, but handle the exception instead of ignoring it.
I don't know if it is good way or not but in my case, it is working fine. I just solved it by applying \ with Exception class and it is going fine.
foreach($credentials as $cred){
try {
$driver = Storage::createSFtpDriver($cred);
if($driver->exists('/report/')){
echo "Found for ".$cred["username"];
}
else{
continue;
}
}
catch (\Exception $e) {
continue;
}
}
I am using PHP 7.2 on a website hosted on Amazon. I have a code similar to this one that writes a record in the MongoDB:
Database connection class:
class Database {
private static $instance;
private $managerMongoDB;
private function __construct() {
#Singleton private constructor
}
public static function getInstance() {
if (!self::$instance) {
self::$instance = new Database();
}
return self::$instance;
}
function writeMongo($collection, $record) {
if (empty($this->managerMongoDB)) {
$this->managerMongoDB = new MongoDB\Driver\Manager(DB_MONGO_HOST ? DB_MONGO_HOST : null);
}
$writeConcern = new MongoDB\Driver\WriteConcern(MongoDB\Driver\WriteConcern::MAJORITY, 1000);
$bulk = new MongoDB\Driver\BulkWrite();
$bulk->insert($record);
try {
$result = $this->managerMongoDB->executeBulkWrite(
DB_MONGO_NAME . '.' . $collection, $bulk, $writeConcern
);
} catch (MongoDB\Driver\Exception\BulkWriteException $e) {
// Not important
} catch (MongoDB\Driver\Exception\Exception $e) {
// Not important
}
return $result->getInsertedCount() > 0;
}
}
Execution:
Database::getInstance()->writeMongo($tableName, $dataForMongo);
The script is working as intended and the records are added in MongoDB.
The problem is that connections are not being closed at all and once there are 500 inserts (500 is the limit of connections in MongoDB on our server) it stops working. If we restart php-fpm the connections are also reset and we can insert 500 more records.
The connection is reused during the request, but we have requests coming from 100s of actual customers.
As far as I can see there is no way to manually close the connections. Is there something I'm doing wrong? Is there some configuration that needs to be done on the driver? I tried setting socketTimeoutMS=1000&wTimeoutMS=1000&connectTimeoutMS=1000 in the connection string but the connections keep staying alive.
You are creating a client instance every time the function is invoked, and never closing it, which would produce the behavior you are seeing.
If you want to create the client instance in the function, close it in the same function.
Alternatively create the client instance once for the entire script and use the same instance in all of the operations done by that script.
So the following code is in its infantile stages, and has a long way to go - but I am working on a database platform that is aware of multiple databases ( slaves, etc ), that is meant to attempt to open a connection to one, and upon failure, continue on to another. The relevant methods in the 'DatabaseConnection' class are as follows:
class DatabaseConnection extends mysqli
{
public function __construct($ConnectAutomatically=false,$Name="database",$Host="127.0.0.1",$Username="root",$Password="",$Catalog="",$Port=3306)
{
$this->CurrentHost = $_SERVER['SERVER_NAME'];
$this->Name = $Name;
$this->Host = $Host;
$this->Username = $Username;
$this->Password = $Password;
$this->Port = $Port;
$this->Catalog = $Catalog;
if( $ConnectAutomatically )
parent::__construct($this->Host, $this->Username, $this->Password, $this->Catalog, $this->Port);
else
parent::init();
return;
}
public function Open()
{
try
{
if(!parent::real_connect($this->Host,$this->Username,$this->Password,$this->Catalog,$this->Port))
{
return false;
}
else
return true;
}
catch( Exception $e )
{
return false;
}
}
}
The idea is that the database could be queried at any time, without having been initialized - therefore, the Query method checks for connectivity, and upon a lack thereof, runs the 'Open' method.
I have intentionally put a bad character in the password to force it to fail to connect, to see if it would connect to the second server in the list - but on the 'parent::real_connect' line, I am getting the obvious response that I cannot connect due to an invalid password.
I am wondering if there is a different approach I can take in this 'Open' method to NOT stop execution upon failure in the real_connect operation. I've taken a few swings at this and come up short, so I had to ask. It seems from my perspective that mysqli won't allow program execution to continue if the connect fails...
I have custom error catching already in place, which seems to catch the fault - but without interpreting the error and re-executing the DatabaseHandler connect method with the next in the list from Within my error handler, I don't see many other approaches at the moment.
I'll be glad to share more if need be.
This question already has answers here:
Synchronized functions in PHP
(6 answers)
Closed 9 years ago.
I've got a php application that requires to make a connection to a server which authenticates with a token, this token stays valid until connection is lost.
When another connection is made while the first is still open my application crashes because the token is different from the currently connected one...
public function connect()
{
$Socket = fsockopen("192.168.1.1", 1234);
if ($Socket !== false) {
stream_set_timeout($Socket, static::TIMEOUT_SEC, static::TIMEOUT_USEC);
$this->socket = $Socket;
$this->sendeverything;
}
}
How am I able to run a function like:
function gogogo() {
connect();
}
multiple times without having them running simultaneously
Sorry for my bad english
Most easy solution would be to have a is_connected function:
function connect() {
if(is_already_connected()) {
return;
}
// ... your connect logic
}
In the is_already_connected() you'll have to write some intelligent code to determine if there is an open connection.
You can also create a kind of singleton connection (although this suggestion would probably instantiate a lot of debate about the use of singletons ;))
Try something like this...
<?php
class Connection {
public $Socket = null;
public function connect(){
// Checking if Socket already has a pointer :P
if((bool)$this->Socket){
return true;
}
$this->Socket = fsockopen("192.168.1.1", 1234);
if ($this->Socket !== false) {
stream_set_timeout($this->Socket, static::TIMEOUT_SEC, static::TIMEOUT_USEC);
$this->sendeverything();
}
}
}
$myconnect = new Connection();
$myconnect->connect();
$myconnect->connect();
?>
As mentioned in this question you can use sem_aquire for this. Something like:
function connect(){
$key = "192.168.1.1:1234" ;
try{
$sem = sem_get( $SEMKey);
sem_acquire($sem);
//Do connecty stuff here
sem_release($sem);
}catch(Exception $ex){
//Exception handling
}finally{
//Finally only available in PHP 5.5 place this in catch and try if < 5.5
sem_release($sem);
}
}
Note that this is entirely untested and wont work on windows. If you are on windows you can use flock - again as mentioned in the above question.
I have a PHP/MySQL website and it is hosted on Shared hosting. This website is not a huge traffic site. But I got an error (Too Many Connections Error) frequently. The hosting provider said that 25 MySQL connections at a time
I am using two functions for database connections, initDB() and closeDB()
function initDB()
{
$connection = mysql_connect($DatabaseURL,$DatabaseUName,$DatabasePWord);
if($connection)
{
$db = mysql_select_db($DatabaseName,$connection);
}
return $connection;
}
function closeDB($connection)
{
mysql_close($connection);
}
gettings data from database
$connection = initDB();
//executing MySQL query
closeDB($connection);
This is working fine, but sometime got Too Many Connections Error.
In my website, initDB() and closeDB() functions are called 76 times.
I am sure the opened connections are closed after execution
If any problem in the above code
How can i solve the Too Many Connections Error
You can use fast-hack of your code using static.
function initDB()
{
static $connection;
if (!$connection) //establish connection only once
{
$connection = mysql_connect($DatabaseURL,$DatabaseUName,$DatabasePWord);
if($connection)
{
$db = mysql_select_db($DatabaseName,$connection);
}
}
return $connection;
}
function closeDB($connection)
{
//no need to close.
}