Connection management in MongoDB using PHP - php

I am using PHP 7.2 on a website hosted on Amazon. I have a code similar to this one that writes a record in the MongoDB:
Database connection class:
class Database {
private static $instance;
private $managerMongoDB;
private function __construct() {
#Singleton private constructor
}
public static function getInstance() {
if (!self::$instance) {
self::$instance = new Database();
}
return self::$instance;
}
function writeMongo($collection, $record) {
if (empty($this->managerMongoDB)) {
$this->managerMongoDB = new MongoDB\Driver\Manager(DB_MONGO_HOST ? DB_MONGO_HOST : null);
}
$writeConcern = new MongoDB\Driver\WriteConcern(MongoDB\Driver\WriteConcern::MAJORITY, 1000);
$bulk = new MongoDB\Driver\BulkWrite();
$bulk->insert($record);
try {
$result = $this->managerMongoDB->executeBulkWrite(
DB_MONGO_NAME . '.' . $collection, $bulk, $writeConcern
);
} catch (MongoDB\Driver\Exception\BulkWriteException $e) {
// Not important
} catch (MongoDB\Driver\Exception\Exception $e) {
// Not important
}
return $result->getInsertedCount() > 0;
}
}
Execution:
Database::getInstance()->writeMongo($tableName, $dataForMongo);
The script is working as intended and the records are added in MongoDB.
The problem is that connections are not being closed at all and once there are 500 inserts (500 is the limit of connections in MongoDB on our server) it stops working. If we restart php-fpm the connections are also reset and we can insert 500 more records.
The connection is reused during the request, but we have requests coming from 100s of actual customers.
As far as I can see there is no way to manually close the connections. Is there something I'm doing wrong? Is there some configuration that needs to be done on the driver? I tried setting socketTimeoutMS=1000&wTimeoutMS=1000&connectTimeoutMS=1000 in the connection string but the connections keep staying alive.

You are creating a client instance every time the function is invoked, and never closing it, which would produce the behavior you are seeing.
If you want to create the client instance in the function, close it in the same function.
Alternatively create the client instance once for the entire script and use the same instance in all of the operations done by that script.

Related

Too many connections with PDO persistent connection?

I've been struggling with this for quite a while, and it's to the point where I need to ask for help because even with all the research I've done I can't get a handle on why this is happening.
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000] [1040] Too many connections'
This happens upon loading a single page (index.php), and I am the only user (dev). As you can see here, the MySQL connection limit is set # 50 but I am narrowly surpassing that. This is an improvement over the 100~ connections that were being created before I refactored the code.
Here are the stats after the page has loaded once.
I've narrowed the issue down to several causes:
I don't fully understand how PDO/MySQL connections work.
I am creating too many connections in my code even though I am trying to only create one that I can share.
I need to increase the connection limit (seems unlikely).
Most of the SO questions I've found, tell the OP to increase the connection limit without truly knowing if that's the best solution so I'm trying to avoid that here if it's not needed. 50 connections for one page load seems like way too many.
These are the classes I am instantiating on the page in question.
$DataAccess = new \App\Utility\DataAccess();
$DataCopyController = new App\Controllers\DataCopyController($DataAccess);
$DriveController = new App\Controllers\DriveController($DataAccess);
$Helper = new App\Utility\Helper();
$View = new App\Views\View();
I am creating the DAL object then injecting it into the classes that need it. By doing it this way I was hoping to only create one object and one connection, however this is not what's happening obviously. Inside the DAL class I've also added $this->DbConnect->close() to each and every query method.
Here is the constructor for the DataAccess() class.
public function __construct() {
$this->DbConnect = new \App\Services\DbConnect();
$this->db = $this->DbConnect->connect("read");
$this->dbmod = $this->DbConnect->connect("write");
$this->Helper = new Helper();
}
Here is the DbConnect() class.
class DbConnect {
private $db;
private $dbmod;
private function isConnected($connection) {
return ($connection) ? TRUE : FALSE;
}
public function connect($access) {
$options = [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
PDO::ATTR_EMULATE_PREPARES => false
];
if ($access == "read") {
if ($this->isConnected($this->db)) {
return $this->db;
} else {
if (strpos($_SERVER['SERVER_NAME'], DBNAME_DEV) === false) {
$this->db = new PDO("mysql:host=127.0.0.1; dbname=".DBNAME,
DBUSER,
DBPASS,
$options
);
} else {
$this->db = new PDO("mysql:host=" . DBHOST_DEV ."; dbname=".DBNAME_DEV,
DBUSER,
DBPASS,
$options
);
}
return $this->db;
}
} elseif ($access == "write") {
if ($this->isConnected($this->dbmod)) {
return $this->dbmod;
} else {
if (strpos($_SERVER['SERVER_NAME'], DBNAME_DEV) === false) {
$this->dbmod = new PDO("mysql:host=127.0.0.1; dbname=".DBNAME,
DBUSER_MOD,
DBPASS,
$options
);
} else {
$this->dbmod = new PDO("mysql:host=" . DBHOST_DEV . "; dbname=".DBNAME_DEV,
DBUSER_MOD,
DBPASS,
$options
);
}
}
return $this->dbmod;
}
}
public function close() {
$this->db = null;
$this->dbmod = null;
}
}
I've also tried instantiating the DbConnect() class on index.php and injecting that rather than DataAccess() but the result was the same.
EDIT:
I also want to add that this MySQL server has two databases, prod and dev. I suppose the connection limit is shared between both. However, the prod database gets very little traffic and I am not seeing this error there. When I refreshed the stats, there were no connections to the prod database.
From the PHP manual ~ http://php.net/manual/en/pdo.connections.php
Many web applications will benefit from making persistent connections to database servers. Persistent connections are not closed at the end of the script, but are cached and re-used when another script requests a connection using the same credentials.
So I would advise removing the DbConnection#close() method as you would not want to ever call this.
Also from the manual...
Note:
If you wish to use persistent connections, you must set PDO::ATTR_PERSISTENT in the array of driver options passed to the PDO constructor. If setting this attribute with PDO::setAttribute() after instantiation of the object, the driver will not use persistent connections.
So you'll want (at least)
new \PDO("mysql:host=127.0.0.1;dbname=" . DBNAME, DBUSER, DBPASS, [
PDO::ATTR_PERSISTENT => true
]);
You can also set your other connection attributes in the constructor.

How can I stop this mysqli extension class from stopping executing on a failed 'real_connect'?

So the following code is in its infantile stages, and has a long way to go - but I am working on a database platform that is aware of multiple databases ( slaves, etc ), that is meant to attempt to open a connection to one, and upon failure, continue on to another. The relevant methods in the 'DatabaseConnection' class are as follows:
class DatabaseConnection extends mysqli
{
public function __construct($ConnectAutomatically=false,$Name="database",$Host="127.0.0.1",$Username="root",$Password="",$Catalog="",$Port=3306)
{
$this->CurrentHost = $_SERVER['SERVER_NAME'];
$this->Name = $Name;
$this->Host = $Host;
$this->Username = $Username;
$this->Password = $Password;
$this->Port = $Port;
$this->Catalog = $Catalog;
if( $ConnectAutomatically )
parent::__construct($this->Host, $this->Username, $this->Password, $this->Catalog, $this->Port);
else
parent::init();
return;
}
public function Open()
{
try
{
if(!parent::real_connect($this->Host,$this->Username,$this->Password,$this->Catalog,$this->Port))
{
return false;
}
else
return true;
}
catch( Exception $e )
{
return false;
}
}
}
The idea is that the database could be queried at any time, without having been initialized - therefore, the Query method checks for connectivity, and upon a lack thereof, runs the 'Open' method.
I have intentionally put a bad character in the password to force it to fail to connect, to see if it would connect to the second server in the list - but on the 'parent::real_connect' line, I am getting the obvious response that I cannot connect due to an invalid password.
I am wondering if there is a different approach I can take in this 'Open' method to NOT stop execution upon failure in the real_connect operation. I've taken a few swings at this and come up short, so I had to ask. It seems from my perspective that mysqli won't allow program execution to continue if the connect fails...
I have custom error catching already in place, which seems to catch the fault - but without interpreting the error and re-executing the DatabaseHandler connect method with the next in the list from Within my error handler, I don't see many other approaches at the moment.
I'll be glad to share more if need be.

PHP - MongoClient connections stacking up, not closing

I've got a class that is pulling and aggregating data from a Mongo DB. This is all working fine...I have multiple queries being run for a connection under the same connection (in the same class). However, every time I refresh the page, a new connection is opened...I can see this in my mongod output:
[initandlisten] connection accepted from 127.0.0.1:46770 #12 (6 connections now open)
I see this count up and up with every page refresh. This should be fine, I think; however, the connections never seem to close.
After a while, the connections / locks take up too much memory in Mongo, and I can no longer run the queries.
This dev environment is on a 32-bit version of Mongo, so I don't know if this is only happening because of that. (Our prod env is 64-bit, and I cannot change the dev env right now.)
My question is: Should I be closing the connection after all the queries have been made for a particular user? How should I be handling the connection pool?
The service class:
class MongoService{
protected $mongoServer;
public function setSpokenlayerMongoServer($mongoServer)
{ $this->mongoServer = $mongoServer; }
protected $mongoUser;
public function setSpokenlayerMongoUser($mongoUser)
{ $this->mongoUser = $mongoUser; }
protected $mongoPassword;
public function setSpokenlayerMongoPassword($mongoPassword)
{ $this->mongoPassword = $mongoPassword; }
protected $conn;
public function setServiceConnection($conn)
{ $this->conn = $conn; }
public function connect(){
try {
$this->conn = $this->getMongoClient();
} catch(Exception $e) {
/* Can't connect to MongoDB! */
// logException($e);
die("Can't do anything :(");
}
}
public function getDatabase($name){
if(!isset($this->conn)){
$this->connect();
}
return $this->conn->$name;
}
protected function getMongoClient($retry = 3) {
$connectString= "mongodb://".$this->mongoUser.":".$this->mongoPassword."#". $this->mongoServer."";
try {
return new MongoClient($connectString);
} catch(Exception $e) {
/* Log the exception so we can look into why mongod failed later */
// logException($e);
}
if ($retry > 0) {
return $this->getMongoClient(--$retry);
}
throw new Exception("I've tried several times getting MongoClient.. Is mongod really running?");
}
}
And in the service class where the queries are:
protected function collection(){
if(!isset($this->collection)){
$this->collection = $this->db()->selectCollection($this->collectionName);
}
return $this->collection;
}
And then a query is done like so:
$results = $this->collection()->aggregate($ops);
Is this correct behavior?
Known issue if you're using Azure IaaS, possible issue on other platforms.
One option would be to change the Mongo configuration:
MongoDefaults.MaxConnectionIdleTime = TimeSpan.FromMinutes(5);
This would kill all idle connections after 5 minutes.

Singleton holding multiple ADODB connections

A legacy website is exhibiting unexpected behavior with it's database connections. The application connects to several MySQL databases on the same server and the original developer created a "singleton" class that holds connection objects for each of them.
Lately we have been encountering a strange behavior with the class: when a connection to www is created after creating extra, getting the instance of extra returns a connection that has the correct parameters when viewed with var_dump() but is actually connected to the www database.
What could cause this? The code has worked before at some stage. Creating a new connection on each call to getInstance() fixes the problem but I'd like to solve this the right way if possible.
<?php
class DBConnection
{
private static $default;
private static $extra;
private static $intra;
private static $www;
public static function getInstance($dbname = "default")
{
global $db; // This is an array containing database connection parameters
if(!($dbname == "default" || $dbname == "extra" || $dbname == "www" || $dbname == "intra"))
{
$dbname = "default";
}
if (empty(self::$$dbname)) // Making this pass every time fixes the problem
{
try
{
self::$$dbname = ADONewConnection('mysqlt');
if(isset($db[$dbname]))
{
self::$$dbname->connect(DBHOSTNAME, $db[$dbname]["dbusername"], $db[$dbname]["dbpassword"], $db[$dbname]["dbname"]);
}
else
{
// fallback
self::$$dbname->connect(DBHOSTNAME, DBUSERNAME, DBPASSWORD, DBNAME);
}
self::$$dbname->SetFetchMode(ADODB_FETCH_ASSOC);
self::$$dbname->execute("SET NAMES utf8");
return self::$$dbname;
}
catch(Exception $e)
{
exit("DB connection failed");
}
}
else
{
return self::$$dbname;
}
}
}
Here's a simplified example of the class misbehaving:
$cn = DBConnection::getInstance("extra");
$cn->debug = true;
$rs = $cn->execute("SELECT * FROM messages WHERE id = ".$this->id);
The last line fails with the error message "Table www.messages does not exist".
1: You don't need to establish connection in getInstance().
getInstance must not do anything, but return instance of DB class.
2: when you do self::$$dbname->connect(, if (empty(self::$$dbname)) will return you false next time you call it.
Singleton is described here: http://en.wikipedia.org/wiki/Singleton_pattern
What you have - it's just a static method.

Should I keep reconnecting to mysql in PHP?

I have a pretty large site and every page is built from several included files, my site is 100% in a procedural format and I am trying to learn to use classes and a more OOP approach in PHP.
Currently my site has a header file that is included into every page, in this header is a mysql connection that is made and last the duration of the page, so if I need to run 10 different queries from different files, they all run without needing to make a new connection, so the connection is only made once.
Now that I am trying to convert to a more OO way, I am starting with writing a mysql class to connect and run queries, so I am thinking of using the classes __construct function to make a connection to mysql, I am just curious how this would work though, everytime that class gets called it would make or try to make a connection to mysql instead of just once.
Maybe I am not thinking it out clearly. Should I just initiate this class in the header 1 time and then I wont have to worry anymore?
You could create a single global object of your MySQL class and use that object everywhere. Then your constructor would only be called once.
Or you could create new objects of your MySQL class everywhere. mysql_connect doesn't open new connections if there already is one open:
If a second call is made to mysql_connect() with the same arguments, no new link will be established, but instead, the link identifier of the already opened link will be returned.
The best way I think is to use a special class to handle mysql connections and use it as a singleton. Make the constructor private and get it to return an instance of either an existing connection or a new one.
Here's my example:
class db
{
public $host;
public $user;
public $pass;
public $database;
private static $instance = false;
private function __construct()
{
}
public static function getInstance()
{
if (self::$instance === false)
{
self::$instance = new db;
}
return self::$instance;
}
public function db_connect()
{
}
public function db_disconnect()
{
}
}
This way, whenever you call: db::getInstance()->db_connect(), you are certain there's only going to be ONE instance of that connection everywhere.
Yes, you shouldn't connect multiple times. The overhead of opening and closing the connection all the time is bigger than the cost of keeping it open during the relative small time your scripts run. So you should create an instance of the class at the start and keep it in a global variable.
It's certainly not a bad idea to write your own classes as a exercise, but maybe you should look into one of the existing solutions for database connection management (Zend_Db etc.).
You can always store the database link reference in a STATIC class variable and call it whenever needed. However, PHP by itself tries to use an existing link if it exists in the memory.
I've a sample database handler code for you, ofcourse its PHP 5 and uses PDO.
<?php
// Class providing generic data access functionality
class DatabaseHandler
{
// Hold an instance of the PDO class
private static $_mHandler;
// Private constructor to prevent direct creation of object
private function __construct()
{
}
// Return an initialized database handler
private static function GetHandler()
{
// Create a database connection only if one doesn’t already exist
if (!isset(self::$_mHandler))
{
// Execute code catching potential exceptions
try
{
// Create a new PDO class instance
self::$_mHandler =
new PDO(PDO_DSN, DB_USERNAME, DB_PASSWORD);
// Configure PDO to throw exceptions
self::$_mHandler->setAttribute(PDO::ATTR_ERRMODE,
PDO::ERRMODE_EXCEPTION);
self::$_mHandler->setAttribute(PDO::ATTR_EMULATE_PREPARES, true);
}
catch (PDOException $e)
{
// Close the database handler and trigger an error
self::Close();
trigger_error($e->getMessage(), E_USER_ERROR);
}
}
// Return the database handler
return self::$_mHandler;
}
// Clear the PDO class instance
public static function Close()
{
self::$_mHandler = null;
}
// Wrapper method for PDO::prepare
private static function Prepare($queryString)
{
// Execute code catching potential exceptions
try
{
// Get the database handler and prepare the query
$database_handler = self::GetHandler();
$statement_handler = $database_handler->prepare($queryString);
// Return the prepared statement
return $statement_handler;
}
catch (PDOException $e)
{
// Close the database handler and trigger an error
self::Close();
trigger_error($e->getMessage(), E_USER_ERROR);
}
}
// Wrapper method for PDOStatement::execute()
public static function Execute($sqlQuery, $params = null)
{
// Try to execute an SQL query or a stored procedure
try
{
$statement_handler = self::Prepare($sqlQuery);
// Execute query
$statement_handler->execute($params);
}
// Trigger an error if an exception was thrown when executing the SQL query
catch(PDOException $e)
{
// Close the database handler and trigger an error
self::Close();
trigger_error($e->getMessage(), E_USER_ERROR);
}
}
// Wrapper method for PDOStatement::fetchAll()
public static function GetAll($sqlQuery, $params = null,
$fetchStyle = PDO::FETCH_ASSOC)
{
// Initialize the return value to null
$result = null;
// Try to execute an SQL query or a stored procedure
try
{
$statement_handler = self::Prepare($sqlQuery);
// Execute the query
$statement_handler->execute($params);
// Fetch result
$result = $statement_handler->fetchAll($fetchStyle);
}
// Trigger an error if an exception was thrown when executing the SQL query
catch(PDOException $e)
{
// Close the database handler and trigger an error
self::Close();
trigger_error($e->getMessage(), E_USER_ERROR);
}
// Return the query results
return $result;
}
// Wrapper method for PDOStatement::fetch()
public static function GetRow($sqlQuery, $params = null,
$fetchStyle = PDO::FETCH_ASSOC)
{
// Initialize the return value to null
$result = null;
// Try to execute an SQL query or a stored procedure
try
{
$statement_handler = self::Prepare($sqlQuery);
// Execute the query
$statement_handler->execute($params);
// Fetch result
$result = $statement_handler->fetch($fetchStyle);
}
// Trigger an error if an exception was thrown when executing the SQL query
catch(PDOException $e)
{
// Close the database handler and trigger an error
self::Close();
trigger_error($e->getMessage(), E_USER_ERROR);
}
// Return the query results
return $result;
}
// Return the first column value from a row
public static function GetOne($sqlQuery, $params = null)
{
// Initialize the return value to null
$result = null;
// Try to execute an SQL query or a stored procedure
try
{
$statement_handler = self::Prepare($sqlQuery);
// Execute the query
$statement_handler->execute($params);
// Fetch result
$result = $statement_handler->fetch(PDO::FETCH_NUM);
/* Save the first value of the result set (first column of the first row)
to $result */
$result = $result[0];
}
// Trigger an error if an exception was thrown when executing the SQL query
catch(PDOException $e)
{
// Close the database handler and trigger an error
self::Close();
trigger_error($e->getMessage(), E_USER_ERROR);
}
// Return the query results
return $result;
}
}
?>
You should pass a connection object (probably PDO) around, and the various places should be able to pick that up, either as a parameter, or as a property of some central object which the others have a reference to, or something.
Having multiple connections can be useful, it seems insane that mysql_connect picks up an existing connection when you might have wanted a new one - but it's insane anyway. Just use PDO.
You can use that method if you use mysql_pconnect() function, wich will search if there's already a mysql connection and in case it finds it, it wouldn't create another one.
In alternative, if you consider nor to use instances in php, you can call php database object directly, like :
class DB {}
DB::connect($host, $user, $pass);
If you use this method, you don't need to worry about multiple connections. Of course that if you need to use several connections to more than one database at a time, you can make use of object instances, or make your class so it can take several parameters and store them all at once (not very recomemded this one)

Categories