I've been struggling with this for quite a while, and it's to the point where I need to ask for help because even with all the research I've done I can't get a handle on why this is happening.
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000] [1040] Too many connections'
This happens upon loading a single page (index.php), and I am the only user (dev). As you can see here, the MySQL connection limit is set # 50 but I am narrowly surpassing that. This is an improvement over the 100~ connections that were being created before I refactored the code.
Here are the stats after the page has loaded once.
I've narrowed the issue down to several causes:
I don't fully understand how PDO/MySQL connections work.
I am creating too many connections in my code even though I am trying to only create one that I can share.
I need to increase the connection limit (seems unlikely).
Most of the SO questions I've found, tell the OP to increase the connection limit without truly knowing if that's the best solution so I'm trying to avoid that here if it's not needed. 50 connections for one page load seems like way too many.
These are the classes I am instantiating on the page in question.
$DataAccess = new \App\Utility\DataAccess();
$DataCopyController = new App\Controllers\DataCopyController($DataAccess);
$DriveController = new App\Controllers\DriveController($DataAccess);
$Helper = new App\Utility\Helper();
$View = new App\Views\View();
I am creating the DAL object then injecting it into the classes that need it. By doing it this way I was hoping to only create one object and one connection, however this is not what's happening obviously. Inside the DAL class I've also added $this->DbConnect->close() to each and every query method.
Here is the constructor for the DataAccess() class.
public function __construct() {
$this->DbConnect = new \App\Services\DbConnect();
$this->db = $this->DbConnect->connect("read");
$this->dbmod = $this->DbConnect->connect("write");
$this->Helper = new Helper();
}
Here is the DbConnect() class.
class DbConnect {
private $db;
private $dbmod;
private function isConnected($connection) {
return ($connection) ? TRUE : FALSE;
}
public function connect($access) {
$options = [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
PDO::ATTR_EMULATE_PREPARES => false
];
if ($access == "read") {
if ($this->isConnected($this->db)) {
return $this->db;
} else {
if (strpos($_SERVER['SERVER_NAME'], DBNAME_DEV) === false) {
$this->db = new PDO("mysql:host=127.0.0.1; dbname=".DBNAME,
DBUSER,
DBPASS,
$options
);
} else {
$this->db = new PDO("mysql:host=" . DBHOST_DEV ."; dbname=".DBNAME_DEV,
DBUSER,
DBPASS,
$options
);
}
return $this->db;
}
} elseif ($access == "write") {
if ($this->isConnected($this->dbmod)) {
return $this->dbmod;
} else {
if (strpos($_SERVER['SERVER_NAME'], DBNAME_DEV) === false) {
$this->dbmod = new PDO("mysql:host=127.0.0.1; dbname=".DBNAME,
DBUSER_MOD,
DBPASS,
$options
);
} else {
$this->dbmod = new PDO("mysql:host=" . DBHOST_DEV . "; dbname=".DBNAME_DEV,
DBUSER_MOD,
DBPASS,
$options
);
}
}
return $this->dbmod;
}
}
public function close() {
$this->db = null;
$this->dbmod = null;
}
}
I've also tried instantiating the DbConnect() class on index.php and injecting that rather than DataAccess() but the result was the same.
EDIT:
I also want to add that this MySQL server has two databases, prod and dev. I suppose the connection limit is shared between both. However, the prod database gets very little traffic and I am not seeing this error there. When I refreshed the stats, there were no connections to the prod database.
From the PHP manual ~ http://php.net/manual/en/pdo.connections.php
Many web applications will benefit from making persistent connections to database servers. Persistent connections are not closed at the end of the script, but are cached and re-used when another script requests a connection using the same credentials.
So I would advise removing the DbConnection#close() method as you would not want to ever call this.
Also from the manual...
Note:
If you wish to use persistent connections, you must set PDO::ATTR_PERSISTENT in the array of driver options passed to the PDO constructor. If setting this attribute with PDO::setAttribute() after instantiation of the object, the driver will not use persistent connections.
So you'll want (at least)
new \PDO("mysql:host=127.0.0.1;dbname=" . DBNAME, DBUSER, DBPASS, [
PDO::ATTR_PERSISTENT => true
]);
You can also set your other connection attributes in the constructor.
Related
I am using PHP 7.2 on a website hosted on Amazon. I have a code similar to this one that writes a record in the MongoDB:
Database connection class:
class Database {
private static $instance;
private $managerMongoDB;
private function __construct() {
#Singleton private constructor
}
public static function getInstance() {
if (!self::$instance) {
self::$instance = new Database();
}
return self::$instance;
}
function writeMongo($collection, $record) {
if (empty($this->managerMongoDB)) {
$this->managerMongoDB = new MongoDB\Driver\Manager(DB_MONGO_HOST ? DB_MONGO_HOST : null);
}
$writeConcern = new MongoDB\Driver\WriteConcern(MongoDB\Driver\WriteConcern::MAJORITY, 1000);
$bulk = new MongoDB\Driver\BulkWrite();
$bulk->insert($record);
try {
$result = $this->managerMongoDB->executeBulkWrite(
DB_MONGO_NAME . '.' . $collection, $bulk, $writeConcern
);
} catch (MongoDB\Driver\Exception\BulkWriteException $e) {
// Not important
} catch (MongoDB\Driver\Exception\Exception $e) {
// Not important
}
return $result->getInsertedCount() > 0;
}
}
Execution:
Database::getInstance()->writeMongo($tableName, $dataForMongo);
The script is working as intended and the records are added in MongoDB.
The problem is that connections are not being closed at all and once there are 500 inserts (500 is the limit of connections in MongoDB on our server) it stops working. If we restart php-fpm the connections are also reset and we can insert 500 more records.
The connection is reused during the request, but we have requests coming from 100s of actual customers.
As far as I can see there is no way to manually close the connections. Is there something I'm doing wrong? Is there some configuration that needs to be done on the driver? I tried setting socketTimeoutMS=1000&wTimeoutMS=1000&connectTimeoutMS=1000 in the connection string but the connections keep staying alive.
You are creating a client instance every time the function is invoked, and never closing it, which would produce the behavior you are seeing.
If you want to create the client instance in the function, close it in the same function.
Alternatively create the client instance once for the entire script and use the same instance in all of the operations done by that script.
So the following code is in its infantile stages, and has a long way to go - but I am working on a database platform that is aware of multiple databases ( slaves, etc ), that is meant to attempt to open a connection to one, and upon failure, continue on to another. The relevant methods in the 'DatabaseConnection' class are as follows:
class DatabaseConnection extends mysqli
{
public function __construct($ConnectAutomatically=false,$Name="database",$Host="127.0.0.1",$Username="root",$Password="",$Catalog="",$Port=3306)
{
$this->CurrentHost = $_SERVER['SERVER_NAME'];
$this->Name = $Name;
$this->Host = $Host;
$this->Username = $Username;
$this->Password = $Password;
$this->Port = $Port;
$this->Catalog = $Catalog;
if( $ConnectAutomatically )
parent::__construct($this->Host, $this->Username, $this->Password, $this->Catalog, $this->Port);
else
parent::init();
return;
}
public function Open()
{
try
{
if(!parent::real_connect($this->Host,$this->Username,$this->Password,$this->Catalog,$this->Port))
{
return false;
}
else
return true;
}
catch( Exception $e )
{
return false;
}
}
}
The idea is that the database could be queried at any time, without having been initialized - therefore, the Query method checks for connectivity, and upon a lack thereof, runs the 'Open' method.
I have intentionally put a bad character in the password to force it to fail to connect, to see if it would connect to the second server in the list - but on the 'parent::real_connect' line, I am getting the obvious response that I cannot connect due to an invalid password.
I am wondering if there is a different approach I can take in this 'Open' method to NOT stop execution upon failure in the real_connect operation. I've taken a few swings at this and come up short, so I had to ask. It seems from my perspective that mysqli won't allow program execution to continue if the connect fails...
I have custom error catching already in place, which seems to catch the fault - but without interpreting the error and re-executing the DatabaseHandler connect method with the next in the list from Within my error handler, I don't see many other approaches at the moment.
I'll be glad to share more if need be.
I had a problem, which I solved but I don't fully understand why the solution works:
Without the '$conn = null;' statement in the code below the number of database connections was increasing and I got a "too many database connections" error. With that statement it works fine - no increasing nr of dbase connections, but I would have assumed that the '$conn = new Database($dbname[$day]);' statement would set a new database connection, and therefore automatically close the previous one. Obvious not, but why?
for($day=0;$day<365;$day++){
//code to determine $dbname here...
$conn = new Database($dbname[$day]); //Database() is defined class
//code to access database $dbname
$conn = null; //close connection
}//for
UPDATE: the the constructor for the Database class as requested:
public function __construct($dbname) {
try {
if($dbname != ECIS_DB_NAME){ //access to master database is forbidden
$this->_db = new PDO('mysql:host='.ECIS_DB_HOST.';dbname='.$dbname, ECIS_DB_USER, ECIS_DB_PASS, array(
PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => true
));
$this->_db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$this->_db->query('SET CHARACTER SET utf8');
self::$count++;
}//if
} catch (PDOException $e) {
exit('Error while connecting to database.'.$e->getMessage());
}
}//public function __construct
By unassigning $conn you are removing all references to the class and its __destruct method should be called. See "Is destructor in PHP predictable?".
The __destruct method likely has code to close the database connection. However, you should explicitly close the connection if possible, e.g.
$conn->close();
$conn = null;
If you are only connecting to a single database outside of your tests, you should initialize the database connection outside of your forloop.
When a user clicks a button, my script instantiates this class. so if there are fifty users on my website that click this button then there will be 50 of these classes that have been instantiated. is this the right thing to do? or do i need to check if this class has already been instantiated before and not do anything if it has been.
I connect to my db here.
there is more to this class, this is just a snippet.
class Database{
private $host = "localhost";
private $user = "rt";
private $pass = "";
private $dbname = "db";
public function __construct(){
// Set DSN
$dsn = 'mysql:host=' . $this->host . ';dbname=' . $this->dbname;
// Set options
$options = array(
PDO::ATTR_PERSISTENT => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
);
// Create a new PDO instanace
try{
$this->dbh = new PDO($dsn, $this->user, $this->pass, $options);
}
// Catch any errors
catch(PDOException $e){
$this->error = $e->getMessage();
}
}
}
is this the right thing to do?
Yes.
That's just how PHP works. 50 PHP instances runs, 50 connects to database established, etc.
do i need to check if this class has already been instantiated before
You can, but it would be pretty useless, as other user's click would be handled by a completely separate PHP process and you will have no access to it anyway.
Here is slightly improved version of constructor
public function __construct(){
// Set DSN
$dsn = 'mysql:host=' . $this->host . ';dbname=' . $this->dbname;
// Set options
$options = array(
PDO::ATTR_PERSISTENT => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
);
// Create a new PDO instanace
$this->dbh = new PDO($dsn, $this->user, $this->pass, $options);
}
Your PHP script needs to be run everytime a user loads the page, there is no way around instantiating a class everytime someone loads the page.
This is an inherent property of HTTP-based communications: statelessness; and PHP is run off the HTTP protocol, so it is subjected to it's limitations.
You can simulate states by using databases and sessions (which are basically just files storing information using a unique key as filename), but the PHP script itself will have to be subjected to HTTP's limitations of not having a state.
Hence, there is no way to keep your class in memory over several different connections.
You need to instantiate the class everytime you use different credentials (parameters). So, if every users creates new connection with different credentinals, then you need to instantiate everytime. Otherwise, you have to do proper checks and continue using same connection
So in the interest of not repeating myself, I only want to create one PDO connection and pass it between pages using a session var. But when I setup my PDO connection and set the session var, the var is coming back as not set on my next page?
This is the code on my first page:
session_start();
try
{
$db = new PDO("mysql:host=".$dbHostname.";dbname=".$dbDatabase, $dbUsername, $dbPassword);
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
catch(PDOException $e)
{
echo "It seems there was an error. Please refresh your browser and try again. ".$e->getMessage();
}
$_SESSION['db'] = $db;
Then this test code on my next page comes back as not set.
session_start();
$db = $_SESSION['db'];
if(isset($db))echo "set";
else echo "not set";
Any ideas??
The connection is fine because if I call a function from the first page and pass along $db as a parameter, the function works without any problems. So why would storing the database var as a session not work? Thank you for any help.
PDO does not allow session serialization. In fact, you should not be able to serialize database connections in the session at all. If it's really necessary to do that, you can do something like this:
class DB {
private $db;
private $creds;
public function __construct($host, $dbname, $user, $pass) {
$this->creds = compact('host', 'dbname', 'user', 'pass');
$this->db = self::createLink($host, $dbname, $user, $pass);
}
public function __sleep() {
return array('creds');
}
public function __wakeup() {
$this->db = self::createLink($this->creds['host'] ...
}
public static function createLink($host ...
return new PDO(...
}
}
PHP closes database connection and free the used resources after the page finishes execution.
Secondly, it's a resource/handler and not really a value.
You can use persistent connections.
$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass, array(
PDO::ATTR_PERSISTENT => true
));
Your application will behave exactly as if the connection was being reconstructed. So no need for you to serialize the connection and pass it around.
But you do need to be careful and close it when the user session is over so it can be released.
What exactly is your need for persistent connections?
You can use a persistent connection, but it is a bad idea. It ties up valuable system resources, might actually get you kicked off of a shared host, and is actually not all that useful. Just close the connection at the end of each page and start a new one at the next page. If you really, really want it then the setting for persistent connections is PDO::ATTR_PERSISTENT which must be set to true on your PDO object.