I am using Zend Framework for my PHP developments and here is a small function I used to execute a query. This is not about an error. The code and everything works fine. But I want to know some concept behind this.
/**
* Get dataset by executing sql statement
*
* #param string $sql - SQL Statement to be executed
*
* #return bool
*/
public function executeQuery($sql)
{
$this->sqlStatement = $sql;
if ($this->isDebug)
{
echo $sql;
exit;
}
$objSQL = $this->objDB->getAdapter()->prepare($sql);
try
{
return $objSQL->execute();
}
catch(Exception $error)
{
$this->logMessage($error->getMessage() . " SQL : " .$sql);
return false;
}
return false;
}
Bellow are unclear areas for me.
How Zend_Db_Table_Abstract Maintain database connections?
Is it creating new connection all the time when I call this function or Does it have some connection pooling?
I didn't write any coding to open or close database connection. So will zend framework automatically close connections?
If this open and close connection works all the time if I execute this function, Is there any performance issue?
Thank you and appreciate your suggestions and opinion on this.
Creating Connection
Creating an instance of an Adapter class does not immediately connect to the RDBMS server. The Adapter saves the connection parameters, and makes the actual connection on demand, the first time you need to execute a query. This ensures that creating an Adapter object is quick and inexpensive. You can create an instance of an Adapter even if you are not certain that you need to run any database queries during the current request your application is serving.
If you need to force the Adapter to connect to the RDBMS, use the getConnection() method. This method returns an object for the connection as represented by the respective PHP database extension. For example, if you use any of the Adapter classes for PDO drivers, then getConnection() returns the PDO object, after initiating it as a live connection to the specific database.
It can be useful to force the connection if you want to catch any exceptions it throws as a result of invalid account credentials, or other failure to connect to the RDBMS server. These exceptions are not thrown until the connection is made, so it can help simplify your application code if you handle the exceptions in one place, instead of at the time of the first query against the database.
Additionally, an adapter can get serialized to store it, for example, in a session variable. This can be very useful not only for the adapter itself, but for other objects that aggregate it, like a Zend_Db_Select object. By default, adapters are allowed to be serialized, if you don't want it, you should consider passing the Zend_Db::ALLOW_SERIALIZATION option with FALSE, see the example above. To respect lazy connections principle, the adapter won't reconnect itself after being unserialized. You must then call getConnection() yourself. You can make the adapter auto-reconnect by passing the Zend_Db::AUTO_RECONNECT_ON_UNSERIALIZE with TRUE as an adapter option.
Closing a Connection
Normally it is not necessary to close a database connection. PHP automatically cleans up all resources and the end of a request. Database extensions are designed to close the connection as the reference to the resource object is cleaned up.
However, if you have a long-duration PHP script that initiates many database connections, you might need to close the connection, to avoid exhausting the capacity of your RDBMS server. You can use the Adapter's closeConnection() method to explicitly close the underlying database connection.
Since release 1.7.2, you could check you are currently connected to the RDBMS server with the method isConnected(). This means that a connection resource has been initiated and wasn't closed. This function is not currently able to test for example a server side closing of the connection. This is internally use to close the connection. It allow you to close the connection multiple times without errors. It was already the case before 1.7.2 for PDO adapters but not for the others.
More information
Related
I want to implement connection pooling in Php in a similar way that works in java.
Why I need this :
Let's consider a flow
Step1: Connection To Db --- Resource Id #12
Step2: some computation... time taking .3 seconds
Step3: Query on Solr .... timing taking 2 seconds
Step4: Connection To Db --- Resource Id #12 (i am using same resource id)
Step5: Exit
Though in step4 I am using the same DB resource as of step1. However, the connection will go in the sleep state for both step2 and step3 and therefore can't be used by any other PHP process (other clients) until exit.
Solution:
use mysql_close every time after query get fired: Drawback: need to connect every time and hence time-consuming
Create a java service to handle queries (possible but too time-consuming and I am looking for other solution where I need to migrate queries )
Need to explore SQL relay like the third party but I am not sure will that be a success and not many good companies have used it
mysql_pconnect is not solving my case.
Please suggest
One way that you can apply scalability techniques to this pool model is to allow on the fly changes to your pool distribution. If you have a particular permalink that is extremely popular for some reason, you could move slaves from the primary pool to the comments pool to help it out. By isolating load, you’ve managed to give yourself more flexibility. You can add slaves to any pool, move them between pools, and in the end dial-in the performance that you need at your current traffic level.
There’s one additional benefit that you get from MySQL database pooling, which is a much higher hit rate on your query cache. MySQL (and most database systems) have a query cache built into them. This cache holds the results of recent queries. If the same query is re-executed, the cached results can be returned quickly.
If you have 20 database slaves and execute the same query twice in a row, you only have a 1/20th chance of hitting the same slave and getting a cached result. But by sending certain classes of queries to a smaller set of servers you can drastically increase the chance of a cache hit and get greater performance.
You will need to handle database pooling within your code - a natural extension of the basic load balancing code in Part 1. Let’s look at how we might extend that code to handle arbitrary database pools:
<?php
class DB {
// Configuration information:
private static $user = 'testUser';
private static $pass = 'testPass';
private static $config = array(
'write' =>
array('mysql:dbname=MyDB;host=10.1.2.3'),
'primary' =>
array('mysql:dbname=MyDB;host=10.1.2.7',
'mysql:dbname=MyDB;host=10.1.2.8',
'mysql:dbname=MyDB;host=10.1.2.9'),
'batch' =>
array('mysql:dbname=MyDB;host=10.1.2.12'),
'comments' =>
array('mysql:dbname=MyDB;host=10.1.2.27',
'mysql:dbname=MyDB;host=10.1.2.28'),
);
// Static method to return a database connection to a certain pool
public static function getConnection($pool) {
// Make a copy of the server array, to modify as we go:
$servers = self::$config[$pool];
$connection = false;
// Keep trying to make a connection:
while (!$connection && count($servers)) {
$key = array_rand($servers);
try {
$connection = new PDO($servers[$key],
self::$user, self::$pass);
} catch (PDOException $e) {}
if (!$connection) {
// Couldn’t connect to this server, so remove it:
unset($servers[$key]);
}
}
// If we never connected to any database, throw an exception:
if (!$connection) {
throw new Exception("Failed Pool: {$pool}");
}
return $connection;
}
}
// Do something Comment related
$comments = DB::getConnection('comments');
. . .
?>
I've managed to get a stable load balanced front end servers that can scale horizontally quite well however the next bottle neck would be the db. There was a blog post discussing scaling dbs horizontally however very little detail on it. I'm currently using PostgreSQL and so the only plugin I've found wouldn't work.
Are my only options creating my own HAProxy or rewriting the PostgreSQL plugin to allow connections with read replicas?
I'm using AWS for all my hosting
Firstly - I'd love to be corrected on this!
Having only had a quick look through some of the ORM classes in a SilverStripe 3.5 site, it looks like while the ORM does support multiple database connections (see DB::get_conn with argument for name) it is designed for specific use cases in mind. That is to say, you may have a module that needs to write to a specific database, so this would allow it to.
What you want is native and automatic support for this within the framework, so that all reads go to your slave(s) and writes go to your master. Unfortunately, it doesn't look like this comes out of the box. You might be able to achieve it by overloading a couple of the core SQL classes using the injector.
If you were to try it, this answer outlines how you could separate select statements out from the rest and run them through a different database connector.
As a quick example of how you might go at achieving this with SQLSelect, you will notice that it is injectable, which means you can easily overload it.
File: mysite/_config/injector.yml
Injector:
SQLSelect:
class: ReadOnlySQLSelect
You need to register a new database connection with the DB class:
File: mysite/_config.php
$readDatabaseConfig = array(/** define your DB credentials here, as with the default $databaseConfig **/);
if (!DB::connect($readDatabaseConfig, 'default_read')) {
user_error('Failed to connect to read replica DB!', E_USER_ERROR);
}
Now, overload the SQLSelect class and replace the parts of it that call the DB class methods. This class inherits from SQLExpression which is the class the contains the methods you actually care about in this instance:
File: mysite/code/ReadOnlySQLSelect.php
class ReadOnlySQLSelect extends SQLSelect
{
public function sql(&$parameters = array())
{
// Changed from SQLExpression: third parameter passed as connection name
$sql = DB::build_sql($this, $parameters, 'default_read');
if (empty($sql)) {
return null;
}
if ($this->replacementsOld) {
$sql = str_replace($this->replacementsOld, $this->replacementsNew, $sql);
}
return $sql;
}
public function execute()
{
$sql = $this->sql($parameters);
// Changed from SQLExpression: skip DB::prepared_query since it doesn't allow
// you to provide the connection name - replace it with its contents instead.
$conn = DB::get_conn('default_read');
return $conn->preparedQuery($sql, $parameters);
}
}
Note: SQLSelect::unlimitedRowCount should technically be replaced where it calls DB::prepared_query, since the prepared query method calls DB::get_conn with no arguments, so will always return the default connection. You could replace the DB::prepared_query line the same as used above:
$conn = DB::get_conn('default_read');
$result = $conn->preparedQuery($sql, $innerParameters);
If you implement the above method, also change new SQLSelect() to SQLSelect::create(), otherwise you'll end up with some queries that still hit the master server because it'll bypass your class by not using the injector.
There's also an instance in SQLConditionalExpression that you should replace too (::toSelect) but that is likely to affect query transformations from other child implementations of that class, and you won't be able to do much about it without either (A) PRing a fix to the framework or (B) overloading all the other SQL* classes.
At this point you should have everything you need to route select queries to your default_read connection.
Infrastructure
On the infrastructure side, you should be able to set up read replicas through the RDS console. When you do so it will provide you with a DNS endpoint for your replica node(s), which you can use in your _config.php to configure the connection to the read replica database.
If this works for you, you should create a module for it and put it up on GitHub - this would definitely be useful for others in future!
You may also consider making pull requests to the framework to add additional arguments to methods like DB::prepared_query to accept a connection name.
Also worth noting is that if you're using the mysqlnd database adapter you may be able to take advantage of read/write splitting, implemented with some sort of injector overloading but all handled at a lower level than the application layer.
I am doing project using mongodb and zendframework 2 so here I create connection in the constructor
private $conn;
public function __construct(){
$this->conn = new \MongoClient('mongodb://example.com:27017', array("connect" => TRUE));
}
It contain several actions to perform database operations like createdb, dropdb, renamedb like wise. so I close that connection within the __distruct() method
public function __destruct(){
$this->conn->close();
}
my code works fine. but I would like to know is this ok?
PHP closes database connections automatically.
You can read more about this in this post
however
read this
quote:
"Open connections (and similar resources) are automatically destroyed at the end of script execution. However, you should still close or free all connections, result sets and statement handles as soon as they are no longer required. This will help return resources to PHP and MySQL faster."
So basically, in case you have tons of things going in and out of the database, then yes, you might want to close right after.
In my opinion it is rather optional and your own choice wether to kill the process or not.
I have a webpage with some mysql querys,
I must start new db connection and db close after every query or its more efficiently start an only one db connection at the top of the code and close at the bottom of the script?
Thanks!
Never create a new connection for every query; it's very wasteful and will overburden your MySQL server.
Creating one connection at the top of your script is enough - all subsequent queries will use that connection until the end of the page. You can even use that connection in other scripts that are include()ed into the script with your mysql_connect() call in it.
I prefer to create some sort of class called Database of which upon creation will create a database connection using mysqli (OOP version of mysql).
This way i do not have to worry about that and i have a static class that has access to the database.
class MyAPI {
private static $database = false;
public static function GetDatabase() {
if (MyAPI::$database === false) {
MyAPI::$database = new Database(credentials can go here)
}
return MyAPI::$database;
}
}
Now your system with the includsion of your API and your database can access the database and its initialization code does not have to be peppered around your files depending on what part of the program is running and your database/connection will not be created if it is not needed (only until it is called).
Just for fun i also like to have a function in my Database class that will perform a query and return its results, this way i do not have to have the SAME while loop over and over again throughout my code depending on where i make these calls.
I also forgot to answer your question. No multiple connections! its baaaad.
Only create one connection. You don't even have to close it manually.
multiple connexion is better for security and tracking.
if you're going though a firewall between front and back there will be a timeout defined so a single connection will be a problem.
but if your web server is on the same host as mysql, a single connection will be more efficient.
I'm using PDO to access two SQLite 3 databases in PHP. I want to switch the database files during a query by renaming them but I can't do that while the files are open as it gives an error that the file is being used by another process. I've tried turning off persistent connections and setting the handles to null but neither work.
Is there really no way to close a PDO handle and release the lock on the database file?
I believe unset($var) does that, I use it on my pdo sqlite project and it works like I want it to :)
Set all references to the handle to null (or to anything except the PDO object, really) and the runtime will destruct the object, which will close the connection.
$db = new PDO('...');
// Do some stuff
$db = null;
// Assuming this was the last reference to that PDO
// object, the runtime will destroy the object and
// its connection.