This is my first time asking a question on here. I've scoured Google, stackoverflow, etc. looking for help with the issue I'm having. We're currently using PHP 5.3.10 & MySQL 5.0.95 with Apache 2.2.21 (CentOS).
We're in the process of starting to cut over from the old mysql library to mysqli in new code, and I'm leading the charge. I've tried
making sure I explicitly close the connection to the database when I'm done with it
freeing result sets when I'm done with them
upping the connection limit to 250 from 150
There are included files (having to do with session checking, etc.) that use the old style mysql_pconnect() to validate certain things. These are included in nearly all of our code.
something like the code:
$mysqli = new mysqli('p:'.DBHOST, DBUSER, DBPASS, $_SESSION['dbname']);
if ($mysqli->connect_error) {
throw new Exception($mysqli->connect_error, $mysqli->connect_errno);
exit;
}
// do my stuff here, a bunch of SQL queries like:
$sql = 'SELECT * FROM MyTable';
$result = $mysqli->query($sql);
if (!$result) {
throw new SQLException($sql, $mysqli);
exit;
// SQLException is an extension to mysqli_sql_exception that adds the
// query into the messaging internally
}
while ($result && $row = $result->fetch_assoc()) {
// do stuff here, like show it on screen, etc., all works normally
}
$result->free(); // free up the result
$mysqli->close(); // close the connection to the database
freeing the results and closing the connection were things I did after getting a "Too many connections" error. Before doing that, I would get 3-4 new database connections each time I ran my program. (viewed in the back end with SHOW PROCESSLIST)
The problem is lessened somewhat (it adds 0 to 3 new connections, rather than 3 new connections each time).
Some of my reading suggests that this could have something to do with Apache threading + the new persistent connections added if there are no existing idle ones in the current thread. Is it this? Are persistent connections not supported well with mysqli? (should I give up on persistence?)
Thanks for any suggestions you might have.
I have no experience with mysqli persistent connections but some of your questions and expectations looks strange to me.
mysqli opens multiple new processes with p: connect option
Yes, that's what permanent connections are for
making sure I explicitly close the connection to the database when I'm done with it
You cannot make sure you closed it explicitly as you just can't do that. Again because the only point of permanent connection is to lasts open
I would get 3-4 new database connections each time I ran my program.
So, you have to make sure you're opening only one.
You have at least two connects from your script - one old style and some from mysqli?
How many mysqli objects being instantiated?
How many php scripts being run to serve one HTTP request? Are you sure?
After all, if it bothers you so much, why you're using persistent connections? Are you any real (not imaginary) benefits from it?
After all, if your version is 5.3, why bother with rewriting from mysql at all?
Related
I currently create a DB connection via PDO for my website like so:
try {
self::$dbh = new PDO("mysql:host={$host};dbname={$dbName}", $dbUser, $dbPass);
self::$dbh->setAttribute( PDO::ATTR_ERRMODE, PDO::ERRMODE_WARNING );
}
catch (PDOException $e) {
return $e->getMessage();
}
I've been reading about persistent connections so I wanted to add the persistent flag like so:
self::$dbh = new PDO("mysql:host={$host};dbname={$dbName}", $dbUser, $dbPass,
array(PDO::ATTR_PERSISTENT => true
));
From what I've been reading this could be dangerous if something happens mid query etc. It sounds like this isn't really a recommended method.
Are there any other alternatives to maintain a persistent DB connection with MySQL?
The reason to use persistent connections is that you have a high number of PHP requests per second, and you absolutely need every last fraction of a percent of performance.
Even though creating a new MySQL connection is really pretty inexpensive (compared to connecting to Oracle or something), you may be trying to cut down this overhead. Keep in mind, though, that most sites get along just fine without doing this. It depends on how heavy your traffic is. Also, MySQL 5.6 and 5.7 have made it even more efficient to create a new connection, so the overhead is lower already if you upgrade.
The risk described in the post you linked to was that session-specific states didn't get cleaned up as a given DB connection was inherited by a subsequent PHP request.
Examples of session state include:
Unfinished transactions
Temporary tables
User variables
Connection character set
This can even be a security problem, for instance if one PHP user populates a temp table with privileged information, and then another PHP user finds they can read it.
Fortunately, in the 4 years since #Charles gave his answer, the mysqlnd driver has addressed this. It now uses mysql_change_user(), which is like a "soft disconnect" that resets all those session state details, but without releasing the socket. So you can get the benefit of persistent connections without risking leaking session state from one PHP request to another. See http://www.php.net/manual/en/mysqlnd.persist.php
This needs the mysqlnd driver to be enabled, which it should be if you use any reasonably up to date version of PHP.
Why do you need a persistent connection? PHP is stateless and reinitializes every time you make a request, so there is mostly no advantages and quite more disadvantages (i.e. sudden disconnectons with no handlers) in working with a persistent connections.
This is my first time asking a question on here. I've scoured Google, stackoverflow, etc. looking for help with the issue I'm having. We're currently using PHP 5.3.10 & MySQL 5.0.95 with Apache 2.2.21 (CentOS).
We're in the process of starting to cut over from the old mysql library to mysqli in new code, and I'm leading the charge. I've tried
making sure I explicitly close the connection to the database when I'm done with it
freeing result sets when I'm done with them
upping the connection limit to 250 from 150
There are included files (having to do with session checking, etc.) that use the old style mysql_pconnect() to validate certain things. These are included in nearly all of our code.
something like the code:
$mysqli = new mysqli('p:'.DBHOST, DBUSER, DBPASS, $_SESSION['dbname']);
if ($mysqli->connect_error) {
throw new Exception($mysqli->connect_error, $mysqli->connect_errno);
exit;
}
// do my stuff here, a bunch of SQL queries like:
$sql = 'SELECT * FROM MyTable';
$result = $mysqli->query($sql);
if (!$result) {
throw new SQLException($sql, $mysqli);
exit;
// SQLException is an extension to mysqli_sql_exception that adds the
// query into the messaging internally
}
while ($result && $row = $result->fetch_assoc()) {
// do stuff here, like show it on screen, etc., all works normally
}
$result->free(); // free up the result
$mysqli->close(); // close the connection to the database
freeing the results and closing the connection were things I did after getting a "Too many connections" error. Before doing that, I would get 3-4 new database connections each time I ran my program. (viewed in the back end with SHOW PROCESSLIST)
The problem is lessened somewhat (it adds 0 to 3 new connections, rather than 3 new connections each time).
Some of my reading suggests that this could have something to do with Apache threading + the new persistent connections added if there are no existing idle ones in the current thread. Is it this? Are persistent connections not supported well with mysqli? (should I give up on persistence?)
Thanks for any suggestions you might have.
I have no experience with mysqli persistent connections but some of your questions and expectations looks strange to me.
mysqli opens multiple new processes with p: connect option
Yes, that's what permanent connections are for
making sure I explicitly close the connection to the database when I'm done with it
You cannot make sure you closed it explicitly as you just can't do that. Again because the only point of permanent connection is to lasts open
I would get 3-4 new database connections each time I ran my program.
So, you have to make sure you're opening only one.
You have at least two connects from your script - one old style and some from mysqli?
How many mysqli objects being instantiated?
How many php scripts being run to serve one HTTP request? Are you sure?
After all, if it bothers you so much, why you're using persistent connections? Are you any real (not imaginary) benefits from it?
After all, if your version is 5.3, why bother with rewriting from mysql at all?
My application will be hosted in a shared hosting provider with a limit of 25 Mysql concurrent connections. Below is my code for connecting to DB
function dbConnect() {
$dbHost = 'localhost';
$dbUser = 'user';
$dbPass = 'pass';
$dbName = 'dbase';
mysql_connect($dbHost, $dbUser, $dbPass)or die(mysql_error());
mysql_select_db($dbName) or die(mysql_error());
}
My application is more on database query. The home page alone has atleast 20 mysq_query in it. The way I code my home page is like below
include 'config.php';
dbConnect();
query1 ... //to get initial variables
process variables that result in multiple query
query2... // process result
query3...// process result
and so on up to....
query 20...// process result
I cant minimize the query anymore coz most of the query is prerequisite of the other query. I am expecting of at least 1000 users daily, and the possibility of 50% of the users will connect on the same time or at lest seconds apart.
On the design of connecting to DB will I easily reach the connection limit set?
Is php closing the connection right after the query, meaning it closes after the first query, then open on the second and closes again right after, and so on and son, even if I'm not closing the connection myself
Is my assumption correct that the above scenario automatically consumed 20 concurrent connections?
Thanks in advance.
For what it's worth, some answers to questions 2. and 3. of genpet's original post:
PHP closes your connection to MySQL if and only if (assuming it wasn't openned with mysql_pconnect()):
you call mysql_close()
you reach the end of the script
Therefore, if you send 20 queries during the same script, you only consume 1 connection
One thought just comes to my mind. Consider this pattern:
open/poll a connection with *_pconnect()
run a query, deal with the results
release the connection ASAP with *_close()
open/poll again if you need to query the DB later in the script
This way, I believe one could work around the low concurrent connections limit (even though I agree with people advising your hosting plan may be inappropriate to fulfill your needs)
This is theory, though, I haven't tried that in the real.
It won't let me post this as an comment above, so this isn't an answer, but an extension of my first comment ("upgrade your hosting plan")
IMO, your connection design is not ok. "die" is a terminal function, if you're expecting failure, then build in a retry option and a user friendly error handling function. Also, opening and closing the DB each time is time intensive, you might be able to squeeze three queries faster than closing down and opening again for one (but that depends on several factors, which can be mitigated worth shared pools add suggested by others.) But my advice is upgrade hosting plan and develop "normally". Then your style is ok, except use a friendlier error screen than "die" :-) as well as pdo or mysqli, not mysql_ functions
In my PHP program I need to hit the database between 0 and 3 times on any given webpage request. I am using PDO to interact with MySQL. First I create a database connection using something like this:
$dbh = new PDO("mysql:host=$hostname;dbname=animals", $username, $password);
Then I do what I need to do and close the connection like this:
$dbh = null;
I open and close a connection 0-3 times right now, the same number of times I need to interact with MySQL.
My question is, should I be re-using this connection instead? My queries are not one after another, they are scattered throughout my program and I don't really think it would be easy to run them one right after another.
So is it better to create a new database connection and close it for each interaction (query) or to just leave the connection open and reuse it? Does it make any difference?
Thanks!
For a typical website page, you should be reusing the same connection for all queries.
It's not worth it to spend time disconnecting and reconnecting.
Unless your pages take a huge amount of time to run (huge being relative), then there's no point in relinquishing a connection. You'll end up wasting more cycles on connectiong/disconnecting than you do actually executing queries. MySQL's pretty lightweight as far as connections go, but it still adds up.
The site says: "You must have 50 reputations to comment". My reputation is pretty bad so I can't leave a comment, but I want to add a notice on:
For a typical website page, you should be reusing the same connection
for all queries.
Absolutely right, but in case of need atypical script (e.g. compare tables between a remote database and local database) one can't use the same connection. Must close first connection and establish new one or have 2 separate connections (using different names) at the same time.
I prefer the second option in this case because of closing connection using "$dbh = null;" it can be a bit difficult in some cases. (PDO closing connection)
Here is explanation [from documentation: https://www.php.net/manual/en/pdo.connections.php ] why should be careful with "$dbh = null;" (it should be closed all instances too):
<?php
$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass);
// use the connection here
$sth = $dbh->query('SELECT * FROM foo');
// and now we're done; close it
$sth = null;
$dbh = null;
?>
So, only $dbh = null; (without nullifying instances) is not a good idea.
I'm using PHP with MYSQL database as both are open source and easy to use.
I'm getting problem when I execute insert and/or update of millions of row one after another
while this operation perform I got the MYSQL error that:
'max_user_connections' active connections
which is the best way to solve this problem.
I don't want to use another database or language other then PHP.
connect_db();
$query = "insert into table(mobno,status,description,date,send_deltime,sms_id,msg,send_type) values('".$to."','".$status."','".$report."','','".$timewsha1."','".$smsID."','','".$type."')";
$result = mysql_query($query) or ("Query failed : " . mysql_error());
this query will execute thousand of times.
and then server give connection error.
First of all, try to know from your hosting server administrator about the max consecutive active connections available to the MySQL database. This is the most basic & primary information to have knowledge about.
If your page(s) load in a decent amount of time and release the connection once the page is loaded, it should be fine. The problem occurs when your script takes some long time to retrieve information from the database or maintains the connections.
Since you are executing INSERT and / or UPDATE operations of millions of rows, so you may have some problem.
Additionally, if you fail to close connections in your script(s), it is possible that someone will load a page and instead of closing the connection when the page is loaded, it is left open. No one else can then use that connection. So please make sure that at the end of execution of all the MySQL / SQL queries, the database connection is closed. Also please make sure that your server provides more than 250 connections, since 100 connections is available in almost all the servers generally.
Also make sure that you are not using the persistent connections (which is available when using the built-in function "mysql_pconnect()"), since this will lock up the user until the connection is manually closed.
Hope it helps.
//this loop is for preparing the subquery for mutiple records
for(// this loop for getting data for mutiple records){
$sub_query[] = "('".$to."','".$status."','".$report."','','".$timewsha1."','".$smsID."','','".$type."')";
}
$query = "insert into table(mobno,status,description,date,send_deltime,sms_id,msg,send_type) values ";
$query .= implode(',',$sub_query);
mysql_query($query );
So, a remote app calls into this script, sends it a list of values, and then this query is executed once, right? It's not in a foreach or for or while loop? When you say it will be executed millions of times, you don't mean in one sitting I mean. If it is in a loop, then move the db connect outside of the loop, otherwise it will attempt to connect again each time the loop iterates, and also, remember to call mysql_close at the end of the script, just in case.
mysql_pconnect() would create a persistent connection and that is the way to go if you don't want to exhaust your server's connection pool.