My application will be hosted in a shared hosting provider with a limit of 25 Mysql concurrent connections. Below is my code for connecting to DB
function dbConnect() {
$dbHost = 'localhost';
$dbUser = 'user';
$dbPass = 'pass';
$dbName = 'dbase';
mysql_connect($dbHost, $dbUser, $dbPass)or die(mysql_error());
mysql_select_db($dbName) or die(mysql_error());
}
My application is more on database query. The home page alone has atleast 20 mysq_query in it. The way I code my home page is like below
include 'config.php';
dbConnect();
query1 ... //to get initial variables
process variables that result in multiple query
query2... // process result
query3...// process result
and so on up to....
query 20...// process result
I cant minimize the query anymore coz most of the query is prerequisite of the other query. I am expecting of at least 1000 users daily, and the possibility of 50% of the users will connect on the same time or at lest seconds apart.
On the design of connecting to DB will I easily reach the connection limit set?
Is php closing the connection right after the query, meaning it closes after the first query, then open on the second and closes again right after, and so on and son, even if I'm not closing the connection myself
Is my assumption correct that the above scenario automatically consumed 20 concurrent connections?
Thanks in advance.
For what it's worth, some answers to questions 2. and 3. of genpet's original post:
PHP closes your connection to MySQL if and only if (assuming it wasn't openned with mysql_pconnect()):
you call mysql_close()
you reach the end of the script
Therefore, if you send 20 queries during the same script, you only consume 1 connection
One thought just comes to my mind. Consider this pattern:
open/poll a connection with *_pconnect()
run a query, deal with the results
release the connection ASAP with *_close()
open/poll again if you need to query the DB later in the script
This way, I believe one could work around the low concurrent connections limit (even though I agree with people advising your hosting plan may be inappropriate to fulfill your needs)
This is theory, though, I haven't tried that in the real.
It won't let me post this as an comment above, so this isn't an answer, but an extension of my first comment ("upgrade your hosting plan")
IMO, your connection design is not ok. "die" is a terminal function, if you're expecting failure, then build in a retry option and a user friendly error handling function. Also, opening and closing the DB each time is time intensive, you might be able to squeeze three queries faster than closing down and opening again for one (but that depends on several factors, which can be mitigated worth shared pools add suggested by others.) But my advice is upgrade hosting plan and develop "normally". Then your style is ok, except use a friendlier error screen than "die" :-) as well as pdo or mysqli, not mysql_ functions
Related
This is my first time asking a question on here. I've scoured Google, stackoverflow, etc. looking for help with the issue I'm having. We're currently using PHP 5.3.10 & MySQL 5.0.95 with Apache 2.2.21 (CentOS).
We're in the process of starting to cut over from the old mysql library to mysqli in new code, and I'm leading the charge. I've tried
making sure I explicitly close the connection to the database when I'm done with it
freeing result sets when I'm done with them
upping the connection limit to 250 from 150
There are included files (having to do with session checking, etc.) that use the old style mysql_pconnect() to validate certain things. These are included in nearly all of our code.
something like the code:
$mysqli = new mysqli('p:'.DBHOST, DBUSER, DBPASS, $_SESSION['dbname']);
if ($mysqli->connect_error) {
throw new Exception($mysqli->connect_error, $mysqli->connect_errno);
exit;
}
// do my stuff here, a bunch of SQL queries like:
$sql = 'SELECT * FROM MyTable';
$result = $mysqli->query($sql);
if (!$result) {
throw new SQLException($sql, $mysqli);
exit;
// SQLException is an extension to mysqli_sql_exception that adds the
// query into the messaging internally
}
while ($result && $row = $result->fetch_assoc()) {
// do stuff here, like show it on screen, etc., all works normally
}
$result->free(); // free up the result
$mysqli->close(); // close the connection to the database
freeing the results and closing the connection were things I did after getting a "Too many connections" error. Before doing that, I would get 3-4 new database connections each time I ran my program. (viewed in the back end with SHOW PROCESSLIST)
The problem is lessened somewhat (it adds 0 to 3 new connections, rather than 3 new connections each time).
Some of my reading suggests that this could have something to do with Apache threading + the new persistent connections added if there are no existing idle ones in the current thread. Is it this? Are persistent connections not supported well with mysqli? (should I give up on persistence?)
Thanks for any suggestions you might have.
I have no experience with mysqli persistent connections but some of your questions and expectations looks strange to me.
mysqli opens multiple new processes with p: connect option
Yes, that's what permanent connections are for
making sure I explicitly close the connection to the database when I'm done with it
You cannot make sure you closed it explicitly as you just can't do that. Again because the only point of permanent connection is to lasts open
I would get 3-4 new database connections each time I ran my program.
So, you have to make sure you're opening only one.
You have at least two connects from your script - one old style and some from mysqli?
How many mysqli objects being instantiated?
How many php scripts being run to serve one HTTP request? Are you sure?
After all, if it bothers you so much, why you're using persistent connections? Are you any real (not imaginary) benefits from it?
After all, if your version is 5.3, why bother with rewriting from mysql at all?
I'm a fairly new php programmer. I've been learning for several weeks and over those weeks i've tinkered from hello world scripts to taking other scripts online and installing them and modifying them. Now i'm writing my own and in the course of writing it, I had a bit of an epiphany.. because the scripts ive been modifying and toying with were done a certain way so i started that way as well. But now im thinking it could be inefficient coding or my idea is just wrong for some reason however..
my script is say index.php it's an interactive site and has users. i have 4 different databases that store some form of data. the first database is focused on users and their data. the second on something else, and so on.
The way I have been writing it is that anytime i need to do a query to the database, at the beginning of the script i have my database connection variables all defined such as
$servername = "127.0.0.1";
$dbusername = "dbusername";
$dbpassword = "passwordhere";
$dbname = "database1";
$roomdb = "database2";
etc
etc
so at all sorts of different points in my script there are needs to run database queries so i've been opening a connection like so
$link = mysqli_connect("$servername", "$dbusername", "$dbpassword", "$roomdb");
then running the query
then closing the database connection.
But then i was having a shower and thought suddenly.. why can't i just at the top of my script where i define the database connection info, just open ONE connection to each of the databases. then run the entire script, and just use queries where i need to use them.. and just name the queries differently if i need to have multiple queries to one database. like $query1 , $query2 etc where a query would be like
$query = "INSERT INTO `$room` (room, message, color) VALUES ('$room', '$randomvariable', '$randomvariable')";
does this make sense? it just seems to me that if i almost assuredly have to make multiple connections to each of these databases each time the script is run, and even if theres a chance i dont need one of the connections once in a while, that it would be more efficient to just connect it once at the beginning, have it available then use unique query names for each different query, and then close the connections at the end of the script?
any insights as to why i might be wrong or why it might be better to open the full connection right where you need it then close it again would be appreciated. i want to know how it functions best so i can make the best coding practice as possible....
but lets pretend i even have one database. the same logic applies. is it more efficient to just open one db connection at the top of my script, keep it open the whole run of the script and just use different $querynames to execute queries to the same db rather than open it, run query, close it, and do that 10 different times for all the queries. it seems pointless to me to open and close the connection so many times and adding overhead. did the examples i learn from just code bad or is there a reason for that
We need to save each connection handler in different variable
$link1 = mysqli_connect("$servername", "$dbusername", "$dbpassword", "$roomdb");
$link2 = mysqli_connect("$servername", "$dbusername", "$dbpassword", "$dbname");
Then use the $link_identifier separately for each, so that we know what are connecting to
mysqli_query($link1,"INSERT INTO `$room` (room, message, color) VALUES ('$room', '$randomvariable', '$randomvariable'");
mysqli_close($link1);
Note : Opening a connection for too long is not good coding standard. We would only connect when we are needed to query anything in that database, and close the connection immediately after the transaction is over.
This is my first time asking a question on here. I've scoured Google, stackoverflow, etc. looking for help with the issue I'm having. We're currently using PHP 5.3.10 & MySQL 5.0.95 with Apache 2.2.21 (CentOS).
We're in the process of starting to cut over from the old mysql library to mysqli in new code, and I'm leading the charge. I've tried
making sure I explicitly close the connection to the database when I'm done with it
freeing result sets when I'm done with them
upping the connection limit to 250 from 150
There are included files (having to do with session checking, etc.) that use the old style mysql_pconnect() to validate certain things. These are included in nearly all of our code.
something like the code:
$mysqli = new mysqli('p:'.DBHOST, DBUSER, DBPASS, $_SESSION['dbname']);
if ($mysqli->connect_error) {
throw new Exception($mysqli->connect_error, $mysqli->connect_errno);
exit;
}
// do my stuff here, a bunch of SQL queries like:
$sql = 'SELECT * FROM MyTable';
$result = $mysqli->query($sql);
if (!$result) {
throw new SQLException($sql, $mysqli);
exit;
// SQLException is an extension to mysqli_sql_exception that adds the
// query into the messaging internally
}
while ($result && $row = $result->fetch_assoc()) {
// do stuff here, like show it on screen, etc., all works normally
}
$result->free(); // free up the result
$mysqli->close(); // close the connection to the database
freeing the results and closing the connection were things I did after getting a "Too many connections" error. Before doing that, I would get 3-4 new database connections each time I ran my program. (viewed in the back end with SHOW PROCESSLIST)
The problem is lessened somewhat (it adds 0 to 3 new connections, rather than 3 new connections each time).
Some of my reading suggests that this could have something to do with Apache threading + the new persistent connections added if there are no existing idle ones in the current thread. Is it this? Are persistent connections not supported well with mysqli? (should I give up on persistence?)
Thanks for any suggestions you might have.
I have no experience with mysqli persistent connections but some of your questions and expectations looks strange to me.
mysqli opens multiple new processes with p: connect option
Yes, that's what permanent connections are for
making sure I explicitly close the connection to the database when I'm done with it
You cannot make sure you closed it explicitly as you just can't do that. Again because the only point of permanent connection is to lasts open
I would get 3-4 new database connections each time I ran my program.
So, you have to make sure you're opening only one.
You have at least two connects from your script - one old style and some from mysqli?
How many mysqli objects being instantiated?
How many php scripts being run to serve one HTTP request? Are you sure?
After all, if it bothers you so much, why you're using persistent connections? Are you any real (not imaginary) benefits from it?
After all, if your version is 5.3, why bother with rewriting from mysql at all?
I'm using PHP with MYSQL database as both are open source and easy to use.
I'm getting problem when I execute insert and/or update of millions of row one after another
while this operation perform I got the MYSQL error that:
'max_user_connections' active connections
which is the best way to solve this problem.
I don't want to use another database or language other then PHP.
connect_db();
$query = "insert into table(mobno,status,description,date,send_deltime,sms_id,msg,send_type) values('".$to."','".$status."','".$report."','','".$timewsha1."','".$smsID."','','".$type."')";
$result = mysql_query($query) or ("Query failed : " . mysql_error());
this query will execute thousand of times.
and then server give connection error.
First of all, try to know from your hosting server administrator about the max consecutive active connections available to the MySQL database. This is the most basic & primary information to have knowledge about.
If your page(s) load in a decent amount of time and release the connection once the page is loaded, it should be fine. The problem occurs when your script takes some long time to retrieve information from the database or maintains the connections.
Since you are executing INSERT and / or UPDATE operations of millions of rows, so you may have some problem.
Additionally, if you fail to close connections in your script(s), it is possible that someone will load a page and instead of closing the connection when the page is loaded, it is left open. No one else can then use that connection. So please make sure that at the end of execution of all the MySQL / SQL queries, the database connection is closed. Also please make sure that your server provides more than 250 connections, since 100 connections is available in almost all the servers generally.
Also make sure that you are not using the persistent connections (which is available when using the built-in function "mysql_pconnect()"), since this will lock up the user until the connection is manually closed.
Hope it helps.
//this loop is for preparing the subquery for mutiple records
for(// this loop for getting data for mutiple records){
$sub_query[] = "('".$to."','".$status."','".$report."','','".$timewsha1."','".$smsID."','','".$type."')";
}
$query = "insert into table(mobno,status,description,date,send_deltime,sms_id,msg,send_type) values ";
$query .= implode(',',$sub_query);
mysql_query($query );
So, a remote app calls into this script, sends it a list of values, and then this query is executed once, right? It's not in a foreach or for or while loop? When you say it will be executed millions of times, you don't mean in one sitting I mean. If it is in a loop, then move the db connect outside of the loop, otherwise it will attempt to connect again each time the loop iterates, and also, remember to call mysql_close at the end of the script, just in case.
mysql_pconnect() would create a persistent connection and that is the way to go if you don't want to exhaust your server's connection pool.
I use lazy connection to connect to my DB within my DB object. This basically means that it doesn't call mysql_connect() until the first query is handed to it, and it subsequently skips reconnecting from then on after.
Now I have a method in my DB class called disconnectFromDB() which pretty much calls mysql_close() and sets $_connected = FALSE (so the query() method will know to connect to the DB again). Should this be called after every query (as a private function) or externally via the object... because I was thinking something like (code is an example only)
$students = $db->query('SELECT id FROM students');
$teachers = $db->query('SELECT id FROM teachers');
Now if it was closing after every query, would this slow it down a lot as opposed to me just adding this line to the end
$db->disconnectFromDB();
Or should I just include that line above at the very end of the page?
What advantages/disadvantages do either have? What has worked best in your situation? Is there anything really wrong with forgetting to close the mySQL connection, besides a small loss of performance?
Appreciate taking your time to answer.
Thank you!
As far as I know, unless you are using persistent connections, your MySQL connection will be closed at the end of the page execution.
Therefore, you calling disconnect will add nothing and because you do the lazy connection, may cause a second connection to be created if you or another developer makes a mistake and disconnects at the wrong time.
Given that, I would just allow my connection to close automatically for me. Your pages should be executing quickly, therefore holding the connection for that small amount of time shouldn't cause any problems.
I just read this comment on PHP website regarding persistent connection and it might be interesting to know:
Here's a recap of important reasons
NOT to use persistent connections:
When you lock a table, normally it is unlocked when the connection
closes, but since persistent
connections do not close, any tables
you accidentally leave locked will
remain locked, and the only way to
unlock them is to wait for the
connection to timeout or kill the
process. The same locking problem
occurs with transactions. (See
comments below on 23-Apr-2002 &
12-Jul-2003)
Normally temporary tables are dropped when the connection closes,
but since persistent connections do
not close, temporary tables aren't so
temporary. If you do not explicitly
drop temporary tables when you are
done, that table will already exist
for a new client reusing the same
connection. The same problem occurs
with setting session variables. (See
comments below on 19-Nov-2004 &
07-Aug-2006)
If PHP and MySQL are on the same server or local network, the
connection time may be negligible, in
which case there is no advantage to
persistent connections.
Apache does not work well with persistent connections. When it
receives a request from a new client,
instead of using one of the available
children which already has a
persistent connection open, it tends
to spawn a new child, which must then
open a new database connection. This
causes excess processes which are just
sleeping, wasting resources, and
causing errors when you reach your
maximum connections, plus it defeats
any benefit of persistent connections.
(See comments below on 03-Feb-2004,
and the footnote at
http://devzone.zend.com/node/view/id/686#fn1)
(I was not the one that wrote the text above)
Don't bother disconnecting. The cost of checking $_connected before each query combined with the cost of actually calling $db->disconnectFromDB(); to do the closing will end up being more expensive than just letting PHP close the connection when it is finished with each page.
Reasoning:
1: If you leave the connection open till the end of the script:
PHP engine loops through internal array of mysql connections
PHP engine calls mysql_close() internally for each connection
2: If you close the connection yourself:
You have to check the value of $_connected for every single query. This means PHP has to check that the variable $_connected A) exists B) is a boolean and C) is true/false.
You have to call your 'disconnect' function, and function calls are one of the more expensive operations in PHP. PHP has to check that your function A) exists, B) is not private/protected and C) that you provided enough arguments to your function. It also has to create a copy of the $connection variable in the new local scope.
Then your 'disconnect' function will call mysql_close() which means PHP A) checks that mysql_close() exists and B) that you have provided all needed arguments to mysql_close() and C) that they are the correct type (mysql resource).
I might not be 100% correct here but I believe the odds are in my favour.
You may want to look at a using persistent connections. Here are two links to help you out
http://us2.php.net/manual/en/features.persistent-connections.php
http://us2.php.net/manual/en/function.mysql-pconnect.php
The basic unit of execution presumably is an entire script. What you first of all are wanting to apply resources (i.e. the database) to, efficiently and effectively, is the entirety of a single script.
However, PHP, Apache/IIS/whatever, have lives of their own; and they are capable of using the connections you open beyond the life of your script. That's the signficance of persistent (or pooled) connections.
Back to your script. It turns out you have a great deal of opportunity to be creative about using that connection during its execution.
The typical naive script will tend to hit the connection again and again, picking up locally appropriate scraps of data associated with given objects/modules/selected options. This is where procedural methodology can inflict a penalty on that connection by opening, requesting, receiving, and closing. (Note that any single query will remain alive until it is explicitly closed, or the script ends. Be careful to note that a connection and a query are not the same thing at all. Queries tie up tables; connections tie up ... connections (in most cases mapped to sockets). So you should be conscious of proper economy in the use of both.
The most economical strategy with regard to queries is to have as few as possible. I'll often try to construct a more or less complex joined query that brings back a full set of data rather than parceling out the requests in small pieces.
Using a lazy connection is probably a good idea, since you may not need the database connection at all for some script executions.
On the other hand, once it's open, leave it open, and either close it explicitly as the script ends, or allow PHP to clean up the connection - having an open connection isn't going to harm anything, and you don't want to incur the unnecessary overhead of checking and re-establishing a connection if you are querying the database a second time.