I have pages on my site that use the database and pages that don't.
When ever I need the database, I connect using $conn = connect(). But this means I need to put it in everywhere its needed. If I put it in an include file and put that file into every page it would connect even when the database is not needed. Would this be a good idea? Would creating a connection cause problems or other issues when not needed, or should I connect only when needed?
Connecting to a database when you don't need to introduces a small amount of overhead that is avoidable. If you need your pages to run as fast as possible, then you could optimize by avoiding the unnecessary db connection.
How much overhead this represents as a proportion of your total PHP execution time varies, for instance if your PHP script is simple and quick, then proportionally the db connection is a larger percentage of the total time wasted. If your PHP script does a lot of other things, then the db connection is a smaller percentage of total time.
Also the speed of a db connection can vary, depending on the speed of your server, whether MySQL is configured with DNS dependency, etc.
When I worked on the Zend Framework, we implemented "lazy" connections. You can create an instance of a Zend_Db_Adapter object anytime you want, but that class doesn't connect to the class constructor. It connects to the database when you run your first query (or when you explicitly call the getConnection() method).
Another consideration is how soon do you disconnect from the database when you're done running queries.
Suppose you handle 1000 PHP requests per second (one per millisecond on average), and each of your PHP requests lasts 100ms. So at any given instant, you may have 100 PHP requests in progress on average. If the first thing your PHP code does is connect to the db, and the last thing it does is disconnect from the db and other resources (by automatic request cleanup), then you may also have 100 db connections active at any time.
But if you delay connecting to the db, and disconnect promptly when you are done querying the db, and avoid connecting altogether on some requests, then on average you will have a much lower number of concurrent db sessions.
This can help reduce resource use on the db server, allowing more throughput and a higher number of PHP requests to complete per second.
Related
I want to know something really important for my website, Imagine if I have thousand of users that accessed the website, then, At that time there probably be thousand of queries that might be running at the same, So, Don't that fill up the DB pool as at the same time there are thousand of queries.So, to overcome this , how can I run the MYSQL queries in Queue ?
Also, I am using InnoDB as MYSQL Storage Engine, Any other suggestion is also appreciated.
The problem does not exist.
Properly written queries can come and go in the blink of an eye.
You won't have "thousands" of users actually executing queries at the same time. The real metric is "pages per minute".
If you were to "queue" the queries, latency would be bad, and the users would move to some competing site.
If you do have performance problems, it is almost always because of lack of index, poorly phrasing of queries, and/or improper schema design.
Another, more appropriate approach, for scaling is to use Replication go allow for multiple read-only Slaves and have multiple clients (on other serves) hitting those Slaves through a load-balancer. This is what Yahoo, for example, does to handle its traffic. And it needs only a handful of servers for each property (news, weather, sports, finance, etc). Are you at that scale yet?
RDMS handles the query queues internally. You can use caches systems in order to increase the response time and use nosql databases in order to scale your solution. On the other hand, if your intent is to throttle the visitors, you have to handle it in your application side.
I suggest you monitor a couple of status variables in MySQL:
SHOW GLOBAL STATUS LIKE 'Threads_%';
Threads_connected is the number of clients are currently connected and occupying a thread in the MySQL server. Typically this is one per concurrent PHP request (if you're using PHP).
Threads_running is the number of clients who are connect and actually executing a query at that moment.
You might have hundreds of threads connected, but only a handful of threads running at any given time. This is because a PHP script runs code in between the SQL queries. During this in-between time, the MySQL connection is still open, but not doing anything in the MySQL server. This is normal.
You might have thousands of users, but since human beings typically view a web page for at least a few seconds before they click on something to request the next web page, there is some time in between page views where neither your web server or your database server need to do anything for a given user.
So it's typically true that the MySQL server can handle the traffic even when you have thousands of users, because only a tiny fraction of them are running a query simultaneously.
I am working a new PHP project that calls upon a records class to update the database. Currently every object or script that needs to do so makes a new instance of the object. In theory this could mean as many as six or seven instances per call.
The design currently calls for the object to connect to the database in the constructor method which would mean between 0 and 7 connections?
For the demo I will be using only MySQL but as it is all tucked away behind a single interface one imagines that it could easily be ported to other data stores (and probably will).
Given the nature of the project speed of response is the key optimisation point however with so many connections do I run the risk of overloading the system when demand is high and therefore slowing everything down anyway?
Is this a situation when it would simply be better (and faster overall) to use a singleton?
Or is there some way to have my cake and eat it?
A connection to a database server takes time to connect and will consume memory. Also the number of connections a database server will accept at the same time is limited. That's why the number of connections should be kept as small as possible which means that only a single connection to the same database should be used.
Using just a single connection means not that you have to use the Singleton pattern. Simply create the connection object somewhere at the start of the script and pass it to components which will execute DB queries.
I can suggest you two solutions:
Use stored procedure if you can, it will decrease # of Mysql connections, its more safety, and its cached.
Use at first time Mysql connection and than save data into Redis or Memcached.
It intend to your queries and your needs.
Thanks
Look at this solution. It uses singleton approach for each connection and handles multiple connections across multiple adapters (e.g. mysql, memcached etc.)
If I have a single page, which its contents are generated from a MySQL database (simple query to display the contents of a cell, HTML contents), how many users can hit that page at once without crashing the database?
I have to display the same data across multiple domains, so instead of duplicating the page on three domains, I'm going to load it into mysql and use this route to display it. However, I want to find out how many concurrent connections I could handle without crashing the database.
Can anyone point me in the right directions to finding this out? I'm assuming this small query shouldn't be a huge load, but if 10,000 visitors hit it at once, then what?
You need to check ur setting for max_connections, you could get more information about this by looking at the following link: http://www.electrictoolbox.com/update-max-connections-mysql/
Are you using any framework? If so most have caching built into them which should solve the issue assuming that the information isn't being updated on a moment to moment basis. Even if it is, try to cache whatever parts of the page you assume are not going to change that often.
how many users can hit that page at once without crashing the database?
All of them.
It's a bad question.
The database should not crash because users are trying to connect. There are lots of limits on how many php clients can open concurrent connections to the database. Really you should have the DBMS correctly configured so it handles the situation gracefully - i.e. max_connections should limit the number of clients - not the amount of memory available to the DBMS.
If you mean how many concurrent connections can you support without connections being rejected / queued, that's something VERY different. And it's nearly a sensible question.
In that case, it's going to depend on the amount of memory you've got, how you've configured the DBMS, how fast the CPU is, what size the content is, how many content items there are, how PHP connects to the DBMS, how far away the PHP is from the DBMS....
You're more likely to run into connection limits on the webserver before you hit them on the DBMS - but the method for establishing what those limits are is the same for both - generate a controlled number of connection and measure how memory it uses - repeat for a different number, get lots of data. Draw a graph, see where the line crosses your resource limits.
so instead of duplicating the page on three domains, I'm going to load it into mysql
The number of domains is pretty much irrelevant - unless you're ecxplicitly looking for a method of avoiding ESI.
but if 10,000 visitors
Really? On a typical site that would mean perhaps a million concurrent HTTP sessions.
Right now the setup for my javascript chat works so it's like
function getNewMessage()
{
//code would go here to get new messages
getNewMessages();
}
getNewMessages();
And within the function I would use JQuery to make a get post to retrieve the messages from a php scrip which would
1. Start SQL connection
2. Validate that it's a legit user through SQL
3. retrieve only new message since the last user visit
4. close SQL
This works great and the chat works perfectly. My concern is that this is opening and closing a LOT of SQL connections. It's quite fast, but I'd like to make a small javascript multiplayer game now, and transferring user coordinates as well as the tens of other variables 3 times a second in which I'm opening and closing the sql connection each time and pulling information from numerous tables each time might not be efficient enough to run smoothly, and might be too much strain on the server too.
Is there any better more efficient way of communicating all these variables that I should know about which isn't so hard on my server/database?
Don't use persistent connections unless it's the only solution available to you!
When MySQL detects the connection has been dropped, any temporary tables are dropped, any active transaction is rolled back, and any locked tables are unlocked. Persistent connections only drop when the Apache child exits, not when your script ends, even if the script crashes! You could inherit a connection in the middle of a transaction. Worse, other requests could block, waiting for those tables to unlock, which may take quite a long time.
Unless you have measured how long it takes to connect and identified it as a very large percentage of your script's run time, you should not consider using persistent connections. In fact, that should be what you do here, if you're worried about performance. Check out xhprof or xdebug, profile your code, then start optimizing.
Maybe try to use a different approach to get the new messages from the server: Comet.
Using this technique you do not have to open that much new connections.
http://www.php.net/manual/en/features.persistent-connections.php
and
http://www.php.net/manual/en/function.mysql-pconnect.php
A couple of dozen players at the same time won't hurt the database or cause noticeable lag if you have efficient SQL statements. Likely your database will be hosted on the same server or at least the same network as your game or site, so no worries. If your DB happens to be hosted on a separate server running an 8-bit 16mz board loaded with MSDOS, located in the remote Amazon, connected by radio waves hooked up to a crank-powered generator operatated by a drunk monkey, you're on your own with this one.
Otherwise, really you should be more worried about exactly how much data you're passing back and forth to your players. If you're passing back and forth coordinates for all objects in an entire world, page load could take a painfully long time, even though the DB query takes a fraction of a second. This is sometimes overcome in games by a "fog of war" feature which doesn't bother notifying the user of every single object in the entire map, only those which are in immediate range of the player. This can easily be done with a single SQL query where object coordinates are in proximity to a player. Though, if you have a stingy host, they will care about the number of connects and queries.
If you're concerned about attracting even more players than that, consider exploring cache methods like pre-building short files storing commonly fetched records or values using fopen(), fgets(), fclose(), etc. Or, use php extensions like apc to store values in memory which persist from page load to page load. memcache or memcached also act similarly, but in a way which acts like a separate server you can connect to, store values which can be shared with other page hits, and query.
To update cached pages or values when you think they might become stale, you can run a cron job every so often to update these files or values. If your host doesn't allow cron jobs, consider making your guests do that legwork: a line of script on a certain page will refresh the cache with new values from a database query after a certain number of page hits. Or cache a date value to check against on every page hit, and if so much time has passed, refresh the cache.
Again, unless you're under the oppressive thumb of a stingy host, or unless you're getting a hundred or more page hits at a time, no need to even be concerned about your database. Databases are not that fragile. If they crashed in a hysterical fit of tears anytime more than one query came their way, the engineers who made it wouldn't have a job for very long.
I know this is quite an annoying "answer" but perhaps you should be thinking about this a different way, after all this is really not the strongest use of a relational database. Have you considered an XMPP solution? IMO this would be the best tool for the job and both ejabberd and openfire are trivial to set up these days. The excellent Strophe library can make the front end story easy, and as an added bonus you get HTTP binding (like commet) so you won't need to poll the server, your latency will go down and you'll be generating less HTTP traffic.
I know it's highly unlikely you're going to change your whole approach just cos I said so, but wanted to provide an alternative perspective.
http://www.ejabberd.im/
http://code.stanziq.com/strophe/
I use lazy connection to connect to my DB within my DB object. This basically means that it doesn't call mysql_connect() until the first query is handed to it, and it subsequently skips reconnecting from then on after.
Now I have a method in my DB class called disconnectFromDB() which pretty much calls mysql_close() and sets $_connected = FALSE (so the query() method will know to connect to the DB again). Should this be called after every query (as a private function) or externally via the object... because I was thinking something like (code is an example only)
$students = $db->query('SELECT id FROM students');
$teachers = $db->query('SELECT id FROM teachers');
Now if it was closing after every query, would this slow it down a lot as opposed to me just adding this line to the end
$db->disconnectFromDB();
Or should I just include that line above at the very end of the page?
What advantages/disadvantages do either have? What has worked best in your situation? Is there anything really wrong with forgetting to close the mySQL connection, besides a small loss of performance?
Appreciate taking your time to answer.
Thank you!
As far as I know, unless you are using persistent connections, your MySQL connection will be closed at the end of the page execution.
Therefore, you calling disconnect will add nothing and because you do the lazy connection, may cause a second connection to be created if you or another developer makes a mistake and disconnects at the wrong time.
Given that, I would just allow my connection to close automatically for me. Your pages should be executing quickly, therefore holding the connection for that small amount of time shouldn't cause any problems.
I just read this comment on PHP website regarding persistent connection and it might be interesting to know:
Here's a recap of important reasons
NOT to use persistent connections:
When you lock a table, normally it is unlocked when the connection
closes, but since persistent
connections do not close, any tables
you accidentally leave locked will
remain locked, and the only way to
unlock them is to wait for the
connection to timeout or kill the
process. The same locking problem
occurs with transactions. (See
comments below on 23-Apr-2002 &
12-Jul-2003)
Normally temporary tables are dropped when the connection closes,
but since persistent connections do
not close, temporary tables aren't so
temporary. If you do not explicitly
drop temporary tables when you are
done, that table will already exist
for a new client reusing the same
connection. The same problem occurs
with setting session variables. (See
comments below on 19-Nov-2004 &
07-Aug-2006)
If PHP and MySQL are on the same server or local network, the
connection time may be negligible, in
which case there is no advantage to
persistent connections.
Apache does not work well with persistent connections. When it
receives a request from a new client,
instead of using one of the available
children which already has a
persistent connection open, it tends
to spawn a new child, which must then
open a new database connection. This
causes excess processes which are just
sleeping, wasting resources, and
causing errors when you reach your
maximum connections, plus it defeats
any benefit of persistent connections.
(See comments below on 03-Feb-2004,
and the footnote at
http://devzone.zend.com/node/view/id/686#fn1)
(I was not the one that wrote the text above)
Don't bother disconnecting. The cost of checking $_connected before each query combined with the cost of actually calling $db->disconnectFromDB(); to do the closing will end up being more expensive than just letting PHP close the connection when it is finished with each page.
Reasoning:
1: If you leave the connection open till the end of the script:
PHP engine loops through internal array of mysql connections
PHP engine calls mysql_close() internally for each connection
2: If you close the connection yourself:
You have to check the value of $_connected for every single query. This means PHP has to check that the variable $_connected A) exists B) is a boolean and C) is true/false.
You have to call your 'disconnect' function, and function calls are one of the more expensive operations in PHP. PHP has to check that your function A) exists, B) is not private/protected and C) that you provided enough arguments to your function. It also has to create a copy of the $connection variable in the new local scope.
Then your 'disconnect' function will call mysql_close() which means PHP A) checks that mysql_close() exists and B) that you have provided all needed arguments to mysql_close() and C) that they are the correct type (mysql resource).
I might not be 100% correct here but I believe the odds are in my favour.
You may want to look at a using persistent connections. Here are two links to help you out
http://us2.php.net/manual/en/features.persistent-connections.php
http://us2.php.net/manual/en/function.mysql-pconnect.php
The basic unit of execution presumably is an entire script. What you first of all are wanting to apply resources (i.e. the database) to, efficiently and effectively, is the entirety of a single script.
However, PHP, Apache/IIS/whatever, have lives of their own; and they are capable of using the connections you open beyond the life of your script. That's the signficance of persistent (or pooled) connections.
Back to your script. It turns out you have a great deal of opportunity to be creative about using that connection during its execution.
The typical naive script will tend to hit the connection again and again, picking up locally appropriate scraps of data associated with given objects/modules/selected options. This is where procedural methodology can inflict a penalty on that connection by opening, requesting, receiving, and closing. (Note that any single query will remain alive until it is explicitly closed, or the script ends. Be careful to note that a connection and a query are not the same thing at all. Queries tie up tables; connections tie up ... connections (in most cases mapped to sockets). So you should be conscious of proper economy in the use of both.
The most economical strategy with regard to queries is to have as few as possible. I'll often try to construct a more or less complex joined query that brings back a full set of data rather than parceling out the requests in small pieces.
Using a lazy connection is probably a good idea, since you may not need the database connection at all for some script executions.
On the other hand, once it's open, leave it open, and either close it explicitly as the script ends, or allow PHP to clean up the connection - having an open connection isn't going to harm anything, and you don't want to incur the unnecessary overhead of checking and re-establishing a connection if you are querying the database a second time.