In certain functions of the code, php will execute hundreds or in some cases thousands of queries on the same tables using a loop. Currently, it creates a new database connection for each query. How expensive is that operation? Would I see a significant speed increase by reusing the same connection? It could take quite a bit of refactoring to change this behavior and use the same database.
The php uses mysql_connect to connect to the database.
Just based on what I've said here, are there other obvious optimizations that you would recommend (I've read about locking tables for example...)?
EDIT:
My question is more about the benefit of using a single connection, not how to avoid using more than one.
The documentation for mysql_connect states:
If a second call is made to mysql_connect() with the same arguments, no new link will be established, but instead, the link identifier of the already opened link will be returned.
So, unless you're connecting with different credentials, changing that part of your code will not affect performance.
I use Zend_Framework and my database profiling shows that the connection itself takes nearly 10x longer than most of my queries. I have two different databases that I connect to, and only connect once to each for each request.
I'd say reconnecting for every query is poor design, but the question of refactoring is more complex than that. Questions that need to be asked:
Are there current performance problems?
Have you done code profiling to narrow down where the performance issues are occurring?
How much time will be required for this refactoring? Take into account the testing involved, not just coding time.
The answer to the original question should be obvious. If its not obvious to you then it should still be obvious how to find out for yourself how much impact it has.
are there other obvious optimizations
No - because you've not provided any details of the table's structure nor the queries you are running.
Related
I am wanting to hear what others think about this? Currently, I make a mysql database connection inside of a header type file that is then included in the top of every page of my site. I then can run as many queries as I want on that 1 open connection. IF the page is built from 6 files included and there is 15 different mysql queries, then they all would run on this 1 connection.
Now sometimes I see classes that make multiple connections, like 1 for each query.
Is there any benefit of using one method over the other? I think 1 connection is better then multiple but I could be wrong?
Creating connections can be expensive (I don't have a reference for this statement as yet Edit: Aha! Here it is) so it seems as if the consensus is to use fewer connections. Using a single connection for all queries on a single page seems to be a better choice than multiple connections.
In PHP+MySQL usually there is no much sence to use multiple connections per page (just slower and a little more RAM consumed).
The only way it might be useful is when you alter connection paremters which might interfer with other pages (like collation). But good PHP programs usually never do that kind of stuff.
Also, it is a good idea to enable persistent connections, so that 1 MySQL connection would be reused across multiples page executions.
If really depends on the level of activity you suspect the site will generate - if it's a high traffic web site, you'll soon run out of connections (unless you set the adjust MySQLs max connections to a stupidly high level, but that'll eventually grind the server to a halt).
I'd generally recommend that the front end of a web site should use a shared database object (singleton is your friend), as it doesn't require a great deal of discipline to write with this is mind and you won't waste time making connections. If you require additional concurrent queries on the backend, it shouldn't be that much of a deal as this isn't likely to be a highly trafficked area.
Its not recommended to execute multiple small queries where the work can be done using just one query, you can use a single query to get data from multiple tables and ieven multiple databases. see the link below:
http://www.x-developer.com/php-scripts/sql-connecting-multiple-databases-in-a-single-query
I don't see any benefit of using multiple connections, I 'd rather think it is a sign of bad structure. These are the reasons I can think of against using multiple connections:
You have to initialize the database multiple times. Setting conection properties upon connection establishment (like SET NAMES UTF8) would have to be done on multiple line.
It is definitely slower than a single connection.
A non-technical reason: Someone working with your code will most probably not expect it and might spend hours debugging the connection properties he had set in another connection.
Having a global connection object (or a class providing one) is the much better approach in PHP.
Are you sure the classes that make multiple connections aren't just returning a reference to the already open connection when one is open? I've seen a lot of stuff structured that way. It really is better performance-wise to use only one connection per page.
I have been digging around stack and all sorts of other sites looking for the best answer to my questions.
I am developing a very large and growing monster of a website, in the form of an information management system. At the core it is running off of PHP and MySQL. I have just updated code, in the more general sense, to mysqli, but without taking full advantage of all of the features. That is part of what I am working on now.
I have read a ton about prepared statements and this is something I certainly need to put to use given the number of statements that get re-used.
I am looking at making in the realm of about 50 prepared statements,
being used across nearly 200 different pages. Is there a recommended
way to do this? All examples I have seen deal with 1 or 2.
Due to the ever growing nature of the site, using databases and such,
one of the things that I liked with the previous mysql is that it
didn't require a connection specified for each query, but does with
mysqli. I had to tweak my functions due to this. Is there a
recommended solution for this?
I built the site in a procedural form rather that object oriented, but I am always open to suggestions, regardless of the format they use.
I'll try to be as accurate as possible, but I'm not an expert.
Your first point: You're probably looking for Stored Procedures. Basically you can store certain logic of your application for repetitive usage.
Prepared Statements, however, are different. They basically mean "Parse once, execute many times" but they're not stored on the server and carried out across connections.
In PHP, each "page load" is a different thread with its own variables and thus its connections to the database, so you cannot really use the Prepared Statement again.
As for your second point, mysql_query() doesn't require a connection handle to be passed to it simply because it assumes you want to use the last created connection.
For example:
mysql_connect();
mysql_query("SELECT * FROM table");
and
$link = mysql_connect();
mysql_query("SELECT * FROM table", $link);
are the same.
So using the connection implicitly doesn't mean scalability.
That's as far as I can write without providing possibly wrong information, so I highly recommend to you really read about this, and then if you have some question everybody here would be happy to answer.
Specifically, I have a DB class that opens and closes multiple MySQL connections every time I call the Query function in the class. Should I open a connection once? or is it ok to open and close connections like this?
My simple-minded (ISAM, no transactions) C-language app runs for eight hours a day, updating multiple tables in one database over one single MySQL connection that stays open the whole time. It works just fine. Anytime there's any kind of MySQL error (not only server gone away), the code just calls mysql_real_connect() again and it picks right up without any trouble. Reconnection is one of the places where, in my opinion, MySQL functions flawlessly.
But there's plenty of controversy and discussion about the goodness/badness of persistent connections. You can find some of it here:
http://www.google.com/webhp?hl=&sourceid=navclient-ff&rlz=1B3GGLL_enUS384US384&ie=UTF-8#rlz=1B3GGLL_enUS384US384&hl=en&source=hp&q=mysql+persistent+connection&aq=0&aqi=g4g-m5&aql=&oq=mysql+persistent+conn&gs_rfai=Ch2c6iCchTO3zG4i6MZ-i7JAOAAAAqgQFT9BAKCs&fp=ff274912d96214e6
-- HTH
If you don't want to change much instead of mysql_connect() use mysql_pconnect()
This way you will use the opened connection. Bu I would agree with #Sarfraz Ahmed - use it only once
Should I open a connection once?
No.
I thought it would be better to release the memory
Actually, connect itself do not consume memory.
And - most important part - you should not worry of such imaginable things.
Don't make decisions on based on empty assumptions.
Here is 2 simple rules to follow:
When you don't know, what to do, do it most general way, as everyone does.
Do necessary things only. Don't try to foresee every problem in the future. Deal only with present problems, not with imaginable ones. Premature optimization is the root of all evil, as it said.
Should I open a connection once? or is
it ok to open and close connections
like this?
You should open multiple connection only when needed otherwise it is not a good idea to open multiple connections thereby consuming a lot of memory which is an overhead.
In general, go back to the simplest MySQL tutorial you can find and do it just that way. Don't deviate unless you have a problem you are trying to solve.
MySQL works just fine when you keep it brain-dead simple. Don't add complexity.
BTW, are you writing yet another MySQL abstraction layer? Why? This question is a good example why reinventing a wheel can be risky.
conecting also uses cpu time. so if you reconnect about 8 times per page, and you have somewhat about 100 visitors a day wich call up 5 pages in average you have 8*100*5=4000 reconnects in one day. and thats a small website. you should realy think about connecting only once or when the connection is getting lost. that would somehow lower your electricity bill also ;-)
I think you should use a Registry Object to open your Database Connection, and make your Database Object a singleton.
Registry::Set('DB', new Database());
$DB = Registry::Get('DB');
I am wanting to hear what others think about this? Currently, I make a mysql database connection inside of a header type file that is then included in the top of every page of my site. I then can run as many queries as I want on that 1 open connection. IF the page is built from 6 files included and there is 15 different mysql queries, then they all would run on this 1 connection.
Now sometimes I see classes that make multiple connections, like 1 for each query.
Is there any benefit of using one method over the other? I think 1 connection is better then multiple but I could be wrong?
Creating connections can be expensive (I don't have a reference for this statement as yet Edit: Aha! Here it is) so it seems as if the consensus is to use fewer connections. Using a single connection for all queries on a single page seems to be a better choice than multiple connections.
In PHP+MySQL usually there is no much sence to use multiple connections per page (just slower and a little more RAM consumed).
The only way it might be useful is when you alter connection paremters which might interfer with other pages (like collation). But good PHP programs usually never do that kind of stuff.
Also, it is a good idea to enable persistent connections, so that 1 MySQL connection would be reused across multiples page executions.
If really depends on the level of activity you suspect the site will generate - if it's a high traffic web site, you'll soon run out of connections (unless you set the adjust MySQLs max connections to a stupidly high level, but that'll eventually grind the server to a halt).
I'd generally recommend that the front end of a web site should use a shared database object (singleton is your friend), as it doesn't require a great deal of discipline to write with this is mind and you won't waste time making connections. If you require additional concurrent queries on the backend, it shouldn't be that much of a deal as this isn't likely to be a highly trafficked area.
Its not recommended to execute multiple small queries where the work can be done using just one query, you can use a single query to get data from multiple tables and ieven multiple databases. see the link below:
http://www.x-developer.com/php-scripts/sql-connecting-multiple-databases-in-a-single-query
I don't see any benefit of using multiple connections, I 'd rather think it is a sign of bad structure. These are the reasons I can think of against using multiple connections:
You have to initialize the database multiple times. Setting conection properties upon connection establishment (like SET NAMES UTF8) would have to be done on multiple line.
It is definitely slower than a single connection.
A non-technical reason: Someone working with your code will most probably not expect it and might spend hours debugging the connection properties he had set in another connection.
Having a global connection object (or a class providing one) is the much better approach in PHP.
Are you sure the classes that make multiple connections aren't just returning a reference to the already open connection when one is open? I've seen a lot of stuff structured that way. It really is better performance-wise to use only one connection per page.
I've seen this question around the internet (here and here, for example), but I've never seen a good answer. Is it possible to find the length of time a given MySQL query (executed via mysql_query) took via PHP?
Some places recommend using php's microtime function, but this seems like it may be inaccurate. The mysql_query may be bogged down by network latency, or a sluggish system which isn't responding to your query quickly, or some other unrelated cause. None of these are directly related to the quality of your query, which is the only thing I really want to test out here. (Please mention in the comments if you disagree!)
My answer is similar, but varied. Record the time before and after the query, but do it within your database query class. Oh, you say you are using mysql_query directly? Well, now you know why you should use a class wrapper around those raw php database functions (pardon the snark). Actually, one is already built called PDO:
http://us2.php.net/pdo
If you want to extend the functionality to do timing around each of your queries... extend the class! Simple enough, right?
I you are only checking the quality of the query itself, then remove PHP from the equation. Use a tool like the MySQL Query Browser or SQLyog.
Or if you have shell access, just connect directly. Any of these methods will be superior in determining the actual performance of your queries.
At the php level you pretty much would need to record the time before and after the query.
If you only care about the query performance itself you can enable the slow query log in your mysql server: http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html That will log all queries longer than a specified number of seconds.
If you really need query information maybe you could make use of SHOW PROFILES:
http://dev.mysql.com/doc/refman/5.0/en/show-profiles.html
Personally, I would use a combination of microtime-ing, the slow query log, mytop, and analyzing problem queries with the MySQL client (command line).