I have a webapp written in PHP that currently creates a DB connection (using mysqli_connect) on every page to pull data from the database.
Recently, my website has slowed down (with some traffic increase), and I was wondering if the creation of so many connections - one for every user that is on any page - is causing the slow down?
Is there any fix for this?
Is it possible to create one sharable connection for the server? I know this is possible in python, but I do not know how I would implement such a model in PHP.
Note: My site is on BlueHost...I don't know if that also makes a difference.
Well two things to do.
Setup the slow query log in MySQL and see if there are queries that
are slow. Optimize these slow queries by using the EXPLAIN command
to identify missing indexes etc.
Setup connection pooling to eliminate opening a connection all the
time. See this link Connection Pooling In PHP for more information.
You can prepend host by "p:" when passing it to mysqli_connect to open a persistent connection.
You can read about Persistent Connections from the links below:
http://us2.php.net/manual/en/features.persistent-connections.php
http://php.net/manual/en/mysqli.persistconns.php
http://us2.php.net/manual/en/function.mysql-pconnect.php
But I would strongly suggest NOT to use Persistent Connections.
You will need to find out what is actually slowing your website.
Ping a website that loads fast for you. eg. google.com.
Ping your webserver, if the ping difference is a lot then you should ask BlueHost to resolve it.
Check whether there is any lag between your Web Server and Database Server. They could probably be:
on the same machine (mostly no lag)
on different machines in the same local network (mostly no lag unless there is a LAN problem, ask BlueHost)
on different machines in different networks (there could be issues, ask BlueHost if they could shift the Database Server within the same Local Network)
If everything above is fine and the pages are still loading slowly.
You could try explicitly calling mysqli_close() after your DB work in the page is done. This will free up your connection immeditely instead of waiting for the page to fully execute. You will then need to figure what is slowing the page after your DB work is over.
You can use mictotime() on your slow pages to see what code is slowing it down.
You could use this link: Accurate way to measure execution times of php scripts
Related
On my website, when an user opens his profile or any other page, (almost all pages use data from mysql), my website makes around 50 connections to mysql in loading the page. This is the case on my development server, running elementary OS.
When I came to know about persistent connections in mysql, I was puzzled. If I am to run this website on a VPS (with low RAM in starting), and considering the overhead produced by a large number of mysql connections, will using persistent connections improve my website's performance?
Currently, I start and end a connection in every function. Is there a better way to connect to mysql?
And, taking into account that if 100 users are using my website simultaneously, what will be the performance if each page makes around 50-60 connections?
You asked so I will answer. You are doing this incorrectly. You should start the processing on each page request (each externally-accessible .php file) by using a common function to establish a single database connection, then you should reuse that connection.
You are getting away with this because you're probably using an automatic connection pool built in to your php database-access library, and because you have not yet scaled up your application.
You won't be able to scale this up very far using this multiconnection strategy because it will perform very badly indeed when you add users.
There are lots of examples of working open-source php-based web app systems you can look at. WordPress is an example. You'll find that most of them start by opening a database connection and storing its handle in a global variable.
You asked:
According to CBroe's comment, I changed my strategy. Actually, I am
using multiple database connections, but the functions are same (don't
ask why lol). So if I open connections on start, and then pass the
handler to the function, will that be an improvement?
Yes, that will be fine. You need to avoid churning connections to get best performance.
If you need connections to more than one different database, you can open them all. But it sounds like you need to hit just one database.
PHP doesn't have significant overhead when passing a handler to a function, so don't worry about that.
As explained wonderfully by Ollie Jones, I opened the connection on start, and my connections dropped from 50-60 per page to 1 per page. Although I don't see any change in performance on my local development server, this will surely we a great improvement when its on a live server. There is no need for me to use persistent connections yet.
Little backstory:
My VPS server is running a webbased game. (PHP/MySQL)
We also have a fairly popular forums and the database of both programs were on the same server. This server also serves the pages with Apache.
Sometimes when the forums were really busy we were seeing that the busy database affected the game also. So we've put the forum database on another server.
Now the forums are connecting to the other server with a TCP/IP connection.
The results of this were awesome, the average server load dropped a lot and the game was rarely slow.
Sometimes though, we get really high spikes of load and then I see that Apache can't serve PHP pages. MySQL's processlist is empty. A lot of Apache connections are there then (which I can see with server-status or top). The requests are normal and there is no sign of a DDoS. Traffic or CPU is not a lot higher than normal.
Graph above show one of these times. Before and after are what the server load is normally.
This occasionally happens right after we put the forum database on the other server.
Any clues where to look?
Please let me know if I need to provide more information.
It might be a bot/spider indexing your site. You can check by logging the user-agent string.
You can give bots an indication on how fast they can make requests to your site by adding
Crawl-delay: [number]
to your robots.txt.
One possibility is that there's a trend in there where people access pages with lots of external resources. External scripts, ajax requests, stylesheets, images, sounds, and sometimes included/required php pages, require new connections. A quick search revealed a question along these lines on ServerFault. A lot of new threads and/or connections running at once could create problems with memory or connectivity.
I'm implementing an application that will have a lot of clients querying lots of small data packages from my webserver. Now I'm unsure whether to use persistent data connections to the database or not. The database is currently on the same system as the webserver and could connect via the socket, but this may change in the near future.
As I know a few releases of PHP ago mysqli_pconnect was removed because it behaved suboptimally. In the meantime it seems to be back again.
Based on my scenario I suppose I won't have an other chance to handle thousands of queries per minute but with loads of persistent connections and a MySQL configuration that reserves only little resources per connection, right?
Thanks for your input!
What happened when you tested it?
With the nest will in the world, there's no practical way you can convey all the informaton required for people to provide a definitive answer in a SO response. However usually there is very little overhead in establishing a mysql connection, particularly if it resides on the same system as the database client (in this case the webserver). There's even less overhead if you use a filesystem rather than a network socket.
So I'd suggest abstracting all your database calls so you can easily switch between connection types, but write your system to use on-demand connections, and ensure you code explicitly releases the connection as soon as practical - then see how it behaves.
C.
Are PHP persistant connections evil?
The problem is there can be only so
many connections active between Host
“Apache” and Host “MySQL”
Persistant connections usually give problems in that you hit the maximum number of connections. Also, in your case it does not give a great benefit since your database server is on the same host. Keep it to normal connections for now.
As they say, your mileage may vary, but I've never had good experiences using persistent connections from PHP, including MySQL and Oracle (both ODBC and OCI8). Every time I've tested it, the system fails to reuse connections. With high load, I end up hitting the top limit while I have hundreds of idle connections.
So my advice is that you actually try it and find out whether your set-up is reusing connections properly. If it isn't working as expected, it won't be a big lose anyway: opening a MySQL connection is not particularly costly compared to other DBMS.
Also, don't forget to reset all relevant settings when appropriate (whatever session value you change, it'll be waiting for you next time to stablish a connection and happen to reuse that one).
I'm maintaining an inherited site built on Drupal. We are currently experiencing "too many connections" to the database.
In the /includes/database.mysql.inc file, #mysql_connect($url['host'], $url['user'], $url['pass'], TRUE, 2) (mysql_connect() documentation) is used to connect to the database.
Should $new_link = TRUE be used? My understanding is that it will "always open a new link." Could this be causing the "too many connections"?
Editing core is a no no. You'll forget you did, upgrade the version, and bam, changes are gone.
Drupal runs several high-performance sites without problems. For instance, the Grammy Awards site switched to Drupal this year and for the first time the site didn't go down during the cerimony! some configuration needs tweaking on your setup. Probably mysql.
Edit your my.cfg and restart your mysql server (/etc/my.cfg in fedora, RH, centos and /etc/mysql/my.cfg on *buntu)
[mysqld]
max_connections=some-number-here
alternatively, to first try the change without restarting the server, login to mysql and try:
show variables like 'max_connections' #this tells you the current number
set global max_connections=some-number-here
Oh, and like another person said: DO. NOT. EDIT. DRUPAL. CORE. It does pay off if you want to keep your site updated, may cause inflexible headache and bring you a world of hurt.
MySQL, just like any RDBMS out there will limit the amount of connections that it accepts at any time. The my.cnf configuration file specifies this value for the server under the max_connections configuration. You can change this configuration, but there are real limitations depending on the capacity of your server.
Persistent connections may help reducing the amount of time it takes to connect to the database, but it has no impact on the total amount of connections MySQL will accept.
Connect to MySQL and use 'SHOW PROCESSLIST'. It will show you the currently open connections and what they do. You might have multiple connections sitting idle or running queries that take way too long. For idle connections, it might just be a matter of making sure your code does not keep connections open when they don't need them. For the second one, they may be parts of your code that need to be optimized so that the queries don't take too long.
If all connections are legitimate, you simply have more load than your current configuration allows for. If you MySQL load is low even with the current connection count, you can increase it a little and see how it evolves.
If you are not on a dedicated server, you might not be able to do much about this. It may just be someone else's code causing trouble.
Sometimes, those failures are just temporary. When it fails, you can simply retry the connection a few milliseconds later. If it still fails, it might be a problem and stopping the script is the right thing to do. Don't put that in an infinite loop (seen it before, terrible idea).
FYI: using permanent connections with Drupal is asking for trouble. Noone uses that as far as I know.
The new_link parameter only has effect, if you have multiple calls to mysql_connect(), during 1 request, which is probably not the case here.
I suspect it is caused by too many users visiting your site, simultaneously, because for each visitor, a new connection to the DB will be made.
If you can confirm that this is the case, mysql_pconnect() might help, because your problem is not with the stress on your database server, but the number of connections. You should also read Persistent database connections to see if it is applicable to your webserver setup, if you choose to go this route.
Are there any tricks to transferring and/or sharing the Resource Links across multiple PHP pages? I do not want the server to continue connecting to the database just to obtain another link to the same user in the same database.
Remember that the link returned from mysql_connect() is automatically closed after the script it originated on completes executing. Is there anyway to close it manually at the end of each session instead?
PHP allows persistent mysql connections, but there are drawbacks. Most importantly, idle apache children end up sitting around, holding idle database connections open. Database connections take up a decent amount of memory, so you really only want them open when they're actually being used.
If your user opens one page every minute, it's far better to have the database connection closed for the 59 seconds out of every minute you're not using it, and re-open it when needed, than to hold it open continually.
Instead, you should probably look into connection pooling.
One of the advantages of Mysql over other heavier-weight database servers is that connections are cheap and quick to setup.
If you are concerned about large numbers of connections to retrieve the save information, you may like to look at such things as caching of information instead, or as well as getting the information from disk. As usual, profiling of the number, and type of calls to SQL that are being made, will tell you are great deal more than anyone here guessing at what you should really be doing next.