Little backstory:
My VPS server is running a webbased game. (PHP/MySQL)
We also have a fairly popular forums and the database of both programs were on the same server. This server also serves the pages with Apache.
Sometimes when the forums were really busy we were seeing that the busy database affected the game also. So we've put the forum database on another server.
Now the forums are connecting to the other server with a TCP/IP connection.
The results of this were awesome, the average server load dropped a lot and the game was rarely slow.
Sometimes though, we get really high spikes of load and then I see that Apache can't serve PHP pages. MySQL's processlist is empty. A lot of Apache connections are there then (which I can see with server-status or top). The requests are normal and there is no sign of a DDoS. Traffic or CPU is not a lot higher than normal.
Graph above show one of these times. Before and after are what the server load is normally.
This occasionally happens right after we put the forum database on the other server.
Any clues where to look?
Please let me know if I need to provide more information.
It might be a bot/spider indexing your site. You can check by logging the user-agent string.
You can give bots an indication on how fast they can make requests to your site by adding
Crawl-delay: [number]
to your robots.txt.
One possibility is that there's a trend in there where people access pages with lots of external resources. External scripts, ajax requests, stylesheets, images, sounds, and sometimes included/required php pages, require new connections. A quick search revealed a question along these lines on ServerFault. A lot of new threads and/or connections running at once could create problems with memory or connectivity.
Related
On my website, when an user opens his profile or any other page, (almost all pages use data from mysql), my website makes around 50 connections to mysql in loading the page. This is the case on my development server, running elementary OS.
When I came to know about persistent connections in mysql, I was puzzled. If I am to run this website on a VPS (with low RAM in starting), and considering the overhead produced by a large number of mysql connections, will using persistent connections improve my website's performance?
Currently, I start and end a connection in every function. Is there a better way to connect to mysql?
And, taking into account that if 100 users are using my website simultaneously, what will be the performance if each page makes around 50-60 connections?
You asked so I will answer. You are doing this incorrectly. You should start the processing on each page request (each externally-accessible .php file) by using a common function to establish a single database connection, then you should reuse that connection.
You are getting away with this because you're probably using an automatic connection pool built in to your php database-access library, and because you have not yet scaled up your application.
You won't be able to scale this up very far using this multiconnection strategy because it will perform very badly indeed when you add users.
There are lots of examples of working open-source php-based web app systems you can look at. WordPress is an example. You'll find that most of them start by opening a database connection and storing its handle in a global variable.
You asked:
According to CBroe's comment, I changed my strategy. Actually, I am
using multiple database connections, but the functions are same (don't
ask why lol). So if I open connections on start, and then pass the
handler to the function, will that be an improvement?
Yes, that will be fine. You need to avoid churning connections to get best performance.
If you need connections to more than one different database, you can open them all. But it sounds like you need to hit just one database.
PHP doesn't have significant overhead when passing a handler to a function, so don't worry about that.
As explained wonderfully by Ollie Jones, I opened the connection on start, and my connections dropped from 50-60 per page to 1 per page. Although I don't see any change in performance on my local development server, this will surely we a great improvement when its on a live server. There is no need for me to use persistent connections yet.
I have a webapp written in PHP that currently creates a DB connection (using mysqli_connect) on every page to pull data from the database.
Recently, my website has slowed down (with some traffic increase), and I was wondering if the creation of so many connections - one for every user that is on any page - is causing the slow down?
Is there any fix for this?
Is it possible to create one sharable connection for the server? I know this is possible in python, but I do not know how I would implement such a model in PHP.
Note: My site is on BlueHost...I don't know if that also makes a difference.
Well two things to do.
Setup the slow query log in MySQL and see if there are queries that
are slow. Optimize these slow queries by using the EXPLAIN command
to identify missing indexes etc.
Setup connection pooling to eliminate opening a connection all the
time. See this link Connection Pooling In PHP for more information.
You can prepend host by "p:" when passing it to mysqli_connect to open a persistent connection.
You can read about Persistent Connections from the links below:
http://us2.php.net/manual/en/features.persistent-connections.php
http://php.net/manual/en/mysqli.persistconns.php
http://us2.php.net/manual/en/function.mysql-pconnect.php
But I would strongly suggest NOT to use Persistent Connections.
You will need to find out what is actually slowing your website.
Ping a website that loads fast for you. eg. google.com.
Ping your webserver, if the ping difference is a lot then you should ask BlueHost to resolve it.
Check whether there is any lag between your Web Server and Database Server. They could probably be:
on the same machine (mostly no lag)
on different machines in the same local network (mostly no lag unless there is a LAN problem, ask BlueHost)
on different machines in different networks (there could be issues, ask BlueHost if they could shift the Database Server within the same Local Network)
If everything above is fine and the pages are still loading slowly.
You could try explicitly calling mysqli_close() after your DB work in the page is done. This will free up your connection immeditely instead of waiting for the page to fully execute. You will then need to figure what is slowing the page after your DB work is over.
You can use mictotime() on your slow pages to see what code is slowing it down.
You could use this link: Accurate way to measure execution times of php scripts
I was recently asked by a client to load test their server to see if it could handle 10,000 concurrent users. To do this I've been using JMeter but getting less than favorable results.
Let me just say that this is my first time using jmeter so I'm not super sure of what Im doing, BUT here's what I've found.
On a test of 1000 concurrent users all launched at once and each user going to 2 pages, the failure rate is 96%. This seems bad...like really really bad.
Is there something that could possibly be going wrong in JMeter? All I'm doing is sending HTTP GET requests to their server.
I don't know what kind of plan the client is on but I do know that they are using GoDaddy as their provider and in my experience GoDaddy's "unlimited" bandwidth is rather limited. Is this the problem OR and I'm really hoping that this is the case, is the Apache server for the website blocking the repeated attempts.
I get an error saying org.apahe.http.com.HttpHostConnectException: Connection to ~~~.com refused.
Is this the server being smart?
Or the server being bogged down?
Thanks in advance for your help, let me know if you need any more information.
Apache can't protect you from ddos attacks, but you can use some modules to reduce risks, they are: mod_qos and mod_evasive.
If you are using shared hosting from GoDaddy, seems you are loading all websites in one server and Godaddy may block your site or they may treat your load testing as ddos attack. For experiments you need isolated VDS server or cloud server.
If you want protect your project you can:
Use load balancer
Use caching tool
Use firewall protection
OS tuning
Use CDN
Here it gets a little complicated. I'm in the last few months to finish a larger Webbased Project, and since I'm trying to keep the budget low (and learn some stuff myself) I'm not touching an Issue that I never touched before: load balancing with NGINX, and scalability for the future.
The setup is the following:
1 Web server
1 Database server
1 File server (also used to store backups)
Using PHP 5.4< over fastCGI
Now, all those servers should be 'scalable' - in the sense that I can add a new File Server, if the Free Disk Space is getting low, or a new Web Server if I need to handle more requests than expected.
Another thing is: I would like to do everything over one domain, so that the access to differend backend servers isnt really noticed in the frontend (some backend servers are basically called via subdomain - for example: the fileserver, over 'http://file.myserver.com/...' where a load balancing only between the file servers happens)
Do I need an additional, separate Server for load balancing? Or can I just use one of the web servers? If yes:
How much power (CPU / RAM) do I require for such a load-balancing server? Does it have to be the same like the webserver, or is it enough to have a 'lighter' server for that?
Does the 'load balancing' server have to be scalable too? Will I need more than one if there are too many requests?
How exactly does the whole load balancing work anyway? What I mean:
I've seen many entries stating, that there are some problems like session handling / synchronisation on load balanced systems. I could find 2 Solutions that maybe would fit my needs: Either the user is always directed to the same machine, or the data is stored inside a databse. But with the second, I basically would have to rebuild parts of the $_SESSION functionality PHP already has, right? (How do I know what user gets wich session, are cookies really enough?)
What problems do I have to expect, except the unsynchronized sessions?
Write scalable code - that's a sentence I read a lot. But in terms of PHP, for example, what does it really mean? Usually, the whole calculations for one user happens on one server only (the one where NGINX redirected the user at) - so how can PHP itself be scalable, since it's not actually redirected by NGINX?
Are different 'load balancing' pools possible? What I mean is, that all fileservers are in a 'pool' and all web servers are in a 'pool' and basically, if you request an image on a fileserver that has too much to do, it redirects to a less busy fileserver
SSL - I'll only need one certificate for the balance loading server, right? Since the data always goes back over the load balancing server - or how exactly does that work?
I know it's a huge question - basically, I'm really just searching for some advices / and a bit of a helping hand, I'm a bit lost in the whole thing. I can read snippets that partially answer the above questions, but really 'doing' it is completly another thing. So I already know that there wont be a clear, definitive answer, but maybe some experiences.
The end target is to be easily scalable in the future, and already plan for it ahead (and even buy stuff like the load balancer server) in time.
You can use one of web servers for load balacing. But it'll be more reliable to set the balacing on a separate machine. If your web servers responds not very quickly and you're getting many requests then load balancer will set the requests in the queue. For the big queue you need a sufficient amount of RAM.
You don't generally need to scale a load balancer.
Alternatively, you can create two or more A (address) records for your domain, each pointing to different web server's address. It'll give you a 'DNS load-balancing' without a balancing server. Consider this option.
This may seem like an obvious question but we have a PHP/MySQL app that runs on Windows 2008 server. The server has about 10 different sites running from it in total. Admin options on the site in question allow an administrator to run reports (through the site) which are huge and can take about 10mins in some cases. These reports are huge mysql queries that display the data on screen. When these reports are running the entire site goes slow for all users. So my questions are:
Is there a simple way to allocate server resources so if a (website) administrator runs reports, other users can still access the site without performance issues?
Even though running the report kills the website for all users of that site, it doesn't affect other sites on the same server. Why is that?
As mentioned, the report can take about 10 minutes to generate - is
it bad practice to make these kinds of reports available on the
website? Would these typically be generated by overnight scheduled tasks?
Many thanks in advance.
The load your putting on the server will most likely have nothing to do with the applications but the mysql table that you are probably slamming. Most people get around this by generating reports in down time or using mysql replication to have a second database which is used purely for reporting.
I recommend trying to get some server monitoring to see what is actually going on. I think Newrelic just released windows versions of its platform and you can try it out for free for 30 days i think.
There's the LOW_PRIORITY flag, but I'm not sure whether that would have any positive effect, since it's most likely a table / row locking issue that you're experiencing. You can get an idea of what's going on by using the SHOW PROCESSLIST; query.
If other websites run fine, it's even more likely that this is due to database locks (causing your web processes to wait for the lock to get released).
Lastly, it's always advisable to run big reporting queries overnight (or when the server load is minimal). Having a read replicated slave would also help.
I strongly suggest you install a replicated MySQL server, then running large administrator queries (SELECT only naturally) on it, to avoid the burden of having your website blocked!
If there's not too much transaction per second, you could even run the replica on a desktop computer remotely from your production server, and thus have a backup off-site of your DB!
Are 100% sure you have added all necessary indexes?
You need to have a insanely large website to have this kinds of problems unless you are missing indexes.
Make sure you have the right indexing and make sure you do not have connection fields of varchar, not very fast.
I have a database with quite a few large tables and millions of records that is working 24/7.
Has loads of activity and automated services processing it without issues due to proper indexing.