I am running a websockets server using https://github.com/ghedipunk/PHP-Websockets/blob/master/websockets.php on an Ubuntu 16 box with PHP7
After 256 users connect to the websocket, it stops taking connections and I can't figure out why. In the client, I get a 1006 error code (connection was closed abnormally (locally) by the browser implementation) and no further information. The websockets request doesn't appear to make it to the websockets server (which normally echos "Client Connected" right after a socket connection is made).
In the connect() function, one of the things I do is echo the count of the number of users, sockets and overall memory usage to the log. This problem occurs whenever the user count hits 256 (at which point the socket count is 257 and memory usage around 4Mb). The fact that it happens at 256 makes me think that a limit is being hit somewhere, but I can't find that limit. If I restart the websockets server, everything works fine again.
From my investigation so far, I have tried and checked:
ulimit (says it's unlimited)
MySQL connection limit (was set to default, now is 1000, but that didn't help)
Increased the PHP memory limit (because why not, just to see)
SOMAXCONN is set to 128, so I don't think this is the problem, but I would have to recompile PHP to test it. I haven't tried this yet.
Apache: The message I get when the problem occurs is: [proxy:error] [pid 16785] (111)Connection refused: AH00957: WS: attempt to connect to 10.0.0.240:9000 (websockets.mydomain.com) failed, which doesn't tell me much about anything. Apache is running MPM prefork and I have increased the spare servers and MaxRequestWorkers
I am open to any suggestions as to where to look next or how to get more detail out of the "Connection Refused" error log from apache!
Thanks
It's Apache that holding you up. Try setting the following in your conf file...
MaxClients 512
ServerLimit 512
(you must set both)
Of course, you can use whatever numbers work for you. In mpm-prefork, you should be able to go to 20,000 but that really shouldn't be necessary.
I've done quite a bit of reading before asking this, so let me preface by saying I am not running out of connections, or memory, or cpu, and from what I can tell, I am not running out of file descriptors either.
Here's what PHP throws at me when MySQL is under heavy load:
Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (11 "Resource temporarily unavailable")
This happens randomly under load - but the more I push, the more frequently php throws this at me. While this is happening I can always connect locally through the console and from PHP through 127.0.0.1 instead of "localhost" which uses the faster unix socket.
Here's a few system variables to weed out the usual problems:
cat /proc/sys/fs/file-max = 4895952
lsof | wc -l = 215778 (during "outages")
Highest usage of available connections: 26% (261/1000)
InnoDB buffer pool / data size: 10.0G/3.7G (plenty o room)
soft nofile 999999
hard nofile 999999
I am actually running MariaDB (Server version: 10.0.17-MariaDB MariaDB Server)
These results are generated both under normal load, and by running mysqlslap during off hours, so, slow queries are not an issue - just high connections.
Any advice? I can report additional settings/data if necessary - mysqltuner.pl says everything is a-ok
and again, the revealing thing here is that connecting via IP works just fine and is fast during these outages - I just can't figure out why.
Edit: here is my my.ini (some values may seem a bit high from my recent troubleshooting changes, and please keep in mind that there are no errors in the MySQL logs, system logs, or dmesg)
socket=/var/lib/mysql/mysql.sock
skip-external-locking
skip-name-resolve
table_open_cache=8092
thread_cache_size=16
back_log=3000
max_connect_errors=10000
interactive_timeout=3600
wait_timeout=600
max_connections=1000
max_allowed_packet=16M
tmp_table_size=64M
max_heap_table_size=64M
sort_buffer_size=1M
read_buffer_size=1M
read_rnd_buffer_size=8M
join_buffer_size=1M
innodb_log_file_size=256M
innodb_log_buffer_size=8M
innodb_buffer_pool_size=10G
[mysql.server]
user=mysql
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
open-files-limit=65535
Most likely it is due to net.core.somaxconn
What is the value of /proc/sys/net/core/somaxconn
net.core.somaxconn
# The maximum number of "backlogged sockets". Default is 128.
Connections in the queue which are not yet connected. Any thing above that queue will be rejected. I suspect this in your case. Try increasing it according to your load.
as root user run
echo 1024 > /proc/sys/net/core/somaxconn
This is something that can and should be solved by analysis. Learning how to do this is a great skill to have.
Analysis to find out just what is happening under a heavy load...number of queries, execution time should be your first step. Determine the load and then make the proper db config settings. You might find you need to optimize the sql queries instead!
Then make sure the PHP db driver settings are in alignment as well to fully utilize the database connections.
Here is a link to the MariaDB threadpool documentation. I know it says version 5.5, but its still relevant and the page does reference version 10. There are settings listed that may not be in your .cnf file that you can use.
https://mariadb.com/kb/en/mariadb/threadpool-in-55/
From the top of my head, I can think of max_connections as a possible source of the problem. I'd increase the limit, to at least eliminate the possibility.
Hope it helps.
I'm trying to upload a CSV into a mysql database using phpmyadmin
When I try with a shortened version of the database, the process works ok, but when I try with the full database, I get the error:
#2006 - MySQL server has gone away
The section of my CSV that is working is:
trans_id,price_paid,date,postcode,property_type,poperty_type_2,hold,add_num,add_flat,add_road,add_area,add_city,add_borough,add_county,add_rand
{33C588EE-BB09-4F6F-BA8C-000312C72B3B},159950,23/05/2014 00:00,SL6 9LX,F,N,L,2,,THE SHAW,COOKHAM,MAIDENHEAD,WINDSOR AND MAIDENHEAD,WINDSOR AND MAIDENHEAD,A
{2C650B8C-57C0-421C-A4A9-00037BDFDCFB},158000,30/05/2014 00:00,NN14 1RJ,T,N,F,4,,MIDLAND COTTAGES,RUSHTON,KETTERING,KETTERING,NORTHAMPTONSHIRE,A
{74FA45D0-CB64-40E1-94C4-00055AEBF72C},470000,30/05/2014 00:00,KT20 5SF,D,N,F,11,,CHAPEL ROAD,,TADWORTH,REIGATE AND BANSTEAD,SURREY,A
{054AB14B-0EED-48FD-B3CD-0005B154A5C3},135000,23/05/2014 00:00,NR27 9AZ,F,N,L,48,,ALBANY COURT,,CROMER,NORTH NORFOLK,NORFOLK,A
{86896E40-68BA-4BA2-8468-0006258B9C41},124995,09/05/2014 00:00,L24 9NA,S,Y,L,131,,ADDENBROOKE DRIVE,SPEKE,LIVERPOOL,LIVERPOOL,MERSEYSIDE,A
{A948BD6F-DD91-4DE9-82D1-0008226FC360},95000,13/06/2014 00:00,HU6 7XE,S,N,F,51,,DOWNFIELD AVENUE,,HULL,CITY OF KINGSTON UPON HULL,CITY OF KINGSTON UPON HULL,A
{7191F69F-7648-4603-9CE7-000882808E16},174000,19/05/2014 00:00,DT5 1HX,T,N,F,2,,LONG ACRE,,PORTLAND,WEYMOUTH AND PORTLAND,DORSET,A
{525BE511-1351-475F-9765-0009645D0B60},328000,11/06/2014 00:00,TW18 2EP,T,N,F,1,,EDGELL ROAD,,STAINES-UPON-THAMES,SPELTHORNE,SURREY,A
I've tried:
increasing the max_packet=64M in /etc/my.cnf to 64M, and the wait_timeout= 1000 but no luck.
I've also made the same changes for the packet size limit on the php.ini but no luck.
Any help would be appreciated
Thanks,
Mo
Check out the documentation on MySQL error #2006. I've included some of the more likely possibilities below:
You have encountered a timeout on the server side and the automatic reconnection in the client is disabled (the reconnect flag in the MYSQL structure is equal to 0).
You can also get these errors if you send a query to the server that is incorrect or too large. If mysqld receives a packet that is too large or out of order, it assumes that something has gone wrong with the client and closes the connection. If you need big queries (for example, if you are working with big BLOB columns), you can increase the query limit by setting the server's max_allowed_packet variable, which has a default value of 1MB. You may also need to increase the maximum packet size on the client end. More information on setting the packet size is given in Section B.5.2.10, “Packet Too Large”. This likely isn't the issue, as you've already done the suggested solution.
You also get a lost connection if you are sending a packet 16MB or larger if your client is older than 4.0.8 and your server is 4.0.8 and above, or the other way around.
There's also an error specific for Windows applications, so if you're using Windows, check that out:
You are using a Windows client and the server had dropped the connection (probably because wait_timeout expired) before the command was issued. The problem on Windows is that in some cases MySQL does not get an error from the OS when writing to the TCP/IP connection to the server, but instead gets the error when trying to read the answer from the connection. Prior to MySQL 5.0.19, even if the reconnect flag in the MYSQL structure is equal to 1, MySQL does not automatically reconnect and re-issue the query as it doesn't know if the server did get the original query or not. The solution to this is to either do a mysql_ping() on the connection if there has been a long time since the last query (this is what Connector/ODBC does) or set wait_timeout on the mysqld server so high that it in practice never times out.
I have a site which is hosted on a dev site for demonstration to the client, and everything works without problem. However, when I download the files and database to my local EasyPHP installation, I receive the following error:
Fatal error: Maximum execution time of
30 seconds exceeded in C:\Program
Files (x86)\EasyPHP-5.3.4.0\www\PC
Estimating\classes\database.class.php
on line 23
The database details for the connection are correct, as the Database object is already used on part of the template before this error is shown.
My question is, why does the system work fine on a live server, but not on EasyPHP?
You should check the max_execution_time setting in the php.ini files on your server and on your local installation.
btw... what is done in line 23 ?
Copied from my comment to make it easier to find the solution:
some things really runs slower on windows... while on mac/unix the php connects to mysql using a file socket while it should use tcpip in windows. Try using "127.0.0.1" instead of "localhost" when connecting to the db
These issue have two possible solutions:
1) Increase the max_execution_time in your php.ini . First of all, locate this file, and then edit it. Locate this line:
max_execution_time=30
and replace by:
max_execution_time=120
And then restart your webserver.
This will increase from 30 seconds to 120 seconds. You can increase even more, according to your application needs.
2) If this setting doesn't solve the problem, you may have to look into your PHP application, because there may be an infinite loop or something similar.
More details about this problem:
https://www.copahost.com/blog/increase-php-max-execution-time/
Because your PC is slow compared to server and/or your code is really badly optimised
I have page listing few records from db.
After upgrading to PHP 5.3 site printing long records list is not displayed - Explorer says "Connection was reset"
I've changed SQL query in code to limit records and then page was shown correctly
So it seems to be some kind of timeout set.
I've tried find some settings in PHP.ini , HTTPD.conf - changed all sounds similar to timeout but nothing happened.
Any idea how to make it working ?
EDIT
Page resets after ~2 secs - so there is no extremely long time....
EDIT-2
I've tried set php vars: max_execution_time, max_input_time, memory_limit
WAMPServer 2 (PHP 5.3, Apache 2.2.11)
At the top of your .php file, insert something like:
set_time_limit(120);
That sets the timeout for the script to 2 minutes. Increase it as you need.
I would recommend avoiding this problem by paginating your results, otherwise you're opening yourself up to a world of trouble. Slow pages are an open door for a denial of service attack by resource exhaustion.