I'm working on a site that connects to many URLs at once, (we are hoping to get to ~600 per minute) and no matter what I try, there are always thousands of TIME_WAIT still open. I understand these are vital to TCP connections, but they are using all the available ports. PHP doesn't have a SO_REUSEPORT, and SO_REUSEADDR doesn't work with remote connections. Here is the beginning of the code:
$s = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_set_option($s, SOL_SOCKET, SO_LINGER,array('l_linger'=>0, 'l_onoff'=>0)); //I have tried l_onoff=1
socket_set_option($s, SOL_SOCKET, SO_RCVTIMEO,array('sec'=>0,'usec'=>500000));
socket_set_option($s, SOL_SOCKET, SO_SNDTIMEO,array('sec'=>0,'usec'=>500000));
socket_set_option($s, SOL_SOCKET, SO_KEEPALIVE,0);
socket_set_option($s, SOL_SOCKET, SO_REUSEADDR,1);
socket_set_nonblock($s);
socket_bind($s,$ip,0);
socket_connect($s,$host,$port);
The $s goes into an array contains all the pending writes, after a write we call socket_shutdown($s,1); to close writing on the socket. Then after a read we:
socket_shutdown($s,2); socket_close($s);
All reading and writing is done in a while loop, the loop has a max of 12 concurrent connections, if that is not hit, then it moves on and proceeds to add another URL to the array. Every loop calls socket_select with a timeout of 0.
Does anyone have any suggestions? I would like to increase the speed as well as reduce the TIME_WAIT's that appear in netstat.
Thanks,
James Hartig
You could send the HTTP header Connection: close along with your request which would cause the server to send a TCP FIN after it sends you your request. Because the other side sends the first FIN, it'll be the other side that gets to wait around in TIME_WAIT, not you.
If you want to reduce TIME_WAIT on Linux, you can modify a proc setting:
echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
This will change the timeout to 30 seconds. Note that TIME_WAIT is a part of standard TCP/IP behavior - unless you are really starved for resources, I wouldn't muck with this much.
Related
I have a function which creates a socket connection and listens on a port number for HL7 messages sent by a laboratory machine via TCP.
If the lab machine is not sending anything, my listen function keeps listening. Is there a way to specify that it should listen only for say 10 seconds and then if there are no messages, an error should be thrown?
$address = '0.0.0.0';
$port = 5600;
// Create a TCP Stream socket
$sock = socket_create(AF_INET, SOCK_STREAM, 0);
// Bind the socket to an address/port
$bind = socket_bind($sock, $address, $port);
// Start listening for connections
socket_listen($sock);
$client = socket_accept($sock);
// Read the input from the client
$input = socket_read($client, 2024);
// Strip all white spaces from input
$segs = explode("|",$input);
// Close the master sockets
$close = socket_close($sock);
This is the solution:
socket_set_option($sock,SOL_SOCKET,SO_RCVTIMEO,array("sec"=>10,"usec"=>0)); // after 10 seconds socket will destroy the connection. Also you can set and uses
This looks like the XY problem.
That the thing you want to measure acts as a client rather implies that you might want to do more than just detect an open TCP connection in your script, e.g. capture some data. Further, the underlying OS has a lot of complex, well tested, reliable and tunable mechanisms for tracking the state of connections.
While you could do as stefo91 suggests and try to manipulate receive timeout, I'm not sure if this is applied in the wait for an initial connection. A better solution would be to set the socket to non-blocking. Don't forget to either:
inject some calls to sleep()/usleep() or
use socket_select()
unless you want your script to be burning a lot of resource with nothing to do.
But depending on a lot of information you've not told us about, the right solution might be to run one script as a server, and a second as a monitor. The second could be polling/parsing the output of netstat to check the connection.
I have a nginx server running a php script 24/7 that receives messages via sockets and stores them using mysql.
Sometimes I have to stop it running for a reason or another and I need a way to have it restart as soon as possible in order to avoid losing messages.
I use this script (.sh) to stop and run the process again:
pkill -f socketserver.php
#<Line that frees the tcp port>
/usr/bin/php /var/www/webroot/ROOT/socketserver.php #uses tcp port: 20491
If I don't use the command that frees the tcp port, I have (and will have) to wait something between 2 and 5 minutes until the OS perceives there is no process using that tcp port and frees/releases it.
Should I use: fuser, tcpkill, what?
I prefer things that are already installed on the ubuntu server 14.04
Thank you.
You cannot forcibly "free" a TCP port. To remove the obligatory wait time, the socket server should set the SO_REUSEADDR socket option when it creates the listening socket.
This way you'll still lose messages if the server is busy when you restart it. A better solution is doing what other people do: have a "master" process create and bind the listening socket, and delegate accepting connections and working on them to a child process. Most reasons you have for restarting, like reloading config, can be done by restarting the child only, while the listening socket stays open for connections.
Building on #Joni's answer, you'll want to use the socket_set_option() function before binding your socket (http://php.net/manual/en/function.socket-set-option.php - and yes, this code is shamelessly copy/pasted from that page):
if (!socket_set_option($socket, SOL_SOCKET, SO_REUSEADDR, 1)) {
echo 'Unable to set option on socket: '. socket_strerror(socket_last_error()) . PHP_EOL;
}
Once you use this, subsequent restarts of the script should work fine, without the need to manually free the socket.
I'm trying to make a simple listener on port 8195. When I try the following code block in PHP CLI conditions, it only shows 'Test' once, then hangs. If I delete the file 'votifier.run', the file designed to be the on/off switch, it still continues to hang. It never shows 'Client connected'.
Furthermore, if I try to connect to the host via Telnet on port 8195 while the script is running, I simply get a connection failed message. It's like it's looking for one connection and just not giving up.
// Set the IP and port to listen to
$address = 'localhost';
$port = 8195;
// Create a TCP Stream socket
$sock = socket_create(AF_INET, SOCK_STREAM, 0);
// Bind the socket to an address/port
socket_bind($sock, $address, $port);
// Start listening for connections
socket_listen($sock);
// Loop continuously
while ( file_exists('votifier.run') ) {
echo 'Test';
$client = socket_accept($sock);
if( $client ) {
echo 'Client connected';
// Don't hang on slow connections
socket_set_timeout($client, 5);
// Send them our version
socket_write("VOTIFIER MCWEBLINK\n");
// Read the 256 byte block
$block = socket_read($client, 256);
...
The answer:
socket_accept() will usually hang until a connection is made. If a connection attempt was made, the script would continue, but because the socket was being created on localhost, it would only accept connections to it from localhost.
The fix is to use your external IP rather than 'localhost' or '127.0.0.1'. Then you can Telnet to it.
I'm just guessing here, but could it be that the address you are trying to bind to should not be a hostname?
If the socket is of the AF_INET family, the address is an IP in dotted-quad notation (e.g. 127.0.0.1).
EDIT
Ok, I've taken your script and tried to reproduce your error but couldn't. There are a couple of flaws in it but none that would cause a telnet client's connection attempt to fail.
Since none of the aforementioned applies, let's go thru the checklist one by one:
sockets module loaded/compiled
localhost does resolve to 127.0.0.1
the port isn't taken by any other application running
there's no rule of any sort of firewall that would prevent communication between the telnet client and your server
the machine which you connect from is allowed to connect to the server host (try the same host if it isn't)
the file that's being checked in the while-loop does exist
you are sure that there isn't another fatal error within your script that would prevent the snippet you posted from running
These are all the possible error sources I can think of, atm. Try fixing up the minor flaws first, then go thru the checklist.
if( $client ) {
echo 'Client connected';
// Don't hang on slow connections
socket_set_option(
$client,
SOL_SOCKET,
SO_RCVTIMEO | SO_SNDTIMEO,
array('sec' => 5, 'usec' => 0)
);
// Send them our version
socket_write($client, "VOTIFIER MCWEBLINK\n");
^^^^^^^
// Read the 256 byte block
$block = socket_read($client, 256);
You should be using threads. If the client never sends anything your code will block in the read() method. Each accepted socket should be completely handled in a new thread.
You may want to check this:
PHP Votifier example for Minecraft Topsites
It explains how the code works, it's the basic function that makes the encryption, fills up the 256 blank spaces and sends the packet too. You can work a little with it as you may want to improve it.
You can see a live demo of the running php for the plugin here: http://topg.org/test_votifier
I have a small application I created to analyze the network connection. It runs from a browser and connects to a local PHP/Apache server. It then asks PHP to send a ping packet through a raw socket. THe problem is that if the host I am trying to ping isn't alive or won't answer to pings, we never get an answer from the server.
I beleave the socket request lives until apache is restarted. I have been getting mixed results from my application lately and I am blaming apache using too many sockets. Currently I have set the AJAX call's timeout and I was happy with it. But I really need to make PHP do the timeouting so that I won't have 500,000 sockets open to an unreachable host.
Some sample code:
$sockconn = #socket_connect($socket, $target, null);
if(!$sockconn)
{
$raw['error'] = socket_strerror(socket_last_error());
$raw['status'] = false;
return $raw;
}
This is the function that won't timeout. I need to get it to timeout. Also PHP script execution time DOES NOT affect sockets.
I am clueless.
You can set timeouts for reading and sending using the following options:
socket_set_option($socket, SOL_SOCKET, SO_RCVTIMEO, array('sec' => 1, 'usec' => 0));
socket_set_option($socket, SOL_SOCKET, SO_SNDTIMEO, array('sec' => 1, 'usec' => 0));
Alternatively, you can use non-blocking sockets and periodically poll the socket to see if the remote host responded.
Try setting default_socket_timeout.
I tryed to create a socket in php and reuse it from other process.
I know this can be done with a daemon script but I want to do this without.
I created a socket and binded it to a specific port.
$sock = socket_create (AF_INET, SOCK_STREAM, SOL_TCP);
socket_set_option ($sock, SOL_SOCKET, SO_REUSEADDR, 1);
socket_bind ($sock, 'xx.xx.xx.xx', 10000);
socket_connect ($sock, $host, $port);
And from another php file I did the same thing. But the packets that I send from the 2 file are not "validated" by host. I sniffed all ports and I see that it uses same local and destination port. I don't understand where is the problem.
Can you help me with this?
It's ok in any other programming language, or any other solution for this.
Andrew
Sockets are not symmetrical. The server side listens on a specific port for a client to conect - the client does not specify the local port - only the remote port and address. Its nothing to do with the language you implement it in.
There's a very good socket server implementation available at http://www.phpclasses.org/browse/package/5758.html with examples.
C.
You can't really use persistent sockets in php. When you execute a php file, a new process is created which cannot access the variables - or sockets - of a different php process so it won't know if there already exists a socket and just creates it.