PHP Sockets, any way to speed this up - php

I'm writing a small script to see if certain ports on certain devices are in use or are open. My code is:
for($c=1;$c<=16;$c++){
echo "<tr><td>Pod " . $c . "</td>";
for ($d=5000;$d<=5010;$d++){
$tmp=fsockopen("10.0.0." . $c,$d,$erstr, $errno, 1);
if($tmp){
echo "<td class='o'>OPEN</td>";
fclose($tmp);
}
else{
echo "<td class='u'>IN USE</td>";
}
}
ob_flush();
flush();
echo "</tr>\n";
}
echo "</table>";
Obviously this is a lot of connections, and currently it's taking about ten seconds to run. I was wondering if there's any way I can get this to be a little bit faster? Thanks for the help!

One way to speed this up tremendously is to get asynchronous. Right now if one of the hosts is slow, it will slow down the entire pipeline because you're doing one right after another. PHP doesn't really have an event-based AIO (select), or even threading. It does, however, have fork in a linux environment. The following example hasn't been tested, but is a general idea for how to do asynchronous IO in php:
<?php
$childrenArr = array();
$childrenLeft = array();
for($c=1;$c<=16;$c++){
for ($d=5000;$d<=5010;$d++){
$pid = pcntl_fork();
if ($pid == -1) {
die("Could not fork");
} else if ($pid) {
$childrenArr[$pid] = array($c, $d);
$childrenLeft[$pid] = 1;
} else {
$tmp=fsockopen("10.0.0." . $c,$d,$erstr, $errno, 1);
exit(($tmp) ? 1 : 0);
}
}
}
$results = array();
while (count($childrenLeft) > 0) {
$oldPid = pcntl_waitpid(-1, $status, WNOHANG);
if (pcntl_wifexited($status )) {
unset($childrenLeft[$oldPid]);
list($c, $d) = $childrenArr[$oldPid];
$results[$c . "_" . $d] = pcntl_wexitstatus($status);
}
usleep(100);
}
for($c=1;$c<=16;$c++){
echo "<tr><td>Pod " . $c . "</td>";
for ($d=5000;$d<=5010;$d++){
if ($results[$c . "_" . $d]) {
echo "<td class='o'>OPEN</td>";
}
else {
echo "<td class='u'>IN USE</td>";
}
}
ob_flush();
flush();
echo "</tr>\n";
}
echo "</table>";

If a given port is not listening/accepting you will suffer all the TCP timeouts on the SYN packet re-transmissions sent out during the three-way handshake. This is the design of the TCP - we can't change that.
One thing I can recommend is switching from streams to the socket functions and trying non-blocking connect - create your 160 sockets, set them to non-blocking, initiate all connections, wait on them in select with a decreasing timeout, flag the ones that return readable. Do that in a loop until you spent, say, a whole second. Now you get a list of open TCP host/port pairs, and a list of likely closed ones, and you spent a fixed amount of time.
Disclaimer: I never do networking in PHP, so this might be totally useless.

Related

How use multi-threading with this

I've been reading up on multi-threading with PHP, but I'm having a tough time integrating it into my command line php script.
I read multithreading
and multithread foreach.
But I'm really not sure. Any thoughts how to apply multi-threading here? The reason I need multi-threading here is that Telnet takes forever (see shell script). But I can't write to my DB concurrently ($stmt2). I'm looping through my list of devices with $stmt->fetch.
Maybe I should do something like run task specifically, with just the telnet/shell script call in the task, like that example:
$task = new class extends Thread {
private $response;
public function run()
{
$content = file_get_contents("http://google.com");
preg_match("~<title>(.+)</title>~", $content, $matches);
$this->response = $matches[1];
}
};
$task->start() && $task->join();
var_dump($task->response); // string(6) "Google"
But, I'm getting the error when I try to add this to my code below:
PHP Parse error: syntax error, unexpected T_CLASS in /opt/IBM/custom/NAC_Dslam/calix_swVerThreaded.php on line 100
this is the line:
$task = new class ...
My script looks like this:
$stmt =$mysqli->prepare("SELECT ip, model FROM TableD WHERE vendor = 'Calix' AND model in ('C7','E7') AND sw_ver IS NULL LIMIT 6000"); //AND ping_reply IS NULL AND software_version IS NULL
$stmt->bind_result($ip, $model); //list of ip's
if(!$stmt->execute())
{
//err
}
$stmt2 = $mysqli2->prepare("UPDATE TableD SET sw_ver = ?
WHERE vendor = 'Calix'
AND ip = ? ");
$stmt2->bind_param("ss", $software, $ip);
while($stmt->fetch()) {
//initializing var's
if(pingAddress($ip)=="alive") { //Ones that don't ping are dead to us.
///////this is the part that takes forever and should be multi-threaded/////
//Call shell script to telnet to calix dslam and get version for that ip
if($model == "C7"){
$task = new class extends Thread {
private $itsOutput;
public function run()
{
exec ("./calix_C7_swVer.sh $ip", $itsOutput);//takes forever/telnet
//in shell script. Can't
//be fixed. Each time I
//call this script it's a
//different ip
}
};
$task->start() && $task->join();
var_dump($task->itsOutput); //should be returned output above //takes forever to telnet
//$output = $task->itsOutput;
$output2=array_reverse($output,true);
if (!(preg_grep("/DENY/", $output2))){
$found = preg_grep("/COMPLD/", $output2);
$ind = key($found);
$version = explode(",", $output[$ind+1]);
if(strlen($version[3])>=1) { //if sw ver came back in an acceptable size
$software = $version[3];
$software = trim($software,'"'); //trim double quote (usually is there)
print "sw ver after trim: " . $software . "\n";
if(!$stmt2->execute()) { //write sw version to netcool db
$tempErr = "Failed to insert into dslam_elements_nac: " . $stmt2->error;
printf($tempErr . "\n"); //show mysql execute error if exists
$err->logThis($tempErr);
}
if(!$stmtX->execute()) { //commit it
$tempErr = "Failed to commit dslam_elements_nac: " . $stmtX->error;
printf($tempErr . "\n"); //show mysql execute error if exists
$err->logThis($tempErr);
}
} //we got a version back
else { //version not retrieved
//error processing
} //didn't get sw ver
} //not deny
} //c7
else if($model == "E7") {
exec ("./calix_E7_swVer.sh $ip", $output);
$output2=array_reverse($output,true);
if (!(preg_grep("/DENY/", $output2))){
$found = preg_grep("/yes/", $output2);
$ind = key($found);
$version = explode(" ", $output[$ind]);
if(strlen($version[5])>=1) { //if sw ver came back in an acceptable size
$software = $version[5];
print "sw ver after trim: " . $software . "\n";
if(!$stmt2->execute()) { //write sw version to netcool db
$tempErr = "Failed to insert into dslam_elements_nac: " . $stmt2->error;
printf($tempErr . "\n"); //show mysql execute error if exists
$err->logThis($tempErr);
}
if(!$stmtX->execute()) { //commit it
//err processing
}
} //we got a version back
else { //version not retrieved
//handle it
} //didn't get sw ver
} //not deny
}
} //while
update
I'm trying this (pcntl_fork), but it doesn't seem to be quite what I need because when I sleep(30), which I think is similar to my shell script call, other processes don't continue and do the next one.
<?php
declare(ticks = 1);
$max=10;
$child=0;
$res = array("aabc", "bcd", "cde", "eft", "ggg", "hhh", "iii", "jjj", "kkk", "lll", "mmm", "nnn", "ooo", "ppp", "qqq", "aabc", "bcd", "cde", "eft", "ggg", "hhh", "iii", "jjj", "kkk", "lll", "mmm", "nnn", "ooo", "ppp", "qqq");
function sig_handler($signo) {
global $child;
switch ($signo) {
case SIGCHLD:
//echo "SIGCHLD receivedn";
// clean up zombies
$pid = pcntl_waitpid(-1, $status, WNOHANG);
$child -= 1;
//exit;
}
}
pcntl_signal(SIGCHLD, "sig_handler");
//$website_scraper = new scraper();
foreach($res as $r){
while ($child >= $max) {
sleep(5); //echo " - sleep $child n";
//pcntl_waitpid(0,$status);
}
$child++;
$pid=pcntl_fork();
if ($pid==-1) {
die("Could not fork:n");
}
elseif ($pid) {
// we're in the parent fork, dont do anything
}
else {
//example of what a child process could do:
print "child process stuff \n";
sleep(30);
//$website_scraper -> scraper("http://foo.com");
exit;
}
while(pcntl_waitpid(0, $status) != -1) { //////???
$status = pcntl_wexitstatus($status);
echo "child $status completed \n";
}
print "did stuff \n";
}
?>
I've been reading up on multi-threading with PHP
Don't. PHP threading has very limited utility, as it cannot be used in a web server environment. It can only be used in command-line scripts.
The author of the PHP pthreads extension has written:
pthreads v3 is restricted to operating in CLI only: I have spent many years trying to explain that threads in a web server just don't make sense, after 1,111 commits to pthreads I have realised that, my advice is going unheeded.
So I'm promoting the advice to hard and fast fact: you can't use pthreads safely and sensibly anywhere but CLI.
If you need to communicate with multiple network devices in parallel, consider using stream_select to perform asynchronous I/O, or running multiple PHP processes as part of a worker queue to manage the connections.

PHP script on host times out before finished

I spoke to the one.com support who said that their servers times out after 50 seconds.
The problem is, that it the script times out before it's finished. How can I make the script loop until it is finished?
This is my script:
<?php
//$id = $_GET['id'];
//$content = file_get_contents("http://www.boligsiden.dk/salg/$id");
$con=mysqli_connect("danico.dk.mysql","danico_dk","password hidden","danico_dk");
// Check connection
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
$result = mysqli_query($con,"SELECT * FROM elements");
$int = 0;
while($row = mysqli_fetch_array($result)) {
if($int < 500 && $row['link'] != "," && $row['link'] != "") {
$link = $row['link'];
$content = file_get_contents("http://www.boligsiden.dk/salg/$link");
preg_match('#"LatLng":{"Lat":(.*?),"Lng":#', $content, $match1);
$lat = $match1[1];
echo "<tr>";
echo "<td>" . $lat . "</td>";
echo "</tr>";
$int = $int + 1;
}
}
?>
The time limit can not be overwritten. The default value on one.com host is always 50 seconds.
Like other answers said, you should use set_time_limit( int $seconds_before_timeout ) but no need to put it in the loop:
set_time_limit(0) will cut off any timing out (except in safe mode).
You should add a caching system: once you've retried your data with file_get_contents(), you could save it to a local file (like /cache/boligsiden/salg/$link.dat) so next time, if the cache is recent (you decide what "recent" means), you'll get the local file instead of doing a long-time-consumming http request.
One solution:
<?php
$con=mysqli_connect("danico.dk.mysql","danico_dk","password hidden","danico_dk");
if (mysqli_connect_errno())
{
echo "Failed to connect to MySQL: " . mysqli_connect_error();
exit();
// else, script will continue...
// You could use a throw new Exception(...) instead
}
$result = mysqli_query($con,"SELECT * FROM elements WHERE `link`!=',' and `link`!='' LIMIT 500");
// Way easier to put the conditions on the query itself
// as your query is not reused anywhere else
set_time_limit(0); // No timeout
while($row = mysqli_fetch_array($result))
{
// No more if there since the query itself managed it faster
$link = $row['link'];
$content = file_get_contents("http://www.boligsiden.dk/salg/$link");
preg_match('#"LatLng":{"Lat":(.*?),"Lng":#', $content, $match1);
$lat = $match1[1];
echo "<tr><td>$lat</td></tr>";
}
set_time_limit(30); // From now on, script will timeout again
// That line can be removed depending on what's coming next
// Aka if you still want the script to not timeout, remove that line
?>
set_time_limit() : http://php.net/set_time_limit
You can change the time the script is allowed to run:
set_time_limit(300); // allow a maximum execution time of 5 minutes
Warning
This function has no effect when PHP is running in safe mode. There is no workaround other than turning off safe mode or changing the time limit in the php.ini.
Warning
Be careful with this option. Wrong usage can lead to scripts that never end. Putting in a definite time limit, at the top of your script is the safest way to go.

can't get stream_set_timeout to work in foreach array loop

I'm not so familiar with PHP but have been able to get a script running that will check if a certain port is open for a number of local network computers.
the downside is that some pc's are offline and I can't get the timeout of the fsockopen function lower then 1 second (0.001 second would be enough)
I've found the *stream_set_timeout* function but can't seem to get it to work.
I'm pretty sure it's just in the wrong place, and hope someone can point out where it should go.
now I get this error:
Warning: stream_set_timeout() expects parameter 1 to be resource,
boolean given
snippet:
$timeout = 1000;
foreach ($farm as $pc){
$check = #fsockopen($pc, $port);
stream_set_timeout($check,0,$timeout);
if (is_resource($check))
{
echo $pc . " online";
fclose($check);
}
else
{
echo $pc . " offline";
}
}
current solution:
foreach ($farm as $pc){
$check = #fsockopen($pc,$port,$errCode,$errStr,0.001);
if (is_resource($check))
{
echo $pc . " online";
fclose($check);
}
else
{
echo $pc . " offline";
}
}

php socket fast way to send message to multiple clients?

I'm running a php server based on the one provided by a tutorial by Raymond Fain on kirupa:
http://www.kirupa.com/developer/flash8/php5sockets_flash8_3.htm
It works great, up to a point. The thing is that when it receives certain messages, it then does some stuff to that message then sends it out to all connected clients. The problem here is that once the number of clients reaches the atmospheric heights of around 12, the loop that sends the message to all clients can take a while (like 4 seconds), and any subsequent messages sent during that 4 second period get queued up and eventually we get timeouts.
This is the loop that sends a message to all clients:
function send_Message($allclient, $socket, $buf)
{
$now = microtime(true);
echo 'sending message to '.count($allclient).' clients ';
$msg = "<mbFeed>$buf</mbFeed>\n\0";
foreach($allclient as $client)
{
socket_write($client, $msg, strlen($msg));
}
$end = microtime(true);
echo 'time was '.($end - $now);
}
You'll notice I've been echoing the time it takes by comparing microtimes. The thing is that the echo claims that the whole process takes a tiny amount of time, like 0.003 seconds, which is what I would have expected, but when I have this script running in the terminal, I see the little spinning icon going for the four seconds, during which everything gets unresponsive.
My questions are as follows: does anyone know what the script is doing during that time? Is there anything I can do it to stop it doing that? Is there a more efficient way to send a message to all connected sockets?
Here's the code for the whole socket file, if you need it, but I'm hoping this is something that might be familiar to somebody...
#!/usr/bin/php -q
<?php
/*
Raymond Fain
Used for PHP5 Sockets with Flash 8 Tutorial for Kirupa.com
For any questions or concerns, email me at ray#obi-graphics.com
or simply visit the site, www.php.net, to see if you can find an answer.
*/
//ini_set('display_errors',1);
ini_set('display_startup_errors',1);
error_reporting(E_ALL);
//ini_set('error_log', 'socket_errors.log');
ini_set('log_errors', 'On');
ini_set('display_errors', '1');
error_log('testing');
stream_set_timeout(1,0);
set_time_limit(0);
ob_implicit_flush();
$address = 'xxx.xxx.x.xxx';
$port = xxxx;
function send_Message($allclient, $socket, $buf)
{
$now = microtime(true);
echo 'sending message to '.count($allclient).' clients ';
$msg = "<mbFeed>$buf</mbFeed>\n\0";
foreach($allclient as $client)
{
socket_write($client, $msg, strlen($msg));
}
$end = microtime(true);
echo 'time was '.($end - $now);
}
echo "connecting...
";
//---- Start Socket creation for PHP 5 Socket Server -------------------------------------
if (($master = socket_create(AF_INET, SOCK_STREAM, SOL_TCP)) < 0)
{
echo "socket_create() failed, reason: " . socket_strerror($master) . "\n";
}
socket_set_option($master, SOL_SOCKET,SO_REUSEADDR, 1);
socket_set_nonblock($master);
if (($ret = socket_bind($master, $address, $port)) < 0)
{
echo "socket_bind() failed, reason: " . socket_strerror($ret) . "\n";
}
echo 'socket bind successfull.
';
if (($ret = socket_listen($master, 5)) < 0)
{
echo "socket_listen() failed, reason: " . socket_strerror($ret) . "\n";
}
$read_sockets = array($master);
echo "connected.";
//---- Create Persistent Loop to continuously handle incoming socket messages ---------------------
while (true)
{
$changed_sockets = $read_sockets;
$num_changed_sockets = socket_select($changed_sockets, $write = NULL, $except = NULL, NULL);
echo 'changed sockets length: '.(count($changed_sockets));
foreach($changed_sockets as $key => $socket)
{
if ($socket == $master)
{
if (($client = socket_accept($master)) < 0)
{
echo "socket_accept() failed: reason: " . socket_strerror($msgsock) . "\n";
continue;
}
else
{
socket_set_nonblock($client);
array_push($read_sockets, $client);
}
}
else
{
$bytes = socket_recv($socket, $buffer, 8192, 0);
if ($bytes == 0)
{
unset($read_sockets[$key]);
unset($changed_sockets[$key]);
socket_close($socket);
}
else
{
if (substr($buffer, 0, 3) == "<->")
{
unset($read_sockets[$key]);
unset($changed_sockets[$key]);
socket_close($socket);
$buffer = substr($buffer, 3);
}
$allclients = $read_sockets;
array_shift($allclients);
if (substr($buffer, 0, 3) == ":::") handleSpecial(substr($buffer, 3));
else
{
echo 'allclients length: '.(count($allclients));
send_Message($allclients, $socket, str_replace("\0","",$buffer));
}
}
}
}
}
?>
I would like to recommend you to look at ZeroMQ its like sockets on steroids
ØMQ in a Hundred Words
ØMQ (also seen as ZeroMQ, 0MQ, zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fanout, pub-sub, task distribution, and request-reply. It's fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multicore applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. ØMQ is from iMatix and is LGPLv3 open source.
using ZMQ::SOCKET_PUB and ZMQ::SOCKET_PULL the same chat server can be as simple as
$ctx = new ZMQContext();
$pub = $ctx->getSocket(ZMQ::SOCKET_PUB);
$pub->bind('tcp://*:5566');
$pull = $ctx->getSocket(ZMQ::SOCKET_PULL);
$pull->bind('tcp://*:5567');
echo "Chat Server Start ", PHP_EOL;
while(true) {
$message = $pull->recv();
echo "Got ", $message, PHP_EOL;
$pub->send($message);
}
This can easily handle10,000 push request per min on a simple system as oppose to your current server implementation.
With ZmqSocket you can talk to zmq sockets from your JavaScript code. You can connect, send and receive string messages. As JavaScript does not support raw TCP connections, it uses Flash as a bridge
Simple JavaScript Bridge Example you can also look at zmqsocket-as which allows you to talk to zmq-sockets from ActionScript code.

PHP - IRC Bot Script Hanging

I am having problem with my IRC Bot script, I have implemented it into my Curl transfer method.
I have a problem, once the IRC bot sends a message to the IRC channel, all of the "echo" at the end of the script does not show and the page hangs. The whole Apache hangs.
<?php
$ircServer = "///";
$ircPort = "6667";
$ircChannel = "#bots";
set_time_limit(0);
$msg = $_POST['msg'];
$paper = $_POST['paper'];
$sizzor = $_POST['sizzor'];
$hand = $_POST['hand'];
$ircSocket = fsockopen($ircServer, $ircPort, $eN, $eS);
if ($ircSocket)
{
fwrite($ircSocket, "USER Lost rawr.test lol :code\n");
fwrite($ircSocket, "NICK Rawr" . rand() . "\n");
fwrite($ircSocket, "JOIN " . $ircChannel . "\n");
ignore_user_abort(TRUE); // Noob Close down page
fwrite($ircSocket, "PRIVMSG " . $ircChannel . " :" . $msg . "\n");
while(1)
{
while($data = fgets($ircSocket, 128))
{
echo nl2br($data);
flush();
// Separate all data
$exData = explode(' ', $data);
// Send PONG back to the server
if($exData[0] == "PING")
{
fwrite($ircSocket, "PONG ".$exData[1]."\n");
}
}
echo $eS . ": " . $eN;
}
}
?>
if ($bootcontents == 'success') {
echo '<center><marquee behavior="alternate" direction="left">Spinning xxx at ' . $power . '% power.</marquee></center>';
This part does not show during the script:
if ($bootcontents == 'success') {
echo '<center><marquee behavior="alternate" direction="left">Spinning xxx at ' . $power . '% power.</marquee></center>';
The page just hangs, if I add the exit(); function onto near the top the whole "echo" info does not show.
Please can someone help.
You are creating an infinite loop:
while (1)
// ...
This loop can never finish, since you did not use an exit statement (like break). Therefore the code after the infinite loop is never executed.
Furthermore is it a busy loop (using a lot of CPU resources), so the whole apache (and computer) will hang.
You're leaving some lines out of the <?php ?> tags, so whatever is outside them will be treated as plain text. You fix it moving the closing ?> tags further down:
[this is the while(1) closing bracket]
}
// code past this line will never run, see below for details
echo $eS . ": " . $eN;
}
}
if ($bootcontents == 'success') {
echo '<center><marquee behavior="alternate" direction="left">Spinning xxx at ' . $power . '% power.</marquee></center>';
}
?> <!-- closing tag goes here -->
The page would anyway not work properly because the while(1) loop is missing an exit condition:
while(1) {
while($data = fgets($ircSocket, 128)) {
// ...
}
}
After the inner while finishes, your script keeps looping, ending up trapped in an empty, infinite loop (which would hang the server up, if it's not configured to detect and kill this kind of loophole).
On a final notice, PHP isn't probably the best tool for the job: you would be much better off with a stand-alone application.
while($data = fgets($ircSocket, 128))
This part blocks the script running until it receives data, and if somehow you're not getting data through that socket... well you're stuck there... forever... lol ok, stuck until the PHP script times out.
If that part doesn't catch, you're still stuck inside the while loop and so there is no way of ever running the part of your code that echos stuff out... so both apfelbox and Alex are correct, just not explained fully...
In order to have a infinite loop but also be able to run code outside, you would need to catch the "event" in which you want to capture and run code. All the events you want to capture would need to sit inside the while loop, or at least dispatched from the while loop to a function that would parse the input from server and respond correctly.
An even better way to do this is to utilise the observer pattern.
I really wouldn't make an IRC bot with PHP, even if you run it via commandline... PHP isn't meant to run as a long-running application.

Categories