PHP Sockets - Accept multiple connections - php

I'm trying to create a simple client/server application and thus I am experimenting with sockets in PHP.
Now I have a simple client in C# which connects to the server well, but i can only connect one client at once to this server (I found this code sample online and tweaked it a bit for testing purposes).
Funny enough I found the same question, based on the same example here: https://stackoverflow.com/questions/10318023/php-socket-connections-cant-handle-multiple-connection
I tried to understand every part of it and I'm close to seeing how it works in detail, but for some reason, when I connect a 2nd client, the first one gets disconnected / crashes.
Can anyone give me some wild ideas or a pointer to where I should look at?
<?php
// Set time limit to indefinite execution
set_time_limit (0);
// Set the ip and port we will listen on
$address = '127.0.0.1';
$port = 9000;
$max_clients = 10;
// Array that will hold client information
$client = array();
// Create a TCP Stream socket
$sock = socket_create(AF_INET, SOCK_STREAM, 0);
// Bind the socket to an address/port
socket_bind($sock, $address, $port) or die('Could not bind to address');
// Start listening for connections
socket_listen($sock);
// Loop continuously
while (true) {
// Setup clients listen socket for reading
$read[0] = $sock;
for ($i = 0; $i < $max_clients; $i++)
{
if (isset($client[$i]))
if ($client[$i]['sock'] != null)
$read[$i + 1] = $client[$i]['sock'] ;
}
// Set up a blocking call to socket_select()
$ready = socket_select($read, $write = NULL, $except = NULL, $tv_sec = NULL);
/* if a new connection is being made add it to the client array */
if (in_array($sock, $read)) {
for ($i = 0; $i < $max_clients; $i++)
{
if (!isset($client[$i])) {
$client[$i] = array();
$client[$i]['sock'] = socket_accept($sock);
echo("Accepting incoming connection...\n");
break;
}
elseif ($i == $max_clients - 1)
print ("too many clients");
}
if (--$ready <= 0)
continue;
} // end if in_array
// If a client is trying to write - handle it now
for ($i = 0; $i < $max_clients; $i++) // for each client
{
if (isset($client[$i]))
if (in_array($client[$i]['sock'] , $read))
{
$input = socket_read($client[$i]['sock'] , 1024);
if ($input == null) {
// Zero length string meaning disconnected
echo("Client disconnected\n");
unset($client[$i]);
}
$n = trim($input);
if ($n == 'exit') {
echo("Client requested disconnect\n");
// requested disconnect
socket_close($client[$i]['sock']);
}
if(substr($n,0,3) == 'say') {
//broadcast
echo("Broadcast received\n");
for ($j = 0; $j < $max_clients; $j++) // for each client
{
if (isset($client[$j]))
if ($client[$j]['sock']) {
socket_write($client[$j]['sock'], substr($n, 4, strlen($n)-4).chr(0));
}
}
} elseif ($input) {
echo("Returning stripped input\n");
// strip white spaces and write back to user
$output = ereg_replace("[ \t\n\r]","",$input).chr(0);
socket_write($client[$i]['sock'],$output);
}
} else {
// Close the socket
if (isset($client[$i]))
echo("Client disconnected\n");
if ($client[$i]['sock'] != null){
socket_close($client[$i]['sock']);
unset($client[$i]);
}
}
}
} // end while
// Close the master sockets
echo("Shutting down\n");
socket_close($sock);
?>

The current top answer here is wrong, you don't need multiple threads to handle multiple clients. You can use non-blocking I/O and stream_select / socket_select to process messages from clients that are actionable. I'd recommend using the stream_socket_* functions over socket_*.
While non-blocking I/O works quite fine, you can't make any function calls with involve blocking I/O, otherwise that blocking I/O blocks the complete process and all clients hang, not just one.
That means all I/O has to be non-blocking or guaranteed to be very fast (which isn't perfect, but might be acceptable). Because not only your sockets need to use stream_select, but you need to select on all open streams, I'd recommend a library that offers to register read and write watchers that are executed once a stream becomes readable / writable.
There are multiple such frameworks available, the most common ones are ReactPHP and Amp. The underlying event loops are pretty similar, but Amp offers a few more features on that side.
The main difference between the two is the approach for APIs. While ReactPHP uses callbacks everywhere, Amp tries to avoid them by using coroutines and optimizing its APIs for such a usage.
Amp's "Getting Started" guide is basically exactly about this topic. You can read the full guide here. I'll include a working example below.
<?php
require __DIR__ . "/vendor/autoload.php";
// Non-blocking server implementation based on amphp/socket.
use Amp\Loop;
use Amp\Socket\ServerSocket;
use function Amp\asyncCall;
Loop::run(function () {
$uri = "tcp://127.0.0.1:1337";
$clientHandler = function (ServerSocket $socket) {
while (null !== $chunk = yield $socket->read()) {
yield $socket->write($chunk);
}
};
$server = Amp\Socket\listen($uri);
while ($socket = yield $server->accept()) {
asyncCall($clientHandler, $socket);
}
});
Loop::run() runs the event loop and watches for timer events, signals and actionable streams, which can be registered with Loop::on*() methods. A server socket is created using Amp\Socket\listen(). Server::accept() returns a Promise which can be used to await new client connections. It executes a coroutine once a client is accepted that reads from the client and echo's the same data back to it. For more details, refer to Amp's documentation.

This script is working perfectly for me
<?php
/*! #class SocketServer
#author Navarr Barnier
#abstract A Framework for creating a multi-client server using the PHP language.
*/
class SocketServer
{
/*! #var config
#abstract Array - an array of configuration information used by the server.
*/
protected $config;
/*! #var hooks
#abstract Array - a dictionary of hooks and the callbacks attached to them.
*/
protected $hooks;
/*! #var master_socket
#abstract resource - The master socket used by the server.
*/
protected $master_socket;
/*! #var max_clients
#abstract unsigned int - The maximum number of clients allowed to connect.
*/
public $max_clients = 10;
/*! #var max_read
#abstract unsigned int - The maximum number of bytes to read from a socket at a single time.
*/
public $max_read = 1024;
/*! #var clients
#abstract Array - an array of connected clients.
*/
public $clients;
/*! #function __construct
#abstract Creates the socket and starts listening to it.
#param string - IP Address to bind to, NULL for default.
#param int - Port to bind to
#result void
*/
public function __construct($bind_ip,$port)
{
set_time_limit(0);
$this->hooks = array();
$this->config["ip"] = $bind_ip;
$this->config["port"] = $port;
$this->master_socket = socket_create(AF_INET, SOCK_STREAM, 0);
socket_bind($this->master_socket,$this->config["ip"],$this->config["port"]) or die("Issue Binding");
socket_getsockname($this->master_socket,$bind_ip,$port);
socket_listen($this->master_socket);
SocketServer::debug("Listenting for connections on {$bind_ip}:{$port}");
}
/*! #function hook
#abstract Adds a function to be called whenever a certain action happens. Can be extended in your implementation.
#param string - Command
#param callback- Function to Call.
#see unhook
#see trigger_hooks
#result void
*/
public function hook($command,$function)
{
$command = strtoupper($command);
if(!isset($this->hooks[$command])) { $this->hooks[$command] = array(); }
$k = array_search($function,$this->hooks[$command]);
if($k === FALSE)
{
$this->hooks[$command][] = $function;
}
}
/*! #function unhook
#abstract Deletes a function from the call list for a certain action. Can be extended in your implementation.
#param string - Command
#param callback- Function to Delete from Call List
#see hook
#see trigger_hooks
#result void
*/
public function unhook($command = NULL,$function)
{
$command = strtoupper($command);
if($command !== NULL)
{
$k = array_search($function,$this->hooks[$command]);
if($k !== FALSE)
{
unset($this->hooks[$command][$k]);
}
} else {
$k = array_search($this->user_funcs,$function);
if($k !== FALSE)
{
unset($this->user_funcs[$k]);
}
}
}
/*! #function loop_once
#abstract Runs the class's actions once.
#discussion Should only be used if you want to run additional checks during server operation. Otherwise, use infinite_loop()
#param void
#see infinite_loop
#result bool - True
*/
public function loop_once()
{
// Setup Clients Listen Socket For Reading
$read[0] = $this->master_socket;
for($i = 0; $i < $this->max_clients; $i++)
{
if(isset($this->clients[$i]))
{
$read[$i + 1] = $this->clients[$i]->socket;
}
}
// Set up a blocking call to socket_select
if(socket_select($read,$write = NULL, $except = NULL, $tv_sec = 5) < 1)
{
// SocketServer::debug("Problem blocking socket_select?");
return true;
}
// Handle new Connections
if(in_array($this->master_socket, $read))
{
for($i = 0; $i < $this->max_clients; $i++)
{
if(empty($this->clients[$i]))
{
$temp_sock = $this->master_socket;
$this->clients[$i] = new SocketServerClient($this->master_socket,$i);
$this->trigger_hooks("CONNECT",$this->clients[$i],"");
break;
}
elseif($i == ($this->max_clients-1))
{
SocketServer::debug("Too many clients... :( ");
}
}
}
// Handle Input
for($i = 0; $i < $this->max_clients; $i++) // for each client
{
if(isset($this->clients[$i]))
{
if(in_array($this->clients[$i]->socket, $read))
{
$input = socket_read($this->clients[$i]->socket, $this->max_read);
if($input == null)
{
$this->disconnect($i);
}
else
{
SocketServer::debug("{$i}#{$this->clients[$i]->ip} --> {$input}");
$this->trigger_hooks("INPUT",$this->clients[$i],$input);
}
}
}
}
return true;
}
/*! #function disconnect
#abstract Disconnects a client from the server.
#param int - Index of the client to disconnect.
#param string - Message to send to the hooks
#result void
*/
public function disconnect($client_index,$message = "")
{
$i = $client_index;
SocketServer::debug("Client {$i} from {$this->clients[$i]->ip} Disconnecting");
$this->trigger_hooks("DISCONNECT",$this->clients[$i],$message);
$this->clients[$i]->destroy();
unset($this->clients[$i]);
}
/*! #function trigger_hooks
#abstract Triggers Hooks for a certain command.
#param string - Command who's hooks you want to trigger.
#param object - The client who activated this command.
#param string - The input from the client, or a message to be sent to the hooks.
#result void
*/
public function trigger_hooks($command,&$client,$input)
{
if(isset($this->hooks[$command]))
{
foreach($this->hooks[$command] as $function)
{
SocketServer::debug("Triggering Hook '{$function}' for '{$command}'");
$continue = call_user_func($function,&$this,&$client,$input);
if($continue === FALSE) { break; }
}
}
}
/*! #function infinite_loop
#abstract Runs the server code until the server is shut down.
#see loop_once
#param void
#result void
*/
public function infinite_loop()
{
$test = true;
do
{
$test = $this->loop_once();
}
while($test);
}
/*! #function debug
#static
#abstract Outputs Text directly.
#discussion Yeah, should probably make a way to turn this off.
#param string - Text to Output
#result void
*/
public static function debug($text)
{
echo("{$text}\r\n");
}
/*! #function socket_write_smart
#static
#abstract Writes data to the socket, including the length of the data, and ends it with a CRLF unless specified.
#discussion It is perfectly valid for socket_write_smart to return zero which means no bytes have been written. Be sure to use the === operator to check for FALSE in case of an error.
#param resource- Socket Instance
#param string - Data to write to the socket.
#param string - Data to end the line with. Specify a "" if you don't want a line end sent.
#result mixed - Returns the number of bytes successfully written to the socket or FALSE on failure. The error code can be retrieved with socket_last_error(). This code may be passed to socket_strerror() to get a textual explanation of the error.
*/
public static function socket_write_smart(&$sock,$string,$crlf = "\r\n")
{
SocketServer::debug("<-- {$string}");
if($crlf) { $string = "{$string}{$crlf}"; }
return socket_write($sock,$string,strlen($string));
}
/*! #function __get
#abstract Magic Method used for allowing the reading of protected variables.
#discussion You never need to use this method, simply calling $server->variable works because of this method's existence.
#param string - Variable to retrieve
#result mixed - Returns the reference to the variable called.
*/
function &__get($name)
{
return $this->{$name};
}
}
/*! #class SocketServerClient
#author Navarr Barnier
#abstract A Client Instance for use with SocketServer
*/
class SocketServerClient
{
/*! #var socket
#abstract resource - The client's socket resource, for sending and receiving data with.
*/
protected $socket;
/*! #var ip
#abstract string - The client's IP address, as seen by the server.
*/
protected $ip;
/*! #var hostname
#abstract string - The client's hostname, as seen by the server.
#discussion This variable is only set after calling lookup_hostname, as hostname lookups can take up a decent amount of time.
#see lookup_hostname
*/
protected $hostname;
/*! #var server_clients_index
#abstract int - The index of this client in the SocketServer's client array.
*/
protected $server_clients_index;
/*! #function __construct
#param resource- The resource of the socket the client is connecting by, generally the master socket.
#param int - The Index in the Server's client array.
#result void
*/
public function __construct(&$socket,$i)
{
$this->server_clients_index = $i;
$this->socket = socket_accept($socket) or die("Failed to Accept");
SocketServer::debug("New Client Connected");
socket_getpeername($this->socket,$ip);
$this->ip = $ip;
}
/*! #function lookup_hostname
#abstract Searches for the user's hostname and stores the result to hostname.
#see hostname
#param void
#result string - The hostname on success or the IP address on failure.
*/
public function lookup_hostname()
{
$this->hostname = gethostbyaddr($this->ip);
return $this->hostname;
}
/*! #function destroy
#abstract Closes the socket. Thats pretty much it.
#param void
#result void
*/
public function destroy()
{
socket_close($this->socket);
}
function &__get($name)
{
return $this->{$name};
}
function __isset($name)
{
return isset($this->{$name});
}
}
Source on github

Typically socket servers need to be multi-threaded if you want to handle > 1 client. You'd create a 'listen' thread and spawn a new 'answer' thread for each client request. Im not sure how PHP would handle a situation like this though. Perhaps it has a fork mechanism?
EDIT: Doesn't appear that PHP offers threading per se (http://stackoverflow.com/questions/70855/how-can-one-use-multi-threading-in-php-applications) If you want to follow the typical paradigm for a socket server you might get away with using 'popen' to spawn a process to handle the child request. Hand off the socket id and let it close itself when the child socket closes. You'd need to keep on top of this list to avoid orphaning these processes if your server process closes.
FWIW: here are some examples of multi-client servers: http://php.net/manual/en/function.socket-accept.php

I found this online. but I would like to share this code on here. as I don't find else where.
<?php
// port number
$port = 5000;
// IP address
$address = '127.0.0.1';
// Maximum client number
$max_clients_number = 10;
// Create master stream sockets.
$master_stream_socket = socket_create(AF_INET, SOCK_STREAM, 0);
// Bind the socket to IP address and Port number.
socket_bind($master_stream_socket, $address, $port);
// Start to listen for the client.
socket_listen($master_stream_socket);
// This variable will hold client informations.
$clients = [$master_stream_socket];
while(true){
$read = $clients;
if( socket_select($read, $write = null, $exp = null, null) ){
if( in_array( $master_stream_socket, $read ) ){
$c_socket = socket_accept($master_stream_socket);
$clients[] = $c_socket;
$key = array_search($master_stream_socket, $read);
unset( $read[ $key ] );
}
if( count($read) > 0 ) {
foreach( $read as $current_socket ) {
$content = socket_read($current_socket, 2048);
foreach( $clients as $client ) {
if( $client != $master_stream_socket && $client != $current_socket ){
socket_write($client, $content, strlen($content));
}
}
}
}
} else {
continue;
}
}
// Close master sockets.
socket_close($master_stream_socket);
?>

Check This
git clone https://github.com/lukaszkujawa/php-multithreaded-socket-server.git socketserver
cd socketserver
php server.php
for more information go to: http://systemsarchitect.net/multi-threaded-socket-server-in-php-with-fork/

Related

Integrating PHP, SSH and ssh-agent

I plan to write a PHP script that makes an SSH connection. I've investigated how to do this and this looks the most promising solution: https://github.com/phpseclib/phpseclib My only issue is how to handle the fact that my SSH key has a passphrase, and I don't want to have to enter it every time I run the script. For every day SSH use I have ssh-agent running in the background, and it's configured to use pinentry. This makes it so that I don't have to enter my passphrase EVERY time. Any ideas as to how I could get PHP and ssh-agent to talk to each other? My only clue is that ssh-agent sets an environment variable, SSH_AUTH_SOCK, pointing to a socket file.
While the documentation for phpseclib addresses this issue, its answer is silly (just put the passphrase in the code): http://phpseclib.sourceforge.net/ssh/2.0/auth.html#encrsakey
UPDATE: I've looked more into phpseclib and written my own simple wrapper class. However, I cannot get it to login either through ssh-agent or by supplying my RSA key. Only password-based authentication works, contrary to my experiences logging in directly with the ssh command. Here is my code:
<?php
// src/Connection.php
declare(strict_types=1);
namespace MyNamespace\PhpSsh;
use phpseclib\System\SSH\Agent;
use phpseclib\Net\SSH2;
use phpseclib\Crypt\RSA;
use Exception;
class Connection
{
private SSH2 $client;
private string $host;
private int $port;
private string $username;
/**
* #param string $host
* #param int $port
* #param string $username
*
* #return void
*/
public function __construct(string $host, int $port,
string $username)
{
$this->host = $host;
$this->port = $port;
$this->username = $username;
$this->client = new SSH2($host, $port);
}
/**
* #return bool
*/
public function connectUsingAgent(): bool
{
$agent = new Agent();
$agent->startSSHForwarding($this->client);
return $this->client->login($this->username, $agent);
}
/**
* #param string $key_path
* #param string $passphrase
*
* #return bool
* #throws Exception
*/
public function connectUsingKey(string $key_path, string $passphrase = ''): bool
{
if (!file_exists($key_path)) {
throw new Exception(sprintf('Key file does not exist: %1$s', $key_path));
}
if (is_dir($key_path)) {
throw new Exception(sprintf('Key path is a directory: %1$s', $key_path));
}
if (!is_readable($key_path)) {
throw new Exception(sprintf('Key file is not readable: %1$s', $key_path));
}
$key = new RSA();
if ($passphrase) {
$key->setPassword($passphrase);
}
$key->loadKey(file_get_contents($key_path));
return $this->client->login($this->username, $key);
}
/**
* #param string $password
*
* #return bool
*/
public function connectUsingPassword(string $password): bool
{
return $this->client->login($this->username, $password);
}
/**
* #return void
*/
public function disconnect(): void
{
$this->client->disconnect();
}
/**
* #param string $command
* #param callable $callback
*
* #return string|false
*/
public function exec(string $command, callable $callback = null)
{
return $this->client->exec($command, $callback);
}
/**
* #return string[]
*/
public function getErrors(): array {
return $this->client->getErrors();
}
}
And:
<?php
// test.php
use MyNamespace\PhpSsh\Connection;
require_once(__DIR__ . '/vendor/autoload.php');
(function() {
$host = '0.0.0.0'; // Fake, obviously
$username = 'user'; // Fake, obviously
$connection = new Connection($host, 22, $username);
$connection_method = 'AGENT'; // or 'KEY', or 'PASSWORD'
switch($connection_method) {
case 'AGENT':
$connected = $connection->connectUsingAgent();
break;
case 'KEY':
$key_path = getenv( 'HOME' ) . '/.ssh/id_rsa.pub';
$passphrase = trim(fgets(STDIN)); // Pass this in on command line via < key_passphrase.txt
$connected = $connection->connectUsingKey($key_path, $passphrase);
break;
case 'PASSWORD':
default:
$password = trim(fgets(STDIN)); // Pass this in on command line via < password.txt
$connected = $connection->connectUsingPassword($password);
break;
}
if (!$connected) {
fwrite(STDERR, "Failed to connect to server!" . PHP_EOL);
$errors = implode(PHP_EOL, $connection->getErrors());
fwrite(STDERR, $errors . PHP_EOL);
exit(1);
}
$command = 'whoami';
$result = $connection->exec($command);
echo sprintf('Output of command "%1$s:"', $command) . PHP_EOL;
echo $result . PHP_EOL;
$command = 'pwd';
$result = $connection->exec($command);
echo sprintf('Output of command "%1$s:"', $command) . PHP_EOL;
echo $result . PHP_EOL;
$connection->disconnect();
})();
The SSH2 class has a getErrors() method, unfortunately it wasn't logging any in my case. I had to debug the class. I found that, whether using ssh-agent or passing in my key, it always reached this spot (https://github.com/phpseclib/phpseclib/blob/2.0.23/phpseclib/Net/SSH2.php#L2624):
<?php
// vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php: line 2624
extract(unpack('Ctype', $this->_string_shift($response, 1)));
switch ($type) {
case NET_SSH2_MSG_USERAUTH_FAILURE:
// either the login is bad or the server employs multi-factor authentication
return false;
case NET_SSH2_MSG_USERAUTH_SUCCESS:
$this->bitmap |= self::MASK_LOGIN;
return true;
}
Obviously the response being returned is of type NET_SSH2_MSG_USERAUTH_FAILURE. There's nothing wrong with the login, of that I'm sure, so per the comment in the code that means the host (Digital Ocean) must use multi-factor authentication. Here's where I'm stumped. What other means of authentication am I lacking? This is where my understanding of SSH fails me.
phpseclib supports SSH Agent. eg.
use phpseclib\Net\SSH2;
use phpseclib\System\SSH\Agent;
$agent = new Agent;
$ssh = new SSH2('localhost');
if (!$ssh->login('username', $agent)) {
throw new \Exception('Login failed');
}
Updating with your latest edits
UPDATE: I've looked more into phpseclib and written my own simple wrapper class. However, I cannot get it to login either through ssh-agent or by supplying my RSA key. Only password-based authentication works, contrary to my experiences logging in directly with the ssh command.
What do your SSH logs look like with both Agent authentication and direct public key authentication? You can get the SSH logs by doing define('NET_SSH2_LOGGING', 2) at the top and then echo $ssh->getLog() after the authentication has failed.
Per neubert, what I had to do was add this line to Connection.php and I was able to get agent-based authentication to work:
$this->client->setPreferredAlgorithms(['hostkey' => ['ssh-rsa']]);
I still can't get key-based authentication to work, but I don't care about that as much.

PHP Warning: ftp_get(): Could not open data connection to port 39997: Connection timed out

I hope i will explain my problem clearly because i feel lost and don't know what is real problem.
Using Symfony 2.8 and https://github.com/iJanki/FtpBundle i'm trying to connect to ftp server and download files. Whole proccess will be launched only through symfony's app/console.
Here's my service which contain whole logic:
<?php
namespace xxx\Services\Import;
use Doctrine\ORM\EntityManager;
use Ijanki\Bundle\FtpBundle\Ftp;
use Ijanki\Bundle\FtpBundle\Exception\FtpException;
use xxx\Entity\Product;
use xxx\Entity\ProductImage;
class ImageImport
{
const FTP_HOST = 'xxx';
const FTP_USER = 'xxx';
const FTP_PASSWORD = 'xxx';
const FTP_PNG_DIR = 'xxx';
const FTP_FILES_EXT = 'png';
/**
*
* #var EntityManager
*/
private $em;
/**
*
* #var string
*/
private $rootDir;
/**
*
* #var Ftp
*/
private $ftp;
/**
*
* #param EntityManager $em
* #param string $rootDir
* #param Ftp $ftp
*/
public function __construct(EntityManager $em, $rootDir, Ftp $ftp)
{
$this->em = $em;
$this->rootDir = $rootDir;
$this->ftp = $ftp;
}
public function import()
{
set_time_limit(10000);
ini_set('max_input_time', 10000);
$this->connectToFtp();
$this->copyAndMatch();
$this->disconnectFtp();
}
/**
*
* #return boolean
*/
public function connectToFtp()
{
try {
$this->ftp->connect(self::FTP_HOST, 21, 10000);
$this->ftp->login(self::FTP_USER, self::FTP_PASSWORD);
$this->ftp->chdir(self::FTP_PNG_DIR);
$this->ftp->pasv(false);
return true;
} catch (FtpException $e) {
echo 'Error: ', $e->getMessage();
return false;
}
}
public function copyAndMatch()
{
$matches = $this->getCsvMatches();
$this->matchWithProducts($matches);
}
public function disconnectFtp()
{
$this->ftp->close();
}
public function __destruct()
{
try {
if (false !== $this->ftp->pwd()) {
$this->ftp->close();
}
} catch (FtpException $ex) {
}
}
/**
*
* #return array|null
*/
private function getCsvMatches()
{
//...
}
/**
*
* #param array $matches
* #return true|null
*/
private function matchWithProducts(array $matches)
{
if (empty($matches) || !is_array($matches)) {
return;
}
foreach ($matches as $pair) {
//...
$filename = $pair['image'] . '.' . self::FTP_FILES_EXT;
if (false === $this->fileAlreadyAdded($product, $filename) and null !== $this->downloadFile($filename)) {
//...
}
}
}
$this->em->flush();
return true;
}
/**
*
* #param Product $product
* #param string $filename
* #return boolean
*/
private function fileAlreadyAdded(Product $product, $filename)
{
//...
}
/**
*
* #param string $filename
* #return string|null
*/
private function downloadFile($filename)
{
try {
$localFile = $this->rootDir . '/../' . ImportHelper::UPLOAD_DIR . '/' . $filename;
if ($this->remoteFileExists($filename)) {
if (false === $this->ftp->pwd()) {
$this->connectToFtp();
}
$this->ftp->get($localFile, $filename, FTP_BINARY);
}
return file_exists($localFile) ? $filename : null;
} catch (FtpException $ex) {
echo $ex->getMessage();
return null;
}
}
/**
*
* #param string $filename
* #return boolean
*/
private function remoteFileExists($filename)
{
return $this->ftp->size($filename) !== -1;
}
}
Commented out or removed code includes Doctrine operations and are not relevant as they don't cause any problems.
As local enviroment I use Xampp with Apache2 server on Windows 7. Also i test FTP connection with Filezilla and i discovered that it works only when using ftp active mode.
Also I discovered that my Windows firewall was reason for warnings and I could connect but do nothing more when it was enabled. So I added exceptions for php.exe and apache httpd.exe so everything works perfectly when php script is launched on my local server.
With Windows firewall enabled, ftp_nlist returned false when using passive mode, and returned warnings when using active mode.
But my goal is to launch that script on production web server which is on VPS which has Ubuntu 14.04.3 LTS installed. My knowledge about Unix is very poor but I was assured by VPS provider that there's no firewall enabled on this server but i get all the time warnings:
PHP Warning: ftp_get(): Could not open data connection to port 39997: Connection timed out
Same warnings that i got when my windows firewall was enabled.
I also called person who's administrating server with FTP (with files i want to dl) and asked him to add exceptions in firewall for ftp connection on port 21 but it didn't help at all.
I'm sure i don't get dicsonnected from ftp server, i never get FtpException. As I can see, download proccess starts because file appears in destination folder but it's empty.
And my problem is that i don't know which side of connection is causing this issues. And what can i do to fix my code to work properly.
Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere multiport dports 7727,http
ACCEPT icmp -- anywhere anywhere icmp echo-request
ACCEPT all -- anywhere anywhere
that is output from sudo iptables -L
output from sudo ufw status was Status: inactive
You know what fixed my problem? sudo ufw enable
I still dont know why it's a fix and what was the problem. But it works now.
"In active mode FTP the client connects from a random unprivileged port (N > 1023) to the FTP server's command port, port 21. Then, the client starts listening to port N+1 and sends the FTP command PORT N+1 to the FTP server. The server will then connect back to the client's specified data port from its local data port, which is port 20."
Looks like you have to add this 39997 port to firewall exceptions.
sudo ufw allow out 39997/tcp
sudo ufw allow in 39997/tcp
sudo ufw reload
I'm not sure if this bundle could start to use some other than "39997" port. But looks like it is constant in this case.

How to avoid race hazard with multiple requests?

In order to protect script form race hazard, I am considering approach described by code sample
$file = 'yxz.lockctrl';
// if file exists, it means that some other request is running
while (file_exists($file))
{
sleep(1);
}
file_put_contents($file, '');
// do some work
unlink($file);
If I go this way, is it possible to create file with same name simultaneously from multiple requests?
I know that there is php mutex. I would like to handle this situation without any extensions (if possible).
Task for the program is to handle bids in auctions application. I would like to process every bid request sequentially. With most possible latency.
From what I understand you want to make sure only a single process at a time is running a certain piece of code. A mutex or similar mechanism could be used for this. I myself use lockfiles to have a solution that works on many platforms and doesn't rely on a specific library only available on Linux etc.
For that, I have written a small Lock class. Do note that it uses some non-standard functions from my library, for instance, to get where to store temporary files etc. But you could easily change that.
<?php
class Lock
{
private $_owned = false;
private $_name = null;
private $_lockFile = null;
private $_lockFilePointer = null;
public function __construct($name)
{
$this->_name = $name;
$this->_lockFile = PluginManager::getInstance()->getCorePlugin()->getTempDir('locks') . $name . '-' . sha1($name . PluginManager::getInstance()->getCorePlugin()->getPreference('EncryptionKey')->getValue()).'.lock';
}
public function __destruct()
{
$this->release();
}
/**
* Acquires a lock
*
* Returns true on success and false on failure.
* Could be told to wait (block) and if so for a max amount of seconds or return false right away.
*
* #param bool $wait
* #param null $maxWaitTime
* #return bool
* #throws \Exception
*/
public function acquire($wait = false, $maxWaitTime = null) {
$this->_lockFilePointer = fopen($this->_lockFile, 'c');
if(!$this->_lockFilePointer) {
throw new \RuntimeException(__('Unable to create lock file', 'dliCore'));
}
if($wait && $maxWaitTime === null) {
$flags = LOCK_EX;
}
else {
$flags = LOCK_EX | LOCK_NB;
}
$startTime = time();
while(1) {
if (flock($this->_lockFilePointer, $flags)) {
$this->_owned = true;
return true;
} else {
if($maxWaitTime === null || time() - $startTime > $maxWaitTime) {
fclose($this->_lockFilePointer);
return false;
}
sleep(1);
}
}
}
/**
* Releases the lock
*/
public function release()
{
if($this->_owned) {
#flock($this->_lockFilePointer, LOCK_UN);
#fclose($this->_lockFilePointer);
#unlink($this->_lockFile);
$this->_owned = false;
}
}
}
Usage
Now you can have two process that run at the same time and execute the same script
Process 1
$lock = new Lock('runExpensiveFunction');
if($lock->acquire()) {
// Some expensive function that should only run one at a time
runExpensiveFunction();
$lock->release();
}
Process 2
$lock = new Lock('runExpensiveFunction');
// Check will be false since the lock will already be held by someone else so the function is skipped
if($lock->acquire()) {
// Some expensive function that should only run one at a time
runExpensiveFunction();
$lock->release();
}
Another alternative would be to have the second process wait for the first one to finish instead of skipping the code.
$lock = new Lock('runExpensiveFunction');
// Process will now wait for the lock to become available. A max wait time can be set if needed.
if($lock->acquire(true)) {
// Some expensive function that should only run one at a time
runExpensiveFunction();
$lock->release();
}
Ram disk
To limit the number of writes to your HDD/SSD with the lockfiles you could crate a RAM disk to store them in.
On Linux you could add something like the following to /etc/fstab
tmpfs /mnt/ramdisk tmpfs nodev,nosuid,noexec,nodiratime,size=1024M 0 0
On Windows you can download something like ImDisk Toolkit and create a ramdisk with that.

How to listen to a TCP port using PHP? [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I have a GPS Tracker that connects and send data to a defined public server:port through GPRS connection.
I can define the ip:port of the GPS device
My question is, can I just open a port in my server and listen/save the data received using PHP?
Thanks.
Edit/Update Aug. 16, 2017 :
User and library author <#Navarr> has commented that he has released a new, updated version of the library the code in my original answer was based from. A link to the new code on his github here. Feel free to explore the new code and refer back to the original example here for insight (I have no personally explored/used the new code).
The below code will use the SocketServer.class.php file found here. It is meant to be run as a standalone process which means under Linux I had to make the file executable, then run it from command line using "php my_server.php".
For more information on running php scripts from command line:
http://www.funphp.com/?p=33
First grab the SocketServer.class.php file here:
http://www.phpclasses.org/browse/file/31975.html
Try this to make use of it, then tweak it to handle receiving your own incoming data as needed. Hope it helps.
<?php
require_once("SocketServer.class.php"); // Include the File
$server = new SocketServer("192.168.1.6",31337); // Create a Server binding to the given ip address and listen to port 31337 for connections
$server->max_clients = 10; // Allow no more than 10 people to connect at a time
$server->hook("CONNECT","handle_connect"); // Run handle_connect every time someone connects
$server->hook("INPUT","handle_input"); // Run handle_input whenever text is sent to the server
$server->infinite_loop(); // Run Server Code Until Process is terminated.
function handle_connect(&$server,&$client,$input)
{
SocketServer::socket_write_smart($client->socket,"String? ","");
}
function handle_input(&$server,&$client,$input)
{
// You probably want to sanitize your inputs here
$trim = trim($input); // Trim the input, Remove Line Endings and Extra Whitespace.
if(strtolower($trim) == "quit") // User Wants to quit the server
{
SocketServer::socket_write_smart($client->socket,"Oh... Goodbye..."); // Give the user a sad goodbye message, meany!
$server->disconnect($client->server_clients_index); // Disconnect this client.
return; // Ends the function
}
$output = strrev($trim); // Reverse the String
SocketServer::socket_write_smart($client->socket,$output); // Send the Client back the String
SocketServer::socket_write_smart($client->socket,"String? ",""); // Request Another String
}
Edit: In keeping things relevant and functional for this answer I found it best not to continue to rely on code from an external source that may not always remain available (or at the given URL provided in my link). Therefore, for convenience, I am adding below the code that corresponds to the SocketServer.class.php file I linked to at the top of this post. (Apologies for length and possible lack of indentation/formatting while copy/pasting, I am not the author of the code below).
<?php
/*! #class SocketServer
#author Navarr Barnier
#abstract A Framework for creating a multi-client server using the PHP language.
*/
class SocketServer
{
/*! #var config
#abstract Array - an array of configuration information used by the server.
*/
protected $config;
/*! #var hooks
#abstract Array - a dictionary of hooks and the callbacks attached to them.
*/
protected $hooks;
/*! #var master_socket
#abstract resource - The master socket used by the server.
*/
protected $master_socket;
/*! #var max_clients
#abstract unsigned int - The maximum number of clients allowed to connect.
*/
public $max_clients = 10;
/*! #var max_read
#abstract unsigned int - The maximum number of bytes to read from a socket at a single time.
*/
public $max_read = 1024;
/*! #var clients
#abstract Array - an array of connected clients.
*/
public $clients;
/*! #function __construct
#abstract Creates the socket and starts listening to it.
#param string - IP Address to bind to, NULL for default.
#param int - Port to bind to
#result void
*/
public function __construct($bind_ip,$port)
{
set_time_limit(0);
$this->hooks = array();
$this->config["ip"] = $bind_ip;
$this->config["port"] = $port;
$this->master_socket = socket_create(AF_INET, SOCK_STREAM, 0);
socket_bind($this->master_socket,$this->config["ip"],$this->config["port"]) or die("Issue Binding");
socket_getsockname($this->master_socket,$bind_ip,$port);
socket_listen($this->master_socket);
SocketServer::debug("Listenting for connections on {$bind_ip}:{$port}");
}
/*! #function hook
#abstract Adds a function to be called whenever a certain action happens. Can be extended in your implementation.
#param string - Command
#param callback- Function to Call.
#see unhook
#see trigger_hooks
#result void
*/
public function hook($command,$function)
{
$command = strtoupper($command);
if(!isset($this->hooks[$command])) { $this->hooks[$command] = array(); }
$k = array_search($function,$this->hooks[$command]);
if($k === FALSE)
{
$this->hooks[$command][] = $function;
}
}
/*! #function unhook
#abstract Deletes a function from the call list for a certain action. Can be extended in your implementation.
#param string - Command
#param callback- Function to Delete from Call List
#see hook
#see trigger_hooks
#result void
*/
public function unhook($command = NULL,$function)
{
$command = strtoupper($command);
if($command !== NULL)
{
$k = array_search($function,$this->hooks[$command]);
if($k !== FALSE)
{
unset($this->hooks[$command][$k]);
}
} else {
$k = array_search($this->user_funcs,$function);
if($k !== FALSE)
{
unset($this->user_funcs[$k]);
}
}
}
/*! #function loop_once
#abstract Runs the class's actions once.
#discussion Should only be used if you want to run additional checks during server operation. Otherwise, use infinite_loop()
#param void
#see infinite_loop
#result bool - True
*/
public function loop_once()
{
// Setup Clients Listen Socket For Reading
$read[0] = $this->master_socket;
for($i = 0; $i < $this->max_clients; $i++)
{
if(isset($this->clients[$i]))
{
$read[$i + 1] = $this->clients[$i]->socket;
}
}
// Set up a blocking call to socket_select
if(socket_select($read,$write = NULL, $except = NULL, $tv_sec = 5) < 1)
{
// SocketServer::debug("Problem blocking socket_select?");
return true;
}
// Handle new Connections
if(in_array($this->master_socket, $read))
{
for($i = 0; $i < $this->max_clients; $i++)
{
if(empty($this->clients[$i]))
{
$temp_sock = $this->master_socket;
$this->clients[$i] = new SocketServerClient($this->master_socket,$i);
$this->trigger_hooks("CONNECT",$this->clients[$i],"");
break;
}
elseif($i == ($this->max_clients-1))
{
SocketServer::debug("Too many clients... :( ");
}
}
}
// Handle Input
for($i = 0; $i < $this->max_clients; $i++) // for each client
{
if(isset($this->clients[$i]))
{
if(in_array($this->clients[$i]->socket, $read))
{
$input = socket_read($this->clients[$i]->socket, $this->max_read);
if($input == null)
{
$this->disconnect($i);
}
else
{
SocketServer::debug("{$i}#{$this->clients[$i]->ip} --> {$input}");
$this->trigger_hooks("INPUT",$this->clients[$i],$input);
}
}
}
}
return true;
}
/*! #function disconnect
#abstract Disconnects a client from the server.
#param int - Index of the client to disconnect.
#param string - Message to send to the hooks
#result void
*/
public function disconnect($client_index,$message = "")
{
$i = $client_index;
SocketServer::debug("Client {$i} from {$this->clients[$i]->ip} Disconnecting");
$this->trigger_hooks("DISCONNECT",$this->clients[$i],$message);
$this->clients[$i]->destroy();
unset($this->clients[$i]);
}
/*! #function trigger_hooks
#abstract Triggers Hooks for a certain command.
#param string - Command who's hooks you want to trigger.
#param object - The client who activated this command.
#param string - The input from the client, or a message to be sent to the hooks.
#result void
*/
public function trigger_hooks($command,&$client,$input)
{
if(isset($this->hooks[$command]))
{
foreach($this->hooks[$command] as $function)
{
SocketServer::debug("Triggering Hook '{$function}' for '{$command}'");
$continue = call_user_func($function,$this,$client,$input);
if($continue === FALSE) { break; }
}
}
}
/*! #function infinite_loop
#abstract Runs the server code until the server is shut down.
#see loop_once
#param void
#result void
*/
public function infinite_loop()
{
$test = true;
do
{
$test = $this->loop_once();
}
while($test);
}
/*! #function debug
#static
#abstract Outputs Text directly.
#discussion Yeah, should probably make a way to turn this off.
#param string - Text to Output
#result void
*/
public static function debug($text)
{
echo("{$text}\r\n");
}
/*! #function socket_write_smart
#static
#abstract Writes data to the socket, including the length of the data, and ends it with a CRLF unless specified.
#discussion It is perfectly valid for socket_write_smart to return zero which means no bytes have been written. Be sure to use the === operator to check for FALSE in case of an error.
#param resource- Socket Instance
#param string - Data to write to the socket.
#param string - Data to end the line with. Specify a "" if you don't want a line end sent.
#result mixed - Returns the number of bytes successfully written to the socket or FALSE on failure. The error code can be retrieved with socket_last_error(). This code may be passed to socket_strerror() to get a textual explanation of the error.
*/
public static function socket_write_smart(&$sock,$string,$crlf = "\r\n")
{
SocketServer::debug("<-- {$string}");
if($crlf) { $string = "{$string}{$crlf}"; }
return socket_write($sock,$string,strlen($string));
}
/*! #function __get
#abstract Magic Method used for allowing the reading of protected variables.
#discussion You never need to use this method, simply calling $server->variable works because of this method's existence.
#param string - Variable to retrieve
#result mixed - Returns the reference to the variable called.
*/
function &__get($name)
{
return $this->{$name};
}
}
/*! #class SocketServerClient
#author Navarr Barnier
#abstract A Client Instance for use with SocketServer
*/
class SocketServerClient
{
/*! #var socket
#abstract resource - The client's socket resource, for sending and receiving data with.
*/
protected $socket;
/*! #var ip
#abstract string - The client's IP address, as seen by the server.
*/
protected $ip;
/*! #var hostname
#abstract string - The client's hostname, as seen by the server.
#discussion This variable is only set after calling lookup_hostname, as hostname lookups can take up a decent amount of time.
#see lookup_hostname
*/
protected $hostname;
/*! #var server_clients_index
#abstract int - The index of this client in the SocketServer's client array.
*/
protected $server_clients_index;
/*! #function __construct
#param resource- The resource of the socket the client is connecting by, generally the master socket.
#param int - The Index in the Server's client array.
#result void
*/
public function __construct(&$socket,$i)
{
$this->server_clients_index = $i;
$this->socket = socket_accept($socket) or die("Failed to Accept");
SocketServer::debug("New Client Connected");
socket_getpeername($this->socket,$ip);
$this->ip = $ip;
}
/*! #function lookup_hostname
#abstract Searches for the user's hostname and stores the result to hostname.
#see hostname
#param void
#result string - The hostname on success or the IP address on failure.
*/
public function lookup_hostname()
{
$this->hostname = gethostbyaddr($this->ip);
return $this->hostname;
}
/*! #function destroy
#abstract Closes the socket. Thats pretty much it.
#param void
#result void
*/
public function destroy()
{
socket_close($this->socket);
}
function &__get($name)
{
return $this->{$name};
}
function __isset($name)
{
return isset($this->{$name});
}
}
Yep, using socket_create, socket_bind, socket_listen, and socket_accept
http://www.php.net/manual/en/function.socket-create.php
http://www.php.net/manual/en/function.socket-bind.php
http://www.php.net/manual/en/function.socket-listen.php
http://www.php.net/manual/en/function.socket-accept.php
There are lots of examples on those pages on how to use them.
Yes, PHP offers a socket_listen() function just for that. http://php.net/manual/en/function.socket-listen.php.

How to make HTTP requests in PHP and not wait on the response

Is there a way in PHP to make HTTP calls and not wait for a response? I don't care about the response, I just want to do something like file_get_contents(), but not wait for the request to finish before executing the rest of my code. This would be super useful for setting off "events" of a sort in my application, or triggering long processes.
Any ideas?
The answer I'd previously accepted didn't work. It still waited for responses. This does work though, taken from How do I make an asynchronous GET request in PHP?
function post_without_wait($url, $params)
{
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;
fwrite($fp, $out);
fclose($fp);
}
If you control the target that you want to call asynchronously (e.g. your own "longtask.php"), you can close the connection from that end, and both scripts will run in parallel. It works like this:
quick.php opens longtask.php via cURL (no magic here)
longtask.php closes the connection and continues (magic!)
cURL returns to quick.php when the connection is closed
Both tasks continue in parallel
I have tried this, and it works just fine. But quick.php won't know anything about how longtask.php is doing, unless you create some means of communication between the processes.
Try this code in longtask.php, before you do anything else. It will close the connection, but still continue to run (and suppress any output):
while(ob_get_level()) ob_end_clean();
header('Connection: close');
ignore_user_abort();
ob_start();
echo('Connection Closed');
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush();
flush();
The code is copied from the PHP manual's user contributed notes and somewhat improved.
You can do trickery by using exec() to invoke something that can do HTTP requests, like wget, but you must direct all output from the program to somewhere, like a file or /dev/null, otherwise the PHP process will wait for that output.
If you want to separate the process from the apache thread entirely, try something like (I'm not sure about this, but I hope you get the idea):
exec('bash -c "wget -O (url goes here) > /dev/null 2>&1 &"');
It's not a nice business, and you'll probably want something like a cron job invoking a heartbeat script which polls an actual database event queue to do real asynchronous events.
You can use this library: https://github.com/stil/curl-easy
It's pretty straightforward then:
<?php
$request = new cURL\Request('http://yahoo.com/');
$request->getOptions()->set(CURLOPT_RETURNTRANSFER, true);
// Specify function to be called when your request is complete
$request->addListener('complete', function (cURL\Event $event) {
$response = $event->response;
$httpCode = $response->getInfo(CURLINFO_HTTP_CODE);
$html = $response->getContent();
echo "\nDone.\n";
});
// Loop below will run as long as request is processed
$timeStart = microtime(true);
while ($request->socketPerform()) {
printf("Running time: %dms \r", (microtime(true) - $timeStart)*1000);
// Here you can do anything else, while your request is in progress
}
Below you can see console output of above example.
It will display simple live clock indicating how much time request is running:
As of 2018, Guzzle has become the defacto standard library for HTTP requests, used in several modern frameworks. It's written in pure PHP and does not require installing any custom extensions.
It can do asynchronous HTTP calls very nicely, and even pool them such as when you need to make 100 HTTP calls, but don't want to run more than 5 at a time.
Concurrent request example
use GuzzleHttp\Client;
use GuzzleHttp\Promise;
$client = new Client(['base_uri' => 'http://httpbin.org/']);
// Initiate each request but do not block
$promises = [
'image' => $client->getAsync('/image'),
'png' => $client->getAsync('/image/png'),
'jpeg' => $client->getAsync('/image/jpeg'),
'webp' => $client->getAsync('/image/webp')
];
// Wait on all of the requests to complete. Throws a ConnectException
// if any of the requests fail
$results = Promise\unwrap($promises);
// Wait for the requests to complete, even if some of them fail
$results = Promise\settle($promises)->wait();
// You can access each result using the key provided to the unwrap
// function.
echo $results['image']['value']->getHeader('Content-Length')[0]
echo $results['png']['value']->getHeader('Content-Length')[0]
See http://docs.guzzlephp.org/en/stable/quickstart.html#concurrent-requests
/**
* Asynchronously execute/include a PHP file. Does not record the output of the file anywhere.
*
* #param string $filename file to execute, relative to calling script
* #param string $options (optional) arguments to pass to file via the command line
*/
function asyncInclude($filename, $options = '') {
exec("/path/to/php -f {$filename} {$options} >> /dev/null &");
}
Fake a request abortion using CURL setting a low CURLOPT_TIMEOUT_MS
set ignore_user_abort(true) to keep processing after the connection closed.
With this method no need to implement connection handling via headers and buffer too dependent on OS, Browser and PHP version
Master process
function async_curl($background_process=''){
//-------------get curl contents----------------
$ch = curl_init($background_process);
curl_setopt_array($ch, array(
CURLOPT_HEADER => 0,
CURLOPT_RETURNTRANSFER =>true,
CURLOPT_NOSIGNAL => 1, //to timeout immediately if the value is < 1000 ms
CURLOPT_TIMEOUT_MS => 50, //The maximum number of mseconds to allow cURL functions to execute
CURLOPT_VERBOSE => 1,
CURLOPT_HEADER => 1
));
$out = curl_exec($ch);
//-------------parse curl contents----------------
//$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
//$header = substr($out, 0, $header_size);
//$body = substr($out, $header_size);
curl_close($ch);
return true;
}
async_curl('http://example.com/background_process_1.php');
Background process
ignore_user_abort(true);
//do something...
NB
If you want cURL to timeout in less than one second, you can use
CURLOPT_TIMEOUT_MS, although there is a bug/"feature" on "Unix-like
systems" that causes libcurl to timeout immediately if the value is <
1000 ms with the error "cURL Error (28): Timeout was reached". The
explanation for this behavior is:
[...]
The solution is to disable signals using CURLOPT_NOSIGNAL
Resources
curl timeout less than 1000ms always fails?
http://www.php.net/manual/en/function.curl-setopt.php#104597
http://php.net/manual/en/features.connection-handling.php
The swoole extension. https://github.com/matyhtf/swoole
Asynchronous & concurrent networking framework for PHP.
$client = new swoole_client(SWOOLE_SOCK_TCP, SWOOLE_SOCK_ASYNC);
$client->on("connect", function($cli) {
$cli->send("hello world\n");
});
$client->on("receive", function($cli, $data){
echo "Receive: $data\n";
});
$client->on("error", function($cli){
echo "connect fail\n";
});
$client->on("close", function($cli){
echo "close\n";
});
$client->connect('127.0.0.1', 9501, 0.5);
let me show you my way :)
needs nodejs installed on the server
(my server sends 1000 https get request takes only 2 seconds)
url.php :
<?
$urls = array_fill(0, 100, 'http://google.com/blank.html');
function execinbackground($cmd) {
if (substr(php_uname(), 0, 7) == "Windows"){
pclose(popen("start /B ". $cmd, "r"));
}
else {
exec($cmd . " > /dev/null &");
}
}
fwite(fopen("urls.txt","w"),implode("\n",$urls);
execinbackground("nodejs urlscript.js urls.txt");
// { do your work while get requests being executed.. }
?>
urlscript.js >
var https = require('https');
var url = require('url');
var http = require('http');
var fs = require('fs');
var dosya = process.argv[2];
var logdosya = 'log.txt';
var count=0;
http.globalAgent.maxSockets = 300;
https.globalAgent.maxSockets = 300;
setTimeout(timeout,100000); // maximum execution time (in ms)
function trim(string) {
return string.replace(/^\s*|\s*$/g, '')
}
fs.readFile(process.argv[2], 'utf8', function (err, data) {
if (err) {
throw err;
}
parcala(data);
});
function parcala(data) {
var data = data.split("\n");
count=''+data.length+'-'+data[1];
data.forEach(function (d) {
req(trim(d));
});
/*
fs.unlink(dosya, function d() {
console.log('<%s> file deleted', dosya);
});
*/
}
function req(link) {
var linkinfo = url.parse(link);
if (linkinfo.protocol == 'https:') {
var options = {
host: linkinfo.host,
port: 443,
path: linkinfo.path,
method: 'GET'
};
https.get(options, function(res) {res.on('data', function(d) {});}).on('error', function(e) {console.error(e);});
} else {
var options = {
host: linkinfo.host,
port: 80,
path: linkinfo.path,
method: 'GET'
};
http.get(options, function(res) {res.on('data', function(d) {});}).on('error', function(e) {console.error(e);});
}
}
process.on('exit', onExit);
function onExit() {
log();
}
function timeout()
{
console.log("i am too far gone");process.exit();
}
function log()
{
var fd = fs.openSync(logdosya, 'a+');
fs.writeSync(fd, dosya + '-'+count+'\n');
fs.closeSync(fd);
}
You can use non-blocking sockets and one of pecl extensions for PHP:
http://php.net/event
http://php.net/libevent
http://php.net/ev
https://github.com/m4rw3r/php-libev
You can use library which gives you an abstraction layer between your code and a pecl extension: https://github.com/reactphp/event-loop
You can also use async http-client, based on the previous library: https://github.com/reactphp/http-client
See others libraries of ReactPHP: http://reactphp.org
Be careful with an asynchronous model.
I recommend to see this video on youtube: http://www.youtube.com/watch?v=MWNcItWuKpI
class async_file_get_contents extends Thread{
public $ret;
public $url;
public $finished;
public function __construct($url) {
$this->finished=false;
$this->url=$url;
}
public function run() {
$this->ret=file_get_contents($this->url);
$this->finished=true;
}
}
$afgc=new async_file_get_contents("http://example.org/file.ext");
Event Extension
Event extension is very appropriate. It is a port of Libevent library which is designed for event-driven I/O, mainly for networking.
I have written a sample HTTP client that allows to schedule a number of
HTTP requests and run them asynchronously.
This is a sample HTTP client class based on Event extension.
The class allows to schedule a number of HTTP requests, then run them asynchronously.
http-client.php
<?php
class MyHttpClient {
/// #var EventBase
protected $base;
/// #var array Instances of EventHttpConnection
protected $connections = [];
public function __construct() {
$this->base = new EventBase();
}
/**
* Dispatches all pending requests (events)
*
* #return void
*/
public function run() {
$this->base->dispatch();
}
public function __destruct() {
// Destroy connection objects explicitly, don't wait for GC.
// Otherwise, EventBase may be free'd earlier.
$this->connections = null;
}
/**
* #brief Adds a pending HTTP request
*
* #param string $address Hostname, or IP
* #param int $port Port number
* #param array $headers Extra HTTP headers
* #param int $cmd A EventHttpRequest::CMD_* constant
* #param string $resource HTTP request resource, e.g. '/page?a=b&c=d'
*
* #return EventHttpRequest|false
*/
public function addRequest($address, $port, array $headers,
$cmd = EventHttpRequest::CMD_GET, $resource = '/')
{
$conn = new EventHttpConnection($this->base, null, $address, $port);
$conn->setTimeout(5);
$req = new EventHttpRequest([$this, '_requestHandler'], $this->base);
foreach ($headers as $k => $v) {
$req->addHeader($k, $v, EventHttpRequest::OUTPUT_HEADER);
}
$req->addHeader('Host', $address, EventHttpRequest::OUTPUT_HEADER);
$req->addHeader('Connection', 'close', EventHttpRequest::OUTPUT_HEADER);
if ($conn->makeRequest($req, $cmd, $resource)) {
$this->connections []= $conn;
return $req;
}
return false;
}
/**
* #brief Handles an HTTP request
*
* #param EventHttpRequest $req
* #param mixed $unused
*
* #return void
*/
public function _requestHandler($req, $unused) {
if (is_null($req)) {
echo "Timed out\n";
} else {
$response_code = $req->getResponseCode();
if ($response_code == 0) {
echo "Connection refused\n";
} elseif ($response_code != 200) {
echo "Unexpected response: $response_code\n";
} else {
echo "Success: $response_code\n";
$buf = $req->getInputBuffer();
echo "Body:\n";
while ($s = $buf->readLine(EventBuffer::EOL_ANY)) {
echo $s, PHP_EOL;
}
}
}
}
}
$address = "my-host.local";
$port = 80;
$headers = [ 'User-Agent' => 'My-User-Agent/1.0', ];
$client = new MyHttpClient();
// Add pending requests
for ($i = 0; $i < 10; $i++) {
$client->addRequest($address, $port, $headers,
EventHttpRequest::CMD_GET, '/test.php?a=' . $i);
}
// Dispatch pending requests
$client->run();
test.php
This is a sample script on the server side.
<?php
echo 'GET: ', var_export($_GET, true), PHP_EOL;
echo 'User-Agent: ', $_SERVER['HTTP_USER_AGENT'] ?? '(none)', PHP_EOL;
Usage
php http-client.php
Sample Output
Success: 200
Body:
GET: array (
'a' => '1',
)
User-Agent: My-User-Agent/1.0
Success: 200
Body:
GET: array (
'a' => '0',
)
User-Agent: My-User-Agent/1.0
Success: 200
Body:
GET: array (
'a' => '3',
)
...
(Trimmed.)
Note, the code is designed for long-term processing in the CLI SAPI.
For custom protocols, consider using low-level API, i.e. buffer events, buffers. For SSL/TLS communications, I would recommend the low-level API in conjunction with Event's ssl context. Examples:
SSL echo server
SSL client
Although Libevent's HTTP API is simple, it is not as flexible as buffer events. For example, the HTTP API currently doesn't support custom HTTP methods. But it is possible to implement virtually any protocol using the low-level API.
Ev Extension
I have also written a sample of another HTTP client using Ev extension with sockets in non-blocking mode. The code is slightly more verbose than the sample based on Event, because Ev is a general purpose event loop. It doesn't provide network-specific functions, but its EvIo watcher is capable of listening to a file descriptor encapsulated into the socket resource, in particular.
This is a sample HTTP client based on Ev extension.
Ev extension implements a simple yet powerful general purpose event loop. It doesn't provide network-specific watchers, but its I/O watcher can be used for asynchronous processing of sockets.
The following code shows how HTTP requests can be scheduled for parallel processing.
http-client.php
<?php
class MyHttpRequest {
/// #var MyHttpClient
private $http_client;
/// #var string
private $address;
/// #var string HTTP resource such as /page?get=param
private $resource;
/// #var string HTTP method such as GET, POST etc.
private $method;
/// #var int
private $service_port;
/// #var resource Socket
private $socket;
/// #var double Connection timeout in seconds.
private $timeout = 10.;
/// #var int Chunk size in bytes for socket_recv()
private $chunk_size = 20;
/// #var EvTimer
private $timeout_watcher;
/// #var EvIo
private $write_watcher;
/// #var EvIo
private $read_watcher;
/// #var EvTimer
private $conn_watcher;
/// #var string buffer for incoming data
private $buffer;
/// #var array errors reported by sockets extension in non-blocking mode.
private static $e_nonblocking = [
11, // EAGAIN or EWOULDBLOCK
115, // EINPROGRESS
];
/**
* #param MyHttpClient $client
* #param string $host Hostname, e.g. google.co.uk
* #param string $resource HTTP resource, e.g. /page?a=b&c=d
* #param string $method HTTP method: GET, HEAD, POST, PUT etc.
* #throws RuntimeException
*/
public function __construct(MyHttpClient $client, $host, $resource, $method) {
$this->http_client = $client;
$this->host = $host;
$this->resource = $resource;
$this->method = $method;
// Get the port for the WWW service
$this->service_port = getservbyname('www', 'tcp');
// Get the IP address for the target host
$this->address = gethostbyname($this->host);
// Create a TCP/IP socket
$this->socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
if (!$this->socket) {
throw new RuntimeException("socket_create() failed: reason: " .
socket_strerror(socket_last_error()));
}
// Set O_NONBLOCK flag
socket_set_nonblock($this->socket);
$this->conn_watcher = $this->http_client->getLoop()
->timer(0, 0., [$this, 'connect']);
}
public function __destruct() {
$this->close();
}
private function freeWatcher(&$w) {
if ($w) {
$w->stop();
$w = null;
}
}
/**
* Deallocates all resources of the request
*/
private function close() {
if ($this->socket) {
socket_close($this->socket);
$this->socket = null;
}
$this->freeWatcher($this->timeout_watcher);
$this->freeWatcher($this->read_watcher);
$this->freeWatcher($this->write_watcher);
$this->freeWatcher($this->conn_watcher);
}
/**
* Initializes a connection on socket
* #return bool
*/
public function connect() {
$loop = $this->http_client->getLoop();
$this->timeout_watcher = $loop->timer($this->timeout, 0., [$this, '_onTimeout']);
$this->write_watcher = $loop->io($this->socket, Ev::WRITE, [$this, '_onWritable']);
return socket_connect($this->socket, $this->address, $this->service_port);
}
/**
* Callback for timeout (EvTimer) watcher
*/
public function _onTimeout(EvTimer $w) {
$w->stop();
$this->close();
}
/**
* Callback which is called when the socket becomes wriable
*/
public function _onWritable(EvIo $w) {
$this->timeout_watcher->stop();
$w->stop();
$in = implode("\r\n", [
"{$this->method} {$this->resource} HTTP/1.1",
"Host: {$this->host}",
'Connection: Close',
]) . "\r\n\r\n";
if (!socket_write($this->socket, $in, strlen($in))) {
trigger_error("Failed writing $in to socket", E_USER_ERROR);
return;
}
$loop = $this->http_client->getLoop();
$this->read_watcher = $loop->io($this->socket,
Ev::READ, [$this, '_onReadable']);
// Continue running the loop
$loop->run();
}
/**
* Callback which is called when the socket becomes readable
*/
public function _onReadable(EvIo $w) {
// recv() 20 bytes in non-blocking mode
$ret = socket_recv($this->socket, $out, 20, MSG_DONTWAIT);
if ($ret) {
// Still have data to read. Append the read chunk to the buffer.
$this->buffer .= $out;
} elseif ($ret === 0) {
// All is read
printf("\n<<<<\n%s\n>>>>", rtrim($this->buffer));
fflush(STDOUT);
$w->stop();
$this->close();
return;
}
// Caught EINPROGRESS, EAGAIN, or EWOULDBLOCK
if (in_array(socket_last_error(), static::$e_nonblocking)) {
return;
}
$w->stop();
$this->close();
}
}
/////////////////////////////////////
class MyHttpClient {
/// #var array Instances of MyHttpRequest
private $requests = [];
/// #var EvLoop
private $loop;
public function __construct() {
// Each HTTP client runs its own event loop
$this->loop = new EvLoop();
}
public function __destruct() {
$this->loop->stop();
}
/**
* #return EvLoop
*/
public function getLoop() {
return $this->loop;
}
/**
* Adds a pending request
*/
public function addRequest(MyHttpRequest $r) {
$this->requests []= $r;
}
/**
* Dispatches all pending requests
*/
public function run() {
$this->loop->run();
}
}
/////////////////////////////////////
// Usage
$client = new MyHttpClient();
foreach (range(1, 10) as $i) {
$client->addRequest(new MyHttpRequest($client, 'my-host.local', '/test.php?a=' . $i, 'GET'));
}
$client->run();
Testing
Suppose http://my-host.local/test.php script is printing the dump of $_GET:
<?php
echo 'GET: ', var_export($_GET, true), PHP_EOL;
Then the output of php http-client.php command will be similar to the following:
<<<<
HTTP/1.1 200 OK
Server: nginx/1.10.1
Date: Fri, 02 Dec 2016 12:39:54 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
X-Powered-By: PHP/7.0.13-pl0-gentoo
1d
GET: array (
'a' => '3',
)
0
>>>>
<<<<
HTTP/1.1 200 OK
Server: nginx/1.10.1
Date: Fri, 02 Dec 2016 12:39:54 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
X-Powered-By: PHP/7.0.13-pl0-gentoo
1d
GET: array (
'a' => '2',
)
0
>>>>
...
(trimmed)
Note, in PHP 5 the sockets extension may log warnings for EINPROGRESS, EAGAIN, and EWOULDBLOCK errno values. It is possible to turn off the logs with
error_reporting(E_ERROR);
Concerning "the Rest" of the Code
I just want to do something like file_get_contents(), but not wait for the request to finish before executing the rest of my code.
The code that is supposed to run in parallel with the network requests can be executed within a the callback of an Event timer, or Ev's idle watcher, for instance. You can easily figure it out by watching the samples mentioned above. Otherwise, I'll add another example :)
I find this package quite useful and very simple: https://github.com/amphp/parallel-functions
<?php
use function Amp\ParallelFunctions\parallelMap;
use function Amp\Promise\wait;
$responses = wait(parallelMap([
'https://google.com/',
'https://github.com/',
'https://stackoverflow.com/',
], function ($url) {
return file_get_contents($url);
}));
It will load all 3 urls in parallel.
You can also use class instance methods in the closure.
For example I use Laravel extension based on this package https://github.com/spatie/laravel-collection-macros#parallelmap
Here is my code:
/**
* Get domains with all needed data
*/
protected function getDomainsWithdata(): Collection
{
return $this->opensrs->getDomains()->parallelMap(function ($domain) {
$contact = $this->opensrs->getDomainContact($domain);
$contact['domain'] = $domain;
return $contact;
}, 10);
}
It loads all needed data in 10 parallel threads and instead of 50 secs without async it finished in just 8 secs.
Here is a working example, just run it and open storage.txt afterwards, to check the magical result
<?php
function curlGet($target){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $target);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec ($ch);
curl_close ($ch);
return $result;
}
// Its the next 3 lines that do the magic
ignore_user_abort(true);
header("Connection: close"); header("Content-Length: 0");
echo str_repeat("s", 100000); flush();
$i = $_GET['i'];
if(!is_numeric($i)) $i = 1;
if($i > 4) exit;
if($i == 1) file_put_contents('storage.txt', '');
file_put_contents('storage.txt', file_get_contents('storage.txt') . time() . "\n");
sleep(5);
curlGet($_SERVER['HTTP_HOST'] . $_SERVER['SCRIPT_NAME'] . '?i=' . ($i + 1));
curlGet($_SERVER['HTTP_HOST'] . $_SERVER['SCRIPT_NAME'] . '?i=' . ($i + 1));
Here is my own PHP function when I do POST to a specific URL of any page....
Sample: *** usage of my Function...
<?php
parse_str("email=myemail#ehehehahaha.com&subject=this is just a test");
$_POST['email']=$email;
$_POST['subject']=$subject;
echo HTTP_POST("http://example.com/mail.php",$_POST);***
exit;
?>
<?php
/*********HTTP POST using FSOCKOPEN **************/
// by ArbZ
function HTTP_Post($URL,$data, $referrer="") {
// parsing the given URL
$URL_Info=parse_url($URL);
// Building referrer
if($referrer=="") // if not given use this script as referrer
$referrer=$_SERVER["SCRIPT_URI"];
// making string from $data
foreach($data as $key=>$value)
$values[]="$key=".urlencode($value);
$data_string=implode("&",$values);
// Find out which port is needed - if not given use standard (=80)
if(!isset($URL_Info["port"]))
$URL_Info["port"]=80;
// building POST-request: HTTP_HEADERs
$request.="POST ".$URL_Info["path"]." HTTP/1.1\n";
$request.="Host: ".$URL_Info["host"]."\n";
$request.="Referer: $referer\n";
$request.="Content-type: application/x-www-form-urlencoded\n";
$request.="Content-length: ".strlen($data_string)."\n";
$request.="Connection: close\n";
$request.="\n";
$request.=$data_string."\n";
$fp = fsockopen($URL_Info["host"],$URL_Info["port"]);
fputs($fp, $request);
while(!feof($fp)) {
$result .= fgets($fp, 128);
}
fclose($fp); //$eco = nl2br();
function getTextBetweenTags($string, $tagname) {
$pattern = "/<$tagname ?.*>(.*)<\/$tagname>/";
preg_match($pattern, $string, $matches);
return $matches[1];
}
//STORE THE FETCHED CONTENTS to a VARIABLE, because its way better and fast...
$str = $result;
$txt = getTextBetweenTags($str, "span"); $eco = $txt; $result = explode("&",$result);
return $result[1];
<span style=background-color:LightYellow;color:blue>".trim($_GET['em'])."</span>
</pre> ";
}
</pre>
ReactPHP async http client
https://github.com/shuchkin/react-http-client
Install via Composer
$ composer require shuchkin/react-http-client
Async HTTP GET
// get.php
$loop = \React\EventLoop\Factory::create();
$http = new \Shuchkin\ReactHTTP\Client( $loop );
$http->get( 'https://tools.ietf.org/rfc/rfc2068.txt' )->then(
function( $content ) {
echo $content;
},
function ( \Exception $ex ) {
echo 'HTTP error '.$ex->getCode().' '.$ex->getMessage();
}
);
$loop->run();
Run php in CLI-mode
$ php get.php
Symfony HttpClient is asynchronous https://symfony.com/doc/current/components/http_client.html.
For example you can
use Symfony\Component\HttpClient\HttpClient;
$client = HttpClient::create();
$response1 = $client->request('GET', 'https://website1');
$response2 = $client->request('GET', 'https://website1');
$response3 = $client->request('GET', 'https://website1');
//these 3 calls with return immediately
//but the requests will fire to the website1 webserver
$response1->getContent(); //this will block until content is fetched
$response2->getContent(); //same
$response3->getContent(); //same
Well, the timeout can be set in milliseconds,
see "CURLOPT_CONNECTTIMEOUT_MS" in http://www.php.net/manual/en/function.curl-setopt

Categories