I'm working on writing a multi-threaded network server to allow for non-direct database access over a network in PHP for security reasons. The problem that I'm having is that PHP pthread has been discontinued in favor of a new method called parallel (https://www.php.net/manual/en/book.parallel.php).
The issue that I'm running into is the documentation states that everything must be passed by value, and that you cannot pass internal objects. Well, sockets are internal objects as well as database link connectors. However, the documentation also says that there are no restrictions in included files. A socket is an internal type, and I can't for the life of me figure out how to bypass that.
In a traditional server, the main thread sits on the incoming socket and waits for clients to connect. When a connection comes in, the main thread spawns a new thread with the socket information and goes back to listening. Based on what PHP is doing now, that does not seem possible.
I've done a fair amount of searching on the internet and I have found -0- parallel examples that use networking with sockets or database connectors. So, is this possible or do I need to write the server in another language, like C++?
Suggestions?
EDIT Jan 18, 2022 # 23:30 PST (-8:00 UTC):
Here is the code that I have thus far...for a non-threading implementation. The networking part does work as I have been able to test it.
FILE: config.php
<?php
/*
Configuration File
*/
// Network Parameters
const LISTEN_IPADDR = '0.0.0.0';
const LISTEN_PORT = 8476;
const MAX_CONNECTION_QUEUE = 1024;
const ACCEPT_MODE = 0;
const ALLOW_HOSTS = array();
const BLOCK_HOSTS = array();
// Operational Parameters
const MEMORY_MAX = '1GB';
const LOG_FILE = './vserver.log';
const DEBUG = false;
// Module List
const MODULE_LIST = array(
'test.php',
);
?>
FILE: module.php
<?php
/*
Module Object File
*/
// All modules must implement this interface and also extend the
// class below.
interface moduleInterface
{
const CMD_READ = 100; // Read: Sends data back to client
const CMD_WRITE = 101; // Write: Writes data to memory
const CMD_CHECK = 102; // Check: Checks a value in memory
const CMD_PURGE = 103; // Purge: Removed expired data
const CMD_AUDIT = 104; // Audit: Data integrity check
public static function initialize();
public function process($socket, $command, $data);
}
class moduleObject extends Thread implements moduleInterface
{
// These have to be set on a per module basis.
const KEY = NULL;
const ID = 0x00000000;
private static $datastore = array();
function __construct()
{
// $class = get_called_class();
// moduleRegister($class, $this, self::ID, self::KEY);
}
function __destruct()
{
}
public static function initialize()
{
$class = get_called_class();
$object = new $class();
moduleRegister($class, $object, self::ID, self::KEY);
}
protected function process($socket, $command, $data, $addr, $port)
{
switch ($command)
{
case self::CMD_READ:
case self::CMD_WRITE:
case self::CMD_CHECK:
case self::CMD_PURGE:
case self::CMD_AUDIT:
default:
$result = $this->processCustom($socket, $command, $data,
$addr, $port);
if ($result == false)
{
writeLog("Invalid command received from $addr:$port",
LOG__WARNING);
}
break;
}
}
private function processCustom($socket, $command, $data, $addr, $port)
{
return false;
}
private function dataRead($socket, $data, $addr, $port)
{
return false;
}
private function dataWrite($socket, $data, $addr, $port)
{
return false;
}
private function dataCheck($socket, $data, $addr, $port)
{
return false;
}
private function dataPurge($socket, $data, $addr, $port)
{
return false;
}
private function dataAudit($socket, $data, $addr, $port)
{
return false;
}
}
?>
FILE: main.php
<?php
/*
Main Server Program
*/
require_once 'config.php';
require_once 'module.php';
// ********************************************************************
// Iinitialize
// ********************************************************************
// This is to work around an issue with PHP on Windows machines.
// Turns out that the Windows Event Viewer has fewer log levels
// than Unix machines, so some of the log levels are mapped to
// the same number. See https://bugs.php.net/bug.php?id=55129
// for details. We can do this since we are not using syslog.
// Logging Levels
define('LOG__EMERG', 0);
define('LOG__ALERT', 1);
define('LOG__CRIT', 2);
define('LOG__ERR', 3);
define('LOG__WARNING', 4);
define('LOG__NOTICE', 5);
define('LOG__INFO', 6);
define('LOG__DEBUG', 7);
// Global Variables
$LOGFILE = NULL;
// Set Parameters
ini_set('memory_limit', MEMORY_MAX);
// Array that holds all the class references.
// The data format of this array is as follows:
// ID => array(
// 'class' => classname,
// 'reference' => class reference,
// 'key' => access key,
// 'id' => class ID,
// ),
$moduleRegisterArray = array();
// ******** Start server
openLogFile();
moduleLoad();
moduleStart();
initiateNetwork();
exit(0);
// ********************************************************************
// Functions
// ********************************************************************
// **** Error Handling/Logging
// Opens the log file.
function openLogFile()
{
global $LOGFILE;
$LOGFILE = fopen(LOG_FILE, 'a');
if ($LOGFILE == false)
{
fprintf(STDERR, "Error opening log file. Aborting.\n");
exit(1);
}
writeLog("Server started", LOG__NOTICE);
}
// Writes a log message to the log file or to the system console,
// depending on debug mode.
function writeLog($msg, $level)
{
global $LOGFILE;
switch ($level)
{
case LOG__EMERG:
$type = '*****EMERGENCY*****';
break;
case LOG__ALERT:
$type = '****ALERT****';
break;
case LOG__CRIT:
$type = '***CRITICAL***';
break;
case LOG__ERR:
$type = '**ERROR**';
break;
case LOG__WARNING:
$type = '*WARNING*';
break;
case LOG__NOTICE:
$type = 'NOTICE';
break;
case LOG__INFO:
$type = 'INFORMATION';
break;
case LOG__DEBUG:
$type = 'DEBUG';
break;
default:
$type = 'UNKNOWN';
break;
}
$date = date('Y-m-d H:m:s');
if (!DEBUG)
{
if ($level != LOG__DEBUG)
{
fprintf($LOGFILE, "%s ::-%s-:: %s\n", $date, $type, $msg);
}
else
{
fprintf(STDOUT, "%s ::-%s-:: %s\n", $date, $type, $msg);
}
}
else
{
fprintf($LOGFILE, "%s ::-%s-:: %s\n", $date, $type, $msg);
}
}
// Handles socket errors
function socketError($socket, $func, $die)
{
$code = socket_last_error($socket);
$msg = socket_strerror($code);
$txt = "Network Error: " . $func . " (" . $code . ") " . $msg;
writeLog($txt, LOG__ERR);
if ($die)
{
socket_close($socket);
exit(1);
}
}
// ******** Module Handling
// Load defined modules.
function moduleLoad()
{
$path = './modules/';
foreach(MODULE_LIST as $kx)
{
$fileMod = $path . $kx;
$fileEx = file_exists($fileMod);
if ($fileEx == true)
{
writeLog("Loading module file: $fileMod", LOG__NOTICE);
require_once $fileMod;
}
else
{
writeLog("Module does not exist: $fileMod", LOG__WARNING);
}
}
}
// Starts off the module registration process.
function moduleStart()
{
$classList = get_declared_classes();
foreach($classList as $kx => $vx)
{
$position = strpos($vx, 'mod_');
if ($position === false) continue;
$vx::initialize();
}
}
// Each module calls this so it can be registered.
function moduleRegister($class, $reference, $id, $key)
{
global $moduleRegisterArray;
$module = array(
'class' => $class,
'reference' => $reference,
'key' => $key,
'id' => $id,
);
$moduleRegisterArray[$id] = $module;
}
// The main server function.
// Does not return.
function initiateNetwork()
{
global $LOGFILE;
$func = 'initiate';
$addr = NULL;
$port = NULL;
// Create the network socket.
$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
if ($socket === false) socketError($socket, $func, true);
// Bind the socket.
$result = socket_bind($socket, LISTEN_IPADDR, LISTEN_PORT);
if ($result === false) socketError($socket, $func, true);
// Now listen using an infinite loop.
// The server will stay in this state until stopped.
writeLog("The server is now listening on port " . LISTEN_PORT .
" on address " . LISTEN_IPADDR, LOG__INFO);
// Close the stdin, stdout, and stderr file descriptors.
if (!DEBUG)
{
fclose(STDIN);
fclose(STDOUT);
fclose(STDERR);
}
while (true)
{
$result = socket_listen($socket, MAX_CONNECTION_QUEUE);
if ($result === false) socketError($socket, $func, true);
$spawn = socket_accept($socket);
if ($spawn === false) socketError($socket, $func, true);
$result = socket_getpeername($spawn, $addr, $port);
if ($result === false) socketError($socket, $func, true);
writeLog("Connection accepted from $addr:$port", LOG__INFO);
process($spawn);
}
}
// Test Function
function process($socket)
{
$func = 'process';
$msg = date('Y-m-d H:m:sP T') . "\r\n";
$bindata = openssl_random_pseudo_bytes(36);
$msg .= bin2hex($bindata) . "\r\n";
$result = socket_write($socket, $msg, strlen($msg));
if ($result === false) socketError($socket, $func, true);
socket_close($socket);
}
?>
This is just a framework that I've been working on. Eventually, I want this to perform the following actions:
Receive commands/data/queries from a client over the network.
Process and convert the information into SQL commands.
Send SQL commands to a database server.
Receive results from said database server.
Process said results.
Send results to client over the network.
The actual data that goes over the network is in JSON format because it is platform neutral. This is also the reason why I'm using PHP. I probably could use Java, but the issue there is that the last time that I checked, Java support on *nix systems is sporadic at best. The only other alternative that I can see is to use C++. I haven't explored what Node.js provides, yet.
What I would like is the traditional server threading model where new threads are spawned when a client connection comes in over the network, and have it perform database processing based on the client's request. So the software needs access to the network and the database at the same time.
Both of these borrow from prior SO answers that I sadly didn't bookmark.
This method is if you want to just split your "threads" by IDs that you can generate in advance. Make sure to close \ reopen all SQL connections prior to executing the following in either example as you don't want them shared.
$some_array_of_ids = [.....];
$max_children = 10; // Max number of forked processes
$pids = []; // Child process tracking array
foreach ($some_array_of_ids as $unqiue_id) {
// Limit the number of child processes
// If $maxChildren or more processes exist, wait until one exits
if (count($pids) >= $max_children) {
$pid = pcntl_waitpid(-1, $status);
unset($pids[$pid]); // Remove PID that exited from the list
}
$pid = pcntl_fork();
if ($pid) { // Parent
if ($pid < 0) {
// Unable to fork process, handle error here
continue;
} else {
// Add child PID to tracker array
// Use PID as key for easy use of unset()
$pids[$pid] = $unqiue_id;
}
} else { // Child
echo "\nChild {$unqiue_id} has begun process...";
.....
echo "\nChild {$unqiue_id} has finished process...";
exit(0);
}
}
// Now wait for the child processes to exit. This approach may seem overly
// simple, but because of the way it works it will have the effect of
// waiting until the last process exits and pretty much no longer
foreach ($pids as $pid => $unqiue_id) {
pcntl_waitpid($pid, $status);
unset($pids[$pid]);
}
If you actually want to stream socket data, there's a more elaborate method:
$some_array_of_data = [.....];
foreach ($some_array_of_data as $group) {
$sockets = array();
if (!socket_create_pair(AF_UNIX, SOCK_STREAM, 0, $sockets)) {
die(socket_strerror(socket_last_error()));
}
list($reader, $writer) = $sockets;
$pid = pcntl_fork();
if ($pid == -1) {
die('cannot fork');
} elseif ($pid) {
socket_close($reader);
$line = sprintf("{$group}\n", getmypid());
if (!socket_write($writer, $line, strlen($line))) {
socket_close($writer);
die(socket_strerror(socket_last_error()));
}
socket_close($writer);
} else {
socket_close($writer);
$line = socket_read($reader, 1024, PHP_NORMAL_READ);
$id = rtrim($line);
$child_id = getmypid();
socket_close($reader);
echo "\nChild {$child_id} has begun process: {$id}.";
....
echo "\nChild {$child_id} has finished process: {$id}.";
exit(0);
}
}
while(pcntl_waitpid(0, $status) != -1);
While forking != threading, pthreads essentially was an abstraction layer over forks anyway vs true threading so.... meh. Either way two different ways to handle this, depending on what your needs specifically are.
I would recommend the top one unless you really actually need to use a stream for something. In which case I would need more details to give you better insight. The bottom example should point you in the right direction if this is the case, and I can show you how to refine it to your needs if this code sample isn't sufficient.
I use thrift php client to connect to java thrift server. But got the error of
TSocket: timed out writing 78 bytes from 10.0.1.151:1234
I dig into the php source of thrift and find it was caused from timeout on function stream_select.
/**
* Write to the socket.
*
* #param string $buf The data to write
*/
public function write($buf)
{
$null = null;
$write = array($this->handle_);
// keep writing until all the data has been written
while (TStringFuncFactory::create()->strlen($buf) > 0) {
// wait for stream to become available for writing
$writable = #stream_select($null, $write, $null, $this->sendTimeoutSec_, $this->sendTimeoutUsec_);
if ($writable > 0) {
// write buffer to stream
$written = #stream_socket_sendto($this->handle_, $buf);
if ($written === -1 || $written === false) {
throw new TTransportException('TSocket: Could not write '.TStringFuncFactory::create()->strlen($buf).' bytes '.
$this->host_.':'.$this->port_);
}
// determine how much of the buffer is left to write
$buf = TStringFuncFactory::create()->substr($buf, $written);
} elseif ($writable === 0) {
throw new TTransportException('TSocket: timed out writing '.TStringFuncFactory::create()->strlen($buf).' bytes from '.
$this->host_.':'.$this->port_);
} else {
throw new TTransportException('TSocket: Could not write '.TStringFuncFactory::create()->strlen($buf).' bytes '.
$this->host_.':'.$this->port_);
}
}
}
That means the socket was blocked and was not able to write. But the socket is just open and should not be blocked. I try select immediately after pfsockopen
if ($this->persist_) {
$this->handle_ = #pfsockopen($this->host_,
$this->port_,
$errno,
$errstr,
$this->sendTimeoutSec_ + ($this->sendTimeoutUsec_ / 1000000));
} else {
$this->handle_ = #fsockopen($this->host_,
$this->port_,
$errno,
$errstr,
$this->sendTimeoutSec_ + ($this->sendTimeoutUsec_ / 1000000));
}
// Connect failed?
if ($this->handle_ === FALSE) {
$error = 'TSocket: Could not connect to '.$this->host_.':'.$this->port_.' ('.$errstr.' ['.$errno.'])';
if ($this->debug_) {
call_user_func($this->debugHandler_, $error);
}
throw new TException($error);
}
$write = array($this->handle_);
$writable = #stream_select($null, $write, $null, $this->sendTimeoutSec_, $this->sendTimeoutUsec_);
if ($writable === 0) {
die('123');
}
The result show that it is block right after it is opened!
When restart php-fpm, the error disappear for a while and come up again.
Here is the client code:
$socket = new TSocket('10.0.1.151', 1234, true);
$framedSocket = new TFramedTransport($socket);
$transport = $framedSocket;
$protocol = new TCompactProtocol($transport);
$transport->open();
$client= new userservice\UserServiceProxyClient($protocol);
$result = $client->findUser($id);
If I adjust the php-fpm configuration pm.max_children from 200 to 2, the error also disappear.
I tried increase timeout to 10 seconds and it timeout after 10 seconds.
Any idea for what's the cause of the problem and how to fix it?
I modify the php thrift lib. Change from
$writable = #stream_select($null, $write, $null, $this->sendTimeoutSec_, $this->sendTimeoutUsec_);
to
$writable = 1;
And the same for read.
The problem get fixed.
That means the connection is established ( I even use tcpdump to confirm the tcp connection is established) and writable, but select timeout. Strange things!
I got a TCP server up and running, which supports many client connections at the same time, together with reading and sending data. However, when client closes the connection, there is a delay on the server to detect a disconnected client.
// TCP socket created here
$clients = array($socket);
while (true) {
$read = $clients; // Copy of $clients
$write = NULL;
$except = NULL;
$num_changed_sockets = socket_select($read, $write, $except, NULL);
if ($num_changed_sockets === false) {
// ...
} else if ($num_changed_sockets > 0) { // Something changed
printb("In here."); // EDIT
if (in_array($socket, $read)) { // For new connections and removing host from the $clients copy
$newsock = socket_accept($socket);
if ($newsock !== false) {
printb("Client $newsock connected");
$clients[] = $newsock;
}
$key = array_search($socket, $read);
unset($read[$key]);
}
foreach ($read as $read_socket) { // The actual problem starts here?
printb("Read part."); // EDIT
$data = socket_read($read_socket, 128);
if ($data === false) {
printb("Client $read_socket disconnected");
$key = array_search($read_socket, $clients);
socket_close($clients[$key]);
unset($clients[$key]);
continue;
}
}
}
EDIT I:
I just added debug prints in the code. When a single client disconnects, server falls into infinite loop from $num_changed_sockets. Is there way to fix it?
EDIT II:
var_dump gives empty string after a client has disconnected.
EDIT III:
According to last edit, disconnecting the client, if socket_read returns an empty string seems to work. I Found a note from PHP's website:
Note: socket_read() returns a zero length string ("") when there is no more data to read.
Should not this return false, when client disconnects?
What I'm trying to achieve is a socket server that broadcasts some data to all connected peers. This is code of server loop:
while(TRUE) {
$read = array();
$read[] = $socket;
for($i = 0; $i < $max_clients; $i++) {
if($client_sockets[$i] != NULL) {
$read[$i+1] = $client_sockets[$i];
}
}
#This is broadcasting loop
foreach($client_sockets as $send_sock)
{
socket_write($send_sock, "broadcasting".PHP_EOL);
}
if( socket_select($read, $write, $except, NULL) === FALSE ) {
$errorcode = socket_last_error();
$errormsg = socket_strerror($errorcode);
die("Could not listen on socket : [$errorcode] $errormsg \n");
}
if( in_array($socket, $read) ) {
for($i = 0; $i < $max_clients; $i++) {
if($client_sockets[$i] == NULL) {
$client_sockets[$i] = socket_accept($socket);
if( socket_getpeername($client_sockets[$i], $client_address, $client_port) ) {
echo "Client $client_address : $client_port is now connected to us. \n";
}
$message = "Connected to php socket server".PHP_EOL;
socket_write($client_sockets[$i], $message);
break;
}
}
}
}
This code works fine, accepts multiple connections and broadcasts data, except for one moment: loop starts only after I type any input from any connected client via telnet. I know this is because socket_select waits for this input, but I don't know how to start broadcasting right after client is connected.
Appreciate any help on this, because I've got feeling that I'm terribly wrong somewhere.
Selfown answer
Problem indeed was in socket_select($read, $write, $except, NULL); and not actually a problem. Last NULL parameter starts endless block, awaiting for client response; to make script work properly, I've changed this timeout, so it looks looks this: socket_select($read, $write, $except, 0, 500000). On the client side, however, parameter should be set to NULL(NOT 0!), so script remains idle while waiting for server broadcasting message.
I've also discovered that such blocking behavior is similar to to socket_read() and socket_write() functions, which endless by default; to change it, use socket_set_option:
socket_set_option($master_socket, SOL_SOCKET, SO_RCVTIMEO, array("sec" => 0, "usec" => 1000));
socket_set_option($master_socket, SOL_SOCKET, SO_SNDTIMEO, array("sec" => 0, "usec" => 1000));
And 1000 microseconds is as low as I could get.
Actually, I've updated answer for somebody who voted up; but I hope that solution will be useful anyway.
I am using sockets in PHP to create a simple command line based chat. It works ok, but there is one main issue that is making it almost unusable. When there are multiple people in the chat and one person is typing a message and the other sends a message the person typing the message gets the message received appended to what they are typing. Is there anyway around this? I'm using stdin and stream select. Here is a piece from the client:
$uin = fopen("php://stdin", "r");
while (true) {
$r = array($socket, $uin);
$w = NULL;
$e = NULL;
if (0 < stream_select($r, $w, $e, 0)) {
foreach ($r as $i => $fd) {
if ($fd == $uin) {
$text = (fgets($uin));
fwrite($socket, $text);
} else {
$text = fgets($socket);
print $text;
}
}
}
}
All help is appreciated! Thanks!
The code outputs a message to stdout everytime a full string is waiting in $socket.
The only way to get around that is to put the text to a variable ($outtext) in stead of printing it. Then you can display it whenever you are ready to read it, such as before writing to the outgoing socket...
$uin = fopen("php://stdin", "r");
while (true) {
$r = array($socket, $uin);
$w = NULL;
$e = NULL;
$outtext = '';
if (0 < stream_select($r, $w, $e, 0)) {
foreach ($r as $i => $fd) {
if ($fd == $uin) {
$text = (fgets($uin));
print $outtext;
$outtext = '';
fwrite($socket, $text);
} else {
$text = fgets($socket);
$outtext .= $text;
}
}
}
}
The downside being that it will only display incoming text when you press enter. The only way around that would be to use something other than fgets().
I'm assuming this is just an experiment - event driven programming with Node.js or similar would be much better for this type of thing.