Related
I work on a somewhat large web application, and the backend is mostly in PHP. There are several places in the code where I need to complete some task, but I don't want to make the user wait for the result. For example, when creating a new account, I need to send them a welcome email. But when they hit the 'Finish Registration' button, I don't want to make them wait until the email is actually sent, I just want to start the process, and return a message to the user right away.
Up until now, in some places I've been using what feels like a hack with exec(). Basically doing things like:
exec("doTask.php $arg1 $arg2 $arg3 >/dev/null 2>&1 &");
Which appears to work, but I'm wondering if there's a better way. I'm considering writing a system which queues up tasks in a MySQL table, and a separate long-running PHP script that queries that table once a second, and executes any new tasks it finds. This would also have the advantage of letting me split the tasks among several worker machines in the future if I needed to.
Am I re-inventing the wheel? Is there a better solution than the exec() hack or the MySQL queue?
I've used the queuing approach, and it works well as you can defer that processing until your server load is idle, letting you manage your load quite effectively if you can partition off "tasks which aren't urgent" easily.
Rolling your own isn't too tricky, here's a few other options to check out:
GearMan - this answer was written in 2009, and since then GearMan looks a popular option, see comments below.
ActiveMQ if you want a full blown open source message queue.
ZeroMQ - this is a pretty cool socket library which makes it easy to write distributed code without having to worry too much about the socket programming itself. You could use it for message queuing on a single host - you would simply have your webapp push something to a queue that a continuously running console app would consume at the next suitable opportunity
beanstalkd - only found this one while writing this answer, but looks interesting
dropr is a PHP based message queue project, but hasn't been actively maintained since Sep 2010
php-enqueue is a recently (2017) maintained wrapper around a variety of queue systems
Finally, a blog post about using memcached for message queuing
Another, perhaps simpler, approach is to use ignore_user_abort - once you've sent the page to the user, you can do your final processing without fear of premature termination, though this does have the effect of appearing to prolong the page load from the user perspective.
When you just want to execute one or several HTTP requests without having to wait for the response, there is a simple PHP solution, as well.
In the calling script:
$socketcon = fsockopen($host, 80, $errno, $errstr, 10);
if($socketcon) {
$socketdata = "GET $remote_house/script.php?parameters=... HTTP 1.1\r\nHost: $host\r\nConnection: Close\r\n\r\n";
fwrite($socketcon, $socketdata);
fclose($socketcon);
}
// repeat this with different parameters as often as you like
On the called script.php, you can invoke these PHP functions in the first lines:
ignore_user_abort(true);
set_time_limit(0);
This causes the script to continue running without time limit when the HTTP connection is closed.
Another way to fork processes is via curl. You can set up your internal tasks as a webservice. For example:
http://domain/tasks/t1
http://domain/tasks/t2
Then in your user accessed scripts make calls to the service:
$service->addTask('t1', $data); // post data to URL via curl
Your service can keep track of the queue of tasks with mysql or whatever you like the point is: it's all wrapped up within the service and your script is just consuming URLs. This frees you up to move the service to another machine/server if necessary (ie easily scalable).
Adding http authorization or a custom authorization scheme (like Amazon's web services) lets you open up your tasks to be consumed by other people/services (if you want) and you could take it further and add a monitoring service on top to keep track of queue and task status.
http://domain/queue?task=t1
http://domain/queue?task=t2
http://domain/queue/t1/100931
It does take a bit of set-up work but there are a lot of benefits.
If it just a question of providing expensive tasks, in case of php-fpm is supported, why not to use fastcgi_finish_request() function?
This function flushes all response data to the client and finishes the request. This allows for time consuming tasks to be performed without leaving the connection to the client open.
You don't really use asynchronicity in this way:
Make all your main code first.
Execute fastcgi_finish_request().
Make all heavy stuff.
Once again php-fpm is needed.
I've used Beanstalkd for one project, and planned to again. I've found it to be an excellent way to run asynchronous processes.
A couple of things I've done with it are:
Image resizing - and with a lightly loaded queue passing off to a CLI-based PHP script, resizing large (2mb+) images worked just fine, but trying to resize the same images within a mod_php instance was regularly running into memory-space issues (I limited the PHP process to 32MB, and the resizing took more than that)
near-future checks - beanstalkd has delays available to it (make this job available to run only after X seconds) - so I can fire off 5 or 10 checks for an event, a little later in time
I wrote a Zend-Framework based system to decode a 'nice' url, so for example, to resize an image it would call QueueTask('/image/resize/filename/example.jpg'). The URL was first decoded to an array(module,controller,action,parameters), and then converted to JSON for injection to the queue itself.
A long running cli script then picked up the job from the queue, ran it (via Zend_Router_Simple), and if required, put information into memcached for the website PHP to pick up as required when it was done.
One wrinkle I did also put in was that the cli-script only ran for 50 loops before restarting, but if it did want to restart as planned, it would do so immediately (being run via a bash-script). If there was a problem and I did exit(0) (the default value for exit; or die();) it would first pause for a couple of seconds.
Here is a simple class I coded for my web application. It allows for forking PHP scripts and other scripts. Works on UNIX and Windows.
class BackgroundProcess {
static function open($exec, $cwd = null) {
if (!is_string($cwd)) {
$cwd = #getcwd();
}
#chdir($cwd);
if (strtoupper(substr(PHP_OS, 0, 3)) == 'WIN') {
$WshShell = new COM("WScript.Shell");
$WshShell->CurrentDirectory = str_replace('/', '\\', $cwd);
$WshShell->Run($exec, 0, false);
} else {
exec($exec . " > /dev/null 2>&1 &");
}
}
static function fork($phpScript, $phpExec = null) {
$cwd = dirname($phpScript);
#putenv("PHP_FORCECLI=true");
if (!is_string($phpExec) || !file_exists($phpExec)) {
if (strtoupper(substr(PHP_OS, 0, 3)) == 'WIN') {
$phpExec = str_replace('/', '\\', dirname(ini_get('extension_dir'))) . '\php.exe';
if (#file_exists($phpExec)) {
BackgroundProcess::open(escapeshellarg($phpExec) . " " . escapeshellarg($phpScript), $cwd);
}
} else {
$phpExec = exec("which php-cli");
if ($phpExec[0] != '/') {
$phpExec = exec("which php");
}
if ($phpExec[0] == '/') {
BackgroundProcess::open(escapeshellarg($phpExec) . " " . escapeshellarg($phpScript), $cwd);
}
}
} else {
if (strtoupper(substr(PHP_OS, 0, 3)) == 'WIN') {
$phpExec = str_replace('/', '\\', $phpExec);
}
BackgroundProcess::open(escapeshellarg($phpExec) . " " . escapeshellarg($phpScript), $cwd);
}
}
}
PHP HAS multithreading, its just not enabled by default, there is an extension called pthreads which does exactly that.
You'll need php compiled with ZTS though. (Thread Safe)
Links:
Examples
Another tutorial
pthreads PECL Extension
UPDATE: since PHP 7.2 parallel extension comes into play
Tutorial/Example
reference manual
This is the same method I have been using for a couple of years now and I haven't seen or found anything better. As people have said, PHP is single threaded, so there isn't much else you can do.
I have actually added one extra level to this and that's getting and storing the process id. This allows me to redirect to another page and have the user sit on that page, using AJAX to check if the process is complete (process id no longer exists). This is useful for cases where the length of the script would cause the browser to timeout, but the user needs to wait for that script to complete before the next step. (In my case it was processing large ZIP files with CSV like files that add up to 30 000 records to the database after which the user needs to confirm some information.)
I have also used a similar process for report generation. I'm not sure I'd use "background processing" for something such as an email, unless there is a real problem with a slow SMTP. Instead I might use a table as a queue and then have a process that runs every minute to send the emails within the queue. You would need to be warry of sending emails twice or other similar problems. I would consider a similar queueing process for other tasks as well.
It's a great idea to use cURL as suggested by rojoca.
Here is an example. You can monitor text.txt while the script is running in background:
<?php
function doCurl($begin)
{
echo "Do curl<br />\n";
$url = 'http://'.$_SERVER['SERVER_NAME'].$_SERVER['REQUEST_URI'];
$url = preg_replace('/\?.*/', '', $url);
$url .= '?begin='.$begin;
echo 'URL: '.$url.'<br>';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
echo 'Result: '.$result.'<br>';
curl_close($ch);
}
if (empty($_GET['begin'])) {
doCurl(1);
}
else {
while (ob_get_level())
ob_end_clean();
header('Connection: close');
ignore_user_abort();
ob_start();
echo 'Connection Closed';
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush();
flush();
$begin = $_GET['begin'];
$fp = fopen("text.txt", "w");
fprintf($fp, "begin: %d\n", $begin);
for ($i = 0; $i < 15; $i++) {
sleep(1);
fprintf($fp, "i: %d\n", $i);
}
fclose($fp);
if ($begin < 10)
doCurl($begin + 1);
}
?>
There is a PHP extension, called Swoole.
Although it might not be enabled, it is available on my hosting for being enabled at click of a button.
Worth checking it out. I haven't had time to use it yet, as I was searching here for info, when I stumbled across it and thought it worth sharing.
Unfortunately PHP does not have any kind of native threading capabilities. So I think in this case you have no choice but to use some kind of custom code to do what you want to do.
If you search around the net for PHP threading stuff, some people have come up with ways to simulate threads on PHP.
If you set the Content-Length HTTP header in your "Thank You For Registering" response, then the browser should close the connection after the specified number of bytes are received. This leaves the server side process running (assuming that ignore_user_abort is set) so it can finish working without making the end user wait.
Of course you will need to calculate the size of your response content before rendering the headers, but that's pretty easy for short responses (write output to a string, call strlen(), call header(), render string).
This approach has the advantage of not forcing you to manage a "front end" queue, and although you may need to do some work on the back end to prevent racing HTTP child processes from stepping on each other, that's something you needed to do already, anyway.
If you don't want the full blown ActiveMQ, I recommend to consider RabbitMQ. RabbitMQ is lightweight messaging that uses the AMQP standard.
I recommend to also look into php-amqplib - a popular AMQP client library to access AMQP based message brokers.
Spawning new processes on the server using exec() or directly on another server using curl doesn't scale all that well at all, if we go for exec you are basically filling your server with long running processes which can be handled by other non web facing servers, and using curl ties up another server unless you build in some sort of load balancing.
I have used Gearman in a few situations and I find it better for this sort of use case. I can use a single job queue server to basically handle queuing of all the jobs needing to be done by the server and spin up worker servers, each of which can run as many instances of the worker process as needed, and scale up the number of worker servers as needed and spin them down when not needed. It also let's me shut down the worker processes entirely when needed and queues the jobs up until the workers come back online.
i think you should try this technique it will help to call as many as pages you like all pages will run at once independently without waiting for each page response as asynchronous.
cornjobpage.php //mainpage
<?php
post_async("http://localhost/projectname/testpage.php", "Keywordname=testValue");
//post_async("http://localhost/projectname/testpage.php", "Keywordname=testValue2");
//post_async("http://localhost/projectname/otherpage.php", "Keywordname=anyValue");
//call as many as pages you like all pages will run at once independently without waiting for each page response as asynchronous.
?>
<?php
/*
* Executes a PHP page asynchronously so the current page does not have to wait for it to finish running.
*
*/
function post_async($url,$params)
{
$post_string = $params;
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "GET ".$parts['path']."?$post_string"." HTTP/1.1\r\n";//you can use POST instead of GET if you like
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
fclose($fp);
}
?>
testpage.php
<?
echo $_REQUEST["Keywordname"];//case1 Output > testValue
?>
PS:if you want to send url parameters as loop then follow this answer :https://stackoverflow.com/a/41225209/6295712
PHP is a single-threaded language, so there is no official way to start an asynchronous process with it other than using exec or popen. There is a blog post about that here. Your idea for a queue in MySQL is a good idea as well.
Your specific requirement here is for sending an email to the user. I'm curious as to why you are trying to do that asynchronously since sending an email is a pretty trivial and quick task to perform. I suppose if you are sending tons of email and your ISP is blocking you on suspicion of spamming, that might be one reason to queue, but other than that I can't think of any reason to do it this way.
I'm having a weird issue with PHP's sockets library: I do not seem to be able to detect/distinguish server EOF, and my code is helplessly going into an infinite loop as a result.
Further explanation below; first of all, some context (there's nothing particularly fancy going on here):
<?php
$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_connect($socket, '127.0.0.1', 8081);
for (;;) {
$read = [$socket];
$except = NULL;
$write = [];
print "Select <";
$n = socket_select($read, $write, $except, NULL);
print ">\n";
if (count($read)) {
print "New data: ";
#socket_recv($socket, $data, 1024, NULL);
$data = socket_read($socket, 1024);
print $data."\n";
}
print "Socket status: ".socket_strerror(socket_last_error())."\n";
}
The above code simply connects to a server and prints what it reads. It's a cut-down version of what I have in the small socket library I'm writing.
For testing, I'm currently using ncat -vvklp 8081 to bind a socket and be a server. With that running, I can fire up the code above and it connects and works - eg, I can type in the ncat window, and PHP receives it. (Sending data from PHP is working too, but I've excluded that code as it's not relevant.)
However, the moment I ^C ncat, the code above enters a hard infinite loop - and PHP says there's no error on the socket.
I am trying to figure out where the button is that whacks PHP upside the head and makes it realize that the peer has disconnected.
socket_get_status() is a great misnomer - it's an alias for stream_get_meta_data(), and it doesn't actually work on sockets!
feof() similarly spouts Warning: feof(): supplied resource is not a valid stream resource.
I can't find a socket_* function for detecting peer EOF.
One of the PHP manual notes for socket_read() initially dissuaded me from using that function so I used socket_recv() instead, but I eventually tried it just in case - but no dice; switching the receive call has no effect.
I have discovered that watching the socket for writing and then attempting to write to it will suddenly make PHP go "oh, wait, right" and start returning Broken pipe - but I'm not interested in writing to the server, I want to read from it!
Finally, regarding the commented part - I would far prefer to use PHP's builtin stream functionality, but the stream_* functions do not provide any means for handling asynchronous connect events (which I want to do, as I'm making multiple connections). I can do stream_socket_client(... STREAM_CLIENT_ASYNC_CONNECT ...) but then cannot find out when the connection has been established (6yo PHP bug #52811).
Okay, I figure I might as well turn the comments above into an answer. All credit goes to Ryan Vincent for helping my thick head figure this out :)
socket_recv will return 0 specifically if the peer has disconnected, or FALSE if any other network error has occurred.
For reference, in C, recv()'s return value is the length of the new data you've just received (which can be 0), or -1 to indicate an error condition (the value of which can be found in errno).
Using 0 to indicate an error condition (and just one arbitrary type of error condition, at that) is not standard and unique to PHP in all the wrong ways. Other network libraries don't work this way.
You need to to handle it like this.
$r = socket_recv($socket, $buf, $len);
if ($r === FALSE) {
// Find out what just happened with socket_last_error()
// (there's a great list of error codes in the comments at
// http://php.net/socket_last_error - considering/researching
// the ramifications of each condition is recommended)
} elseif ($r === 0) {
// The peer closed the connection. You need to handle this
// condition and clean up.
} else {
// You DO have data at this point.
// While unlikely, it's possible the remote peer has
// sent you data of 0 length; remember to use strlen($buf).
}
I am trying to implement a realtime chat application using PHP . Is it possible to do it without using a persistent data storage like database or file . Basically what I need is a mediator written in PHP who
accepts messages from client browsers
Broadcasts the message to other clients
Forgets the message
You should check out Web Sockets of html5. It uses two way connection so you will not need any database or file. Any chat message comes to the server will directly sent to the other users browser without any Ajax call. But you need also to setup web socket server.
Web sockets are used in many real time applications as well. I am shortly planing to write full tutorial on that. I will notify you.
Just tried something I had never done before in response to this question. Seemed to work but I only tested it once. Instead of using a Socket I had an idea of using a shared Session variable. Basically I forced the Session_id to be the same value regardless of the user therefore they are all sharing the same data. From a quick test it seems to work. Here is what I did:
session_id('12345');
session_start();
$session_id = session_id();
$_SESSION['test'] = $_SESSION['test'] + 1;
echo "session: {$session_id} test: {$_SESSION['test']} <br />";
So my thought process was that you could simply store the chat info in a Session variable and force everyone regardless of who they are to use a shared session. Then you can simply use ajax to continually reload the current Session variable, and use ajax to edit the session variable when adding a message. Also you would probably want to set the Session to never expire or have a really long maxlifetime.
As I said I just played around with this for a few minutes to see if it would work.
You will want to use Sockets. This article will cover exactly what you want to do: http://devzone.zend.com/209/writing-socket-servers-in-php/
When I tried to solve the same problem, I went with Nginx's Push Module. I chose to go this way since I had to support older browsers (that usually won't support WebSockets) and had no confidence in setting up an appropriate solution like Socket.io behind a TCP proxy.
The workflow went like this:
The clients connect through long-polling to my /subscriber location, which is open to all.
The /publisher location only accepts connections from my own server
When a client subscribes and talks, it basically just asks a PHP script to handle whatever data is sent.
This script can do validation, authorization, and such, and then forwards (via curl) the message in a JSON format to the /publisher.
Nginx's Push Module handles sending the message back to the subscribers and the client establishes a new long-polling connection.
If I had to do this all over again, then I would definitely go the Socket.io route, as it has proper fallbacks to Comet-style long-polling and has great docs for both Client and Server scripts.
Hope this helps.
If you have a business need for PHP, then adding another language to the mix just means you then have two problems.
It is perfectly possible to run a permanent, constantly-running daemonised PHP IRCd server: I know, because I've done it, to make an online game which ran for years.
The IRC server part I used is a modified version of WaveIRCd:
http://sourceforge.net/projects/waveircd/
I daemonised it using code I made available here:
http://www.thudgame.com/node/254
That code might be overkill: I wrote it to be as rugged as I could, so it tries to daemonise using PHP's pcntl_fork(), then falls back to calling itself recursively in the background, then falls back to perl, and so on: it also handles the security restrictions of PHP's safe mode in case someone turns that on, and the security restrictions imposed by being called through cron.
You could probably strip it down to just a few lines: the bits with the comments "Daemon Rule..." - follow those rules, and you'll daemonize your process just fine.
In order to handle any unexpected daemon deaths, etc, I then ran that daemoniser every minute through cron, where it checked to see if the daemon was already running, and if so either quietly died, or if the daemon was nonresponsive, killed it and took its place.
Because of the whole distributed nature of IRC, it was nicely rugged, and gave me a multiplayer browser game with no downtime for a good few years until bit-rot ate the site a few months back. I should try to rewrite the front end in Flash and get it back up again someday, when I have time...
(I then ran another daemonizer for a PHP bot to manage the game itself, then had my game connect to it as a java applet, and talk to the bot to play the game, but that's irrelevant here).
Since WaveIRCd is no longer maintained, it's probably worth having a hunt around to find if anyone else has forked the project and is supporting it.
[2012 edit: that said, if you want your front end to be HTML5/Javascript, or if you want to connect through the same port that HTTP connects through, then your options are more limited than when using Flash or Java. In that case, take the advice of others, and use "WebSockets" (poor support in most current browsers) or the "Socket.io" project (which uses WebSockets, but falls back to Flash, or various other methods, depending what the browser has available).
The above is for situations where your host allows you to run a service on another port. In particular, many have explicit rules in their ToS against running an IRCd.]
[2019 edit: WebSockets are now widely supported, you should be fine using them. As a relevant case study, Slack is written in PHP (per https://slack.engineering/taking-php-seriously-cf7a60065329), and for some time supported the IRC protocol, though I believe that that has since been retired. As its main protocol, it uses an API based on JSON over WebSockets (https://api.slack.com/rtm). This all shows that a PHP IRCd can deliver enterprise-level performance and quality, even where the IRC protocol is translated to/from another one, which you'd expect to give poorer performance.]
You need to use some kind of storage as a buffer. It IS plausable not to use file or db (which also uses a file). You can try using php's shared memory functions, but I don't know any working solution so you'll have to do it from scratch.
Is it possible to do it without using a persistent data storage like
database or file?
It is possible but you shouldn't use. Database or file based doesn't slows down chat. It will be giving additional security to your chat application. You can make web based chat using ajax and sockets without persistent data.
You should see following posts:
Is database based chat room bad idea?
Will polling from a SQL DB instead of a file for chat application increase performance?
Using memcached as a database buffer for chat messages
persistent data in php question
https://stackoverflow.com/questions/6569754/how-can-i-develop-social-network-chat-without-using-a-database-for-storing-the-c
File vs database for storage efficiency in chat app
PHP is not a good fit for your requirements (in a normal setup like apache-php, fastcgi etc.), because the PHP script gets executed from top to bottom for every request and cannot maintain any state between the requests without the use of external services or databases/files (Except e.g. http://php.net/manual/de/book.apc.php, but it is not intended for implementing a chat and will not scale to multiple servers.)
You should definitely look at Node.js and especially the Node.js module Socket.IO (A Websocket library). It's incredibly easy to use and rocks. Socket.IO can also scale to multiple chat servers with an optional redis backend, which means it's easier to scale.
Trying to use $_SESSION with a static session id as communication channel is not a solution by the way, because PHP saves the session data into files.
One solution to achieving this is by writing a PHP socket server.
<?php
// Set time limit to indefinite execution
set_time_limit (0);
// Set the ip and port we will listen on
$address = '192.168.0.100';
$port = 9000;
$max_clients = 10;
// Array that will hold client information
$clients = Array();
// Create a TCP Stream socket
$sock = socket_create(AF_INET, SOCK_STREAM, 0);
// Bind the socket to an address/port
socket_bind($sock, $address, $port) or die('Could not bind to address');
// Start listening for connections
socket_listen($sock);
// Loop continuously
while (true) {
// Setup clients listen socket for reading
$read[0] = $sock;
for ($i = 0; $i < $max_clients; $i++)
{
if ($client[$i]['sock'] != null)
$read[$i + 1] = $client[$i]['sock'] ;
}
// Set up a blocking call to socket_select()
$ready = socket_select($read,null,null,null);
/* if a new connection is being made add it to the client array */
if (in_array($sock, $read)) {
for ($i = 0; $i < $max_clients; $i++)
{
if ($client[$i]['sock'] == null) {
$client[$i]['sock'] = socket_accept($sock);
break;
}
elseif ($i == $max_clients - 1)
print ("too many clients")
}
if (--$ready <= 0)
continue;
} // end if in_array
// If a client is trying to write - handle it now
for ($i = 0; $i < $max_clients; $i++) // for each client
{
if (in_array($client[$i]['sock'] , $read))
{
$input = socket_read($client[$i]['sock'] , 1024);
if ($input == null) {
// Zero length string meaning disconnected
unset($client[$i]);
}
$n = trim($input);
if ($input == 'exit') {
// requested disconnect
socket_close($client[$i]['sock']);
} elseif ($input) {
// strip white spaces and write back to user
$output = ereg_replace("[ \t\n\r]","",$input).chr(0);
socket_write($client[$i]['sock'],$output);
}
} else {
// Close the socket
socket_close($client[$i]['sock']);
unset($client[$i]);
}
}
} // end while
// Close the master sockets
socket_close($sock);
?>
You would execute this by running it through command line and would always have to run for your PHP clients to connect to it. You could then write a PHP client that would connect to the socket.
<?php
$fp = fsockopen("www.example.com", 80, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
$out = "GET / HTTP/1.1\r\n";
$out .= "Host: www.example.com\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
while (!feof($fp)) {
echo fgets($fp, 128);
}
fclose($fp);
}
?>
You would have to use some type of ajax to call with jQuery posting the message to this PHP client.
http://devzone.zend.com/209/writing-socket-servers-in-php/
http://php.net/manual/en/function.fsockopen.php
Better use a node.js server for this. WebSockets aren't cross-browser nowadays (except socket.io for node.js that works perfect)
in short answer, you can't.
the current HTTP/HTML implementation doesn't support the pushstate so the algorithm of your chat app should follow :
A: sent message
B,C,D: do while a new message has been sent get this message.
so the receivers always have to make a new request and check if a new message has been sent. (AJAX Call or something similar )
so always there are a delay between the sent event and the receive event.
which means the data must be saved in something global, like db or file system.
take a look for :
http://today.java.net/article/2010/03/31/html5-server-push-technologies-part-1
You didn't say it had to all be written it PHP :)
Install RabbitMQ, and then use this chat implementation built on top of websockets and RabbitMQ.
Your PHP is pretty much just 'chat room chrome'. It's possible most of your site would fit within the 5 meg limit of offline HTML5 content, and you have a very flexible (and likely more robust than if you did it yourself) chat system.
It even has 20 messages of chat history if you leave the room.
https://github.com/videlalvaro/rabbitmq-chat
If You need to use just PHP, then You can store chat messages in session variables, session could be like object, storing a lot of information.
If You can use jQuery then You could just append paragraph to a div after message has been sent, but then if site is refreshed, messages will be gone.
Or combining, store messages in session and update that with jQuery and ajax.
Try looking into socket libraries like ZeroMQ they allow for instant transport of the message, and are quicker than TCP, and is realtime. Their infrastructure allows for instant data send between points A and B, without the data being stored anywhere first (although you can still choose to).
Here's a tutorial for a chat client in ZeroMQ
I need to open a log file for writing. Trouble is, many things may do this at the same time, and I don't want conflicts. Each write will be a single line, generally about 150 bytes (and always less than 1K), and getting things in chronological order is not strictly required.
I think what I want is to attempt to flock(), and if it fails, keep trying for a few seconds. If a lock can't be established after a number of tries, then give up.
$fh=fopen($logfile, "a");
if (flock($fh, LOCK_EX|LOCK_NB)) {
$locked=TRUE;
} else {
$locked=FALSE;
// Retry lock every 0.1 seconds for 3 seconds...
$x=0; while($x++ < 30) {
usleep(100000);
if (flock($fh, LOCK_EX|LOCK_NB)) {
$locked=TRUE;
break;
}
}
}
if ($locked) {
if (fwrite($fh, strftime("[%Y-%m-%d %T] ") . $logdata . "\n")) {
print "Success.\n";
} else {
print "Fail.\n";
}
flock($fh, LOCK_UN)
} else {
print "Lock failed.\n";
}
I have two questions, one general and one specific. First, aside from implementing the same solution in different ways (do...while, etc), is there a better general strategy for handling this kind of problem, that runs solely in PHP? Second, is there a better way of implementing this in PHP? (Yes, I separated these because I'm really interested in the strategy part.)
One alternative I've considered is to use syslog(), but the PHP code may need to run on platforms where system-level administration (i.e. adding things to /etc/syslog.conf) may not be available as an option.
UPDATE: added |LOCK_NB to the code above, per randy's suggestion.
In my long experience in PHP making logs (under linux!) I've never experienced problems of conflicts (even with hundreds of simultaneus and concurrent writes). So simple skip lock maagement:
$fh=fopen($logfile, "a");
if (fwrite($fh, strftime("[%Y-%m-%d %T] ") . $logdata . "\n")) {
print "Success.\n";
} else {
print "Fail.\n";
}
fclose($fh);
About this strategy the file logging (with or without lock) is not the best solution, because every fopen with the "a" imply a seek syscall to set the cursor at the end of the file. syslog keeping the file open avoid this overhead.
Of course the overhead become significative (in performance) with "big" files, an easy solution is create log file with date (or date-time) in the name.
ADD
apache package include a tester program: ab, allowing doing query in concurrency, you can test my thesis stressing your server with 1000000 of query done by 10to1000 threads.
ADD - following the comment
No, it'not an impossible task.
I found a note from http://php.net/manual/en/function.fwrite.php
If handle was fopen()ed in append mode, fwrite()s are atomic (unless
the size of string exceeds the filesystem's block size, on some
platforms, and as long as the file is on a local filesystem). That is,
there is no need to flock() a resource before calling fwrite(); all of
the data will be written without interruption.
to know how big is a block in bytes (usual 4k):
dumpe2fs /dev/sd_your_disk_partition |less -i
The "atomicity" of write is realized blocking other "agents" to write (when you see process in "D" status in "ps ax"), BUT PHP stream facility can solve this problem, see: *stream_set_blocking*.
This approach can introduce partial writes, so you will have to validate the integrity of your records.
In any case a fwrite (network or file) is susceptible to block/failure regardless of use of flock. IMHO flock introduce only overhead.
About your original questions AND your goal (trying to implement corporate policy in a highly risk-averse environment), where even a fwrite can be problematic, I can only imagine a simple solution: use a DB
most of complexity is out of PHP and written in C!
total control of the flow of the operation
hi level of concurrency
Well, I've been searching for material for this one, and failed to find anything. I am familiar with the IRC protocol, and I want to write IRC Services for my server (running on UnrealIRCD)
I prefer PHP, as I am most proficient there. Can anyone point me at some good tutorials or even explain here how to start? (I only need the basic protocol/syntax and server settings, I'll do the actual functions and abilities myself (hopefully :P)).
Thanks in advance.
A friend of mine me did this one already for InspIRCd. It never was finished, but the basics were working. We simply connected to the IRC server via a Socket-Connection (fsockopen in the first version, socket_create in the second one).
For communication we implemented an interface to InspIRCd with contained the specific lines as stated in the protocol. These methods could then be called by the core.
Reading uncompiled C source code of the Anope services got me pointed in the right direction, although it's a bit of a trial and error as to where to look inside it. I can't remember since it was quite a while ago when I tried my own PHP services, but if each client the services connects requires a PING/PONG reply to keep the connection alive, then won't PHP kinda fall down? The threading isn't that great, Linux only as far as I can remember.
EDIT: Good Lord, I didn't realize how old this question was. That being said, someone might use it so...answer below.
I've done this in PHP so I can tell you it can be done (source code not released).
That being said, I'll throw a basic core at you here:
$socket = socket_create(AF_INET, SOCK_STREAM, 6);
// Prevent fast reconnects if server down but try to connect every ten seconds.
while ( !socket_connect($socket,ADDRESS,PORT) ) {
sleep(10);
}
// set non blocking to avoid 'waiting' for data
socket_set_nonblock($socket);
// INTENTIONAL INFINITE LOOP
while(1) {
$read = array($socket);
// check if there's anything to read.
$num_changed_sockets = #socket_select($read, null, null, 0, 1);
if ( $num_changed_sockets > '0' ) {
// process the socket data here
processSocket(socket_read($socket,10000,PHP_NORMAL_READ));
}
}
Hope this helps you get started!