in certain apps, you sometimes need to do processing that is irrelevant to response. for example send push notifications after chat message etc. such tasks has no effect on response you will return to user.
what is best approach to run such tasks ?
example in an API for a blog, after post is created i want to send 201 to client and end connection. yet afterwords i want to send a curl call to push notification server, or trigger some data analysis and save it to disk. yet i dont want user to wait for such tasks to end.
methods i can think of
1. is sending connect: closed and content-length headers and flush out response, but this is not compatible with all servers and not all browsers.
2. trigger task using php exec function ! ? but how can i pass a json object to that function then :/ ?
so any ideas how we can accomplish this in async behaviour for php in a manner that would works in any server setup ?
You can take an example of how WordPress triggering wp-cron.php functionality by sending HEAD request to wp-cron.php using curl, which is perfectly fitted to your idea of sending the request and not waiting to respond.
I would do it with this code:
Register as many functions as required with register_shutdown_function of php:
register_shutdown_function('background_function_name_1');
register_shutdown_function('background_function_name_2');
write below lines after html end tag(if any) where all output has been printed (adjust time limit as per upper limit of script execution):
ignore_user_abort(true);
set_time_limit(120);
header('Connection: close');
header('Content-Length: ' . ob_get_length());
ob_end_flush();
flush();
Here the server will send output to browser and all the registered functions will be called in the order they were registered.
Related
I have a Wordpress website with a working order system. Now I want to make an Android app which displays every new order in a list view as soon as the order was made.
The last two days I thought about the following solutions:
Simple HTTP GET requests every 10 seconds
Websockets
MySQL binary log + Pusher Link
Server Sent Events
My thoughts (working with a LAMP stack):
Simple HTTP requests are obviously the most ineffective solution.
I figured out that websockets and Apache aren't working well together.
Feels quite hacky and I want to avoid any 3rd party service if I can.
4. Looks like this is the optimal way for me, however there are some problems with Apache/php and Server Sent Events from what I experienced.
I tried to implement a simple demo script but I don't understand why some of them are using an infinite while loop to keep the connection open and others don't.
Here is an example without a loop and here with an infinite loop, also here
In addition to that, when I tested the variant with the infinite loop, my whole page won't load because of that sleep() function. It looks like the whole server freezes whenever I use it.
Does anyone have an idea how to fix that? Or do you have other suggestions?
That is the code that causes trouble (copied from here) and added a missing curly bracket:
<?php
// make session read-only
session_start();
session_write_close();
// disable default disconnect checks
ignore_user_abort(true);
// set headers for stream
header("Content-Type: text/event-stream");
header("Cache-Control: no-cache");
header("Access-Control-Allow-Origin: *");
// Is this a new stream or an existing one?
$lastEventId = floatval(isset($_SERVER["HTTP_LAST_EVENT_ID"]) ? $_SERVER["HTTP_LAST_EVENT_ID"] : 0);
if ($lastEventId == 0) {
$lastEventId = floatval(isset($_GET["lastEventId"]) ? $_GET["lastEventId"] : 0);
}
echo ":" . str_repeat(" ", 2048) . "\n"; // 2 kB padding for IE
echo "retry: 2000\n";
// start stream
while(true){
if(connection_aborted()){
exit();
}
else{
// here you will want to get the latest event id you have created on the server, but for now we will increment and force an update
$latestEventId = $lastEventId+1;
if($lastEventId < $latestEventId){
echo "id: " . $latestEventId . "\n";
echo "data: Howdy (".$latestEventId.") \n\n";
$lastEventId = $latestEventId;
ob_flush();
flush();
}
else{
// no new data to send
echo ": heartbeat\n\n";
ob_flush();
flush();
}
}
// 2 second sleep then carry on
sleep(2);
}
?>
I'm thankful for every advice I can get! :)
EDIT:
The main idea is to frequently check my MySQL database for new entries and if there is a new order present, format the data nicely and send the information over SSE to my android application.
I already found libraries to receive SSEs on android, the main problem is on the server side.
Based on your question I think you could implement SSE - Server sent events, which is part of HTML5 standard. It is a one-way communication from server to client. It needs html/javascript and a backend language, e.g PHP.
The client will subscribe on events and when subscription is up and running the server will send any updates from the input data. As standard the update will be visible each 3 seconds. This can be adjusted though.
I would recommend you to first create a basic functioning web-browser-client as a start. When and if it is working as you expect, only then you would judge about the effort of building the client as an app.
You would probably need to add functions on the client-side, such as start/stop the subscription.
My understanding of users not recommending the combination of (server sent events) and Apache is the lack of control how many open connections there are and what would control the continuously need of closing of connections. This could lead to sever server performance problems.
Seems using for example node.js would not cause that problem.
Here are some start link:
MDN:
https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
Stream Updates with Server-Sent Events:
https://www.html5rocks.com/en/tutorials/eventsource/basics/
I've recently discovered EventSource, YUI3 has a Gallery module to normalise and fallback behaviour, that's what I've chosen to go with in my example as I use that framework already.
So I've searched about quite a bit, read many blogs, posts and examples, all of which show pretty much the same thing: How to set up basic SSE events. I now have 6 examples of open/message/error/close events firing.
What I don't have (what I'd hoped this link was going to give me) is an example of how to fire SSE events which are more useful to my application, I'm trying one called 'update'.
Here's is my basic test page: http://codefinger.co.nz/public/yui/eventsource/test.php (it might as well be an html file, there's no php code in here yet)
And here's the 'message.php' in the EventSource constructor:
<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache'); // recommended to prevent caching of event data.
/**
* Constructs the SSE data format and flushes that data to the client.
*
* #param string $id Timestamp/id of this connection.
* #param string $msg Line of text that should be transmitted.
*/
function sendMsg($id, $msg) {
echo "id: $id" . PHP_EOL;
echo "data: $msg" . PHP_EOL;
echo PHP_EOL;
ob_flush();
flush();
}
while(true) {
$serverTime = time();
sendMsg($serverTime, 'server time: ' . date("h:i:s", time()));
sleep(10);
}
// I was hoping calling this file with a param might allow me to fire an event,
// which it does dutifully, but no browsers register the 'data : update' - though
// I do see the response in Firebug.
if( $_REQUEST['cmd'] ){
sendMsg($serverTime, $_REQUEST['cmd'] );
}
?>
From the live example above, you can see that I've tried to use YUI's io module to send a request, with param, to fire my 'update' event when I click the 'update' button. It seems to work, as you can see in Firebug's Net panel, but my event isn't handled (I realise the script above will run that loop again, I just want to get my event handled in connected browsers, then I'll remove/cleanup).
Am I doing this part wrong? Or is there something more fundamental I'm doing wrong? I'm trying to push events in response to my UI's state changing.
This SO question seemed to come close, #tomfumb commented that his next question was going to be "how to send new events to the client after the initial connection is made - now I see that the PHP just has to never stop executing." But surely I'd only send events as they happen... and not continuously...
there are several issues in your approach:
The server-side code that reads the cmd parameter is unreachable because of the infinite loop that sends event data to the client.
You are trying to send an event from the client to the server. It is in the specification name - Server-Sent Events - the server is the sender and the client is the receiver of events. You have options here:
Use the appropriate specification for the job called Web Sockets which is a two-way communication API
Write the logic that makes the desired type of communication possible
If you choose to stay with the SSE API I see two possible scenarios
Reuse the same Event Source connection and store a pool of connections on the server. When the user sends subsequent XMLHttpRequest with the update command, get the EventSource connection from the pool, that was made by this visitor, and send response with it that specifies your custom event type, the default type is message. It is important to avoid entering in the infinite loop that would make another EventSource connection to the client, but the client does not handle it because he made the request with XMLHttpRequest and not with EventSource.
Make all requests with EventSource. Before making a new EventSource request, close the previous one - you can do this from the client or from the server. On the server check the parameters and then send data to client.
Also you can use XMLHttpRequest with (long) polling and thus avoiding the need of using EventSource. Because of the simplicity of your example I can't see a reason to mix the two type of requests.
I would like to run a PHP script in the background for logging visitor information (after the user's page has loaded), so that it doesn't slow the user down.
From what I have found, there are a few methods to achieving this, one is to launch a popen command, another is to use exec, and the last I know of is using fsockopen (source):
$socket = fsockopen($_SERVER['HTTP_HOST'],80,$errorno,$errorstr,10);
if($socket) {
$socketdata = “GET /background_script.php HTTP 1.1\r\nHost: “.$_SERVER['HTTP_HOST'].”\r\nConnection: Close\r\n\r\n”;
fwrite($socket,$socketdata);
fclose($socket);
}
My server doesn't allow the use of popen or exec, so that leaves me with fsockopen.
Is this the best method available to me?
Thanks for any support (:
EDIT:
Another possible solution which I have found could be to send Connection: close to the browser to stop the loading on the client side, and then include the background_script.
Is this solution recommended?
E.G:
ob_start();
echo $OUTPUT_DATA;
header("Content-Length: ".ob_get_length());
header('Connection: close');
ob_end_flush();
ob_flush();
flush();
//Do background visitor logging stuff here..
You could implement a queuing solution to fulfill your requirements which would work like this:
User logs in
PHP User function adds work to a 'login' queue
An 'always-on' PHP script (worker) is assigned work from the queue
Work is completed in the background
This approach is a little more advanced, in that you have to introduce a queue manager and php daemon(s) to handle the work, but it works great when completed.
I would suggest reading some docs on 'Gearman', which has built-in php user functions.
Scenario is as follows:
Call to a specified URL including the Id of a known SearchDefinition should create a new Search record in a db and return the new Search.Id.
Before returning the Id, I need to spawn a new process / start async execution of a PHP file which takes in the new Search.Id and does the searching.
The UI then polls a 3rd PHP script to get status of the search (2nd script keeps updating search record in the Db).
This gives me a problem around spawning the 2nd PHP script in an async manner.
I'm going to be running this on a 3rd party server so have little control over permissions. As such, I'd prefer to avoid a cron job/similar polling for new Search records (and I don't really like polling if I can avoid it). I'm not a great fan of having to use a web server for work which is not web-related but to avoid permissions issues it may be required.
This seems to leave me 2 options:
Calling the 1st script returns the Id and closes the connection but continues executing and actually does the search (ie stick script 2 at the end of script 1 but close response at the append point)
Launch a second PHP script in an asynchronous manner.
I'm not sure how either of the above could be accomplished. The first still feels nasty.
If it's necessary to use CURL or similar to fake a web call, I'll do it but I was hoping for some kind of convenient multi-threading approach where I simply spawn a new thread and point it at the appropriate function and permissions would be inherited from the caller (ie web server user).
I'd rather use option 1. This would also keep related functionality closer to each other.
Here is a hint how to send something to user and then close the connection and continue executing:
(by tom ********* at gmail dot com, source: http://www.php.net/manual/en/features.connection-handling.php#93441)
<?php
ob_end_clean();
header("Connection: close\r\n");
header("Content-Encoding: none\r\n");
ignore_user_abort(true); // optional
ob_start();
echo ('Text user will see');
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush(); // Strange behaviour, will not work
flush(); // Unless both are called !
ob_end_clean();
//do processing here
sleep(5);
echo('Text user will never see');
//do some processing
?>
swoole: asynchronous & concurrent extension.
https://github.com/matyhtf/swoole
event-driven
full asynchronous non-blocking
multi-thread reactor
multi-process worker
millisecond timer
async MySQL
async task
async read/write file system
async dns lookup
I first configure my script to run even after the HTTP request is over
ignore_user_abort(true);
then flush out some text.
echo "Thats all folks!";
flush();
Now how can I trick the browser into thinking the HTTP request is over? so I can continue doing my own work without the browser showing "page loading".
header(??) // something like this?
Here's how to do it. You tell the browser to read in the first N characters of output and then close the connection, while your script keeps running until it's done.
<?php
ob_end_clean();
header("Connection: close");
ignore_user_abort(true); // optional
ob_start();
echo ('Text the user will see');
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush(); // Will not work
flush(); // Unless both are called !
// At this point, the browser has closed connection to the web server
// Do processing here
echo('Text user will never see');
?>
Headers won't work (they're headers, so they come first)
I don't know of any way to close the http connection without terminating the script, though I suppose there's some obscure way of doing it.
Telling us what you want to do after the request is done would help us give better suggestions.
But generally, I'd be thinking about one of the following:
1) Execute some simple command-line script (using exec()) that looks like:
#!/bin/sh
php myscript.php <arg1> <arg2> .. <argN> &
Then kick that off from your http-bound script like:
<?PHP
exec('/path/to/my/script.sh');
?>
Or:
2) Write another program (possibly a continuously-running daemon, or just some script that is cronned ever so often), and figure out how your in-request code can pass it instructions. You could have a database table that queues work, or try to make it work with a flat file of some sort. You could also have your web-based script call some command-line command that causes your out-of-request script to queue some work.
At the end of the day, you don't want your script to keep executing after the http request. Assuming you're using mod_php, that means you'll be tying up an apache process until the script terminates.
Maybe this particular comment on php.net manual page will help: http://www.php.net/manual/en/features.connection-handling.php#71172
Theoretically, if HTTP 1.1 keep-alive is enabled and the client receives the amount of characters it expects from the server, it should treat it as the end of the response and go ahead and render the page (while keeping the connection still open.) Try sending these headers (if you can't enable them another way):
Connection: keep-alive
Content-Length: n
Where n is the amount of characters that you've sent in the response body (output buffering can help you count that.) I'm sorry that I don't have the time to test this out myself. I'm just throwing in the suggestion in case it works.
The best way to accomplish this is using output buffering. PHP sends the headers when it's good and ready, but if you wrap your output to the browser with ob_* you can control the headers every step of the way.
You can hold a rendered page in the buffer if you want and send headers till the sun comes up in china. This practice is why you may see a lot of opening <?php tags, but no closing tags nowadays. It keeps the script from sending any headers prematurely since there might some includes to consider.