I wrote a page (cron.php) that uses the imap library to connect to a mailbox, parse messages, and store them in a database, then echos the results in json. I have a few dozen mailboxes that I need to run this same process for, and so I put together a page (mailboxes.php) that lists all these accounts, each with a button that when clicked, essentially hits cron.php via AJAX and parses the json response to update the page when the process is completed.
I've noticed however that if I click each of these boxes, they return as if running serially, not in parallel. Is there a configuration option someplace that might explain this?
Yeah you need to use session_write_close() on the cron.php file. session_write_close
Are you using sessions? Every time you run session_start for given session, it is being locked, until the script finishes, or the session is being 'detached'.
Related
I have a webform that sends data to PHP script.
PHP script may take a while to process the data. What I want to do is to send raw data to database, then redirect the visitor to "thank you" page and then continue processing the data in background. Important thing is that the script must continue working even if the visitor closes "thank you" page.
Can you advise which solution should I look into?
P.S. I use nginx + php-fpm if that matters.
UPDATE. I've found info about using ignore_user_abort(true). Could this be the way to go?
What I want to do is to send raw data to database, then redirect the visitor to "thank you" page and then continue processing the data in background.
That basically describes how I'd do it right there, actually.
Consider two separate applications. One is the web application, which saves the user input to the database and then continues to interact with the user. The other is a scheduled console application (a standalone script invoked by cron most likely) which looks for data in the database to be processed and processes it.
The user uploads the data, receives a "thank you" message, and his/her interaction is complete. The next time the scheduled task runs (every couple minutes, maybe?) it sees the pending data in the database, flags it as being processed (so if another instance of the script runs it doesn't also try to process the same data), processes it, flags it as being done (so it doesn't pick it up again next time), and completes.
You can notify the user of the completed process a couple of different ways. The back-end script can send the user an email (active notification), or perhaps the web application can examine the table for the flagged completed records the next time the user visits the page (passive notification).
Something like this should work :
pclose(popen('php script.php &', 'r'));
http://fr2.php.net/manual/fr/function.popen.php
You can also use more options or others functions to get more control over the execution :
http://fr2.php.net/manual/fr/function.proc-open.php
But use this carefully and be sure you need this way to resolve your problem.
Ajax would be nice.
You need to do the thing asynchronously. Use AJAX to achieve this
I would like to find a way to have my user not having to wait for the output of a php script and being redirected to a page while the script is running on the server.
Basically the user submits a form which takes quite long to process and I would like to redirect the user to a page notifying him that the form is being processed and that its output will be later available (I thought about opening a tab when the output is ready).
Basically I would like something like this, which I tried without success, the
if ($form_valid) {
process_form(); // this would need not to be running on the current page so that the user don't have to wait for it to be ready (timeout problems)
header('Location: http://example.com/form_submitted_output_coming_soon.html');
}
I hope that it is not too vague.
Thank you in advance for any help / advice on how I could do that.
It really depends on the time the script takes to execute if it's seconds, under 10 I would do an ajax request and have a modal progress message
If they take extended amounts of time my approach would be to create or use an existing task scheduler/ report generator
a single system scheduled task entry calling a central management script ( probably not optimal )
You mark a task/report for execution
Concurrency. Count, limit the number currently executing ( so you don't over load the server)
users pool via ajax for their tasks / reports or push to the clients with web sockets
Example on how to fork php to background
Update
I think you would get better performance out of a bot continuously check a database or file for work to do and submitting results back to the database. Alerting users via ajax, web sockets and or email when the work that they need is done / updated.
Here is a good introduction on how to build a web crawler in php
The best approach for solving this kind of problem is to use AJAX to make the request to the server in the background and then update the user once it has finished processing.
You may submit the form with an asynchronous request (ajax) and handle the page forward also with javascript. This way your form is handled asynchronously - you may even wait for the response to tell the user once you have an answer. This asynchronous request will not block the UI.
Just for completeness if you really really want to use php only for this:
Run PHP Task Asynchronously
I have a signup form that calls a PHP script which can interact with our CRM's API, like so:
CRM API <--> PHP script <--> Signup form
The signup form passes some information to the PHP script in one
AJAX call
The PHP script run a dozen API calls to the CRM to create
an account and attach various data
The CRM returns the new account id it just created to the PHP script
The PHP script passes the account id back to the signup form, at which point the AJAX call is complete and the signup form can continue.
The problem is #2, those dozen calls take about 20 seconds to complete, but the data the signup form needs is generated after the first API call so it could in theory return that data much sooner and do the rest of the stuff server side without holding that AJAX call open the whole time.
I tried flush() and ob_flush() which does output account id to the client before processing is complete, but the jQuery AJAX connection remains open so I'm still stuck waiting for the connection to be closed on the signup form side before anything happens.
So what's the easiest route for returning that account id to the form as fast as possible?
Maybe break out using curl and exec?
if(signing up){
stuff
exec(curl myself, notsignup)
}
else {
bunch of api calls
}
You should probably think about creating a seperate process for the rest of the steps that are needed. One way is that you could after the #1 first api calls has been completed. It responds back to the user, and doesn't try to complete the rest of the 20 calls on the user side.
Then create a queue that will finish the rest. You could always create a table in mysql to store the queue.
Next just create a cronjob that will run in the background, knocking the queue out.
Note: You will not want this cronjob to just start and never stop. Maybe have it run every 5 minutes, but before it starts to run, check to see if another cron is still in progress. If it is then it will check in another 5 minutes to see if it is ok to run.
Hope this helps!
If you only need the information from the first API call to return the form, then I would probably try a different workflow:
Form calls PHP Script
PHP Calls first API Call
PHP Returns to Form
Form processes response
Form calls second PHP Script to complete the process
PHP finishes API Calls (the form can abandon at this point since it sounds like you don't care what happens from here on out).
The workflow requires a little more work and co-ordination for the developer, but presents the most responsive interface to the user.
Here is the scenario:
I have a page that is logging data to MYSQL. I have another page that reads that data and allows it to be viewed. When a new piece of data is logged I would like to have the first script check and see if the viewing page is open in the browser, and if so append the newest data to the end of the view. Also - could anyone point to some info giving an overview of how PHP and the browser interact? I think I have the concept of the DOM down for javascript...but as far as PHP it just appears that once the page is sent, that's it...
You're correct in that once the PHP is sent, that's it.
There is no way to send data to a PHP page once the page is loaded. There is another slightly nastier method, but the easiest way of doing this is going to be polling the page via Ajax.
So, have a script that every 20 seconds, sends a message to another PHP script that contains the timestamp of the last MySQL log you received, then get the script to return all the data that has been set by that time.
I'm unsure how new you are to JavaScript, but the easiest way of doing that is probably using JQuery's $.ajax and encoding the new MySQL records as JSON.
No this isn't possible as you describe. The viewing page will have to poll the server for changes, either by periodically reloading itself, or by javascript / AJAX.
You are right that once the page is sent by PHP it can have no further influence. In fact the PHP execution thread on the server is killed as soon as output is complete, so the thing that generated the page no longer even exists.
To expand on Dolondro's suggestion, rather than periodically polling the server for updates, you could use Server-Sent-Events (newly supported in modern browsers).
With these, you basically just send 1 ajax request to the server, and the connection is held open. Then, the server can send updates whenever it wants. When the browser receives an event, it can add the data to the screen. Even still, the connection is held open, and the server can send additional events/updates as they occur.
W3C page:
http://dev.w3.org/html5/eventsource/
Wikipedia:
http://en.wikipedia.org/wiki/Server-sent_events
More Info:
https://www.google.com/search?ix=hcb&sourceid=chrome&ie=UTF-8&q=server+sent+events
I've looked all over for any answer on this and haven't been able to find one, hopefully someone can point me in the right direction, I think I'm close
I have two hosts let's call them host1.mydomain.com and host2.mydomain.com (to get around the 2 concurrent connections per host/per browser issue), so they both point to the same content one is just an alias of the other
User goes to host1.mydomain.com, enters some information to register, clicks Go, which loads an iframe on the same page pointing to a page on host2.mydomain.com which calls a php script via exec("curl") sending the request to the background to start a website scraper, the process ID is then stored in the database for the user. After the iframe has successfully loaded (only takes 1 second since it's creating a background process) I have an AJAX request set on an interval to check the status periodically of the cURL process (by it's process ID in the database) so that I can display the current step of the scraper (there are 6 steps in total). All good so far.
The problem is that the AJAX requests are timing out after step 4 of the scraper (browser default timeout is 115/120 seconds) even though it shouldn't be because I'm working with two different hosts...meaning to say that it's almost as if I'm clogging both connections on host1.mydomain.com when I'm not because I initiated the scraper from host2
The iframe loads this URL: http://host2.mydomain.com/page.php
The contents of the PHP script calls:
exec("curl -o /dev/null 'http://host2.mydomain.com/page.php?method=process' > /dev/null & echo $!", $op);
Then my ajax request is polling http://host1.mydomain.com/status.php?pid=x which looks up in the database to check the status by the process ID
and once the scraper gets to step 4, my ajax requests are timing out
I think I confused myself explaining this, but hopefully someone can help me
Turns out I was successfully getting around the 2 connections per server/browser limitation...however in doing some research I found the reason why my ajax request was hanging is because I was trying to access and write to the session data from both of the requests. Digging a little deeper I found a session_write_close() which closes the session for reading/writing, I basically have to call this after each page request of the scraper and then reinitialize the session, this allows my ajax requests to go through and stops the blocking of the request.
Hopefully someone else finds this useful if you stumble across the same issue
Cheers!
Jeff
Instead of waiting for the request to finish, you should spawn new process which runs in the background on server. And use javascript to "check back" each few seconds to see when the execution has finished. Then all you have to do is pick up the result and display it.
Additionally you might want to make sure that only one php process is spawned.