I am currently working with a really really slow API and in many instances the website users have to wait for those to finish. E.g. when a contact form is sent and the information is sent via the API.
Now, I am wondering how I can speed the API calls up, at least for the user? Is it ok, do make an asynchronous AJAX-call to a separate PHP-file and make the API-call from there? If so, what happens if the user closes the page, while the API-call is still running? He might think, that everything is already sent.
Is it ok, do make an asynchronous AJAX-call to a separate PHP-file and
make the API-call from there?
Yes, definitely; that would be the best way.
If so, what happens if the user closes the page, while the API-call is
still running? He might think, that everything is already sent.
It likely is sent; the PHP script running the API call continues on its merry way, and it's only when it tries to send a response back (a confirmation or error, likely) that it finds the client went away. If the API call generates an email eventually, it will complete whether the user waits or not (unless there's an error in the API call itself).
If you have the ability to run cron or scheduled task, I would convert it to an offline process. E.g. save the database in the db locally and return immediately. Then write a script that will run periodically via cron to process the new entries.
I blogged any article about this a while called that describes pretty much this exact process: Building A Scalable Queueing System With PHP
Related
I have a website (Site A) where visitors arrive every day. Site A is only tracking the user and redirecting them to Site B, when it's done.
I would like a PHP based solution which allows me to start a specific task on the server and redirect the visitor right after it has started. Basically I would not want my users to wait while my PHP scripts (which are taking like 3-4 sec to complete) are finishing their work.
It is very much essential to redirect the user as soon as possible. I have thought about Cronjobs, but it's not good because it can only run specific times, as far as I know. Also cURL isn't good, because it needs to wait the server to finish loading (as I know).
Any solutions for this issue?
You could probably use a CURL call. There is a thing in php called ignore_user_abort(). Combine that with set_time_limit() and you will be able to create a PHP script that will continue running even if curl request is cancelled.
I have this scenario:
User submits a link to my PHP website and closes the browser. Now that the server has got the link it will analyse the submitted link (page) for the broken links and after it has completely analysed the posted link, it will send an email to the user. I have a complete understanding of the second part i.e. how to analyse the page for the broken links and send the mail to the user. Only problem that I have is how may I achieve this first part i.e. make the server keep running the actions on it's own even even if there is no request made by the client end?
I have learned that "Crontab" or a "fork" may work for me. What do you say about these? Is it possible to achieve what I want, using these? What are the alternatives?
crontab would be the way to go for something like this.
Essentially you have two applications:
A web site where users submit data to a database.
An offline script, scheduled to run via cron, which checks for records in the database and performs the analysis, sending notifications of the results when complete.
Both of these applications share the same database, but are otherwise oblivious to each other.
A website itself isn't suited well for this sort of offline work, it's mainly a request/response system. But a scheduled task works for this. Unless the user is expecting an immediate response, a small delay of waiting for the next scheduled run of the offline task is fine.
The server should run the script independently of the browser. Once the request is submitted, the php server runs the script and returns the result to the browser (if it has a result to return)
An alternative would be to add the request to a database and then use crontab run the php script at a given interval. The script would then check the database to see if there's anything that needs to be processed. You could limit the script to run one database entry every minute (or whatever works). This will help prevent performance problems if you have a lot of requests at once, but will be slower to send the email.
A typical approach would be to enter the link into a database when the user submits it. You would then use a cron job to execute a script periodically, which will process any pending links.
Exactly how to setup a cron job (or equivalent scheduled task) depends on your server. If you have a host which provides a web-based admin tool (such as CPanel), there will often be a way to do it in there.
PHP script will keep running after the client closes the broser (terminating the connection).
Only keep in mind PHP scripts maximum execution time is limited to "max_execution_time" directive value.
Of course here I suppose the link submission happens calling your script page... I don't understand if this is your use case...
For the sake of simplicity, a cronjob could do the wonders. User submits a link, the web handler simply saves the link into a DB (let me pretend here that the table is named "queued_links"). Then a cronjob scheduled to run each minute (for example), selects every link from queued_links, does the application logic (finds broken page links) and sends the email. It then also deletes the link from queued_links (or updates a flag to represent the fact that the link has already been processed.
In the sake of scale and speed, a cronjob wouldn't fit as well as a Message Queue (see rabbitmq, activemq, gearman, and beanstalkd (gearman and beanstalk are my favorite 2, simple and fit well with php)). In lieu of spawning a cronjob every minute, a queue processor listens for 'events' and asynchronously processes the 'events' (think 'onLinkSubmission($link)'), and processes the messages ASAP. The cronjob solution is just a simplified implementation of one of these MQ solutions, will result in better / more predictable results, but at the cost of adding new services to maintain, etc.
well, there are couple of ways, simplest of them would be:
When user submit a request, save this request some where, let's call it jobs table, and inform customer that his request has been received, they'll be updated site finish processing your request, or whatever suites you.
Now, create a (or multiple) scripts (depending upon requirement) and run this script from Cron, this script will pick requests from Job table, process it, do whatever required.
Alternatively, you can evaluate possibility of message_queue or may be using a Job server for this.
so, it all depends on your requirement.
I'm not an expert on http request so this question might be trivial for some. I'm sending a request to a php script which takes a lot of time to process a file and return a response. Is there a way to send a response before this script finishes its task to let the user know about the process status? Since this task can take up to several minutes I'd like to notify the user when key parts of the process are finished.
Note: I cannot break this request into several others
I might not have the correct approach here if so do you have other ideas how this could be handled?
Technically yes, but it would require you to have fine grained control of the http-stack, which you may or may not have in a typical php setup. I would suggest you look into other solutions (E.g. Make request to start the task - then poll to get an update on the progress)
http://www.redips.net/javascript/ajax-progress-bar/
here's a great article that goes over creating ajax a progress bar to use with php.
let me know if it doesn't make sense!
I think the best way for long proccessing requests is cron jobs. You can send request which will create 'task' and catch the task by cron job. Cron job can change task status while working and you can check task status via interval requests. I can't imagine another way to inform users about request proccessing. As soon as you response your headers are sent and PHP stops.
EDIT: it should be noted that Cron jobs are only available on Linux servers. windows servers would require access to the task scheduler, which most web hosts will not allow.
I am calling a php script over ajax to do some database maintenance. If the user closes the page, hits back, or clicks a link, will the php script be fully executed? Is there a way to do it?
Maybe if the php script called the exec() method or something similar, which would in turn call a script via the console as such:
$ php /var/www/httpdocs/maintenance.php
?
It's a race condition. PHP will detect at some point (usually upon attempting to do output) that Apache is yelling in its face that the remote user has closed the connection. Whether everything you wanted to do is done at that point depends on how your code's structured.
If you want to ensure that all operations are complete before the script shuts itself down, use ignore_user_abort(TRUE), which keeps PHP running after the connection is severed. It's still subject to the user max_execution_time limits and whatnot, but it will not shut down because you disconnected.
As long as the user agent (browser, etc.) has fully sent the request, the server has all it needs and will complete the request and try to send back a response.
In fact, this sort of "pinging" behavior is often used for "heartbeat"-like processes that keep a service warm or perform periodic maintenance.
Once the web request makes it to your server, it really doesn't matter if the user closes their browser or navigates away. Your server will still respond, but no one will be listening for the response.
Varies on the settings, web server, operating system and so on.
Usually the request will be processed as usual, and the response will just never be read. Occasionally, a write might fail earlier, and the request fails while processing.
Once the ajax call is kicked off, the user is free to do whatever they want. If they close the page they simply won't get the feedback (if any ) from the ajax call that was made.
If the php starts executing then it will continue to execute regardless if the user closes the window or navigates away from the page.
The php script will complete, regardless of browser state. The php is parsed on the server, and that doesn't care about whether the client is still open or not.
If the HTTP request was completed, then yes, the PHP script will be executed fully even if the client's computer is closed.
I'm currently running a Linux based VPS, with 768MB of Ram.
I have an application which collects details of domains and then connect to a service via cURL to retrieve details of the pagerank of these domains.
When I run a check on about 50 domains, it takes the remote page about 3 mins to load with all the results, before the script can parse the details and return it to my script. This causes a problem as nothing else seems to function until the script has finished executing, so users on the site will just get a timer / 'ball of death' while waiting for pages to load.
**(The remote page retrieves the domain details and updates the page by AJAX, but the curl request doesnt (rightfully) return the page until loading is complete.
Can anyone tell me if I'm doing anything obviously wrong, or if there is a better way of doing it. (There can be anything between 10 and 10,000 domains queued, so I need a process that can run in the background without affecting the rest of the site)
Thanks
A more sensible approach would be to "batch process" the domain data via the use of a cron triggered PHP cli script.
As such, once you'd inserted the relevant domains into a database table with a "processed" flag set as false, the background script would then:
Scan the database for domains that aren't marked as processed.
Carry out the CURL lookup, etc.
Update the database record accordingly and mark it as processed.
...
To ensure no overlap with an existing executing batch processing script, you should only invoke the php script every five minutes from cron and (within the PHP script itself) check how long the script has been running at the start of the "scan" stage and exit if its been running for four minutes or longer. (You might want to adjust these figures, but hopefully you can see where I'm going with this.)
By using this approach, you'll be able to leave the background script running indefinitely (as it's invoked via cron, it'll automatically start after reboots, etc.) and simply add domains to the database/review the results of processing, etc. via a separate web front end.
This isn't the ideal solution, but if you need to trigger this process based on a user request, you can add the following at the end of your script.
set_time_limit(0);
flush();
This will allow the PHP script to continue running, but it will return output to the user. But seriously, you should use batch processing. It will give you much more control over what's going on.
Firstly I'm sorry but Im an idiot! :)
I've loaded the site in another browser (FF) and it loads fine.
It seems Chrome puts some sort of lock on a domain when it's waiting for a server response, and I was testing the script manually through a browser.
Thanks for all your help and sorry for wasting your time.
CJ
While I agree with others that you should consider processing these tasks outside of your webserver, in a more controlled manner, I'll offer an explanation for the "server standstill".
If you're using native php sessions, php uses an exclusive locking scheme so only a single php process can deal with a given session id at a time. Having a long running php script which uses sessions can certainly cause this.
You can search for combinations of terms like:
php session concurrency lock session_write_close()
I'm sure its been discussed many times here. I'm too lazy to search for you. Maybe someone else will come along and make an answer with bulleted lists and pretty hyperlinks in exchange for stackoverflow reputation :) But not me :)
good luck.
I'm not sure how your code is structured but you could try using sleep(). That's what I use when batch processing.