I am making a script that scrapes certain data from a website, and may iterate over multiple pages on the site if certain conditions exist, a new page iteration requires a script reload since the function I am using in PHP to get the page I'm scraping can only be called once, when I was running the script in a browser I got around this by simply reloading with javascript. However, now I am trying to set this task with a cronjob and so the javascript won't work...Any suggestions?
Meta refresh is what are you looking for.
<meta http-equiv="refresh" content="5; url=http://example.com/">
If you're running php from CLI/cron you can build one more process that would use popen and open child processes one after another.
The only way to rerun a PHP script is to call it again since the server needs to run the script. if you can't use Javascript and you can't use the suggestion Vyktor offered then you just can't.
Basically the only thing JS and does is call the server to execute the script and return the results to you, just as if you are pressing F5 or hitting the refresh button.
if you can emulate this in another way, then perhaps you can work around it. but I don't think its practical.
Related
Situation:
My php/html page retrieves the contents of another page on a different domain every 5-10 minutes or so. I use a JavaScript setInterval() and a jquery .load() to request content from the other domain into an element on my page. Each time it retrieves content, javascript compares new content with the previous content and then I make an Ajax call to a php script that sends me an email of what the changes are.
Problem:
It's all working fine and dandy except for the fact that I need a browser constantly open, requesting the updates.
Question:
Is there a way to accomplish this with some sort of 'self executing' script on the server? Something that I would only have to start once, and it continues to run on it's own without needing a browser to be open as long as I want the script to run?
Thanks in advance!
P.S. I'm not a php/javascript expert by any means, but I can get my way around.
I believe the thing you are looking for is a cron job.
If your script relies on Javascript for proper execution, you will need to use a browser to accomplish your goals.
However, if you can alter your script to perform all of the functionality via PHP, perhaps using cURL to request the necessary data, you can use a cron job to execute the script at regular intervals.
If you're running a script at an interval, I would recommend using a bash script instead that runs in the background.
#!/bin/bash
while [ 1 ]
do
php "script.php"
sleep 300
done
Then you can run the script like nohup bash.sh. 300 seconds = 5 minutes.
My website runs simplexml commands to pull data from 2 different websites, and doesn't finish loading the page until after the functions have their responses.
This is really only 1-2 seconds, but it is noticable when regular webpages take milliseconds to load.
Since this code is already in PHP functions, how can I most efficiently load the page and execute the code after? I'm assuming that by the time the page loads, the functions will have executed as well, its just that the browser itself won't refresh and finish loading til execution completes.
Hope this makes sense to you.
Unfortunately, php runs on the server side before the page is loaded. That is what allows it to provide dynamically generated content to the page. If you want to load the page and then run the php functions, you should check out AJAX.
Ajax uses javascript to call external functions and change content on the page without a reload.
Create a webpage without calling any of these functions. Add some JavaScript to that page to make AJAX requests to PHP scripts that call the functions, then adds the returned results to the page.
You have a few options.
AJAX call -- once the important stuff loads, have JS send word to the server that it needs to do some process to complete loading. (rennekon and Dan Grossman seem to have already suggested this).
iframe similar to AJAX, but it does not require JS. Placed at the bottom of the HTML it can let the server know something needs to finish without worrying about any other rendering. (this can actually also be accomplished by any number of tags which make HTTP requests. img attacks are notorious for allowing this with vulnerable sites.)
Spawn a new thread. This is a bit more difficult/annoying, but it does not rely on user feedback to finish processing. You also may not be able to do this on most servers, but it is one way to finish processing in the background.
You can create a cron that would talk to the 2 different websites and store the data you need periodically and then when your page runs it would talk to the local version that cron stored for you taking the communication off of page render time
Running a PHP script that is doing a huge mysql query plus some crunching on the results. Because of this the script takes a long time to execute and may appear to be not working to the user.
Is there a way to provide feedback to the user that the script is running?
Perhaps way to print to the browser with each loop - indicating what record it's on... kind of a "live output buffer" or something?
Try using flush(). http://us3.php.net/flush
You could also have a main page, that uses Javascript/jQuery to request the work page. Then, Javascript could show a nice little loader box telling you the the page is still doing stuff!
Do the request in an iframe. That way the user sees a page while the results are still being loaded in the frame. Ajax would work as well.
I have a PHP script that takes about 10 minutes to complete.
I want to give the user some feedback as to the completion percent and need some ideas on how to do so.
My idea is to call the php page with jquery and the $.post command.
Is there a way to return information from the PHP script without ending the script?
For example, from my knowledge of this now, if I return the variable, the PHP script will stop running.
My idea is to split the script into multiple PHP files and have the .post run each after a return from the previous is given.
But this still will not give an accurate assessment of time left because each script will be a different size.
Any ideas on a way to do this?
Thanks!
You can echo and flush() output, but that's suboptimal and rather fragile solution.
For long operations it might be good idea to launch script in the background and store/updte script status in shared location.
e.g. you could lanuch script using fopen('http://… call, proc_open PHP CLI process or even just openg long-running script in an <iframe>.
You could store status in the database or in shared memory (using apc_store()).
This will let user to check status of the script at any time (by refreshing page, or using AJAX) and user won't lose track of the script if browser's connection times out.
It also lets you avoid starting same long script twice.
I am looking for a way to start a function on form submit that would not leave the browser window waiting for the result.
Example:
User fills in the form and press submit, the data from the form via javascript goes to the database and a function in php that will take several seconds will start but I dont want the user to be left waiting for the end of that function. I would like to be able to take him to another page and leave the function doing its thing server side.
Any thoughts?
Thanks
Thanks for all the replies...
I got the ajax part. But I cannot call ajax and have the browser move to another page.
This is what I wanted.
-User fills form and submits
-Result from the form passed to database
-long annoying process function starts
-user carries on visiting the rest of the site, independent of the status on the "long annoying process function"
By the way and before someone suggests it. No, it cannot be done by cron job
Use AJAX to call the php script, and at the top of the script turn on ignore_ user_ abort.
ignore_user_abort(true);
That way if they navigate away from the page the script will continue running in the backround. You can also use
set_time_limit(0);
to set a time limit, useful if you know your script will take a while to complete.
The most common method is:
exec("$COMMAND > /dev/null 2>&1 &");
Ah, ok, well you're essentially asking therefore, does PHP support threading, and the general answer is no... however...
there are some tricks you can perform to mimick this behaviour, one of which is highlighted above and involves forking out to a separate process on the server, this can be acheived in a number of ways, including the;
exec()
method. You also may want to look here;
PHP threading
I have also seen people try to force a flush of the output buffer halfway through the script, attempting to force the response back to the client, I dont know how successful this approach is, but maybe someone else will have some information on that one.
This is exactly what AJAX (shorthand for asynchronous JavaScript + XML) is for;
AJAX Information
It allows you to code using client side code, and send asynchronous requests to your server, such that the user's browser is not interuppted by an entire page request.
There is alot of information relating to AJAX out there on the web, so take a deep breath and get googling!
Sounds like you want to use some of the features AJAX (Asynchronous Javascript and XML - google) have to offer.
Basically, you would have a page with content. When a user clicks a button, javascript would be used to POST data to the server and begin processing. Simultaneously, that javascript might load a page from the server and then display it (eg, load data, and then replace the contents of a DIV with that new page.)
This kind of thing is the premise behind AJAX, which you see everywhere when you have a web page doing multiple things simultaneously.
Worth noting: This doesn't mean that the script is running "in the background on the server." Your web browser is still maintaining a connection with the web server - which means that the code is running in the "background" on the client's side. And by "background" we really mean "processing the HTTP request in parallel with other HTTP requests to give the feel of a 'background' running process"