Situation:
My php/html page retrieves the contents of another page on a different domain every 5-10 minutes or so. I use a JavaScript setInterval() and a jquery .load() to request content from the other domain into an element on my page. Each time it retrieves content, javascript compares new content with the previous content and then I make an Ajax call to a php script that sends me an email of what the changes are.
Problem:
It's all working fine and dandy except for the fact that I need a browser constantly open, requesting the updates.
Question:
Is there a way to accomplish this with some sort of 'self executing' script on the server? Something that I would only have to start once, and it continues to run on it's own without needing a browser to be open as long as I want the script to run?
Thanks in advance!
P.S. I'm not a php/javascript expert by any means, but I can get my way around.
I believe the thing you are looking for is a cron job.
If your script relies on Javascript for proper execution, you will need to use a browser to accomplish your goals.
However, if you can alter your script to perform all of the functionality via PHP, perhaps using cURL to request the necessary data, you can use a cron job to execute the script at regular intervals.
If you're running a script at an interval, I would recommend using a bash script instead that runs in the background.
#!/bin/bash
while [ 1 ]
do
php "script.php"
sleep 300
done
Then you can run the script like nohup bash.sh. 300 seconds = 5 minutes.
Related
I need to provide interaction between js on html-page and php-script.
- Use AJAX.
Ok. But the problem is that php-script executing for a long time, and i need to know the state of this script processing (e.g. 60% complete)
What should i do? Create 2 php-scripts (client&server) and do ajax-request to client.php which will do requests to server.php via sockets or smth?
Are there more elegant solutions?
What if you had the script doing the processing write its status to a file once in awhile. Make a second script that will read the file and return the status of the original one.
You should never have a long-running process being executed entirely within an HTTP session.
A simple and common approach to this problem is message queuing. Basically, you have your UI queue up the request into a database table and then have external daemon(s) process the queue.
To provide feedback, have the daemon periodically update the table with the status for the row it's currently working on. Then, your javascript code can make AJAX requests to a script that retrieves the status for that work item from the database and displays it to the user.
See: Dealing with long server-side operations using ajax?
Ajax call php script and return information that script is runing.
Main script create lock.file.
Script called from cron is checking if lock.file exists and run the correct script.
The correct script saves the current progress into progress.txt.
Ajax is reading progress.txt and when progress is 100% then return information that script processing is finished.
edited: Thanks to Justin for poiting the timeout problem ;)
If you want to be really fancy, write output from the php script to stdout, and capture it via a pipe. This would require running the php script using exec() or proc_open() (http://php.net/manual/en/function.proc-open.php) and pipe the output to a file (or if you want to be extra-extra fancy, use node.JS to listen for that data)
There are quite a few ways to accomplish this:
Node.JS
An Ajax query every x seconds
A META/Javascript page reload
An iframe that is routinely reloading with the status in it.
Good luck!
You could use PHP's output buffering (see ob_flush) to flush the contents at certain points in your script and tailor your JavaScript so that it uses the flushed contents. I believe readyState in your AJAX call won't be set to 4 on flushes so that's where you'll have to handle that yourself (see this article). I think its a much nicer way than writing to a file and checking the contents of that.
on your process.php:
// 1st task
$_SESSION['progress'] = 0;
// your code for the first task here ...
// 2nd task
$_SESSION['progress'] = 10;
// you code for 2nd task ...
// 3rd task
$_SESSION['progress'] = 17;
// continue ...
// everything finished?
$_SESSION['progress'] = 100;
on your progress.php:
// simply output
echo $_SESSION['progress'];
now from your client-side, just make a request to your progress.php, receive the number and give it to your progress bar ...
didn't check that by myself, but hope that it works! :)
I have a cron job running a php script, but theres some html and javascript that I need to execute for the actual script to work.
Converting the javascript to php isnt an option.
Basically I need it to act as though a person is viewing the page every time the cronjob runs.
EDIT:
the script uses javascript from a different site to encrypt some passwords so it is able to log into my account on the site, and the javascript is thousands of lines. The way the script flows is: Send data to website>get the data it sends back>use sites javascript to alter data>set html form value to value of data returned by javascript function>submit html form to get info back to php>send data to log me in. I know the code is very shoddy but its the only way i could think to do it without having to rewrite all the javascript they use to encrypt the password to php
Yau can try Node.JS to run JavaScript code on the server.
install your favorite web browser, and then have the cron job run the browser with the url as an argument.
something like
/usr/bin/firefox www.example.com/foo.html
you'll probably want to wait a minute or so and then kill the processes, or determine a better way to find when it finishes.
cronjobs always runs on server side only. when there is no client side - how can you expect javascript to work really???
anyway solution is: use cronjob to run another php script - which in fact calls the php script you want to run using CURL.
e.g. file1.php - file you want to execute and expect the javascript on that page to work.
file2.php - another file you create ... in this file use curl to call the file1.php ( make sure you provide full http:// path like you type in browser - you can pass values like get/post methods on html forms do as well ). in your cronjob - call file2.php.
Make sure curl is available and not any firewall rule blocking http calls i.e. port 80 calls to same server. Most of the servers both conditions above are fulfilled.
---------- sorry guys - Kristian Antonsen is right - so dont consider this as full answer at the moment. However I am leaving this on as someone might have food for thoughts from this -----
I am making a script that scrapes certain data from a website, and may iterate over multiple pages on the site if certain conditions exist, a new page iteration requires a script reload since the function I am using in PHP to get the page I'm scraping can only be called once, when I was running the script in a browser I got around this by simply reloading with javascript. However, now I am trying to set this task with a cronjob and so the javascript won't work...Any suggestions?
Meta refresh is what are you looking for.
<meta http-equiv="refresh" content="5; url=http://example.com/">
If you're running php from CLI/cron you can build one more process that would use popen and open child processes one after another.
The only way to rerun a PHP script is to call it again since the server needs to run the script. if you can't use Javascript and you can't use the suggestion Vyktor offered then you just can't.
Basically the only thing JS and does is call the server to execute the script and return the results to you, just as if you are pressing F5 or hitting the refresh button.
if you can emulate this in another way, then perhaps you can work around it. but I don't think its practical.
I have a PHP script that takes about 10 minutes to complete.
I want to give the user some feedback as to the completion percent and need some ideas on how to do so.
My idea is to call the php page with jquery and the $.post command.
Is there a way to return information from the PHP script without ending the script?
For example, from my knowledge of this now, if I return the variable, the PHP script will stop running.
My idea is to split the script into multiple PHP files and have the .post run each after a return from the previous is given.
But this still will not give an accurate assessment of time left because each script will be a different size.
Any ideas on a way to do this?
Thanks!
You can echo and flush() output, but that's suboptimal and rather fragile solution.
For long operations it might be good idea to launch script in the background and store/updte script status in shared location.
e.g. you could lanuch script using fopen('http://… call, proc_open PHP CLI process or even just openg long-running script in an <iframe>.
You could store status in the database or in shared memory (using apc_store()).
This will let user to check status of the script at any time (by refreshing page, or using AJAX) and user won't lose track of the script if browser's connection times out.
It also lets you avoid starting same long script twice.
Basically I need to get around max execution time.
I need to scrape pages for info at varying intervals, which means calling the bot at those intervals, to load a link form the database and scrap the page the link points to.
The problem is, loading the bot. If I load it with javascript (like an Ajax call) the browser will throw up an error saying that the page is taking too long to respond yadda yadda yadda, plus I will have to keep the page open.
If I do it from within PHP I could probably extend the execution time to however long is needed but then if it does throw an error I don't have the access to kill the process, and nothing is displayed in the browser until the PHP execute is completed right?
I was wondering if anyone had any tricks to get around this? The scraper executing by itself at various intervals without me needing to watch it the whole time.
Cheers :)
Use set_time_limit() as such:
set_time_limit(0);
// Do Time Consuming Operations Here
"nothing is displayed in the browser until the PHP execute is completed"
You can use flush() to work around this:
flush()
(PHP 4, PHP 5)
Flushes the output buffers of PHP and whatever backend PHP is using (CGI, a web server, etc). This effectively tries to push all the output so far to the user's browser.
take a look at how Sphider (PHP Search Engine) does this.
Basically you will just process some part of the sites you need, do your thing, and go on to the next request if there's a continue=true parameter set.
run via CRON and split spider into chunks, so it will only do few chunks at once. call from CRON with different parameteres to process only few chunks.