I would like to set a cron job in cpanel to run these different pages. I thought it would be easier to put them in one file. I know how to set these up to run individually, but the way it is written below, it won't run.
What do I need to change to get it to run smoothly?
<?php
ini_set('max_execution_time', 18000);
exec('/usr/bin/php -q /home2/sample/public_html/linktest/myapp-page1.php');
sleep (120);
exec('/usr/bin/php -q /home2/sample/public_html/linktest/myapp-page2.php');
sleep (120);
exec('/usr/bin/php -q /home2/sample/public_html/linktest/myapp-page3.php');
sleep (120);
exec('/usr/bin/php -q /home2/sample/public_html/linktest/myapp-page4.php');
sleep (120);
exec('/usr/bin/php -q /home2/sample/public_html/linktest/myapp-page5.php');
sleep (120);
echo 'Cron ran successfully';
?>
Thanks!
Or you could use wget and load a set of URLs from a file
wget -i CronScripts.txt
These would have to be accessible from the outside world though
Is it possible you are running in safe_mode and not allowed to modify max_execution_time?
you can use this technique it will help to call as many as pages you like all pages will run at once independently without waiting for each page response as asynchronous.
cornjobpage.php //mainpage
<?php
post_async("http://localhost/projectname/testpage.php", "Keywordname=testValue");
//post_async("http://localhost/projectname/testpage.php", "Keywordname=testValue2");
//post_async("http://localhost/projectname/otherpage.php", "Keywordname=anyValue");
//call as many as pages you like all pages will run at once independently without waiting for each page response as asynchronous.
?>
<?php
/*
* Executes a PHP page asynchronously so the current page does not have to wait for it to finish running.
*
*/
function post_async($url,$params)
{
$post_string = $params;
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "GET ".$parts['path']."?$post_string"." HTTP/1.1\r\n";//you can use POST instead of GET if you like
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
fclose($fp);
}
?>
testpage.php
<?
echo $_REQUEST["Keywordname"];//case1 Output > testValue
?>
PS:if you want to send url parameters as loop then follow this answer :https://stackoverflow.com/a/41225209/6295712
you need to make sure that the exec function is allowed in your php.
Related
I have a web application I'm working on where the user clicks a button to initiate an ajax request to a php file. The php file takes a very long time to run which forces the user to wait until the php files has finished running before notifying the user the request has been completed.
Here is a sample of the code I'm using:
jQuery caller:
$('#button').click(function(){
$.ajax({
type: "POST",
url: 'index-process.php',
success: function (data) {
console.log(data);
alert('finished');
}
});
});
index-process.php
<?php
/// placeholder for a long script
sleep(60);
echo "finished processing";
?>
I'm looking for some kind of work around that will allow me to notify the user that the request was submitted. Then allow the code to finish running in the background. I don't necessarily need my php script to return any values to the user. It just needs to execute.
So far I've tried something like this using a curl request with 2 different php files, but it still forces the user to wait until both php files have finished running before finishing the ajax request:
index-process.php
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'index-process2.php');
curl_setopt($ch, CURLOPT_WRITEFUNCTION, 'do_nothing');
curl_exec($ch);
curl_close($ch);
echo "finished processing";
function do_nothing($curl, $input) {
return 0; // aborts transfer with an error
}
?>
index-process2.php
<?php
ob_end_clean();
ignore_user_abort(true);
ob_start();
header("Connection: close");
header("Content-Length: " . ob_get_length());
ob_end_flush();
flush();
/// placeholder for a long script
sleep(60);
?>
If there isn't a good solution using php is there a possible solution using jQuery? If so, could somebody show me the proper way of coding this?
fsockopen is the best answer I could come up on this one. Hope this helps somebody down the road. The following code allows me to call file1.php via ajax which will send data to file2.php. Once file1.php sends data to file2.php using fsockopen it does not wait for a response. Additional code can be run in file1.php while file2.php is doing it's thing. File1.php can echo a response to the ajax request right away even while file2.php is still running.
This answer is only useful for someone who needs to run a long executing script that requires data in but no data back in return.
file1.php called via ajax request:
$vars = array('hello' => 'world');
$post_data = http_build_query($vars);
/// ssl and 443 are used for https, change to tls and 80 for http
/// only the main website domain is needed
/// do not put the full path to the file you need to call here
$fp = fsockopen("ssl://www.main-website-domain.com", 443, $errno, $errstr, 1);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
//// this is where the full path to the file you want to reach goes
//// format is (method) (path not including the domain) (HTTP version)
$out = "POST /Full/Path/to-test-file2.php HTTP/1.1\r\n";
$out .= "Host: www.main-website-domain.com\r\n";
$out .= "Content-Type: application/x-www-form-urlencoded\r\n";
$out .= "Content-length: " . strlen($post_data) . "\r\n";
$out .= "User-Agent: What-ever-you-want\r\n";
$out .= "Connection: close\r\n\r\n";
$out .= $post_data . "\r\n\r\n";
fwrite($fp, $out);
fclose($fp);
}
file2.php
$data = $_POST['hello'];
ignore_user_abort(true);
sleep(60);///example of a long running script
// since no data will be returned it's best to store any results in a database
// echo, print_r, var_dump, or any other display mechanism will not work in this file unless directly accessed
For people who use php-fpm (meaning, nginx or apache via mod_fcgi), you can use fastcgi_finish_request to achieve described behavior.
What it does
fastcgi_finish_request will send the output to the client (browser) but it'll execute all code specified after the function.
Example
echo "Request accepted";
fastcgi_finish_request(); // Sends "Request accepted" back to web server, but executed all code below
// Do your long running task
sleep(60);
We use REST for logging - we need more info than a simple textlog can provide. So we implemented a REST service, that accepts some basic information about the event and then asynchronously fetches more detail from the event's source web. So it's practically a ping with very little additional info. The logger machine than verifies API key, connects to DB and writes the basic info along with pointers to more detailed information.
We have a problem though, that logging may slow down the app a lot, because the connection and waiting for answer takes quite some time (it's fast but when you log 10+ events in one request it's a problem).
The question is:
Is there a way in plain PHP (5.3) to do a request to an URL and NOT wait for any answer, or just wait for HTTP 200 header to be sure?
I think I can make the logging server send HTTP 200 header as soon as it gets the request. But it needs to do some more work then ;)
What you want is called a asynchrnous request.
A solution could look like this (quoted):
function curl_post_async($url, $params)
{
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;
fwrite($fp, $out);
fclose($fp);
}
If you need more informations take a look at:
http://petewarden.typepad.com/searchbrowser/2008/06/how-to-post-an.html
Here are two tricks maybe helpful.
I. Continue execute after http connection closed. Which means you can close the connection immediately when the main function finished, then continue with your log process.
<?php
ignore_user_abort(true);//avoid apache to kill the php running
ob_start();//start buffer output
echo "show something to user";//do something you need -- your main function
session_write_close();//close session file on server side if needed
header("Content-Encoding: none");//send header to avoid the browser side to take content as gzip format
header("Content-Length: ".ob_get_length());//send length header
header("Connection: close");//or redirect to some url
ob_end_flush();flush();//really send content, can't change the order:1.ob buffer to normal buffer, 2.normal buffer to output
//continue do something on server side
ob_start();
sleep(5);//the user won't wait for the 5 seconds
echo 'for log';//user can't see this
file_put_contents('/tmp/process.log', ob_get_contents());
// or call remote server like http://your.log.server/log.php?xxx=yyy&aaa=bbb
ob_end_clean();
?>
II. Use function apache_note to write logs is pretty lightweight choice than insert into DB. Because Apache will open the log file and keep the handle during Apache running. It's stable and really fast.
Apache configuration:
<VirtualHost *:80>
DocumentRoot /path/to/your/web
ServerName your.domain.com
ErrorLog /path/to/your/log/error_log
<Directory /path/to/your/web>
AllowOverride All
Order allow,deny
Allow from all
</Directory>
SetEnvIf Request_URI "/log\.php" mylog
LogFormat "%{mylog}n" log1
CustomLog "|/usr/sbin/cronolog /path/to/logs/mylog/%Y%m%d/mysite.%Y%m%d%H.log" log1 env=mylog
</VirtualHost>
PHP code:
<?php
apache_note('mylog', session_id()); //you can log any data you need, see http://www.php.net/manual/en/function.apache-note.php
Then you can use my tricks both I and II, call URL http://your.log.server/log.php?xxx=yyy&aaa=bbb to log your detail data after connection of main page closed. No extra time cost needed at all.
I'd like to run a particularly expensive fql query in the background, log results to the database, and retrieve it later without the user having to wait for each step.
Can you share an example of how to run a facebook request asynchronously?
main.php
$uid = $facebook->getUser();
if ($uid) {
try {
echo $user;
////////////////////////////////////////////////////////
// Run lengthy query here, asynchronously (async.php) //
////////////////////////////////////////////////////////
// //
// For example: $profile = $facebook->api('/me'); //
// (I know this request doesn't take long, but //
// if I can run it in the background, it'll do. //
// //
////////////////////////////////////////////////////////
} catch (FacebookApiException $e) {
echo $e;
}
}
async.php
$profile = $facebook->api('/me');
$run = mysql_query("INSERT INTO table (id) VALUES (" . $profile['id'] . ");";
complete.php
echo getProfileId(); // assume function grabs id from db, as stored via async.php
My solution to running PHP jobs in the background is just to have the script itself make a new request which executes the actual job.
The code I've used before is the following, I'm not sure if there are more elegant solutions... but it has worked more than well enough for me in the past. The reason I use fsock and not file_get_contents() etc is of course so that I won't have to wait for the job to finish (which would defeat the purpose)
$sock = fsockopen("127.0.0.1", 80);
fwrite($sock, "GET /yourjoburl/ HTTP/1.1\r\n");
fwrite($sock, "Host: yourdomain.com\r\n");
fwrite($sock, "Connection: close\r\n");
fwrite($sock, "\r\n");
fflush($sock);
fclose($sock);
So, then you just have the other script write the results and progress to a database or whatever... also remember that MySQL supports mutexes, which means you can easily prevent multiple jobs from running at the same time... or to allow other scripts to wait for the job to finish.
PS. The reason I personally avoid exec and all that stuff is that it just seems like a hassle to work with, different servers, different setups, different OSes, etc. This works the same on all hosts that allow you to open sockets. Although you might want to add a private key to the request that you verify in the job, or check the IP of the caller, to prevent others from being able to start jobs.
EDIT: This is untested but should work if you want to forward the cookies as well, including the session cookie.
$sock = fsockopen("127.0.0.1", 80);
fwrite($sock, "GET /yourjoburl/ HTTP/1.1\r\n");
fwrite($sock, "Host: yourdomain.com\r\n");
fwrite($sock, "Cookie: " . $_SERVER['HTTP_COOKIE'] . "\r\n");
fwrite($sock, "Connection: close\r\n");
fwrite($sock, "\r\n");
fflush($sock);
fclose($sock);
How can I run PHP code asynchronously without waiting? I have a long run (almost infinite) that should run while server starts and should process asynchronously without waiting.
The possible options I guess are:
Running the code in a web page and keep it open to do that task
Calling the script from some command line utility (I am not sure how) which would process in the background.
I am running the PHP scripts on my local server which will send emails when certain events occur, e.g. birthday reminders.
Please suggest how can I achieve this without opening the page in a browser.
If you wanted to run it from the browser (perhaps you're not familiar with the command line) you could still do it. I researched many solutions for this a few months ago and the most reliable and simplest to implement was the following from How to post an asynchronous HTTP request in PHP
<?php
$params['my_param'] = $a_value;
post_async('http:://localhost/batch/myjob.php', $params);
/*
* Executes a PHP page asynchronously so the current page does not have to wait for it to finish running.
*
*/
function post_async($url, array $params)
{
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;
fwrite($fp, $out);
fclose($fp);
}
Let's say the file above is in your web root directory (/var/www) for example and is called runjobs.php. By visiting http://localhost/runjobs.php your myjob.php file would start to run. You'd probably want to add some output to the browser to let you know it was submitted successfully and it wouldn't hurt to add some security if your web server is open to the rest of the world. One nice thing about this solution if you add some security is that you can start the job anywhere you can find a browser.
Definitely sounds like a job for a cron task. You can set up a php script to do your task once and have the cron run as often as you like. Here's a good writeup on how to have a php script run as a cron task; it's very easy to do.
This isn't really what PHP is designed for. You have to use the PECL threading library to spin off threads that run asynchronously, and I don't recommend it. The new hotness in the async department is node.js - I recommend you look into that and see if you can utilize it. It's designed for light weight, asynchronous network operations, and can be used to fire PHP scripts.
How Can I run PHP code asynchronously
without waiting. I have a long run
(almost inifinite) that should run
while server starts and should process
asynchrously without waiting.
Assuming a typical LAMP system, you can start a PHP daemon from the command line with
root# php yourscript.php &
where yourscript.php contains something similar to
<?php
$log = fopen('/var/log/yourscript.log', 'a+');
// ### check if we are running already omitted
while(true) {
// do interesting stuff and log it.
// don't be a CPU hog
sleep(1);
}
?>
Embellishments:
To make your script directly executable: chmod +x yourscript.php
and add #!/usr/bin/php to the beginning of yourscript
To start with apache, you should add that command to your apache startup script (usually apachectl) and be sure to add code to kill it when apache stops.
The check if you are already running involves a file with your PID in /var/locks/
and something like system('/bin/ps '.$thePID); It also makes the kill instruction easier to write.
thanks Todd Chaffee but it is not working for me so i edited your code i hope you will not mind and may be it will also help others with this technique
cornjobpage.php //mainpage
<?php
//if you want to call page for multiples time w.r.t array
//then uncomment loop start & end)
?>
<?php
//foreach ($inputkeywordsArr as $singleKeyword) {
$url="http://localhost/projectname/testpage.php";
$params['Keywordname'] = "testValue";//$singleKeyword
post_async($url, $params);
//}//foreach ($inputkeywordsArr end
?>
<?php
/*
* Executes a PHP page asynchronously so the current page does not have to wait for it to finish running.
*
*/
function post_async($url, array $params)
{
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "GET ".$parts['path']."?$post_string"." HTTP/1.1\r\n";//you can use POST instead of GET if you like
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
fclose($fp);
}
?>
testpage.php
<?
echo $_REQUEST["Keywordname"];//Output > testValue
?>
How can I have PHP 5.2 (running as apache mod_php) send a complete HTTP response to the client, and then keep executing operations for one more minute?
The long story:
I have a PHP script that has to execute a few long database requests and send e-mail, which takes 45 to 60 seconds to run. This script is called by an application that I have no control over. I need the application to report any error messages received from the PHP script (mostly invalid parameter errors).
The application has a timeout delay shorter than 45 seconds (I do not know the exact value) and therefore registers every execution of the PHP script as an error. Therefore, I need PHP to send the complete HTTP response to the client as fast as possible (ideally, as soon as the input parameters have been validated), and then run the database and e-mail processing.
I'm running mod_php, so pcntl_fork is not available. I could work my way around this by saving the data to be processed to the database and run the actual process from cron, but I'm looking for a shorter solution.
I had this snippet in my "special scripts" toolbox, but it got lost (clouds were not common back then), so I was searching for it and came up with this question, surprised to see that it's missing, I searched more and came back here to post it:
<?php
ob_end_clean();
header("Connection: close");
ignore_user_abort(); // optional
ob_start();
echo ('Text the user will see');
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush(); // Strange behaviour, will not work
flush(); // Unless both are called !
session_write_close(); // Added a line suggested in the comment
// Do processing here
sleep(30);
echo('Text user will never see');
?>
I actually use it in few places. And it totally makes sense there: a banklink is returning the request of a successful payment and I have to call a lot of services and process a lot of data when that happens. That sometimes takes more than 10 seconds, yet the banklink has fixed timeout period. So I acknowledge the banklink and show him the way out, and do my stuff when he is already gone.
Have the script that handles the initial request create an entry in a processing queue, and then immediately return. Then, create a separate process (via cron maybe) that regularly runs whatever jobs are pending in the queue.
What you need is this kind of setup
One can to use "http fork" to oneself or any other script. I mean something like this:
// parent sript, called by user request from browser
// create socket for calling child script
$socketToChild = fsockopen("localhost", 80);
// HTTP-packet building; header first
$msgToChild = "POST /sript.php?¶m=value&<more params> HTTP/1.0\n";
$msgToChild .= "Host: localhost\n";
$postData = "Any data for child as POST-query";
$msgToChild .= "Content-Length: ".strlen($postData)."\n\n";
// header done, glue with data
$msgToChild .= $postData;
// send packet no oneself www-server - new process will be created to handle our query
fwrite($socketToChild, $msgToChild);
// wait and read answer from child
$data = fread($socketToChild, $dataSize);
// close connection to child
fclose($socketToChild);
...
Now the child script:
// parse HTTP-query somewhere and somehow before this point
// "disable partial output" or
// "enable buffering" to give out all at once later
ob_start();
// "say hello" to client (parent script in this case) disconnection
// before child ends - we need not care about it
ignore_user_abort(1);
// we will work forever
set_time_limit(0);
// we need to say something to parent to stop its waiting
// it could be something useful like client ID or just "OK"
...
echo $reply;
// push buffer to parent
ob_flush();
// parent gets our answer and disconnects
// but we can work "in background" :)
...
The main idea is:
parent script called by user request;
parent calls child script (same as parent or another) on the same server (or any other server) and gives request data to them;
parent says ok to user and ends;
child works.
If you need to interact with child - you can use DB as "communication medium": parent may read child status and write commands, child may read commands and write status. If you need that for several child scripts - you should keep child id on the user side to discriminate them and send that id to parent each time you want to check status of respective child.
I've found that here - http://linuxportal.ru/forums/index.php/t/22951/
What about calling a script on the file server to execute as if it had been triggered at the command line? You can do this with PHP's exec.
You can use the PHP function register-shutdown-function that will execute something after the script has completed its dialog with the browser.
See also ignore_user_abort - but you shouldn't need this function if you use the register_shutdown_function. On the same page, set_time_limit(0) will prevent your script to time out.
Using a queue, exec or cron would be an overkill to this simple task.
There is no reason not to stay within the same script.
This combination worked great for me:
ignore_user_abort(true);
$response = "some response";
header("Connection: close");
header("Content-Length: " . mb_strlen($response));
echo $response;
flush(); // releasing the browser from waiting
// continue the script with the slow processing here...
read more in:
How to continue process after responding to ajax request in PHP?
It is possible to use cURL for that, with a very short timeout. This would be your main file:
<?php>
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://example.com/processor.php");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT_MS, 10); //just some very short timeout
curl_exec($ch);
curl_close($ch);
?>
And this your processor file:
<?php
ignore_user_abort(true); //very important!
for($x = 0; $x < 10; $x++) //do some very time-consuming task
sleep(10);
?>
As you can see, the upper script will timeout after a short time (10 milliseconds in this case). It is possible that CURLOPT_TIMEOUT_MS will not work like this, in this case, it would be equivalent to curl_setopt($ch, CURLOPT_TIMEOUT, 1).
So when the processor file has been accessed, it will do its tasks no matter that the user (i.e. the calling file) aborts the connection.
Of course you can also pass GET or POST parameters between the pages.
You can create an http request between server and server. (not browser is needed).
The secret to create a background http request is setting a very small timeout, so the response is ignored.
This is a working function that I have used for that pupose:
MAY
31
PHP asynchronous background request
Another way to create an asynchronous request in PHP (simulating background mode).
/**
* Another way to make asyncronous (o como se escriba asincrono!) request with php
* Con esto se puede simpular un fork en PHP.. nada que envidarle a javita ni C++
* Esta vez usando fsockopen
* #author PHPepe
* #param unknown_type $url
* #param unknown_type $params
*/
function phpepe_async($url, $params = array()) {
$post_params = array();
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;
fwrite($fp, $out);
fclose($fp);
}
// Usage:
phpepe_async("http://192.168.1.110/pepe/feng_scripts/phprequest/fork2.php");
For more info you can take a look at
http://www.phpepe.com/2011/05/php-asynchronous-background-request.html
You can split these functions into three scripts.
1. Initiate process and call second one via exec or command, this is also possible to run via http call.
2. second one will run database processing and at the end will start last one
3. last one will email
Bah, I misunderstood your requirements. Looks like they're actually:
Script receives input from an external source you do not control
Script processes and validates the input, and lets the external app know if they're good or not and terminates the session.
Script kicks off a long-running proccess.
In this case, then yes, using an outside job queue and/or cron would work. After the input is validated, insert the job details into the queue, and exit. Another script can then run, pick up the job details from the queue, and kick off the longer process. Alex Howansky has the right idea.
Sorry, I admit I skimmed a bit the first time around.
I would recommend spawning a new async request at the end, rather than continuing the process with the user.
You can spawn the other request using the answer here:
Asynchronous PHP calls?
In your Apache php.ini config file, make sure that output buffering is disabled:
output_buffering = off