I have a script which crawls info using API. What I want is to run this script continuously to grab data and store it on my local machine. I configured machine as localhost and installed phpmyadmin and MySQL.
You probably will want to run it from the command line. Put your code in some kind of loop to always keep it running, or schedule it to run as a cron job / scheduled task.
Wrap your main code, except for functions, classes, etc, in this:
while (true) {
and this:
sleep(2);
}
Then run it from the command line, using:
$ php myscript.php
If using Windows, you manually have to add php to your PATH. I don't know Windows (using a Mac all my life) but I guess you can do this in My Machine's Properties. :)
Read about set_time_limit, or modify max_execution_time value in your php.ini file .
Instead of a php loop, use a batch file loop. .
This gets around any set_time_limit , maximum memory etc problems. If php crashes, no problem, a new instance will be started immediately.
#echo off
:loop
php myScrapeScript.php
goto loop;
Related
I have centos VPS hosting and installed WHM/cPanel . I want to run a php script using command line for unlimited time.
My script is look like:
<?php
set_time_limit(0);
while(true)
{
//code to send me email
sleep(600);
}
?>
I know that this script should be run for unlimited time.
I have used these commands:
php myfile.php &
nohup php myfile.php &
I found these commands on stackoverflow. And these are running fine. But after one hours, It stop automatically.
I think, i am doing right. But i do not know, which is killing that process.
If not,
i want to know that How to run this script for unlimited time.
What you are doing is correct. It should run. I have PHP scripts that run for much longer than an hour. They run for days on end. I also have programs that are not PHP that should run forever, but don't. It is because they die due to a bug in the program. For example, xscreensaver dies on me about once a week. To keep it running, I use this shell script (which you can use to keep your PHP running):
while:
do
xscreensaver &
wait
done
Now, running that shell script will start the program again if it ever dies for any reason.
If you have console access try using cronjob to run this file.
see : https://www.centos.org/docs/5/html/Deployment_Guide-en-US/ch-autotasks.html
also : http://alvinalexander.com/linux/linux-crontab-file-format-example
Make sure you use php-command in cron-job
Note : You'll require admin rights to edit/work with cron-jobs
I have two php files. In one php file there is simple html form in which I created some drop down for select time and days for cronjob when user set time and day and submit form then all the drop down values stored in database.
Now with the help of this stored values I need to set cronjob and when cron will execute then it run another php file in which I write some code for generate xml file.
And this file run every time which time is stored by user in database.
By using Cpanel I can do that but I don't want to use cpanel.
Please give me some proper and working solution, I really need it.
Thanks in advance.
Use phps system() or exec() function to call the crontab command to replace or modify the existing crontab for the account the web server runs under. You might have to make sure that user is allowed to use the cron system, this depends on the operating system you use.
From within the crontab you can use one of two strategies to run a php script_
- call the php cli interpreter like any normal command, so something like: /usr/bin/php and give it the script to interpret. As an alternative you also can use shebang inside your php script and call it as a simple executable.
- use wget to call an url pointing to your webserver (maybe localhost) when you want to execute the script inside your web server.
This might act as a starting point for you to experiment with:
#!/usr/bin/php
<?php
// the time tokens to be fed into the crontab line
$time=array('10','*','*','*','*');
// the actual command to be executed
$command=sprintf("crontab -l | (cat;echo \"%s\t%s\") | crontab",
implode(' ',$time),
'/usr/bin/beep');
// execute command
$result=system($command);
// output result of execution
if ($result)
echo "Result: $result\n";
else
echo "FAILURE!\n";
?>
Works for me when called from CLI.
I am having an issue using the PHP function shell_exec().
I have an application which I can run from the linux command line perfectly fine. The application takes several hours to run, so I am trying to spawn a new instance using shell_exec() to manage better. However, when I run the exact same command (which works on the command line) through shell_exec(), it returns an empty string, and it doesn't look like any new processes were started. Plus it completes almost instantly. shell_exec() is suppose to wait until the command has finished correct?
I have also tried variations of exec() with the same outcome.
Does anyone have any idea what could be going on here?
There are no symbolic links or anything funky in the command: just the path to the application and a few command line parametes.
Some thing with you env
See output of env from cli (command line interface) and php script
Also see what your shell interpreter?
And does script and cli application runs from one user?
If so, se option safe_mode
Make sure the user apache is running on (probably www-data) has access to the files and that they are executable (ls -la). A simple chmod 777 [filename] would fix that.
By default PHP will timeout after 30 sec. You can disable the limit like this:
<?php
set_time_limit(0);
?>
Edit:
Also consider this: http://www.rabbitmq.com/
I have a PHP website and I would like to execute a very long Python script in background (300 MB memory and 100 seconds). The process communication is done via database: when the Python script finishes its job, it updates a field in database and then the website renders some graphics, based on the results of the Python script.
I can execute "manually" the Python script from bash (any current directory) and it works. I would like to integrate it in PHP and I tried the function shell_exec:
shell_exec("python /full/path/to/my/script") but it's not working (I don't see any output)
Do you have any ideas or suggestions? It worths to mention that the python script is a wrapper over other polyglot tools (Java mixed with C++).
Thanks!
shell_exec returns a string, if you run it alone it won't produce any output, so you can write:
$output = shell_exec(...);
print $output;
First off set_time_limit(0); will make your script run for ever so timeout shouldn't be an issue. Second any *exec call in PHP does NOT use the PATH by default (might depend on configuration), so your script will exit without giving any info on the problem, and it quite often ends up being that it can't find the program, in this case python. So change it to:
shell_exec("/full/path/to/python /full/path/to/my/script");
If your python script is running on it's own without problems, then it's very likely this is the problem. As for the memory, I'm pretty sure PHP won't use the same memory python is using. So if it's using 300MB PHP should stay at default (say 1MB) and just wait for the end of shell_exec.
A proplem could be that your script takes longer than the server waiting time definied for a request (can be set in the php.ini or httpd.conf).
Another issue could be that the servers account does not have the right to execute or access code or files needed for your script to run.
Found this before and helped me solve my background execution problem:
function background_exec($command)
{
if(substr(php_uname(), 0, 7) == 'Windows')
{
pclose(popen('start "background_exec" ' . $command, 'r'));
}
else
{
exec($command . ' > /dev/null &');
}
}
Source:
http://www.warpturn.com/execute-a-background-process-on-windows-and-linux-with-php/
Thanks for your answers, but none of them worked :(. I decided to implement in a dirty way, using busy waiting, instead of triggering an event when a record is inserted.
I wrote a backup process that runs forever and at each iteration checks if there is something new in database. When it finds a record, it executes the script and everything is fine. The idea is that I launch the backup process from the shell.
I found that the issue when I tried this was the simple fact that I did not compile the source on the server I was running it on. By compiling on your local machine and then uploading to your server, it will be corrupted in some way. shell_exec() should work by compiling the source you are trying to run on the same server your are running the script.
I'm attempting to get PHP to call a batch file which will take an RTF file and convert it to a PDF using an OpenOffice macro. I've tested the batch file on the command line and it works fine, but I'm not having any luck calling and using the same batch file from PHP.
My machine OS is XP professional SP 3. I'm running IIS 6 and PHP version 5.2.9.
I've granted execute permissions to the internet user on c:\windows\system32\cmd.exe.
I specified the full path to the batch file being executed and the full path to the RTF file to be converted.
The PHP looks like this where $arg is the RTF to be converted:
$arg = "C:\\web_root\\whatever\\tempOutput.rtf";
$command = "c:\\windows\\system32\\cmd.exe /c c:\\web_root\\whatever\\convert.bat $arg";
Then inside a try-catch I call the exec command:
exec("$command 2>&1 && exit", $ret, $err);
I echo the results after the catch:
echo "ret: ";
print_r ($ret);
print "<br>";
echo "err is ";
echo $err;
print "<br>";
echo "DONE!";
And this is what I see:
ret: Array ( )
err is 0
DONE!
The RTF file does not get converted and I'm not seeing the errors. Any ideas on what I can try next? Thanks!!!
I'm going to bet this is about permissions.
In a typical setup, PHP runs as apache - so you'll want to make sure apache has the rights to execute the batch file.
also, check this relevant SO question, and this google search.
Looks like the output array is empty. Is your batch script supposed to have output?
Also, escapeshellcmd and escapeshellarg should be used
Are you using IIS as your webserver? If so, the PHP exec function will not work by default and you should NOT circumvent the security measures that prevent it from running.
Check your event viewer and you should find some errors pertaining to your problem. Run a query through google for: IIS PHP exec. This should give you a large selection of information about the problem.
Basically, the PHP exec function tries to fork a new cmd.exe instance. IIS prohibits this because it could open a security hole in the system.
The best solution that I have come up with is to have your php script either write the command that you want to execute to a flat file or make a database entry. You will then need to write a seperate script that is launched by the windows scheduler to run every 10 minutes or so that will check your flat file or database for commands to run. The new script will then run the commands and then place either the results or an execution confirmation that your web app will be able to access at a later time.
It's a kludge for sure.
Is PHP running in safe-mode? If so, shell commands are escaped with escapeshellcmd. Perhaps this is the problem?
Do you have control of the server running the PHP script?