I have a PHP script that is called from a cron job every minute. This script takes some info from the database and then calls another PHP script using the System function (passing it some parameters).
That means that I can start up to 10 scripts from this "main" one. And what I would like to do is that I would call the script and continue the execution of the main script, that is, not wait for the System call to complete and then call the next one.
How can this be done?
You may be able to use proc_open(), stream_select() and stream_set_blocking() in concert to achieve this kind of thing.
If that sounds vague, I was going to paste a big chunk of code in here that I used in a recent project that did something similar, but then felt it may hinder rather than help! In summary though, the code worked like this:
cronjob calls cronjob_wrapper.php
cronjob_wrapper.php creates a new Manager class and then calls start on it.
Manager class start method check to see how many instances are running (looking for pid files in a particular location). If it's less than a given max number of instances it writes out it's own process id to a pid file and then carries on
Manage class creates an instance of an appropriate Encoder class and calls exec on it.
The exec method uses proc_open, stream_select and stream_set_blocking to run a system command in a non-blocking fashion (running ffmpeg in this case - and could take quite a while!)
When it has finally run it cleans up its PID file and bails out.
Now the reason I'm being vague and handwavy is that our multiple instances here are being handled by the cronjob not by PHP. I was trying to do very much the kind of thing you are talking about, and got something working pretty well with pcntl_fork() and friends, but in the end I encountered a couple of problems (if I recall at least one was a bug in PHP) and decided that this approach was a much more rock-solid way to achieve the same thing. YMMV.
Well worth a look at those functions though, you can achieve a lot with them. Though somehow I don't think PHP will ever become the sockets programming language of choice... :)
If your OS supports it, you can use the pcntl_fork() function to spin off child processes that the parent doesn't wait for. Be careful though, it is easy to accidentally create too many child processes, especially if they take longer than expected to run!
I think the answer would be very similar to those already provided for Asynchronous PHP calls.
http://php.net/pcntl_fork
It's *NIX only but you can fork your script using the PCNTL extension.
I'm not sure that PHP supports threading. Check here.
You could run them in the background:
system('php yourscript.php &');
You just have to make sure that you check on the total number of processes running. All in all, not a super elegant solution. Instead cron you could let one script run for forever, I am thinking something like this:
<?php
while(true) {
// do whatever needs to be done.
}
?>
Careful though. PHP is not exactly known to be used as a daemon.
use php's version of fork or threads.
Related
The question is very easy, I want to execute several php files every "N" minutes. For example:
every N minutes
{
execute(script1.php)
execute(script2.php)
execute(script3.php)
}
I know about crontab but i was trying to find another solution. Any suggestions? Thanks in advance.
Using a Cron job is the usual solution. Can you explain why you don't want to use CRON? I've also seen libraries that add cron-like features to your system. For example in the Java/Groovy/Grails world there's the Quartz library/plugin. A quick Google search yielded a PHP library called phpJobScheduler that seems similar to Quartz. I have never used phpJobScheduler so I can't vouch for it.
I'd be interested in why you don't want to use crontabs for this? Are you going to be the primary web operations person running this server or will you be relying on an existing sysop team? You may want to get their input since they are the ones who will be most impacted by what method you choose. I've found they tend to be fond of cron for simple scheduling.
On windows I used built-in proggie called Task Scheduler. As for linux, yes, cron jobs is your answer.
You could create a php script to loop forever do X every N minutes and run the command with a & to make it a background process.
/path/to/php /home/user/bgscript.php &
If you want it to always run, you'd then have to add it to startup init.d or services depending on flavor of *nix.
This solution is possible but personally I would highly recommend going with crontab, its established, proven and works well! Why are you avoiding it?
You could build a script and let it run as a daemon and perform certain tasks on a set interval, but that's actually just simulating cron ... and if you have the ability to run a php script as a daemon you should really also be able to run it as a cronjob since that's what crons are made for.
If you need more info on how to run a php script as a daemon read this great intro. There is also a great comparison between daemon and cron inthere, worth the read.
I have a PHP script launched from a command-line (aka CLI mode, no webserver involved).
In this script I launch a command that will run from some time, usually exiting after a couple minutes. But at times, this command will run for hours because of various issues, and the best I can do is kill it and wait for a while before launching it again.
Two things I want to emphasize :
I have no control of the code inside that command, and can't improve failure detection.
It's not an important task, and it's perfectly ok to have it working that way.
That being said, I would like to improve things in my code, so that I can kill the child process if the command has been running for more than N seconds. But I still want to get the return code from the command, when it runs fine.
Pseudo-code should be something like this :
Launch command
While command is running
{
If the command is done running
{
echo return code
}
else
{
If the command has been running for more than N seconds
{
Kill the child process
}
}
}
How would you implement this in PHP ?
Thank you !
Solution : I ended up using the SIG_ALERT signal. More info on the signals handling and the pcntl lib in the pages provided by Gordon in his post.
Read through this Chapter of Practical PHP Programming.
It covers interoperating with Processes extensively. And also have a look at the PHP Manual's pages on Process Control.
You might also be interested in Gearman
Sorry for not providing any code. I never had the need for Process Control stuff, so I just kept in memory where to look in case I'd ever need it.
Hello I have a couple questions about PHP exec() and passthru().
1)
I never used exec() in PHP but I have seen it is sometimes used with imagemagick. I am now curious, what is some other common uses where exec is good in a web application?
2)
About 6 years ago when I first started playing around with PHP I did not really know anything, just very basic stuff and I had a site that got compromised and someone setup there own PHP file that was using the passthru() function to pass a bunch of traffic throught my site to download free music or video and I got hit with a 4,000$ bandwidth charge from my host! 6 years later, I know soo much more about how to use PHP but I still don't know how this ever happened to me before. How can someone beable to add a file to my server through bad code?
1] Exec() is really useful when you:
A) Want to run a program/utility on the server that php doesn't have a command equivalent for. For example ffmpeg is common utility run via an exec call (for all sorts of media conversion).
B) Running another process - which you can block or NOT block on - that's very powerful. Sometimes you qant a pcnt_fork though, or similar, along with the correct CL args for non blocking.
C) Another example is when I have to process XSLT 2.0 - I have to exec() a small java service I have running to handle the transformations. Very handy. PHP doesn't support XSLT 2.0 transformations.
2] Damn that's a shame.
Well, lots of ways. Theres a family of vulnerability called, "remote file include vulns", that basically allow an attacker to include arbitrary source and thus execute it on your server.
Take a look at: http://lwn.net/Articles/203904/
Also, mentioned above, say your doing something like (Much simplified):
exec("someUnixUtility -f $_GET['arg1']");
Well, imagine the attacker does, url.come?arg1="blah;rm -rf /", your code will basically boil down to:
exec("someUnixUtility -f blah; rm -rf /");
Which in unix, you separate commands w/the ; So yeah - that could be a lot of damage.
Same with a file upload, imagine you strip the last four chars (.ext), to find the extension.
Well, what about something like this "exploit.php.gif", then you strip the extension, so you have exploit.php and you move it into your /users/imgs/ folder. Well, all the attacker has to do now is browse to users/imgs/exploit.php and they can run any code they want. You've been owned at that point.
Use exec or when you want to run a different program.
The documentation for passthru says:
Warning
When allowing user-supplied data to be passed to this function, use escapeshellarg() or escapeshellcmd() to ensure that users cannot trick the system into executing arbitrary commands.
Someone had probably found a security hole in your script which allowed them to run arbitrary commands. Use the given functions to sanitise your inputs next time. Remember, nothing sent from the client can ever be trusted.
exec() allows you to use compiled code that is on your server, which would run faster than php, which is interpreted.
So if you have a large amount of processing that needs to be done quickly, exec() could be useful.
I have cron job - php script which is called one time in 5 minutes. I need to be sure that previously called php script has finished execution - do not want to mix data that's being processed.
There are three approaches I used to apply:
Creation of auxiliary text file which contains running-state flag. Executed script analyzes the contents of the file and breaks if flag is set to true. It's the simplest solution, but every time I create such script, I feel that I invented a bike one more time. Is there any well-known patterns or best-practices which would satisfy most of the needs?
Adding UNIX service. This approach is the best for the cron jobs. But it's more time consuming to develop and test UNIX service: good bash scripting knowledge is required.
Tracking processes using database. Good solution, but sometimes database usage is not encouraged and again - do not want to invent a bike, hope there is a good flexible solution already.
Maybe you have other suggestions how to manage single-processing of php scripts? Would be glad to hear your thoughts about this.
I'd recommend using the file locking mechanism. You create a text file, and you make your process lock it exclusively (see php flock function: http://us3.php.net/flock). If it fails to lock, then you exit because there is another instance running.
The advantage of using file locking is that if your PHP scripts dies unexpectedly or gets killed, it will automatically release the lock. This will not happen if you use plain text files for the status (if the script is set to update this file at the end of execution and it terminates unexpectedly, you will be left with untrue data).
http://php.net/flock with LOCK_EX should be enough in your case.
You could check wether or not your script is currently running using the ps command, helped by the grep command. "man ps" and "man grep" will tell you all about these unix/linux commands if you need informations about these.
Let's assume your script is called 'my_script.php'. This unix command :
ps aux | grep my_script.php
...will tell you if your script is running. You can run this command with shell_exec() at the start of your script, and exit() if it's already running.
The main advantage of this method is that it can't be wrong, where the script could have crashed, leaving your flag file in a state that would let you think it's still running.
I'd stick to version number 1. It's simple and works out. As long as you only wan't to check whether the script has finished or not it should be sufficent. If more complex data is to be remembered I'd go for version 3 in order to be able to 'memorize' the relevant data...
hth
K
My question is whether or not Flex's fcsh can be called from within a PHP script. Here is the background:
I have been created a simple process that creates a simple quiz/tutorial by converting a text file into a .mxml file and compiling to a .swf file using the mxmlc compiler. This works well from the command line, but I wanted to make the process easier by creating a web-interface to do this. My initial attempts with PHP's exec() function have not worked. The Python scripts I use to create the .mxml file work fine (using exec()), but I have not been able to get the mxmlc compiler to work.
After searching on the Web and on this site, I believe that using fcsh (instead of mxmlc) may be the way to go. Using fcsh would certainly compile the .mxml file faster (after the first run), and I think that fcsh can be launched as a service that might be able to be called from PHP.
On the other hand, maybe I am approaching this the wrong way. Would it be better to write a Flex application that calls fcsh and avoid using PHP?
Edit: Using fcshctl as hasseg suggested in his answer below worked very well. Thanks Ali.
The problem with calling fcsh from within scripts is that it works as an interactive shell instead of taking command-line arguments, compiling, and returning an exit status. There are different ways to get around this, which I've listed in this blog post of mine, where I mainly talk about fcshctl (which is my own solution for this,) but at the bottom of the post I've also listed other similar solutions to get fcsh integrated into nonstandard build workflows.
There are a few other ways in php to execute an external script. They are exec(), passthru(), system(), and backticks i.e. the key to the left of the 1 key. Each one has a different purpose and return mechanism.
You may have to put the command that executes your executable into a script and call that script via one of these functions.
Is there a particular reason why you can't use mxmlc directly? It seems like it would be easier to call than fcsh. Just specify all your compiler options in a XML file run it like mxmlc -load-config path/to/config.xml. You can find an example of the XML configuration format in FLEX_HOME/frameworks/flex-config.xml.