Part of my web application is a background script that polls from a beanstalkd server and process data.
This script needs to run continuously (like a daemon). If it crashes, it needs to be started again. It also can't be started twice (more precisely run twice).
As I want to ease the deployment and development process, I want to avoid using pcntl_fork. It's not available on Windows, it necessitates recompiling PHP on Mac, sometimes on Linux too...
Can I do this simply using a bash script to launch the PHP script in background?
# verify that the script is not already running
...
/usr/bin/php myScript.php &
If I execute this batch with crontab every hour or so, my process should run continuously and be restarted in maximum one hour if it crashes?
Assuming blindly that you control the server(s) on which your scripts run, Supervisor is probably a good solution for you.
It's a process control daemon, written in Python. You can configure it to start your PHP script and keep it running. The PHP script itself doesn't need to do anything special. No forking, no manual process control, nothing.
On the other hand, you've also expressed concern about pcntl_fork not being available on Windows. If you're really running this thing on Windows, Supervisor isn't going to work out for you, as it isn't Windows friendly. Keep in mind that Windows isn't really friendly to Unix-style daemonization either, as it would want to control the daemon as a Service. While that's possible, it's not exactly an easy or elegant solution.
Related
What are the bad points when you execute a python script with php?
Also, how is it different from executing python through the cgi method
I found an this interesting method from Running python script from php and i thought it will be great to just use the
exec("python ../cgi-bin/form.py");
and closely-related methods.
Please explain properly and tell me what we have to keep in mind when using this method.
You problem is very common - and in general it's not about executing python scripts - but to execute some external commands. To do that, you'll need some conditions to be fulfilled:
Normally, PHP is run by web-server. So to execute some script, web-server must be able to do that. It means - that OS user, from which web-server was launched, must have enough permissions to execute external command
In many cases, external execution functions, like exec() or system() are treated as unsafe - and, thus, are disabled in common case (I'm speaking about hostings). So, relying on those functions will make your application's technical requirements more strict - and you'll not be able to use such hostings.
Besides described above, PHP script will "hang" until full data will be passed from exec() back to script. That means slow execution and low-predictable response-time. And, more, in Win systems execution of external scripts is included to total script execution time (unlike in *nix systems) - and, therefore, you may have good chances to catch time limit error if your external script was too long to response.
If you want to make some "comparison" with launching python script as CGI - then you should choose CGI. At least because it's intended to serve that purpose. Launching python script with CGI will definitely win in terms of speed - because there will be no overhead for launching PHP script (and you may, for example, disable PHP support if you want to only use python). Permissions level in common case will not be a problem, since, at end point, executable will be launched from web-server user, thus, they will be same in both cases. And, as I've mentioned above, launching via CGI will not bind you to PHP's time limits - you'll not care about what's happening in PHP.
So the main idea is - yes, it is a way to launch. But no - that's not a thing that you should do if you can do that natively via CGI launch.
I have a php program that does extensive curl requests to scrape web pages. It could be up to a million requests. I need to completely stop the script from running. Even though I stopped it in my browser, it is still processing requests. How can I stop it permanently?
You are just killing the request, you will need to stop apache to stop it for now. In the future redesign it so that the process looks for a kill switch (like the presence of a file) and stops processing if it finds it. Sounds like you are jamming a long running process into a php script, why not run it as a normal system process directly?
Assuming you are running the typical lamp stack, SSH into your machine, if necessary, and restart Apache.
If you are really going to perform long running tasks with PHP, I must suggest you consider using cron to run them or implement a task queue of some sort. It's generally a really bad idea to have these sort of things fired from a browser request.
Restart Apache. If you're using XAMP, stop and start it from the control panel.
If not, on Windows, go to task manager and end the apache.exe process. Then start it again.
Why the hell is everyone assuming you're running Apache? Restart your web server and it should be dandy. In the future, you could have a kill switch like (example):
while(!file_exists('stop.txt'))
Then just make that file when you're ready to stop ^.^ Or have a finite number of iterations before cutting off.
We have a web app which allows us to monitor and control our server applications. The web pages start applications by executing a shell script to start them. The problem we have run into is that if we need to restart apache, it kills any of the processes that were started by the web app.
The web pages are PHP, and are using the exec() command to call the start scripts. The start scripts start Java apps, and and run the apps with something like this:
nohup java ... &
As mentioned, PHP is running in Apache on Linux. Is there some other switch or way to start these processes which would not have them be child processes of Apache (and killed when it stops)?
CLARIFICATION
I am more familiar with Windows than with Linux. In Windows, if you want to accomplish what we are trying add the start keyword in the shell, i.e.:
start <batchfile>
When you use start, the new shell/process can be unhooked from the one that started it. Is there a Linux equivalent to the start command?
Starting long-lasting processes by PHP sounds like asking for big trouble.
You will have problems like yours, and you will have huge security implications.
Much better solution is to have your PHP pages save their intent that something needs to be run in batch mode into database table (MySQL or PostgreSQL).
Another process (probably running under more advanced credentials than apache www user) should run as daemon and constantly check database for new stuff to do and execute necessary tasks (also it could be fired by cron every few minutes).
This way, you will kill two birds with one stone.
I wrote up how to create long running processes with php on my blog however I've got to agree with mvp that this approach is far from ideal for your purposes - not just from the point of view of privilege seperation (using a setuid program or sudo solves that easily enough).
Depending opn what you're trying to achieve here, I suspect that the additional functionality in DJ Bernsteins daemontools will be a better fit.
You could use batch(1) to start your long lasting server processes.
In shell, you could do
batch << END
java yourjava.jar
END
if you have some batch shell script file, start it with
batch -f yourbatchfile
If you can improve the Java programs, you might have them call daemon(3) at their start time, perhaps using the daemon thing from Apache.
You probably want to store the daemons' process pid somewhere (e.g. in some file or database), to be able to stop them (first with kill -TERM, then with kill -QUIT, at last with kill -KILL).
Using daemon function or Java thing is probably better than using a batch
I'm planning the development of a server written in PHP that can service socket requests. I use a free host (Heliohost) for testing, and it has cPanel. So far the only thing I've been able to think of to have a PHP script always running is to write a cron job that runs a bash script to check ps to see if the PHP is already running, and if it isn't, start it.
Is there a better way? Perhaps a way for a PHP thread to be started on an HTTP request and continue to run in Apache after the request has been serviced?
You will almost certainly not have success running persistent processes from Apache. It is designed to prevent that scenario (though if you can get to the fork(2) system call, it is probably do-able). I wouldn't recommend trying it though.
What would make more sense is if you use a hosting provider that gives you the ability to write your own crontab(5) specifications and run the PHP interpreter directly. Then you could just add a line to your crontab(5) like:
#reboot /path/to/php /path/to/script.php
Your script should probably perform the usual daemonization tasks so that cron(8) isn't stuck waiting for your process to exit.
I need an application to be running in the background on my web server, but I need to be able to start/stop the application with root privileges.
In order to do this, I want to have a service running that has root privileges, so that it can kill the application, and start it up again if need be.
Finally, I need to be able to send the start and kill commands to the service via Apache/PHP, so that it can be indirectly controlled through the web.
How do I create a Linux service?
How do I communicate with a Linux service in this manner?
Thanks in advance!
Use the exec command in your PHP script to call shell files. The shell files can be setup with the "setuser" bit so it will run as its owner (instead of running with the web server's permissions).
Of course, you'll need to be very careful--lots of testing, monitoring, etc.
Finally, think about the service running as a dedicated user, not as root. Eg like apache and most other well done services do.
Added: Re: running a service in Linux. Depends on your flavor of Linux. If you want to be sure that your app service will be automatically re-started if it fails, plus logging, checkout Runit:
https://web.archive.org/web/1/http://blogs.techrepublic%2ecom%2ecom/opensource/?p=202
http://smarden.org/runit
Added: Instead of setuid bit, I think Frank's suggestion (in comment) of using sudo system is better.
So, you have three pieces here :
Your web server without root privilege
An application
A daemon that is monitoring the application
Your problem is not launching the daemon, it is writing it, and communicating with it from the web server, without needing root privilege.
A daemon can be as simple as a non interactive application launched in the background :
# my_dameon &
I am not a php developper, but searching for message queue and php, I discovered beanstalkd
Looking at the example on the first page it seems you can use it to do the following :
The apache/php sends some message to beanstalkd
Your daemon reads the message from beanstalkd. Based on the command it starts or kill or reload the background application.
You can write your daemon in php, since there are client in many languages
You can also check this question
You can create a daemon which accepts the following commands:
daemon_name start
daemon_name stop
daemon_name restart
deamon_name reload
Starting the daemon should not be hard. Just executing daemon_name start from a PHP script should run it. After starting, you can write the PID of the process to a lock file (for stopping, restarting or reloading later). The daemon should handle signals.
In a PHP script, you can then invoke daemon_name stop. This should fire up a new daemon which would check the lock file and get the PID of the running daemon and send a signal which would be handled by the running daemon. The lock file should be removed/cleared and then the stop initiating daemon/process can quit.
I think you should look at inetd, which can be configured to run all sorts of services, and it runs as root. You could then write a relatively simple program that is not itself root privileged, but which when run by root does the tasks you need done.
As far as communicating with the service you did not say what type of service it is, however assuming you're writing it yourself the most common methods would be to comunicate via UNIX sockets or MMAP. Depends on your needs really.
Oh yeah, should point out there are already applications for web management of linux systems. Webmin is a really good one which can be configured to allow as much or as little control as you need.
As #shodanex suggests, using Beanstalkd would be an excellent way to disconnect a web-front-end from a running-as-root command line worker. It could trivially be set to only run exactly what was required.
To run the worker, Pear's System_Daemon can generate and run a daemon-running script, with start/stop/restart.
When doing this be very very careful. Never use any user submitted data from the web in the exec command. this could allow someone to arbitrarily execute commands on your machine.
Also I second Frank use a sudo rule so you can run that specific command with the permissions you need but nothing else. It will be more secure that way.
Of course with
sudo apt-get install openbsd-inetd
you can create a service you want