What are the bad points when you execute a python script with php?
Also, how is it different from executing python through the cgi method
I found an this interesting method from Running python script from php and i thought it will be great to just use the
exec("python ../cgi-bin/form.py");
and closely-related methods.
Please explain properly and tell me what we have to keep in mind when using this method.
You problem is very common - and in general it's not about executing python scripts - but to execute some external commands. To do that, you'll need some conditions to be fulfilled:
Normally, PHP is run by web-server. So to execute some script, web-server must be able to do that. It means - that OS user, from which web-server was launched, must have enough permissions to execute external command
In many cases, external execution functions, like exec() or system() are treated as unsafe - and, thus, are disabled in common case (I'm speaking about hostings). So, relying on those functions will make your application's technical requirements more strict - and you'll not be able to use such hostings.
Besides described above, PHP script will "hang" until full data will be passed from exec() back to script. That means slow execution and low-predictable response-time. And, more, in Win systems execution of external scripts is included to total script execution time (unlike in *nix systems) - and, therefore, you may have good chances to catch time limit error if your external script was too long to response.
If you want to make some "comparison" with launching python script as CGI - then you should choose CGI. At least because it's intended to serve that purpose. Launching python script with CGI will definitely win in terms of speed - because there will be no overhead for launching PHP script (and you may, for example, disable PHP support if you want to only use python). Permissions level in common case will not be a problem, since, at end point, executable will be launched from web-server user, thus, they will be same in both cases. And, as I've mentioned above, launching via CGI will not bind you to PHP's time limits - you'll not care about what's happening in PHP.
So the main idea is - yes, it is a way to launch. But no - that's not a thing that you should do if you can do that natively via CGI launch.
Related
On a web server running PHP, I need the PHP scripts to be able to run one specific external program through exec(), system() or similar.
For that reason, the execution function must be enabled at PHP level.
However, to mitigate the impact of malicious PHP code I would like to limit the execution from PHP to the program needed only.
In other words, I don't want PHP to be able to run any (dangerous) program found in the filesystem, such as /bin/ls, /usr/bin/wget and so on.
Edit: It's not a problem to trust or not the user input. I know that my PHP scripts runs the legitimate program. I worry about the capability offered to malicious PHP scripts to run everything through a shell exec.
It seems that PHP doesn't allow to specify the file names that can be executed, but is there any workaround to limit the execution? Maybe writing a customized PHP extension?
I have tried reading and have understood PHP console to be a command-line interface (CLI) like one used in composer. I do not understand the difference between a web script and a console script. I do not see the use of having the two.
I want to crawl data from a certain link. Should I use a console script or web script and why?
Please explain in the simplest manner possible.
There is no difference between the two. In most instances, the same PHP script will run whether you execute it from the command line or via the web.
There is, however, a difference between the environment the script will execute within. A CLI script is initiated from and executed within your shell on your computer. It is very self-contained. A web script, on the other hand, is (typically) initiated via a HTTP request from a browser, passes over the web to a web server, is executed on that remote server and a result (typically a web page) is passed back to your browser. In the latter case, there are special environment variables related to the web request made available to the script.
It's a bit hard to know which is the best case for your web crawler script without knowing more detail. But I'd say a command line script is what you're after.
One difference between a web page and a CLI instance is the way the script is executed: webpages will be loaded via a web container, while CLI's will be usually executed by the shell used to launch the PHP. Due to this, a CLI might not have access to all $_SERVER variables as the webpage as practically there is no HTTP request involved.
CLI scripts are useful for doing background tasks that are not initiated by the web server, for example a cron job that periodically cleans your database, on one that executes queued jobs. Think of CLI as shell scripts, you can write a PHP script instead of a bash one.
The PHP interpreter is the same in both cases, and it's up to you to decide which one suits best your needs: webpages are more common, however if you need to have you server do some work without waiting for a web request, the you can go with CLI.
Well, basically a console script is the way for your task.
The difference resides in the fact a Webscript will block your browser, will not show your progress real-time, etc.
I was able to crawl and download about 6000 images from my beloved anime with a console script, showing the progress status, something harder with a Web script as the browser will cache the output. Also you can chain your script and also make some cron magic(assuming you are on nix box)
I have a script that will update some columns on my database. It is written in PHP, I execute it via URL (eg. http://foo.com/xyz/yzx/dbupt8r). I did this using crontab -e then curl on the script URL, because on my mind it is like somehow similar of what I am doing: accessing it via URL. Is there any advisable or better way of doing this? Am I at security or threat flaws?
There are two ways to do this, the way that you're already doing it: (curling a publicly accessible URL); or executing the PHP script directly from your crontab.
Cron Curling
As you mentioned, this is often very convenient and comfortable since you're already developing a web application in PHP and so it's the way you're already working. There are a few risks:
Most importantly, security: You'll want to ensure that input is properly filtered, there are no risks of SQL injection, etc., in case someone discovers the URL and tries to take advantage of it. Hopefully you're covering most of this anyway since you're developing a web application.
Frequency & concurrency: You're scheduling it's execution from cron, so are there any issues if someone else starts calling the URL and making it fire more frequently or at the same time as a scheduled run is occurring?
Relying on curl: It also means you're relying on curl to execute your script, so you're opening yourself up to many points of failure (curl itself, DNS, etc.).
Running PHP from Cron
Alternatively, you may be able to run the script directly from your crontab. There are two ways of doing this:
Passing the PHP script to the PHP interpreter binary, which would look something like this (note the path to your PHP binary varies by platform, but should be specified as an absolute path as cron doesn't have access to many environment variables):
*/15 * * * * /usr/bin/php -f /path/to/php/script.php
Alternatively, you can add a hashbang/shebang line to the first line of the PHP script as follows:
#!/usr/bin/php
Make it executable, for example:
chmod 755 /path/to/php/script.php
And add it directly to your crontab:
*/15 * * * * /path/to/php/script.php
The advantages of this method are that you can put the script in a location that's not publicly accessible so you can ensure tighter control over its access & execution. It may also mean you can write lighter code if you don't have to handle the web side of things. That said, if you're using a PHP framework, you may find it difficult to develop a stand-alone script such as this.
You can always run it using the php command. Have your crontab run a "/path/to/script.sh" that contains:
#!/bin/bash
cat "/path/to/phpscript.php" | php -e
You can have it save the output if you want. You could also have CRON run "php -f /path/to/script.php"
It depends on what you have access to. Personally, I wouldn't like to depend on an external curl script for required periodic jobs. One of the downsides to this approach is that you risk giving permission to the world to run your dbupt8r script. Please bear in mind that you can run PHP scripts without them being in the context of a web server so you could create a cron job on the web server that does
php /my/folder/dbupt8r.php
In this case, your periodic job will run regardless of whether the web server is available and without any risk of exposing it to the outside world.
Calling a URL exposes you to timeout problems which could lead to transaction errors in your database. I suggest you use command line interface (CLI) for this kind of process.
Part of my web application is a background script that polls from a beanstalkd server and process data.
This script needs to run continuously (like a daemon). If it crashes, it needs to be started again. It also can't be started twice (more precisely run twice).
As I want to ease the deployment and development process, I want to avoid using pcntl_fork. It's not available on Windows, it necessitates recompiling PHP on Mac, sometimes on Linux too...
Can I do this simply using a bash script to launch the PHP script in background?
# verify that the script is not already running
...
/usr/bin/php myScript.php &
If I execute this batch with crontab every hour or so, my process should run continuously and be restarted in maximum one hour if it crashes?
Assuming blindly that you control the server(s) on which your scripts run, Supervisor is probably a good solution for you.
It's a process control daemon, written in Python. You can configure it to start your PHP script and keep it running. The PHP script itself doesn't need to do anything special. No forking, no manual process control, nothing.
On the other hand, you've also expressed concern about pcntl_fork not being available on Windows. If you're really running this thing on Windows, Supervisor isn't going to work out for you, as it isn't Windows friendly. Keep in mind that Windows isn't really friendly to Unix-style daemonization either, as it would want to control the daemon as a Service. While that's possible, it's not exactly an easy or elegant solution.
I am writing a Wordpress plug in php in and next step is some kind of add on to this plug in.
The add on would scrape data from web, sending forms etc. I have this part almost ready from the time before I had any thoughts about Wordpress plugin - it's coded in ruby using mechanize. I haven't found anything similar to mechanize in php anyway.
But I do not know what is the best way to call my ruby script from Wordpress. Some tasks will be managed by cron. What about the ones based on user request?
php script only triggers the ruby script. It won't wait/require anything from ruby's output
Wordpress plugin is fully portable and functional without ruby script. Ruby adds on something more. If somebody requires it.
everything will be running on my linux server where I have a root access
A WordPress plugin that depends on Ruby isn't going to be portable. That's OK if you're the only one who will be using it, though.
If the Ruby script needs to return a result that will be used immediately by the PHP script that's calling it, then something like exec() is the only way. Make sure you escape any arguments you pass to the Ruby script; otherwise you'll be vulnerable to injection attacks.
If the Ruby script doesn't need to return a result immediately (e.g. some background processing, such as thumbnail generation) then I think the best way would be for the PHP script to insert a row into a MySQL database or something similar. The Ruby script can work in the background or run from cron, check the database periodically for new jobs, and do whatever processing it needs to do. This approach avoids the performance overhead and security issues of exec(), and it's arguably also more scalable. (A similar approach would have the Ruby script listen on a socket, and your PHP scripts would connect to the socket. But this requires more work to get it right.)
If i were you i would handle all the ruby stuff from the cron. Make a queue in the DB to hand user requests then make the script (in ruby?) invoked by cron grab all the unprocessed jobs from the queue and start running them, then remove the job from the queue (or set some kind of flag for it being done). This way you dont have to call exec which in most cases is going to be off limits unless the user is running on VPS/Dedicated server where they have root access.
You could also make this a seperate job and have it poll the DB for unprocessed jobs more regularly than the primary job... if necessary.
Still, this begs the question... why use ruby in a php blog/cms app??????
Use exec() to run the ruby interpreter, giving it the path to your ruby script.
http://php.net/manual/en/function.exec.php