I have a script that will update some columns on my database. It is written in PHP, I execute it via URL (eg. http://foo.com/xyz/yzx/dbupt8r). I did this using crontab -e then curl on the script URL, because on my mind it is like somehow similar of what I am doing: accessing it via URL. Is there any advisable or better way of doing this? Am I at security or threat flaws?
There are two ways to do this, the way that you're already doing it: (curling a publicly accessible URL); or executing the PHP script directly from your crontab.
Cron Curling
As you mentioned, this is often very convenient and comfortable since you're already developing a web application in PHP and so it's the way you're already working. There are a few risks:
Most importantly, security: You'll want to ensure that input is properly filtered, there are no risks of SQL injection, etc., in case someone discovers the URL and tries to take advantage of it. Hopefully you're covering most of this anyway since you're developing a web application.
Frequency & concurrency: You're scheduling it's execution from cron, so are there any issues if someone else starts calling the URL and making it fire more frequently or at the same time as a scheduled run is occurring?
Relying on curl: It also means you're relying on curl to execute your script, so you're opening yourself up to many points of failure (curl itself, DNS, etc.).
Running PHP from Cron
Alternatively, you may be able to run the script directly from your crontab. There are two ways of doing this:
Passing the PHP script to the PHP interpreter binary, which would look something like this (note the path to your PHP binary varies by platform, but should be specified as an absolute path as cron doesn't have access to many environment variables):
*/15 * * * * /usr/bin/php -f /path/to/php/script.php
Alternatively, you can add a hashbang/shebang line to the first line of the PHP script as follows:
#!/usr/bin/php
Make it executable, for example:
chmod 755 /path/to/php/script.php
And add it directly to your crontab:
*/15 * * * * /path/to/php/script.php
The advantages of this method are that you can put the script in a location that's not publicly accessible so you can ensure tighter control over its access & execution. It may also mean you can write lighter code if you don't have to handle the web side of things. That said, if you're using a PHP framework, you may find it difficult to develop a stand-alone script such as this.
You can always run it using the php command. Have your crontab run a "/path/to/script.sh" that contains:
#!/bin/bash
cat "/path/to/phpscript.php" | php -e
You can have it save the output if you want. You could also have CRON run "php -f /path/to/script.php"
It depends on what you have access to. Personally, I wouldn't like to depend on an external curl script for required periodic jobs. One of the downsides to this approach is that you risk giving permission to the world to run your dbupt8r script. Please bear in mind that you can run PHP scripts without them being in the context of a web server so you could create a cron job on the web server that does
php /my/folder/dbupt8r.php
In this case, your periodic job will run regardless of whether the web server is available and without any risk of exposing it to the outside world.
Calling a URL exposes you to timeout problems which could lead to transaction errors in your database. I suggest you use command line interface (CLI) for this kind of process.
Related
On a web server running PHP, I need the PHP scripts to be able to run one specific external program through exec(), system() or similar.
For that reason, the execution function must be enabled at PHP level.
However, to mitigate the impact of malicious PHP code I would like to limit the execution from PHP to the program needed only.
In other words, I don't want PHP to be able to run any (dangerous) program found in the filesystem, such as /bin/ls, /usr/bin/wget and so on.
Edit: It's not a problem to trust or not the user input. I know that my PHP scripts runs the legitimate program. I worry about the capability offered to malicious PHP scripts to run everything through a shell exec.
It seems that PHP doesn't allow to specify the file names that can be executed, but is there any workaround to limit the execution? Maybe writing a customized PHP extension?
I have tried reading and have understood PHP console to be a command-line interface (CLI) like one used in composer. I do not understand the difference between a web script and a console script. I do not see the use of having the two.
I want to crawl data from a certain link. Should I use a console script or web script and why?
Please explain in the simplest manner possible.
There is no difference between the two. In most instances, the same PHP script will run whether you execute it from the command line or via the web.
There is, however, a difference between the environment the script will execute within. A CLI script is initiated from and executed within your shell on your computer. It is very self-contained. A web script, on the other hand, is (typically) initiated via a HTTP request from a browser, passes over the web to a web server, is executed on that remote server and a result (typically a web page) is passed back to your browser. In the latter case, there are special environment variables related to the web request made available to the script.
It's a bit hard to know which is the best case for your web crawler script without knowing more detail. But I'd say a command line script is what you're after.
One difference between a web page and a CLI instance is the way the script is executed: webpages will be loaded via a web container, while CLI's will be usually executed by the shell used to launch the PHP. Due to this, a CLI might not have access to all $_SERVER variables as the webpage as practically there is no HTTP request involved.
CLI scripts are useful for doing background tasks that are not initiated by the web server, for example a cron job that periodically cleans your database, on one that executes queued jobs. Think of CLI as shell scripts, you can write a PHP script instead of a bash one.
The PHP interpreter is the same in both cases, and it's up to you to decide which one suits best your needs: webpages are more common, however if you need to have you server do some work without waiting for a web request, the you can go with CLI.
Well, basically a console script is the way for your task.
The difference resides in the fact a Webscript will block your browser, will not show your progress real-time, etc.
I was able to crawl and download about 6000 images from my beloved anime with a console script, showing the progress status, something harder with a Web script as the browser will cache the output. Also you can chain your script and also make some cron magic(assuming you are on nix box)
So, ok, I'm using PHP for my website, and suppose I know quite a bit of MySQL and a little bit of MS SQL.
Now, I want to parse some XML with PHP/USD exchange rate and store it in the database.
The simplest way one could think of would be, perhaps this:
$XMLContent=file("http://www.webservicex.net/CurrencyConvertor.asmx/ConversionRate?FromCurrency=USD&ToCurrency=PHP");
foreach($XMLContent as $line){
$a=&strip_tags($line);
if (is_numeric($a))
{
// output the exchange rate
echo $a;
// THen I would probably go like this:
mssql_query('update table set xchange_rate='.(float)floatval($a));
// break cycle once found;
break;
}
}
That would perfectly do... if it didn't take 0.8 seconds to run this script, due to external XML GET request.
So I suppose I should use Windows Scheduler, and make a task to run, let's say every hour to update the records. Now the question is, I've no idea what script I would use. I mean, I can't just simply run browser with a PHP script - that would be rediculous.
So, what would be the easiest way to make a script/ application to run it outside PHP, without a need to actually open a browser.
Thanks!
EDIT
Good point, right now I'm testing it on Win7, but later it is going to be implemented on Windows Server (2008?).
php has a binary executable interpretor too. On win, its called php.exe
http://php.net/manual/en/features.commandline.php
get it working first from a shell prompt, then make a scheduled task for it.
keep in mind, it is not the same php as you get when running a script through a webserver. Some settings are different, and the version may be different, and may use a different php.ini file(and so may have different extensions loaded).
consider using absolute file paths to start until you get things working.
You don't need a browser to run a PHP script. PHP can happily run as a console application. Without knowing what system you're deploying this on (i.e. which OS), it's hard to give you more directions. For example, on a unix-like system (UN*X, linux, OS X, etc.) you can create your PHP file:
somefile.php
<?php
.....
?>
save this file into some pre-defined directory, for example, /opt/myproject and then schedule it in /etc/crontab (assuming your php is installed in /usr/bin):
7 * * * * /usr/bin/php /opt/myproject/somefile.php
This would run your script with PHP, without any browser, once an hour, at 7 minutes past each hour.
EDIT If you're to deploy this on a windows server, you can either use at command on the command prompt or use the Task Scheduler snap-in in Server Manager to configure your job. The PHP script would be saved where you like it to be (e.g. D:\Projects\MyProject\myfile.php) and the command you would schedule to run would be C:\WAMP\bin\php.exe - or whereve your php.exe is located).
I have written a PHP script which generates an SQL file containing all tables in my database.
What I want to do is execute this script daily or every n days. I have read about cron jobs but I am using Windows. How can I automate the script execution on the server?
You'll need to add a scheduled task to call the URL.
First of all, read up here:
MS KB - this is for Windows XP.
Second, you'll need some way to call the URL - i'd recommend using something like wget - this way you can call the URL and save the output to a file, so you can see what the debug output is. You can get hold of wget on this page.
Final step is, as Gabriel says, write a batch file to tie all this up, then away you go.
e: wget is pretty simple to use, but if you have any issues, leave a comment and I'll help out.
ee: thinking about it, you don't even really need a batch file, and could just call wget directly..
add a scheduled task to request the url. either using a batch file or a script file (WSH).
http://blog.netnerds.net/2007/01/vbscript-download-and-save-a-binary-file/
this script will allow you to download binary data from a web source. Modify it to work for you particular case. This vbs file can either be run directly or executed from within a script. Alternately you do not have to save the file using the script, you can just output the contents (WScript.Echo objXMLHTTP.ResponseBody) and utilize the CMD out to file argument:
cscript download.vbs > logfile.log
save that bad boy in a .bat file somewhere useful and call it in the scheduler: http://lifehacker.com/153089/hack-attack-using-windows-scheduled-tasks
Cron is not always available on many hosting accounts.
But try this:
http://www.phpjobscheduler.co.uk/
its free, has a useful interface so you can see all the scheduled tasks and will run on any host that provides php and mysql.
You can use ATrigger scheduling service. A PHP library is also available to create scheduled tasks without overhead. Reporting, Analytics, Error Handling and more benefits.
Disclaimer: I was among the ATrigger team. It's a freeware and I have not any commercial purpose.
Windows doesn't have cron, but it does come with the 'at' command. It's not as flexible as cron, but it will allow you to schedule arbitrary tasks for execution from the command line.
Yes, You can schedule and execute your php script on windows to run automatically. In linux like os u will have cron but on windows u can schedule task using task scheduler.
If your code is in remote hosted server then create a cron-job for the same.
Else if in local then use a scheduled task in windows.Its easy to implement.I am having servers with so many scheduled tasks running.
I am working on a site that require a php script running on a server without any request,
it is a bot script that keeps (not full time but at least once a day) checking client accounts and send alert messages to clients when something happens.
any ideas are appreciated.
Assuming you need to do this on linux, you may run any php script from the browser and from the CLI as well.
You may run a simple php script:
<? echo "Ana are mere"; ?>
like this:
php -f ./index.php
Be careful about file-permissions, and any bug that may creep inside your code, memory leaks or unallocated variables will become VERY visible now, as the process will run continuously.
If you dont want it running in the background all the time, take a look at crontab (http://unixgeeks.org/security/newbie/unix/cron-1.html) to be able to start jobs regularly.
-- edit--
take a look at php execute a background process and PHP: How to return information to a waiting script and continue processing
Basically you want to start a background process, and you may do this by either using exec() or fsockopen() or a file_get_contents() on your own script probably in this order, if don't have access to exec, or socket functions.
Also take a look at http://us2.php.net/manual/en/function.session-write-close.php so the "background script" won't "block" the request and http://us2.php.net/manual/en/function.ignore-user-abort.php
Use a cron job to do it http://www.cronjobs.org/
You can automatically call a script at any interval you like indefinitely. Your hosting provider should support them if they are good.
You should also consider putting a unique key on the end of the page
ie. www.yoursite.com/cronjob.php?key=randomstring
and then only run the script if the key is correct, to prevent bots and other users from running the script when you don't want it run.
If you can't create a cron job, then create a page that does what you want and create a scheduled task on another machine (maybe your PC?) that just goes out and hits that page at a certain time every day.
It's really a hack, but if you absolutely can't set up a cron job, it would be an option.
As Evernoob and Quamis said, you want to have a cron job (UNIX/Linux/Mac OS) or a scheduled task (MS Windows). Furthermore, you can either have the PHP script run using the PHP command line interface (CLI), in which case you can invoke the PHP executable and then your script name. As an alternate, you can use a tool like wget (availble on all platforms) to invoke the PHP script as if someone had typed the URL in the location bar of a web browser.
A php script could not be used like you imagine here. Because it's executed through apache after a request from somewhere.
Even if you do while(1) in your script, apache/php will automaticly stop your script.
Responding to your comment, yes you'll need ssh access to do this, except if your web interface allow you to add cronjob.
Maybe you can write a service which can be executed with a program on another server and do the job.
If you have no access to the server the easiest way would probably be to hit it through the browser, but that would require you or an external script hitting the URL at the same interval each day when you wanted it to one. You may also be able to setup a Selenium test suite that runs locally on a schedule and hits the page. I'm not 100% if that's possible with Selenium though, you may need some 3rd-party apps to make it happen.
Something else you could try would be to see about using PHP's Process Control Functions (link). These will let you create a script that is a deamon and runs in the background. You may be able to do this to keep the script running on the server and firing off commands at programmed intervals. You will still need some way to get it running the first time (browser request or via command line) though.