Codeigniter CLI command with Tsohost not working - php

I currently have a PHP website built with codeigniter, and i'm having issues with CLI and cron jobs.
The CLI is setup so the controller running the script is found in the /application/controllers/scrape on the server (looking via the ftp) this would be /public_html/application/controllers/scrape, the function to run is called all_sites.
I'm hosted with TSOhost and can successfully run the command using the browser via URL (website.com/index.php/scrape/all_sites)however the script times out, hence the need to use a cron job to run the script.
So far i have tried the following raw cron commands in the advanced mode in the TSOhost control panel when trying to get the script to run daily:
The TSOhost technician set this up
03 19 * * * /usr/bin/php-5.3 /var/sites/s/website.com/public_html/application/controllers/scrape.php (didn't work)
0 6 * * * /usr/bin/wget -O /dev/null -o /dev/null http://www.speeddatemate.com/index.php/scrape/all_sites
0 6 * * * /usr/bin/php-5.3 /var/sites/s/speeddatemate.com/public_html/application/controllers/scrape/all_sites
03 19 * * * /usr/bin/php-5.3 /var/sites/s/speeddatemate.com/public_html/application/controllers/scrape.php
03 19 * * * /usr/bin/php-5.3 /var/sites/s/speeddatemate.com/public_html/application/controllers/scrape/all_sites
10 18 * * * /usr/bin/php-5.3 /var/sites/s/speeddatemate.com/public_html/index.php scrape all_sites
TSO host have stated:
For referencing your site path, use /var/sites/s/website.com
The path to PHP 5.2 is /usr/bin/php and for 5.3 it's /usr/bin/php-5.3
The technician also said:
"To run from CLI you would need to find a way to get those parameters into the script, they can't be appended to the command."
Although is this not the Point of the command?
I've also tried running it via the "make a http request" option which creates the raw job as:
0 6 * * * /usr/bin/wget -O /dev/null -o /dev/null http://www.speeddatemate.com/index.php/scrape/all_sites
Again this does not work.
I've searched high and low to find a way to get this working and read various posts and tried various methods nothing has worked. Can anyone help?

First off, don't bother setting up a cron job unless you have it working on the command line. You will end up mixing different things and generating a plethora of unwanted possibilities for debugging which will confuse you further.
Now, you are saying that
I can successfully run the command using the browser via URL (website.com/index.php/scrape/all_sites), however the script times out
This could be because the script takes longer than 30 seconds to complete, and thus is hitting the max-execution-time in php. Check this.
If this is the case, check this and answers over here and here to resolve it by increasing the limit.
Once you have a working version of the script, either via browser or via command line, you can go ahead and schedule the cron jobs, like already shared by your TSOhost tech support.
03 19 * * * /usr/bin/php-5.3 /var/sites/s/website.com/public_html/application/controllers/scrape.php (didn't work)
0 6 * * * /usr/bin/wget -O /dev/null -o /dev/null http://www.speeddatemate.com/index.php/scrape/all_sites
Once again, setup the crons only after you have the other pieces working.
If it still doesn't work, give more info regarding:
What exactly the script does? Does it download something, does it backup something; update the question with whatever it does.
What parameters does it require? and how were you passing them via the url?
What do the logs say? Assuming your webserver is apache, you can usually find them at /var/log/apache2/error.log and /var/log/apache2/access.log.
EDIT 1
OP says in comments that php index.php scrape all_sites works for him.
Assuming that works from the root of of his app, where path to index.php can be asssumed to be /var/sites/s/website.com/public_html/application/index.php, try this cron job then
03 19 * * * cd /var/sites/s/website.com/public_html/application/ && /usr/bin/php-5.3 index.php scrape all_sites
If possible, schedule it for a time closer to current time rather than a fixed job for 19:03
If this still doesn't work, and assuming the max-execution-time has already been taken care of, the problem could be with one of your environment variables - your cli shell environment is having some variables that are missing from your cron environment.
In my experience, I have found that PATH variable causes the most troubles, so run echo $PATH on your shell, and if the path value you get is /some/path:/some/other/path:/more/path/values, run your cron job like
03 19 * * * export PATH="/some/path:/some/other/path:/more/path/values" && cd /var/sites/s/website.com/public_html/application/ && /usr/bin/php-5.3 index.php scrape all_sites
If this doesn't work out, check all the environment variables next.
Use printenv > ~/shell_environment.txt from a normal shell to get all the environment variables set in the shell. Then use the following cron entry * * * * * printenv > ~/cron_environment.txt to get the variables from the cron environment.
Compare the two - shell_environment.txt and cron_environment.txt for any unset variable which you need to tinker with in cron environment.

First of all make sure your script run as expected from your browser. If so then you can try to run it from command line. Lets assume following is your controller.
<?php
class Scrape extends CI_Controller {
public function all_sites($some_parameter = 'working')
{
echo "Its {$some_parameter}!".PHP_EOL;
}
}
?>
As per your provided information to run the command you can run
/usr/bin/php-5.3 /var/sites/s/website.com/public_html/index.php scrape all_sites
And set cron as
03 19 * * * /usr/bin/php-5.3 /var/sites/s/website.com/public_html/index.php scrape all_sites
If you need to pass a parameter, pass the parameter like:
/usr/bin/php-5.3 /var/sites/s/website.com/public_html/index.php scrape all_sites "Working fine"
Enjoy!
You can read reference document about Running via the CLI from codeigniter here

Related

Execute PHP scraper script in cronjob

I am trying to execute this script that will go fetch data from a site and import it into my database.
I have created the cronjob and waited for 20 minutes. There is no error or result, it is just silent like nothing happened.
I am also not getting an email showing the result of the command. How can I execute this script and also receive the result via email?
This is the cronjob I am currently using:
20 * * * * /usr/bin/GET http://example.com/wp-content/plugins/ScriptName/scrap_data2.php?request_type=import_animes&site=2
Although Hostgator does use GET as one of its cron examples; the usual way to "run" a script from cron via its http link is to use "wget" or "curl".
I beleive "GET" is part of "libwww-perl" package and it may depend on where or whether this is installed. Try using WGET instead.
20 * * * * wget -O /dev/null http://example.com/wp-content/plugins/ScriptName/scrap_data2
'-O /dev/null' above is used to ditch cron wgets output as you won't need it since your script emails success.

Self Executing Script on timed schedule

I'm Working on a php app where I need to have a number count in a sql data base reset at the first of the month and at 12am everyday via PHP and MYSQL.
I have googled but found nothing. As far as I know PHP needs to be accessed by a client to be manipulated.
What is the best method of having the server do this on its own?
You can use crontab
* 0 * * * /path/to/my/script
You can use cronjobs. On windows 'scheduled tasks', on linux crontab. For linux:
crontab -e
A line have to look like this to be called every day 0 am:
* 0 * * * /bin/php -q /script/path
Or, to call the file via web but do not get the content:
* 0 * * * wget --spider "http://url.com/cron.php"
If you do not have a dedicated server, check your webspace control panel for this or ask your webhoster.

How to write a Cron Job to execute simple php script?

I have a simply php script on my server and want it to be run every 2 minutes using a cron job.
*/2 * * * * http://mydomain.com/_adder.php
I suspect the command syntax is wrong.
Do I need to add a command before the script url? Another way to run the script?
Any help is very much appreciated.
the cron-job will simply execute a program on the (local) machine.
a URL is NOT a program. it's a link to a ressource.
whether this ressource triggers a PHP-script execution is not of cron's business.
in any case, you could run a cron-job that will periodically visit a given URL. e.g. using the wget command (a "non-interactive web-page downloader")
*/2 * * * * wget --quiet -O /dev/null http://mydomain.com/_adder.php
you can do it as umläute suggest, but being a local file it's actually faster to access from command line php like this:
*/2 * * * * php /path/to/file/_adder.php
there are differences running a script from the command line vs via a browser that may effect the script.
you may need the full path to php on some systems
use php-cgi to make GET request similar to a web browser .
/usr/bin/php-cgi /your_path/_adder.php
if you're using Linux you can use which php-cgi to locate for php-cgi or search for php7.2-cgi if php 7.2 installed on your machine or search using different version . for Windows users search for php-cgi.exe .

How do I use cron to run a database cleaning script?

I have a database with a bunch of links that I want to keep updated. Basically if a link returns a 404 error code I want to remove it from the database. I have a script that I am using however I need to run it manually. How can I make this work using CRON?
in your shell as cron user (or root):
crontab -e
This will bring up your crontab file in your editor. Add a new line something like this:
* */12 * * * /path/to/script
Save/exit the file.
Now for a quick lesson on cronjobs:
-The first 5 arguments in the line tell how often, or when the cron daemon will execute the 6th argument.
-From left-right, arguments represent: minutes, hours, days, weeks, months
-An asterix (*) tells the cron to run on all values of it's associated time measurement (example * * * * * means to run every minute, of every hour, of every week, and of every month!)
In my example, * */12 * * * means to run every 12 hours.
Check out: http://kevin.vanzonneveld.net/techblog/article/schedule_tasks_on_linux_using_crontab/
To run a PHP script with cron you can use the PHP executable and the path to the script.
On most linux systems you want to edit your cron file (the crontab) with the command crontab -e. This will open up a command line based editor and you can just append your new job to the bottom of the file using this format.
<minute> <hour> <day_of_month> <month> <day_of_week> php /path/to/script
If the commands dont work for you let me know what distribution you are using and I can modify the instructions.
/usr/bin/php -q /home/user/public_html/script.php

Why is crontab not executing my PHP script?

I have built one php file to check some result, so that I need to setup a cronjob.
I set one to run every 30 minute, so that the results will be send. However, I don't know why my crontab did not run after every 30 minute.
Here is how I set the crontab:
*/30 * * * * php /var/www/html/result.php
I have confirmed my file directory is correct. What I not sure is about the timing part: isn't it possible to use */30 * * * * or 30 * * * * ? I set */30 * * * * and did not work.
Given
*/30 * * * * php /var/www/html/result.php
There are multiple possibilities why it is not working:
First of all it is important to check if the simple execution of php /var/www/html/result.php. This is required. But unfortunately, accomplishing this does not mean that the problem is solved.
The path of the php binary has to be added.
*/30 * * * * php /var/www/html/result.php
to be changed to
*/30 * * * * /usr/bin/php /var/www/html/result.php
or whatever coming from which php.
Check the permission of the script to the user running the crontab.
Give execution permission to the file: chmod +x file. And make sure the crontab is launched by a user having rights to execute the script. Also check if the user can access the directory in which the file is located.
To be safer, you can also add the php path in the top of the script, such as:
#!/usr/bin/php -q
<?php
...
?>
Make sure the user has rights to use crontab. Check if he is in the /etc/cron.d/deny file. Also, make a basic test to see if it is a crontanb or php problem.
* * * * * touch /tmp/hello
Output the result of the script to a log file, as William Niu suggested.
*/30 * * * * /usr/bin/php /var/www/html/result.php > /tmp/result
Use the -f option to execute the script:
*/30 * * * * /usr/bin/php -f /var/www/html/result.php > /tmp/result
Make sure the format in crontab is correct. You can do so for example using the site Crontab.guru.
To sum up, there are many possible reasons. One of them should solve the problem.
It may be because php is not in the path. crontab has a very minimal path. So, include the full path for your php program.
you can test your cron commands by piping the output to a file, e.g.
*/30 * * * * php /var/www/html/result.php > /tmp/result.log
From this reference page, under "Crontab Environment":
cron invokes the command from the user’s HOME directory with the
shell, (/usr/bin/sh). cron supplies a default environment for every
shell, defining:
HOME=user’s-home-directory
LOGNAME=user’s-login-id
PATH=/usr/bin:/usr/sbin:.
SHELL=/usr/bin/sh
Also, /30 syntax might not be supported by all platforms, so, try to change it to 0,30 instead.
Had a similar issue; from command line, it worked, but from cron, no go.
had a "include ("./connect.php"); in my php code for the db stuff.
Removed that, and added the connect.php code directly into the php script, and it worked from cron.
I had a similar issue on Ubuntu 14.04.1 and the problem turned out to be the way I was modifying the crontab:
I was using sudo crontab -e instead of just crontab -e and this caused my changes to be ignored.
I had a funny one regarding this. Although my scripts would run manually, they wouldn't run from crontab.
Turns out that because the script was being run from /usr/bin/php rather that the location of the file (as it does when I run it manually) my php require wasn't finding the files I wanted. Changing that to reflect the full address fixed it.
troubleshooting by running the script as /usr/bin/php -f /var/www/myfile.php helped me find the issue
You can use */30 * * * * wget http://my.domain.com/path/to/php/result.php
But Crontab executes the task using the current user that ran crontab -e. When you use wget it’s handled by Apache using the www-data user/group pair
First, make sure the script works as expected.
$ php /var/www/html/result.php
Second, edit the crontab for the Apache user account
$ sudo crontab -u www-data -e
or
$ sudo crontab -u root -e
Now add the crontab and output to a log file.
*/30 * * * * php /var/www/html/result.php > /tmp/result.log
After a day of puzzling why my script would work directly (to send data in an email to a gmail account) I discovered that all the deliberate sends worked when I clicked the url and all the cron sends went into spam. No idea why but I thought I'd share it.
Willem's answer showed me the way. In my case, I have a "include("connection.php")" inside my code. I changed connection.php to /my/full/path/connection.php. I have some rename() calls with the relative path, and I changed to the absolute path. That worked for me. I hope it can help someone else.
Easy and logical way:
Checking the cron logs at /var/log/cron will give you very useful info
less /var/log/cron
Eg.,
My cron entry is * * * * * /usr/bin/php /cat.php <== Run cat.php every minute
The log file will contain an entry similar to the one below every time a cron entry is run
Jan 24 08:06:01 OlaTower CROND[13641]: (root) CMD (/usr/bin/php /cat.php)
Jan 24 08:07:01 OlaTower CROND[13641]: (root) CMD (/usr/bin/php /cat.php)
Here, the php command will be executed every minute and there will be an entry in the log file every minute
If the entry is not there then crond is not even picking that cronjob. If the log entry is there and still you are not getting the desired output then there is something wrong with the command/application logic
Are you sure it is not running? If you use exec, realize that you are running from cron and the full path for everything is required, so instead of cp, you'll need to use /bin/cp.
Centos 7
For the record (and it could work for other distros)
I had the next script
* * * * * /usr/bin/php -f /var/www/html/cron.php >/tmp/result.txt
But it failed to execute.
In the /var/log/cron log file, I found the next line
crond[2213]: (/usr/bin/php) ERROR (getpwnam() failed)
What is that?
It's simple, the syntax of corn is * * * * * user command (check user)
* * * * * someuser /usr/bin/php -f /var/www/html/cron.php >/tmp/result.txt
Using Ubuntu w/ Vesta :
The following command works perfect for me,
/usr/bin/php /home/admin/web/mydomain.com/public_html/mycode.php
Feel free to comment if you have any question, have a nice day :)
I was stuck too. I am using centos 7 and had to run few php scripts. I initially tried this
$crontab -e
& inserted the scripts to be executed at 12 midnight.
0 0 * * * usr/bin/php /var/www/html/cronjob/myscript.php
However in var /var/spool/mail/centos, it gave me an error in the mail there
/bin/sh: usr/bin/php: No such file or directory
So then I used wget like this,
$ crontab -e
0 0 * * * wget https://myapplicationurl/var/www/html/cronjob/myscript.php
This also gave me an error.
ERROR 404: Not Found.
Then I realized my mistake of specifying the folder, since the url will already be pointing to html folder, the folder from there i to be specified, like this
0 0 * * * wget http://myapplicationurl/cronjob/myscript.php
and it worked !!!
Hope this helps any newbie like me :)
if you php script has an include or require, you must provide the full path yours includes
wrong way
// relative path
require_once("../library/PHPMailer/PHPMailerAutoload.php");
Correct Way
// full path
require_once("/home/bitnami/library/PHPMailer/PHPMailerAutoload.php");
I had same problem with my php. Then I test execute php from root dir:
php -f /var/www/html/my_proj_folder/test.php
and got some errors regarding path to lib (included files), such as parse_ini with argument 'config.ini' <- has been taken from my global lib and lose path when it has been started from root.
So,
try to run your php-file (php -f your.file)
check relative path and try to run with absolute path
check permissions to your.php - it has to have executable flag x (you can see it ls -l your.php and set by chmod +x your.php)
put #!/usr/bin/php -q before <?php in your main/executable file

Categories