I want to run a cronjob every minute to detect all files that were changed in the last minute in a specific directory (with about 300.000 inodes) and export this file list to a csv.
Is it possible to run an optimized command to do that? I cant run a "find" with sort flag in this directory cause it is huge and it will probably take more than 1 minute to run all files.
Is there any command I can do that? Or run any specific program on the background of the server that logs every changed file as it is changed? If there is a command using PHP to do this I am fine, I can create a cron to execute a PHP script, no problem.
There is a Linux utility called incron that can be used similar to normal cron, but rather than events being time based, they work off of inotify and are fired from file events.
You can find the Ubuntu man page here: http://manpages.ubuntu.com/manpages/intrepid/man5/incrontab.5.html
I personally have not had to use it for anything too complex, but it roughly goes like this:
Install it:
sudo apt-get install incron
Open the editor to add an entry:
incrontab -e
Put something like this:
/var/www/myfolder IN_MODIFY curl https://www.example.com/api/file-updated/$#
The first part is the file or folder to watch. The second part is the event. And the third part is the command.
I think that $# is the placeholder for the file in question.
Related
I need to run a php script at midnight every night on my server. On a linux system I'd set up a cron job, but I'm stuck with a windows system.
I know I have to set up a task using the windows task scheduler, and that the task will need to run a .bat file which in turn will run the php file, but I'm stuck trying to write the .bat file.
What I currently have is:
#echo off
REM this command runs the nightly cron job
start "C:\Program Files (x86)\PHP\v5.3\php.exe" -f C:\inetpub\wwwroot\sitename\crons\reminder-email.php
But when I try to manually run the .bat file to test it, I get a windows alert saying
"Windows cannot find '-f'. Make sure you typed the name correctly, and then try again.
What have I missed?
The START command optionally accepts a title for the created window as its first argument; in this case, it thinks that C:\Program Files (x86)\PHP\v5.3\php.exe is the title to display and -f (the second argument) is the executable you want to run.
You can therefore fix this by providing a placeholder title, e.g.
start "email reminder task" "C:\Program Files (x86)\PHP\v5.3\php.exe" -f C:\inetpub\wwwroot\sitename\crons\reminder-email.php
Or, preferably, you can ditch the START command altogether (you aren't using any of its unique facilities) and just run PHP directly:
"C:\Program Files (x86)\PHP\v5.3\php.exe" -f C:\inetpub\wwwroot\sitename\crons\reminder-email.php
Actually, you don't even need a batch-file.
You can run the php-script from the task scheduler.
Just let the task scheduler run php.exe and set the location of the php-file as the argument of the task.
Can I suggest a small change.
echo off
REM This adds the folder containing php.exe to the path
PATH=%PATH%;C:\Program Files (x86)\PHP\v5.3
REM Change Directory to the folder containing your script
CD C:\inetpub\wwwroot\sitename\crons
REM Execute
php reminder-email.php
PS. Putting Apache,MySQL or PHP in Program Files is a bad idea. Dont use windows folders with spaces in their names.
How about this?
set php="C:\Program Files (x86)\PHP\v5.3\php.exe"
%php% -f C:\inetpub\wwwroot\sitename\crons\reminder-email.php
Sadly this took close to 10 hours to figure out. Some protocols such as WinRM and PSEXEC must pass the logged in user account regardless of the credentials supplied (that or PHP overrides whatever you type in). At any rate, to get PSEXEC, WinRM or just kicking off batch files that connect to remote computers, you will need to change the IIS run as account to a user with rights to those resources (service account). I realize this is a huge security hole, but that is by design. PHP is secure so that it can't be hacked easily, you have to override their security by specifying a run as account. Not the same thing as your app pool account - although your IIS credentials can use your app pool account.
Try like this guys!
cd E:\xampp\htdocs\my-project
E:
php artisan schedule:run
I'm working with Yii and have to implement a script for cron.
I've got a script file, which just calls Yii and starts my php-script file.
Until this point everything is fine. If I'm updating the php-script, Cron just continues executing the old one.
Restart of cron-service, reboot of the server etc didn't help.
I also uninstalled cron and installed it again, but nothing changed. He still executes the old version of this php-script.
Anyone an idea what's wrong or what I could do to solve this? I'm using Ubuntu 12.04.
EDIT:
The cronjob script is running:
#!/bin/bash
cd ../www/protected/ ./yiic Cron ProcessPayments
The php-script
class CronCommand extends CConsoleCommand {
public function actionProcessPayments() {
...
}}
This works, but any change I make on this script is ignored by Cron.
And now I'm on this point: he executes both. My old version and the new version. I've never been this confused by something.
Assuming you have sudo on the machine it's located on...
First check to see if it's located in the usual cron directories /etc/cron.*
sudo find /etc/cron* -mount -iname yiic
If nothing shows up there then search the entire filesystem because otherwise you'll start having to search a lot of other places manually and it takes more time than running a find
sudo find / -mount -iname yiic
In both find commands if the filename is or has ever been anything other than yiic do yiic* so that it searchs for anything that starts with yiic
Once you have found the copies, if there are any others, remove every single one except the newest one, or even that one and then reinstall the script on the machine from version control.
Edit:
Probably more than you really care about, but the find command breaks down like this:
sudo - elevated priviledges so you can search everywhere as opposed to where you're user can read
find - command to look for thigs
/ - the starting point for the search, can be anywhere
-mount - this tells find not to search other filesystems
-iname - case insensitive name to search for
yiic - the search term
Locate all your copies of the script using the find command and delete those copies not needed. Restart cron after you have chosen the copy of the script you want to execute. My guess is a copy of the script might be hanging around in a directory (e.g. config.daily or something similar).
HTH
I had a similar problem. A script in /etc/cron.d was still executed in the old fashion after modifying it. I solved the problem by sending a SIGHUP to cron:
sudo pkill -SIGHUP /usr/sbin/cron
There is a folder in which xml files are beeing copied at no particular time, when an event is happening. I want a php way to inspect the folder's status and when an xml file arrives, an event will be triggered.(ex.call to the xml parser). So which is the best way (in php) to monitor a folder and trigger events according to it's status? Thanx!
Haven't tried it, but maybe Inotify can help you:
inotify is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications.
There's a PHP extension for inotify, see InotifyDocs and inotifyPECL.
Another alternative if you're running on linux is to use a PHP-independent daemon to monitor a directory for changes. You can use dnotify for it (obsoleted by inotify), something like:
dnotify -a -r -b -s /path/ -e <command>;
It will execute the command each time one of the files in other folder are modified (-a -r -b -s = any access/recursive directory lookup/run in background/no output).
Related:
Read file change in php (linux equivalent of tail -f )
How to efficiently monitor a directory for changes on linux?
I think the most simple way to do this is to use cron job to examine the folder every minute. The other option is to trigger your php script from another script/program that copies new xml file to the directory.
Cron enables you to run your script every minute. If you want instant response you should write a shell script(http://aplawrence.com/Unixart/watchdir.html) that constanlty runs in the background or maybe pearl daemon to you detect new file and trigger your php script to examine changes.
You should take a look at FAM (File Alteration Monitor). PHP 4 based binary extension (beta status); documentation.
Use cron to regularly scandir() the directory.
I have a php script that triggers some magento actions, and I set it to a cron of:
cd /home/dir/public_html; php -f file.php;
This starts the script, however it does not finish executing for some reason, the cron runs as the user "user", and when I run the command from the terminal as root it works perfectly. All the files it uses are chowned to user however. I thought it was an issue with paths which is why I added the CD command to the front of it, however that wasn't it.
I'm thinking this may be an issue with the creation of a lock file, I have it create a lock file, run the script, then delete the lock file in order to prevent it from running if it already is. The lock file is generated but never deleted, my knowledge is if it creates it as user "user" then it should be able to delete it as that user as well.
Any thoughts? Much appreciated.
Try to put the full path of PHP, or define the PATH variable in the first lines of crontab:
PATH=/usr/local/bin:/usr/bin:/bin:/usr/games:/sbin:/usr/sbin
Edit : moreover, you can log your script like that :
* * * * * cd /home/dir/public_html; /usr/bin/php -f file.php; &>/tmp/file.log
Instead of calling the php from within the cronjob, call a shell script which is calling the php file then.
You can then change the environment the script is running in w/o the need to change the cronjob and you can easier test-drive the cron command (as you can just call the shell-script).
You can then, in the shell script, change the directory where the php-script expects to be in which will most certainly solve your issue.
Additionally you can change ini directives, handle logging and shell error handling, like piping STDERR to a file etc.
This is not exactly your problem, but the information given in this question might solve your issue: How can I force PHP Version for Command Line?.
When accessing http://www.example.net, a CSV file is downloaded with the most current data regarding that site. I want to have my site, http://www.example.com, access http://www.example.net on an hour by hour basis in order to get updated information.
I want to then use the updated information stored in the CSV file to compare changes from data in previous CSV files. I obviously have no idea what the best plan of attack would be so any help would be appreciated. I am just looking for a general outline of how I should proceed, but the more information the better.
By the way, I'm using a LAMP bundle so PHP and mySQL solutions are preferred.
I think the most easy way for you to handle this would be to have a cron job running every hour (or scheduled task if are on windows), downloading the CSV with curl or file_get_contents(manual). When you have downloaded the CSV you can import new data in your MySQL database.
The CSV should have some kind of timestamp on every row so you can easily separate new and old data.
Also handling XML would be better then plain CSV.
A better way to setup that would be you to create a webservice on http://www.example.com and update in real time from your http://www.example.net. But it requires you to have access to both websites.
Depending on the OS you're using, you're looking at a scheduled task (Windows) or a cron job (*nix) to kick up a service/app that would pull the new CSV and compare it to an older copy.
You'll definitely want to go the route of a cron job. I'm not exactly sure what you want to do with the differences, however, if you just want an email, here is one potential (and simplified) option:
wget http://uri.com/file.txt && diff file.txt file_previous.txt | mail -s "Differences" your#email.com && mv file.txt file_previous.txt
Try this command by itself from your command line (I'm guessing you are using a *nix box) to see if you can get it working. From there, I would save this to a shell file in the directory where you want to save your CSV files.
cd /path/to/directory
vi process_csv.sh
And add the following:
#!/bin/bash
cd /path/to/directory
wget http://uri.com/file.txt
diff file.txt file_previous.txt | mail -s "Differences" your#email.com
mv file.txt file_previous.txt
Save and close the file. Make the new shell script executable:
chmod +x process_csv.sh
From there, start investigating the cronjob route. It could be as easy as checking to see if you can edit your crontab file:
crontab -e
With luck, you'll be able to enter your cronjob and save/close the file. It will look something like the following:
01 * * * * /path/to/directory/process_csv.sh
I hope you find this helpful.