How to resolve multiple simultaneous when i use Linux task-crontab - php

Here has a question: I need execute a task to put many data to another mysql database per minute; if the first task hasn't finish, the second has start; so,there has a multiple concurrent problem; how to resolve the problem??
I have some ideas, first, Let the task has a execute-time which less than the start time of next task;second, let the task support multi-process; but,i don't the how to write the code?
public function execute(Input $input, Output $output)
{
$tele_data = Telesales::field('*')->where([['create_time','<',time()-48*3600],['customer_label','in',[2,6,7]],['virtual_sale','=','0']])->whereRaw('phone is not null')->select()->toArray();
foreach($tele_data as $key=>$value) {
static::pushTeleToIdc($value);
}
}
private static function pushTeleToIdc($data = []) {
$res = Telesales::where('id',$value['id'])->update(['virtual_sale'=>'1']);
if(!$res) {
return;
}
$url = config('idc.tele_url');
$key = config('idc.tele_key');
$channel = config('idc.tele_channel');
$time = time();
$sign = md5($key.$channel.$time);
$urls = $url."?channel=".$channel."&sign=".$sign."&time=".$time;
$require_params = config('idc.require_params');
foreach($require_params as $key=>$value) {
if(array_key_exists($key,$data) && !empty($data[$key])) {
$d[$key] = $data[$key];
}else{
$d[$key] = empty($value)?'':$value[array_rand($value,1)];
}
}
$d['register_time'] = $d['create_time'];
$res = post_url($urls,$d);
$result = json_decode($res,true);
if (isset($result['code']) && $result['code'] != 0){
Log::init(['single'=>'tpushidc'])->error($res);
}
}
Could you help me resolve the problem?

The easiest thing to do is to setup a flag to tell that the process is already in progress and check if that's the case at the start of the function. I don't know how you want to setup the visibility of your code, so I leave it to you to extract $myFile to the file/class scope (same goes for the file path, you probably want to use some /var or /log folder for such stuff).
So the gist is: we create a file, if it doesn't exist or there is a 0 in it - it means we can start working. On other hand, if the contents of the file is 1, the process will die and it will be so every time you run it, until the first one finishes and rewrites the contents of the file to 0 (which means the process is not in progress anymore).
public function execute(Input $input, Output $output)
{
if ($this->isProcessInProgress()) {
die('Process is in progress');
}
$this->startProcess();
$tele_data = [...];
foreach($tele_data as $key=>$value) {
static::pushTeleToIdc($value);
}
$this->finishProcess();
}
private function isProcessInProgress() {
$myFile = 'tele_to_idc_process.txt';
$handle = fopen($myFile, 'r');
if (!$handle)
return false;
$status = fread($handle, 1);
fclose($handle);
return (bool) $status;
}
private function startProcess() {
$myFile = 'tele_to_idc_process.txt';
$handle = fopen($myFile, 'w');
if (!$handle)
return;
$status = fwrite($handle, '1');
fclose($handle);
}
private function finishProcess() {
$myFile = 'tele_to_idc_process.txt';
$handle = fopen($myFile, 'w');
if (!$handle)
return;
$status = fwrite($handle, '0');
fclose($handle);
}
You might get a warning if the file doesn't exist, you can suppress it with #fopen instead of fopen

Related

Why does PHP deletes the content of the file on input?

I have one file: configuration.txt.
This file gets read by PHP, then wrote by the same PHP, while a C++ program reads the content of the same file at a regular interval.
PHP:
$closeFlag = false;
$arrayInputs = new SplFixedArray(3);
$arrayInputs[0] = "URL not entered";
$arrayInputs[1] = "3";
$arrayInputs[2] = "50";
$configFilePath = "/var/www/html/configuration.txt";
$currentSettingsFile = fopen($configFilePath, "r");
if(flock($currentSettingsFile, LOCK_SH)) {
$arrayInputs = explode(PHP_EOL, fread($currentSettingsFile, filesize($configFilePath)));
flock($currentSettingsFile, LOCK_UN);
$closeFlag = fclose($currentSettingsFile);
}
if(isset( $_POST['save_values'])) {
if(!empty($_POST['getURL'])) {
$arrayInputs[0] = $_POST['getURL'];
}
if(!empty($_POST['getURR'])) {
$arrayInputs[1] = $_POST['getURR'];
}
if(!empty($_POST['getBrightness'])) {
$arrayInputs[2] = $_POST['getBrightness'];
}
}
if(!$closeFlag) fclose($currentSettingsFile);
$currentSettingsFile = fopen($configFilePath, "w");
if(flock($currentSettingsFile, LOCK_SH)) {
foreach ($arrayInputs as $key => $value) {
if($value != '')
fwrite($currentSettingsFile,$value.PHP_EOL);
}
flock($currentSettingsFile, LOCK_UN);
fclose($currentSettingsFile);
}
?>
C++
char configFilePath[]="/var/www/html/configuration.txt";
std::fstream configFile;
configFile.open(configFilePath, std::fstream::in);
if(configFile.is_open()){
// do stuff
} else {
std::cout<<"Error ! Could not open Configuration file to read"<<std::endl;
}
The c++ returned no error so far. It can open the file. And php will return Warning: fread(): Length parameter must be greater than 0 because the file is empty.
I believe that PHP is deleting the file's content.
When locking a file in PHP, you lock a LOCK file, not the main file. Example:
$myfile = 'myfile.txt';
$lockfile = 'myfile.lock';
$lock = fopen($lockfile,'a');
if(flock($lock, LOCK_EX)) // The lock file is locked in exclusive mode - so I can write to it.
{
$fp = fopen($myfile,'w');
fputs($fp, "I am writing safely!");
fclose($fp);
flock($lock, LOCK_UN); // Always unlock it!
}
fclose($lock);
You work similarly in C++ because PHP is not locking the actual file. It is locking a lock file. The exact syntax depends heavily on your version of C/C++ and the operating system. So, I will use minimal syntax.
int lock=fopen(lockfile, "r+");
if(flock(fileno(lock), LOCK_EX))
{
//Locked. You can open a stream to ANOTHER file and play with it.
flock(fileno(lock), LOCK_UN));
}
fclose(lock);

Check if file is locked by concurrent process

I have a process that writes a file using file_put_contents():
file_put_contents ( $file, $data, LOCK_EX );
I have added the LOCK_EX parameter to prevent concurrent processes from writing to the same file, and prevent trying to read it when it's still being written to.
I'm having difficulties testing this properly due to the concurrent nature, and I'm not sure how to approach this. I've got this so far:
if (file_exists($file)) {
$fp = fopen($file, 'r+');
if (!flock($fp, LOCK_EX|LOCK_NB, $wouldblock)) {
if ($wouldblock) {
// how can I wait until the file is unlocked?
} else {
// what other reasons could there be for not being able to lock?
}
}
// does calling fclose automatically close all locks even is a flock was not obtained above?
fclose($file);
}
Questions being:
Is there a way to wait until the file is not locked anymore, while keeping the option to give this a time limit?
Does fclose() automatically unlock all locks when there would be another process that had locked the file?
I wrote a small test that uses sleep() so that I could simulate concurrent read/write processes with a simple AJAX call. It seems this answers both questions:
when the file is locked, a sleep that approximates estimated write duration and subsequent lock check allow for waiting. This could even be put in a while loop with an interval.
fclose() does indeed not remove the lock from the process that's already running as confirmed in some of the answers.
PHP5.5 and lower on windows does not support the $wouldblock parameter according to the docs,
I was able to test this on Windows + PHP5.3 and concluded that the file_is_locked() from my test still worked in this scenario:
flock() would still return false just not have the $wouldblock parameter but it would still be caught in my else check.
if (isset($_POST['action'])) {
$file = 'file.txt';
$fp = fopen($file, 'r+');
if ($wouldblock = file_is_locked($fp)) {
// wait and then try again;
sleep(5);
$wouldblock = file_is_locked($fp);
}
switch ($_POST['action']) {
case 'write':
if ($wouldblock) {
echo 'already writing';
} else {
flock($fp, LOCK_EX);
fwrite($fp, 'yadayada');
sleep(5);
echo 'done writing';
}
break;
case 'read':
if ($wouldblock) {
echo 'cant read, already writing';
} else {
echo fread($fp, filesize($file));
}
break;
}
fclose($fp);
die();
}
function file_is_locked( $fp ) {
if (!flock($fp, LOCK_EX|LOCK_NB, $wouldblock)) {
if ($wouldblock) {
return 'locked'; // file is locked
} else {
return 'no idea'; // can't lock for whatever reason (for example being locked in Windows + PHP5.3)
}
} else {
return false;
}
}
I often use a small class... that is secure and fast, basically you have to write only when you obtain exclusive lock on the file otherwise you should wait until is locked...
lock_file.php
<?php
/*
Reference Material
http://en.wikipedia.org/wiki/ACID
*/
class Exclusive_Lock {
/* Private variables */
public $filename; // The file to be locked
public $timeout = 30; // The timeout value of the lock
public $permission = 0755; // The permission value of the locked file
/* Constructor */
public function __construct($filename, $timeout = 1, $permission = null, $override = false) {
// Append '.lck' extension to filename for the locking mechanism
$this->filename = $filename . '.lck';
// Timeout should be some factor greater than the maximum script execution time
$temp = #get_cfg_var('max_execution_time');
if ($temp === false || $override === true) {
if ($timeout >= 1) $this->timeout = $timeout;
set_time_limit($this->timeout);
} else {
if ($timeout < 1) $this->timeout = $temp;
else $this->timeout = $timeout * $temp;
}
// Should some other permission value be necessary
if (isset($permission)) $this->permission = $permission;
}
/* Methods */
public function acquireLock() {
// Create the locked file, the 'x' parameter is used to detect a preexisting lock
$fp = #fopen($this->filename, 'x');
// If an error occurs fail lock
if ($fp === false) return false;
// If the permission set is unsuccessful fail lock
if (!#chmod($this->filename, $this->permission)) return false;
// If unable to write the timeout value fail lock
if (false === #fwrite($fp, time() + intval($this->timeout))) return false;
// If lock is successfully closed validate lock
return fclose($fp);
}
public function releaseLock() {
// Delete the file with the extension '.lck'
return #unlink($this->filename);
}
public function timeLock() {
// Retrieve the contents of the lock file
$timeout = #file_get_contents($this->filename);
// If no contents retrieved return error
if ($timeout === false) return false;
// Return the timeout value
return intval($timeout);
}
}
?>
Simple use as follow:
include("lock_file.php");
$file = new Exclusive_Lock("my_file.dat", 2);
if ($file->acquireLock()) {
$data = fopen("my_file.dat", "w+");
$read = "READ: YES";
fwrite($data, $read);
fclose($data);
$file->releaseLock();
chmod("my_file.dat", 0755);
unset($data);
unset($read);
}
If you want add more complex level you can use another trick... use while (1) to initialize a infinite loop that breaks only when the exclusive lock is acquired, not suggested since can block your server for an undefined time...
include("lock_file.php");
$file = new Exclusive_Lock("my_file.dat", 2);
while (1) {
if ($file->acquireLock()) {
$data = fopen("my_file.dat", "w+");
$read = "READ: YES";
fwrite($data, $read);
fclose($data);
$file->releaseLock();
chmod("my_file.dat", 0755);
unset($data);
unset($read);
break;
}
}
file_put_contents() is very fast and writes directly into file but as you say has a limit... race condition exists and may happen even if you try to use LOCK_EX. I think that a php class is more flexible and usable...
See you this thread that treats a similar question: php flock behaviour when file is locked by one process
The first question is answered here How to detect the finish with file_put_contents() in php? and beacuse PHP is single-threaded, only solution is to use extension of core PHP using PTHREADS and one good simple article about it is https://www.mullie.eu/parallel-processing-multi-tasking-php/
The second question is answered here Will flock'ed file be unlocked when the process die unexpectedly?
The fclose() will unlock only valid handle that is opened using fopen() or fsockopen() so if handle is still valid, yes it will close file and release lock.
Here is a fix for #Alessandro answer to work correctly and not lock the file forever
lock_file.php
<?php
/*
Reference Material
http://en.wikipedia.org/wiki/ACID
*/
class Exclusive_Lock {
/* Private variables */
public $filename; // The file to be locked
public $timeout = 30; // The timeout value of the lock
public $permission = 0755; // The permission value of the locked file
/* Constructor */
public function __construct($filename, $timeout = 1, $permission = null, $override = false) {
// Append '.lck' extension to filename for the locking mechanism
$this->filename = $filename . '.lck';
// Timeout should be some factor greater than the maximum script execution time
$temp = #get_cfg_var('max_execution_time');
if ($temp === false || $override === true) {
if ($timeout >= 1) $this->timeout = $timeout;
set_time_limit($this->timeout);
} else {
if ($timeout < 1) $this->timeout = $temp;
else $this->timeout = $timeout ;
}
// Should some other permission value be necessary
if (isset($permission)) $this->permission = $permission;
if($this->timeLock()){
$this->releaseLock();
}
}
/* Methods */
public function acquireLock() {
// Create the locked file, the 'x' parameter is used to detect a preexisting lock
$fp = #fopen($this->filename, 'x');
// If an error occurs fail lock
if ($fp === false) return false;
// If the permission set is unsuccessful fail lock
if (!#chmod($this->filename, $this->permission)) return false;
// If unable to write the timeout value fail lock
if (false === #fwrite($fp, time() + intval($this->timeout))) return false;
// If lock is successfully closed validate lock
return fclose($fp);
}
public function releaseLock() {
// Delete the file with the extension '.lck'
return #unlink($this->filename);
}
private function timeLock() {
// Retrieve the contents of the lock file
$timeout = #file_get_contents($this->filename);
// If no contents retrieved return true
if ($timeout === false) return true;
// Return the timeout value
return (intval($timeout) < time());
}
}
use as follow:
include("lock_file.php");
$file = new Exclusive_Lock("my_file.dat", 2);
if ($file->acquireLock()) {
$data = fopen("my_file.dat", "w+");
$read = "READ: YES";
fwrite($data, $read);
fclose($data);
$file->releaseLock();
chmod("my_file.dat", 0755);
unset($data);
unset($read);
}
hope that save some else time

php warning fclose() expects parameter 1 to be resource boolean given

I use newrelic to keep track of anything on my website and I always get this error:
Error message: E_WARNING: fclose() expects parameter 1 to be resource, boolean given
Stack trace: in fclose called at /etc/snmp/bfd-stats.php (68)
This is how /etc/snmp/bfd-stats.php looks like
<?php
$a = 0;
$ptr = 0;
$any = 0;
$mx = 0;
$ns = 0;
$cname = 0;
$soa = 0;
$srv = 0;
$aaaa = 0;
$txt = 0;
$total = 0;
if(file_exists('/etc/snmp/bfd-log-pos.stat')) {
$lfh = fopen('/etc/snmp/bfd-log-pos.stat','r');
$string = fread($lfh,2087);
$res = explode(',',$string);
fclose($lfh);
}
else {
$res = array();
$res[0] = 0;
$res[1] = 0;
}
if(file_exists("/var/log/bfd_log.1")) {
$stats = stat('/var/log/bfd_log.1');
if($stats[10] > $res[0]) {
$res[0] = 0;
$res[1] = 0;
}
}
$fh = fopen('/var/log/bfd_log', 'r');
fseek($fh,$res[1]);
$blocks = 0;
if(!$fh) {
echo "Error! Couldn't open the file.";
} else {
while (!feof($fh)) {
$data = fgets($fh);
if(preg_match('/executed\sban/',$data)) {
$blocks++;
}
}
}
$lfh = fopen('/etc/snmp/bfd-log-pos.stat','w');
$timestamp = time();
$pos = ftell($fh);
fwrite($lfh,"$timestamp,$pos");
fclose($lfh);
if(!fclose($fh)) {
echo "Error! Couldn't close the file.";
}
print("bfd_blocks\n$blocks");
?>
On line 40: $fh = fopen('/var/log/bfd_log', 'r'); I looked at the directory /var/log and there is no file called bfd_log, I dont know if I have to create it by myself or it is automatically created.
Can anyone help me on fixing this error, Thanks in advance.
The error indicates that you are trying to pass a variable with a boolean value (true/false) to a function that needs a resource instead of a boolean value.
Please make sure that before you use resources from variables, the function that returns the resource has not run into trouble. Only on success perform the other functions that use this resource/variable.
$fh = fopen('/var/log/bfd_log', 'r');
// check fh before other functions use this variable
if (!$fh) {
echo "Error! Couldn't open the file.";
} else {
// perform task with resource $fh
fseek($fh, $res[1]);
[...]
$lfh = fopen('/etc/snmp/bfd-log-pos.stat', 'w');
// check before other code block is executed and use this variable
if( $lfh )
{
// perform task with resource $lfh
$pos = ftell($fh);
fwrite($lfh, "$timestamp,$pos");
fclose($lfh);
fclose($fh);
[...]
} else {
// lfh error
}
}
If you always check before using variables, you won't run into this error anymore.
I wrestled with this problem and could not find the answer until I separated my write check (put it first) from the actual file write code. So before I would open the file fopen/fwrite then do the is_writable check and then do the fclose and i would get this error.
To resolve I moved the is_writable and variable declaration before the fopen/fwrite and the error went away. Shown below (former php code position shown in comments) The first comment did help me realize this... Thank you.
$myfile = "/var/www/html/newfile.txt";
if (is_writable($myfile)) {
echo "The file is writable";
}
else {
echo "The file is not writable";
}
$txt = "$name, $email, $command, $searchtype, $keyword \n";
$myfile = fopen('/var/www/html/newfile.txt', 'w') or die("Unable to open file!");
fwrite($myfile, $txt);
// $myfile = "/var/www/html/newfile.txt";
// if (is_writable($myfile)) {
// echo "The file is writable";
// }
// else {
// echo "The file is not writable";
// }
fclose($myfile);
Try
$fh = fopen('/var/log/bfd_log', 'a+');
a+ mode will create the file if it does not exists

PHP die stops everything. What else can I use?

I hope this is the last time I have to bother you good people. I'm a newbie hack who is working on a unique hit counter for each page of a web site. I seem to have it working properly but after the first time when it hits and adds the IP to the file it stops the whole page from loading on refresh or coming back. I know the problem is with the 'die' statement which ends the loop of checking for the IP. I have also tried 'break' and 'exit' but the same thing happens. I have searched for anything else but I can't find anything. Is there a way of getting out of the php code without stopping everything else from loading? Thanks in advance.
<?php
// Declare string names
$ip_file = "ip_index.txt";
// get ip address of user
$user_ip = $_SERVER['REMOTE_ADDR'];
$ip_handle = fopen($ip_file, "r");
while (!feof($ip_handle) ) {
$line_of_text = fgets($ip_handle);
$ip = trim($line_of_text);
if ($user_ip==$ip){
die();
}
}
$count_file = 'count_index.txt';
// read contents of count.txt
$count_file = "count_index.txt";
ini_set('display_errors', 'On');
error_reporting(E_ALL);
$handle = fopen($count_file, "r");
$old_count=fgets($handle);
fclose($handle);
// write contents of count.txt
$fp = fopen($count_file, 'ab');
if (false === $fp) {
throw new RuntimeException('Unable to open log file for writing');
}
$handle = fopen($count_file, "w");
$new_count = $old_count +1;
fwrite($handle, $new_count);
fclose($handle);
// write new IP to ip.txt file
$fp = fopen($ip_file, 'r');
if (false === $fp) {
throw new RuntimeException('Unable to open log file for writing');
}
$handle = fopen($ip_file, 'a+');
$w_user_ip=$user_ip . "\n";
fwrite($handle, $w_user_ip);
fclose($handle);
?>
Don't exit the whole script when you find a match, just exit the loop. Set a variable that allows you to skip over the code that increments the unique hit counter.
$ip_exists = false;
while (!feof($ip_handle) ) {
$line_of_text = fgets($ip_handle);
$ip = trim($line_of_text);
if ($user_ip==$ip){
$ip_exists = true;
break;
}
}
if (!$ip_exists) {
// Update all the files
...
}
Not sure what you mean. You can reverse the condition of the if block and then just wrap the remaining code in the braces.
eg
if ($user_ip != $ip) {
$count_file = 'count_index.txt';
// read contents of count.txt
$count_file = "count_index.txt";
// ... etc
}
}
When you have:
if ($user_ip==$ip){
die();
}
Put everything you don't want to run if the IPs are the same in an else{} block
Do this:
....
if ($user_ip != $ip) {
$count_file = 'count_index.txt';
// read contents of count.txt
$count_file = "count_index.txt";
ini_set('display_errors', 'On');
error_reporting(E_ALL);
$handle = fopen($count_file, "r");
$old_count=fgets($handle);
fclose($handle);
// write contents of count.txt
$fp = fopen($count_file, 'ab');
if (false === $fp) {
throw new RuntimeException('Unable to open log file for writing');
}
$handle = fopen($count_file, "w");
$new_count = $old_count +1;
fwrite($handle, $new_count);
fclose($handle);
// write new IP to ip.txt file
$fp = fopen($ip_file, 'r');
if (false === $fp) {
throw new RuntimeException('Unable to open log file for writing');
}
$handle = fopen($ip_file, 'a+');
$w_user_ip=$user_ip . "\n";
fwrite($handle, $w_user_ip);
fclose($handle);
}
If this is included by other code you can use return:
if ($user_ip==$ip){
return;
}
This will stop further execution of your script but code which is including wont die.
If on the other hand you want to terminate the while loop only then break is what you need (what issue did you have when using it?)
if ($user_ip==$ip){
break;
}

Define array of file locations, parse and replace. Where's my error?

I'm trying to define an array with a list of file urls, and then have each file parsed and if a predefined string is found, for that string to be replaced. For some reason what I have isn't working, I'm not sure what's incorrect:
<?php
$htF = array('/home/folder/file.extension', '/home/folder/file.extension', '/home/folder/file.extension', '/home/folder/file.extension', '/home/folder/file.extension');
function update() {
global $htF;
$handle = fopen($htF, "r");
if ($handle) {
$previous_line = $content = '';
while (!feof($handle)) {
$current_line = fgets($handle);
if(stripos($previous_line,'PREDEFINED SENTENCE') !== FALSE)
{
$output = shell_exec('URL.COM');
if(preg_match('#([0-9]{1,3}\.){3}[0-9]{1,3}#',$output,$matches))
{
$content .= 'PREDEFINED SENTENCE '.$matches[0]."\n";
}
}else{
$content .= $current_line;
}
$previous_line = $current_line;
}
fclose($handle);
$tempFile = tempnam('/tmp','allow_');
$fp = fopen($tempFile, 'w');
fwrite($fp, $content);
fclose($fp);
rename($tempFile,$htF);
chown($htF,'admin');
chmod($htF,'0644');
}
}
array_walk($htF, 'update');
?>
Any help would be massively appreciated!
Do you have permissions to open the file?
Do you have permissions to write to /tmp ?
Do you have permissions to write to the destination file or folder?
Do you have permissions to chown?
Have you checked your regex? Try something like http://regexpal.com/ to see if it's valid.
Try adding error messages or throw Exceptions for all of the fail conditions for these.
there's this line:
if(stripos($previous_line,'PREDEFINED SENTENCE') !== FALSE)
and I think you just want a != in there. Yes?
You're using $htF within the update function as global, which means you're trying to fopen() an array.
$fh = fopen($htF, 'r');
is going to get parsed as
$fh = fopen('Array', 'r');
and return false, unless you happen to have a file named 'Array'.
You've also not specified any parameters for your function, so array_walk cannot pass in the array element it's dealing with at the time.

Categories