PHP MySQL get_lock - php

In a script I'm trying to check whether the same script is already running using MySQL GET_LOCK. The problem is, when a script tries to get lock, which isn't free, it blocks forever regardless of the parameter I provide.
<?php
class Controller_Refresher extends Controller {
public function action_run($key) {
echo date('H:i:s'."\n", time());
$this->die_if_running();
$this->run();
echo date('H:i:s'."\n", time());
}
private function die_if_running() {
$result = DB::query(Database::SELECT, "SELECT IS_FREE_LOCK('refresher_running') AS free")->execute();
if (! intval($result[0]['free'])) die('Running already');
$result = DB::query(Database::SELECT, "SELECT GET_LOCK('refresher_running', 1)")->execute();
}
private function run() {
echo "Starting\n";
ob_flush();
sleep(10);
DB::query(Database::SELECT, "SELECT RELEASE_LOCK('refresher_running')")->execute();
}
}
When I run this in 2 tabs in browser, I get e.g.:
-tab 1-
20:48:16
Starting
20:48:26
-tab 2-
20:48:27
Starting
20:48:37
While what I want to do is to make the second tab die('Running already');.

Watch out - this problem might actually be caused by php locking the session file:
https://stackoverflow.com/a/5487811/539149
So you should call session_write_close() before any code that needs to run concurrently. I discovered this after trying this Mutex class:
http://code.google.com/p/mutex-for-php/
The class worked great but my php scripts were still running one by one!
Also, you don't need IS_FREE_LOCK(). Just call GET_LOCK('refresher_running', 0) and it will either return 1 if it gives you the lock or 0 if the lock is taken. It's more atomic that way. Of course, lock timeouts can still be useful in certain situations, like when you want to queue up tasks, but watch out for the script timing out if you get too many simultaneous requests.
Zack Morris

One option would be to rely on a filesystem lock instead of a database. Since it's the script execution that needs handling, it should not matter. A sample from the manual with a non-blocking exclusive lock:
$fp = fopen('/tmp/lock.txt', 'r+');
/* Activate the LOCK_NB option on an LOCK_EX operation */
if(!flock($fp, LOCK_EX | LOCK_NB)) {
die('Running already');
}
/* ... */
fclose($fp);
Edit
Another option would be to use a status file that gets created at the beginning of each exection and will be automatically deleted by register_shutdown_function upon script completion.
The script would simply check the existence of the status file and if it's already there, execution would stop:
define('statusFile', sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'myjob.running');
//
// If a process is already running then exit
//
if (file_exists(statusFile)) {
die('Running already');
} else {
file_put_contents(date('Y-m-d H:i:s'), statusFile);
}
//
// Other code here
//
function shutdown() {
unlink(statusFile);
}
//
// Remove the status file on completion
//
register_shutdown_function('shutdown');

Related

Request to launch background php script

I'm currently working on an internal website displaying a lot of statistics, and some pages or ajax scripts are extremely slow due to large datas.
What I'm searching is a way to launch theses scripts in background with a request, and then launch ajax requests to know the progress of the background script.
Is there any way to achieve this? I work with php7.0 and an apache2 server (I don't have direct access to apache server configuration, so if it's possible, I search for a client-side option)
If ever someone else is searching for a way to achieve this, here is the solution I found:
I call in Ajax a script, it forks itself and save the PID of the child process in the database.
Then I call session_write_close() in the child process to allow user making new requests, and the father process exits (not waiting for child end).
As the father exits, the user receive an answer to his request, and the child process continue his job.
Then in Ajax I call another script to get the evolution of the worker, and finally I get the result and kill the child process when everything is done.
Here is the code of my worker class:
class AsyncWorker
{
private $pid;
private $worker;
private $wMgr;
public function __construct($action, $content, $params = NULL)
{
$this->wMgr = new WorkersManager();
$pid = pcntl_fork(); // Process Fork
if ($pid < 0) {
Ajax::Response(AJX_ERR, "Impossible de fork le processus");
} else if ($pid == 0) { // In the child, we start the job and save the worker properties
sleep(1);
$this->pid = getmypid();
$this->worker = $this->wMgr->fetchBy(array("pid" => $this->pid));
if (!$this->worker) {
$this->worker = $this->wMgr->getEmptyObject();
$this->wMgr->create($this->worker);
}
$this->worker->setPid($this->pid);
$this->worker->setAction($action);
$this->worker->setContent($content);
$this->worker->setPercent(0.00);
$this->worker->setResult("");
$this->wMgr->update($this->worker);
$this->launch($params);
} else { // In the father, we save the pid to DB and answer the request.
$this->worker = $this->wMgr->fetchBy(array("pid" => $this->pid));
if (!$this->worker) {
$this->worker = $this->wMgr->getEmptyObject();
$this->worker->setPid($pid);
$this->wMgr->create($this->worker);
}
Ajax::Response(AJX_OK, "Worker started", $this->worker->getId());
}
}
// Worker job
private function launch($params = NULL)
{
global $form, $_PHPPATH, $url, $session;
session_write_close(); // This is useful to let the user make new requests
ob_start(); // Avoid writing anything
/*
** Some stuff specific to my app (include the worker files, etc..)
*/
$result = ob_get_contents(); // Get the wrote things and save them to DB as result
$this->worker->setResult($result);
$this->worker->setPercent(100);
ob_end_clean();
}
}
It's a bit tricky but I had no choices, as I have no access to server plugins and libraries.
you can make php script to execute shell bash script , or using exec() method for that

How do you delete or edit a file in PHP when it is being read?

Supposing that there is a file on PHP. The file is constantly being read.
I want to stop users from accessing the file first, then delete or edit the file.
How can I do this?
Please refer to this answer.
file locking in php
That covers the locking part. However, to access the file you need to do a loop until the lock is released. Here is a sample algorithm.
define(MAX_SLEEP, 3); // Decide a good value for number of tries
$sleep = 0; // Initialize value, always a good habit from C :)
$done = false; // Sentinel value
$flock = new Flock; // You need to implement this class
do {
if (! $flock->locked()) { // We have a green light
$flock->lock(); // Lock right away
//DO STUFF;
$flock->unlock(); // Release the lock so others can access
$done = true; // Allows the loop to exit
} else if ($sleep++ > MAX_SLEEP) { // Giving up, cannot write
// Handle exception, there are many possibilities:
// Log exception and do nothing (definitely log)
// Force a write
// See if another process has been running for too long
// Check for timestamp of the lock file, maybe left behind after a reboot
} else {
sleep(SLEEP_TIME);
}
} while(! $done);

Can I read an external file every set amount of time?

I am writing a script where it checks for an updated version from an external server. I use this code in the config.php file to check for latest version.
$data = get_theme_data('http://externalhost.com/style.css');
$latest_version = $data['Version'];
define('LATEST_VERSION', $latest_version);
This is fine and I can fetch the latest version (get_theme_data is WordPress function) but the problem is that it will be executed on every single load which I do not want. I also do not want to only check when a form is submitted for example. Alternatively I was looking into some sort of method to cache the result or maybe check the version every set amount of hours? Is such thing possible and how?
Here, gonna make it easy for you. Store the time you last checked for the update in a file.
function checkForUpdate() {
$file = file_get_contents('./check.cfg', true);
if ($file === "") {
$fp = fopen('./check.cfg', 'w+');
fwrite($fp, time() + 86400);
fclose($fp);
}
if ((int)$file > time()) {
echo "Do not updatE";
} else {
echo "Update";
$fp = fopen('./check.cfg', 'w+');
fwrite($fp, time() + 86400);
fclose($fp);
}
}
You can obviously make this much more secure/efficient if you want to.
Edit: This function will check for update once every day.
A scheduled task like this should be set up as a separate cron or at job. You can still write everything in PHP, just make a script that runs from the command line and does the updating. Checkout "man crontab" for details, and/or check which scheduling services your server is running.

forking php process and tying to specific web user

I have a web app that has a few processes that can take up to 10 minutes to run. Sometimes these processes are triggered by a user and they need the output as it is processed.
For instance, the user is looking for
a few records that they need. The
click the button to retrieve the
records (this is the part that can
take 10 minutes). They can continue
to work on other things but when they
click back to view the returns, it is
updated as the records are downloaded
into the system.
Right now, the user is locked while the process runs. I know about pcntl_fork() to fork a child process so that the user doesn't have to wait until the long process completes.
I was wondering if it's possible to tie that forked process to the specific user that triggered the request in a $_SESSION variable so that I can update the user when the process is complete. Also, is this the best way to update a user on a long-running process?
I think gearman fits your needs. Look at this sample code, taken from the doc :
<?php
/* create our object */
$gmclient= new GearmanClient();
/* add the default server */
$gmclient->addServer();
/* run reverse client */
$job_handle = $gmclient->doBackground("reverse", "this is a test");
if ($gmclient->returnCode() != GEARMAN_SUCCESS)
{
echo "bad return code\n";
exit;
}
$done = false;
do
{
sleep(3);
$stat = $gmclient->jobStatus($job_handle);
if (!$stat[0]) // the job is known so it is not done
$done = true;
echo "Running: " . ($stat[1] ? "true" : "false") . ", numerator: " . $stat[2] . ", denomintor: " . $stat[3] . "\n";
}
while(!$done);
echo "done!\n";
?>
If you store the $job_handle in the session, you can adapt the sample to make a control script.

How to properly implement a custom session persister in PHP + MySQL?

I'm trying to implement a custom session persister in PHP + MySQL. Most of the stuff is trivial - create your DB table, make your read/write functions, call session_set_save_hander(), etc. There are even several tutorials out there that offer sample implementations for you. But somehow all these tutorials have conveniently overlooked one tiny detail about session persisters - locking. And now that's where the real fun starts!
I looked at the implementation of session_mysql PECL extension of PHP. That uses MySQL's functions get_lock() and release_lock(). Seems nice, but I don't like the way it's doing it. The lock is acquired in the read function, and released in the write function. But what if the write function never gets called? What if the script somehow crashes, but the MySQL connection stays open (due to pooling or something)? Or what if it the script enters a deadly deadlock?
I just had a problem where a script opened a session and then tried to flock() a file over an NFS share, while the other computer (that hosted the file) was also doing the same thing. The result was that the flock()-over-NFS call was blocking the script for about 30 seconds on each call. And it was in a loop of 20 iterations! Since that was an external operation, PHP's script timeouts didn't apply, and the session got locked for over 10 minutes every time this script was accessed. And, as luck would have it, this was the script that got polled by an AJAX shoutbox every 5 seconds... Major showstopper.
I already have some ideas on how to implement it in a better way, but I would really like to hear what other people suggest. I haven't had that much experience with PHP to know what subtle edge cases loom in the shadows which could one day jeopardize the whole thing.
Added:
OK, seems that nobody has anything to suggest. OK then, here's my idea. I'd like some opinon on where this could go wrong.
Create a session table with InnoDB storage engine. This should ensure some proper locking of rows even under clustered scenarios. The table should have the columns ID, Data, LastAccessTime, LockTime, LockID. I'm omitting the datatypes here because they follow quite directly from the data that needs to be stored in them. The ID will be the ID of the PHP session. Data will of course contain the session data. LastAccessTime will be a timestamp which will be updated on each read/write operation and will be used by GC to delete old sessions. LockTime will be a timestamp of the last lock that was acquired on the session, and LockID will be a GUID of the lock.
When a read operation is requested, there will be the following actions taken:
Execute INSERT IGNORE INTO sessions (id, data, lastaccesstime, locktime, lockid) values ($sessid, null, now(), null, null); - this will create the session row if it is not there, but do nothing if it is already present;
Generate a random lock id in the variable $guid;
Execute UPDATE sessions SET (lastaccesstime, locktime, lockid) values (now(), now(), $guid) where id=$sessid and (lockid is null or locktime < date_add(now(), INTERVAL -30 seconds)); - this is an atomic operation which will either obtain a lock on the session row (if it's not locked or the lock is expired), or will do nothing.
Check with mysql_affected_rows() if the lock was obtained or not. If it was obtained - proceed. If not - re-attempt the operation every 0.5 seconds. If in 40 seconds the lock is still not obtained, throw an exception.
When a write operation is requested, execute UPDATE sessions SET (lastaccesstime, data, locktime, lockid) values (now(), $data, null, null) where id=$sessid and lockid=$guid; This is another atomic operation which will update the session row with the new data and remove the lock if it still has the lock, but do nothing if the lock was already taken away.
When a gc operation is requested, simply delete all rows with lastaccesstime too old.
Can anyone see flaws with this?
Ok. The answer is going to be a bit longer - so patience!
1) Whatever I am going to write is based on the experiments I have done over last couple of days. There may be some knobs/settings/inner working I may not be aware of. If you spot mistakes/ or do not agree then please shout!
2) First clarification - WHEN SESSION DATA is READ and WRITTEN
The session data is going to be read exactly once even if you have multiple $_SESSION reads inside your script. The read from session is a on a per script basis. Moreover the data fetch happens based on the session_id and not keys.
2) Second clarification - WRITE ALWAYS CALLED AT END OF SCRIPT
A) The write to session save_set_handler is always fired, even for scripts that only "read" from session and never do any writes.
B) The write is only fired once, at the end of the script or if you explicitly call session_write_close. Again, the write is based on session_id and not keys
3) Third Clarification : WHY WE NEED Locking
What is this fuss all about?
Do we really need locks on session?
Do we really Need a Big Lock wrapping READ + WRITE
To explain the Fuss
Script1
1: $x = S_SESSION["X"];
2: sleep(20);
3: if($x == 1 ) {
4: //do something
5: $_SESSION["X"] = 3 ;
6: }
4: exit;
Script 2
1: $x = $_SESSION["X"];
2: if($x == 1 ) { $_SESSION["X"] = 2 ; }
3: exit ;
The inconsistency is that script 1 is doing something based on a session variable (line:3) value that has changed in by another script while script-1 was already running. This is a skeleton example but it illustrates the point. The fact that you are taking decisions based on something that is no longer TRUE.
when you are using PHP default session locking (Request Level locking) script2 will block on line 1 because it cannot read from the file that script 1 started reading at line1. So the requests to session data are serialized. When script2 reads a value, it is guaranteed to read the new value.
Clarification 4: PHP SESSION SYNCHRONIZATION IS DIFFERENT FROM VARIABLE SYNCHRONIZATION
Lot of people talk about PHP session synchronization as if it is like a variable synchronization, the write to memory location happening as soon as you overwrite variable value and the next read in any script will fetch the new value. As we see from CLARIFICATION #1 - That is not true. The script uses the values read at the start of the script throughout the script and even if some other script has changed the values, the running script will not know about new values till next refresh. This is a very important point.
Also, keep in mind that values in session changes even with PHP big locking. Saying things like, "script that finishes first will overwrite value" is not very accurate. Value change is not bad, what we are after is inconsistency, namely, it should not change without my knowledge.
CLARIFICATION 5: Do we REALLY NEED BIG LOCK?
Now, do we really need Big Lock (request level)? The answer, as in the case of DB isolation, is that it depends on how you want to do things. With the default implementation of $_SESSION, IMHO, only the big lock makes sense. If I am going to the use the value that I read at the beginning throughout my script then only the big lock makes sense. If I change the $_SESSION implementation to "always" fetch "fresh" value then you do not need BIG LOCK.
Suppose we implement a session data versioning scheme like object versioning. Now, script 2 write will succeed because script-1 has not come to write point yet. script-2 writes to session store and increments version by 1. Now, when script 1 tries to write to session, it will fail (line:5) - I do not think this is desirable, though doable.
===================================
From (1) and (2), it follows that no matter how complicated your script, with X reads and Y writes to session,
the session handler read() and write() methods are only called once
and they are always called
Now, there are custom PHP session handlers on net that try to do a "variable"-level locking etc. I am still trying to figure some of them. However I am not in favor of complex schemes.
Assuming that PHP scripts with $_SESSION are supposed to be serving web pages and are processed in milli-seconds, I do not think the additional complexity is worth it. Like Peter Zaitsev mentions here, a select for update with commit after write should do the trick.
Here I am including the code that I wrote to implement locking. It would be nice to test it with some "Race simulation" scripts. I believe it should work. There are not many correct implementations I found on net. It would be good if you can point out the mistakes. I did this with bare mysqli.
<?php
namespace com\indigloo\core {
use \com\indigloo\Configuration as Config;
use \com\indigloo\Logger as Logger;
/*
* #todo - examine row level locking between read() and write()
*
*/
class MySQLSession {
private $mysqli ;
function __construct() {
}
function open($path,$name) {
$this->mysqli = new \mysqli(Config::getInstance()->get_value("mysql.host"),
Config::getInstance()->get_value("mysql.user"),
Config::getInstance()->get_value("mysql.password"),
Config::getInstance()->get_value("mysql.database"));
if (mysqli_connect_errno ()) {
trigger_error(mysqli_connect_error(), E_USER_ERROR);
exit(1);
}
//remove old sessions
$this->gc(1440);
return TRUE ;
}
function close() {
$this->mysqli->close();
$this->mysqli = null;
return TRUE ;
}
function read($sessionId) {
Logger::getInstance()->info("reading session data from DB");
//start Tx
$this->mysqli->query("START TRANSACTION");
$sql = " select data from sc_php_session where session_id = '%s' for update ";
$sessionId = $this->mysqli->real_escape_string($sessionId);
$sql = sprintf($sql,$sessionId);
$result = $this->mysqli->query($sql);
$data = '' ;
if ($result) {
$record = $result->fetch_array(MYSQLI_ASSOC);
$data = $record['data'];
}
$result->free();
return $data ;
}
function write($sessionId,$data) {
$sessionId = $this->mysqli->real_escape_string($sessionId);
$data = $this->mysqli->real_escape_string($data);
$sql = "REPLACE INTO sc_php_session(session_id,data,updated_on) VALUES('%s', '%s', now())" ;
$sql = sprintf($sql,$sessionId, $data);
$stmt = $this->mysqli->prepare($sql);
if ($stmt) {
$stmt->execute();
$stmt->close();
} else {
trigger_error($this->mysqli->error, E_USER_ERROR);
}
//end Tx
$this->mysqli->query("COMMIT");
Logger::getInstance()->info("wrote session data to DB");
}
function destroy($sessionId) {
$sessionId = $this->mysqli->real_escape_string($sessionId);
$sql = "DELETE FROM sc_php_session WHERE session_id = '%s' ";
$sql = sprintf($sql,$sessionId);
$stmt = $this->mysqli->prepare($sql);
if ($stmt) {
$stmt->execute();
$stmt->close();
} else {
trigger_error($this->mysqli->error, E_USER_ERROR);
}
}
/*
* #param $age - number in seconds set by session.gc_maxlifetime value
* default is 1440 or 24 mins.
*
*/
function gc($age) {
$sql = "DELETE FROM sc_php_session WHERE updated_on < (now() - INTERVAL %d SECOND) ";
$sql = sprintf($sql,$age);
$stmt = $this->mysqli->prepare($sql);
if ($stmt) {
$stmt->execute();
$stmt->close();
} else {
trigger_error($this->mysqli->error, E_USER_ERROR);
}
}
}
}
?>
To register the object session Handler,
$sessionHandler = new \com\indigloo\core\MySQLSession();
session_set_save_handler(array($sessionHandler,"open"),
array($sessionHandler,"close"),
array($sessionHandler,"read"),
array($sessionHandler,"write"),
array($sessionHandler,"destroy"),
array($sessionHandler,"gc"));
ini_set('session_use_cookies',1);
//Defaults to 1 (enabled) since PHP 5.3.0
//no passing of sessionID in URL
ini_set('session.use_only_cookies',1);
// the following prevents unexpected effects
// when using objects as save handlers
// #see http://php.net/manual/en/function.session-set-save-handler.php
register_shutdown_function('session_write_close');
session_start();
Here is another version done with PDO. This one checks for existence of sessionId and does update or Insert. I have also removed the gc function from open() as it unnecessarily fires a SQL query on each page load. The stale session cleanup can easily be done via a cron script. This should be the version to use if you are on PHP 5.x. Let me know if you find any bugs!
=========================================
namespace com\indigloo\core {
use \com\indigloo\Configuration as Config;
use \com\indigloo\mysql\PDOWrapper;
use \com\indigloo\Logger as Logger;
/*
* custom session handler to store PHP session data into mysql DB
* we use a -select for update- row leve lock
*
*/
class MySQLSession {
private $dbh ;
function __construct() {
}
function open($path,$name) {
$this->dbh = PDOWrapper::getHandle();
return TRUE ;
}
function close() {
$this->dbh = null;
return TRUE ;
}
function read($sessionId) {
//start Tx
$this->dbh->beginTransaction();
$sql = " select data from sc_php_session where session_id = :session_id for update ";
$stmt = $this->dbh->prepare($sql);
$stmt->bindParam(":session_id",$sessionId, \PDO::PARAM_STR);
$stmt->execute();
$result = $stmt->fetch(\PDO::FETCH_ASSOC);
$data = '' ;
if($result) {
$data = $result['data'];
}
return $data ;
}
function write($sessionId,$data) {
$sql = " select count(session_id) as total from sc_php_session where session_id = :session_id" ;
$stmt = $this->dbh->prepare($sql);
$stmt->bindParam(":session_id",$sessionId, \PDO::PARAM_STR);
$stmt->execute();
$result = $stmt->fetch(\PDO::FETCH_ASSOC);
$total = $result['total'];
if($total > 0) {
//existing session
$sql2 = " update sc_php_session set data = :data, updated_on = now() where session_id = :session_id" ;
} else {
$sql2 = "insert INTO sc_php_session(session_id,data,updated_on) VALUES(:session_id, :data, now())" ;
}
$stmt2 = $this->dbh->prepare($sql2);
$stmt2->bindParam(":session_id",$sessionId, \PDO::PARAM_STR);
$stmt2->bindParam(":data",$data, \PDO::PARAM_STR);
$stmt2->execute();
//end Tx
$this->dbh->commit();
}
/*
* destroy is called via session_destroy
* However it is better to clear the stale sessions via a CRON script
*/
function destroy($sessionId) {
$sql = "DELETE FROM sc_php_session WHERE session_id = :session_id ";
$stmt = $this->dbh->prepare($sql);
$stmt->bindParam(":session_id",$sessionId, \PDO::PARAM_STR);
$stmt->execute();
}
/*
* #param $age - number in seconds set by session.gc_maxlifetime value
* default is 1440 or 24 mins.
*
*/
function gc($age) {
$sql = "DELETE FROM sc_php_session WHERE updated_on < (now() - INTERVAL :age SECOND) ";
$stmt = $this->dbh->prepare($sql);
$stmt->bindParam(":age",$age, \PDO::PARAM_INT);
$stmt->execute();
}
}
}
?>
I just wanted to add (and you may already know) that PHP's default session storage (which uses files) does lock the sessions files. Obviously using files for sessions has plenty of shortcomings which is probably why you are looking at a database solution.
Check with mysql_affected_rows() if the lock was obtained or not. If it was obtained - proceed. If not - re-attempt the operation every 0.5 seconds. If in 40 seconds the lock is still not obtained, throw an exception.
I see a problem in blocking script execution with this continual check for a lock. You're suggesting that PHP run for up to 40 seconds looking for this lock everytime the session is initialized (if I'm reading that correctly.)
Recommendation
If you have a clustered environment, I would highly recommend memcached. It supports a server/client relationship so all clustered instances can defer to the memcached server. It doesn't have locking issues you're fearful of, and is plenty fast. Quote from their page:
Regardless of what database you use (MS-SQL, Oracle, Postgres, MySQL-InnoDB, etc..), there's a lot of overhead in implementing ACID properties in a RDBMS, especially when disks are involved, which means queries are going to block. For databases that aren't ACID-compliant (like MySQL-MyISAM), that overhead doesn't exist, but reading threads block on the writing threads. memcached never blocks.
Otherwise, if you're still committed to an RDBMS session store (and worried that locking will become a problem), you could try some sort of sharding based on a sticky session identifier (grasping at straws here.) Knowing nothing else about your architecture, that's about as specific as I can get.
My question is why lock at all? Why not just let the last write succeed? You shouldn't be using session data as a cache, so writes tend to be infrequent, and in practice never trample each other.

Categories