I have a couple of functions used to prevent concurrent execution of some PHP scripts that are being run via crontab on a debian 10.13. Essentially, if an instance of that script is already running, the new instance is supposed to quit.
The problem is that every now and then (about 2-3 times per day for a script that runs every 5 minutes so perhaps in 1% of the cases), the semaphore handle seems to be gone at the time the script attempts to release it. This happens even though the script takes less than 30s to run, typically 10-15s.
The setup is as follows, in the script, right after including the library:
if (!singleInstanceHelper()) {
die("The script ".__file__." is already running.\n");
}
In the included library:
function singleInstanceHelper($wait = false, $suffix = '') {
global $semdebug;
$bt = debug_backtrace();
$calling_file = $bt[count($bt)-1]['file'];
$semaphore_key = crc32($calling_file.$suffix); // unique integer key for this semaphore
$semaphore_max = 1; // The number of processes that can acquire this semaphore
$semaphore_permissions = 0666; // Unix style permissions for this semaphore
$semaphore_autorelease = 1; // Auto release the semaphore if the request shuts down
// open a new or get an existing semaphore
$semaphore_handle = sem_get($semaphore_key, $semaphore_max, $semaphore_permissions, $semaphore_autorelease);
$semdebug = "semaphore key $semaphore_key handle ".print_r($semaphore_handle, true);
if(!$semaphore_handle)
{
die("Failed to get semaphore - sem_get().\nCalling script: $calling_file.\n");
}
// acquire exclusive control
if (sem_acquire($semaphore_handle, !$wait)) {
register_shutdown_function('semaphoreRelease', $semaphore_handle);
return true;
}
return false;
}
function semaphoreRelease($semaphore_handle) {
global $semdebug;
if (!sem_release($semaphore_handle)) {
print $semdebug."\n";
system('ipcs -s');
}
}
This is what crond mails me when the failure occurs (the script runs as user, line 1790 is sem_release):
PHP Warning: sem_release(): failed to release key 0x9efb366f: Invalid argument in /path/to/lib.php on line 1790
semaphore key 2667263599 handle Resource id #25
------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 957349888 user 600 1
0x00000000 1083801604 user 600 1
0x00000000 1083834373 user 600 1
0x00000000 1083867142 user 600 1
0x00000000 1083899911 user 600 1
0x00000000 1083932680 user 600 1
0x1a48fa0e 455868425 root 666 3
0x474b7f2f 470319114 root 666 3
0x30614a01 1002373236 root 666 3
Any idea what might be causing this? Or, perhaps, any suggestions on how to further debug it or how to modify the code so that it no longer happens?
Related
I have a PHP script that someone wrote for me that takes a Google Alert feed and strips the Google portions of the URL for each feed item, producing a standard RSS feed with links directly to the original source URLs.
For instance, it takes a Google Alert URL and runs it through the script like this:
http://foo.com/feeds/xml-converter.php?file_name=[path to output file]&tempfile=[path to tempfile]&link=[Google Alert URL]
The result is a clean RSS feed minus the Google redirect information.
Here's the script:
<?php
set_time_limit(60);
$link = 'variable';
$tempfile = 'variable';
$file_name = 'variable';
extract($_GET);
//load xml and save to temp file
$xml=simplexml_load_file($link) or die("Error: Cannot create object");
$xml->asXml($tempfile);
//load xml from temp file
$xml=simplexml_load_file($tempfile) or die("Error: Cannot create object");
//iterate all the linkmy
foreach($xml->entry as $entry) {
$url = $entry->link['href'];
preg_match("/url=(.*?)&/",$url,$program);
$entry->link['href'] =$program[1];
}
if (file_exists($file_name) && isset($link) && isset($file_name)) {
unlink($file_name);
}
$xml->asXml($file_name);
unlink($tempfile);
?>
The script generally works fine and runs in much less than a second in most cases.
However, the problem I'm experiencing is that on occasion the process that runs the script on the server keeps running endlessly at 100% processor usage.
When I use strace to watch what's happening, I get these three steps over and over again for hours at a time, endlessly:
wait4(-1, 0x7ffcc3390550, WNOHANG|__WALL, NULL) = -1 ECHILD (No child processes)
lseek(5, 0, SEEK_SET) = 0
getdents(5, /* 170 entries */, 32768) = 5088
wait4(-1, 0x7ffcc3390550, WNOHANG|__WALL, NULL) = -1 ECHILD (No child processes)
lseek(5, 0, SEEK_SET) = 0
getdents(5, /* 170 entries */, 32768) = 5088
wait4(-1, 0x7ffcc3390550, WNOHANG|__WALL, NULL) = -1 ECHILD (No child processes)
lseek(5, 0, SEEK_SET) = 0
getdents(5, /* 170 entries */, 32768) = 5088
etc.
I run this script on a cron job, and while it works just fine most of the time, it occasionally creates a never-ending process. Over time, multiple processes get stuck running this way, all pegged at 100% of processor.
I'm hoping someone can tell me how I can improve the script to prevent this unfortunate and unexpected occasional result from happening.
I want to send ~50 requests to different pages on the same domain and then, I'm using DOM object to gain urls to articles.
The problem is that this number of requests takes over 30 sec.
for ($i = 1; $i < 51; $i++)
{
$url = 'http://example.com/page/'.$i.'/';
$client = new Zend_Http_Client($url);
$response = $client->request();
$dom = new Zend_Dom_Query($response); // without this two lines, execution is also too long
$results = $dom->query('li'); //
}
Is there any way to speed this up?
It's a generel problem by design - not the code itself. If you're doing a for-loop over 50 items each opening an request to an remote uri, things get pretty slow since every requests waits until responde from the remote uri. e.g.: a request takes ~0,6 sec to been completed, multiple this by 50 and you get an exection time of 30 seconds!
Other problem is that most webserver limits its (open) connections per client to an specific amount. So even if you're able to do 50 requests simultaneously (which you're currently not), things won't speed up measurably.
In my option there is only one solution (without any deep going changes):
Change the amout of requests per exection. Make chunks from e.g. only 5 - 10 per (script)-call and trigger them by an external call (e.g. run them by cron).
Todo:
Build a wrapper function which is able to save the state of its current run ("i did request 1 - 10 at my last run, so now I have to call 11 - 20) into a file or database and trigger this function by an cron.
Example Code (untested) for better declaration;
[...]
private static $_chunks = 10; //amout of calls per run
public function cronAction() {
$lastrun = //here get last run parameter saved from local file or database
$this->crawl($lastrun);
}
private function crawl($lastrun) {
$limit = $this->_chunks + $lastrun;
for ($i = $lastrun; $i < limit; $i++)
{
[...] //do stuff here
}
//here set $lastrun parameter to new value inside local file / database
}
[...]
I can't think of a way to speed it up but you can increase the timeout limit in PHP if that is your concern:
for($i=1; $i<51; $i++) {
set_time_limit(30); //This restarts the timer to 30 seconds starting now
//Do long things here
}
In a script I'm trying to check whether the same script is already running using MySQL GET_LOCK. The problem is, when a script tries to get lock, which isn't free, it blocks forever regardless of the parameter I provide.
<?php
class Controller_Refresher extends Controller {
public function action_run($key) {
echo date('H:i:s'."\n", time());
$this->die_if_running();
$this->run();
echo date('H:i:s'."\n", time());
}
private function die_if_running() {
$result = DB::query(Database::SELECT, "SELECT IS_FREE_LOCK('refresher_running') AS free")->execute();
if (! intval($result[0]['free'])) die('Running already');
$result = DB::query(Database::SELECT, "SELECT GET_LOCK('refresher_running', 1)")->execute();
}
private function run() {
echo "Starting\n";
ob_flush();
sleep(10);
DB::query(Database::SELECT, "SELECT RELEASE_LOCK('refresher_running')")->execute();
}
}
When I run this in 2 tabs in browser, I get e.g.:
-tab 1-
20:48:16
Starting
20:48:26
-tab 2-
20:48:27
Starting
20:48:37
While what I want to do is to make the second tab die('Running already');.
Watch out - this problem might actually be caused by php locking the session file:
https://stackoverflow.com/a/5487811/539149
So you should call session_write_close() before any code that needs to run concurrently. I discovered this after trying this Mutex class:
http://code.google.com/p/mutex-for-php/
The class worked great but my php scripts were still running one by one!
Also, you don't need IS_FREE_LOCK(). Just call GET_LOCK('refresher_running', 0) and it will either return 1 if it gives you the lock or 0 if the lock is taken. It's more atomic that way. Of course, lock timeouts can still be useful in certain situations, like when you want to queue up tasks, but watch out for the script timing out if you get too many simultaneous requests.
Zack Morris
One option would be to rely on a filesystem lock instead of a database. Since it's the script execution that needs handling, it should not matter. A sample from the manual with a non-blocking exclusive lock:
$fp = fopen('/tmp/lock.txt', 'r+');
/* Activate the LOCK_NB option on an LOCK_EX operation */
if(!flock($fp, LOCK_EX | LOCK_NB)) {
die('Running already');
}
/* ... */
fclose($fp);
Edit
Another option would be to use a status file that gets created at the beginning of each exection and will be automatically deleted by register_shutdown_function upon script completion.
The script would simply check the existence of the status file and if it's already there, execution would stop:
define('statusFile', sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'myjob.running');
//
// If a process is already running then exit
//
if (file_exists(statusFile)) {
die('Running already');
} else {
file_put_contents(date('Y-m-d H:i:s'), statusFile);
}
//
// Other code here
//
function shutdown() {
unlink(statusFile);
}
//
// Remove the status file on completion
//
register_shutdown_function('shutdown');
Time to time, I receive this strange warning message. It is usually gone on page reload. What does that mean. I googled but to no avail.
Warning: include(): GC cache entry '/.../...class.php' (dev=2049 ino=37120489) was on gc-list for 3840 seconds in /.../...class.php on line 111
Definitely this issue goes from APC, source code from package apc-3.1.6-r1. When item is inserted into user cache or file cache, this function is called.
static void process_pending_removals(apc_cache_t* cache TSRMLS_DC)
{
slot_t** slot;
time_t now;
/* This function scans the list of removed cache entries and deletes any
* entry whose reference count is zero (indicating that it is no longer
* being executed) or that has been on the pending list for more than
* cache->gc_ttl seconds (we issue a warning in the latter case).
*/
if (!cache->header->deleted_list)
return;
slot = &cache->header->deleted_list;
now = time(0);
while (*slot != NULL) {
int gc_sec = cache->gc_ttl ? (now - (*slot)->deletion_time) : 0;
if ((*slot)->value->ref_count <= 0 || gc_sec > cache->gc_ttl) {
slot_t* dead = *slot;
if (dead->value->ref_count > 0) {
switch(dead->value->type) {
case APC_CACHE_ENTRY_FILE:
apc_warning("GC cache entry '%s' (dev=%d ino=%d) was on gc-list for %d seconds" TSRMLS_CC,
dead->value->data.file.filename, dead->key.data.file.device, dead->key.data.file.inode, gc_sec);
break;
case APC_CACHE_ENTRY_USER:
apc_warning("GC cache entry '%s'was on gc-list for %d seconds" TSRMLS_CC, dead->value->data.user.info, gc_sec);
break;
}
}
*slot = dead->next;
free_slot(dead TSRMLS_CC);
}
else {
slot = &(*slot)->next;
}
}
}
From APC configuration ( http://cz.php.net/manual/en/apc.configuration.php#ini.apc.gc-ttl )
apc.gc_ttl integer
The number of seconds that a cache entry may remain on the garbage-collection list. This value provides a fail-safe in the event that a server process dies while executing a cached source file; if that source file is modified, the memory allocated for the old version will not be reclaimed until this TTL reached. Set to zero to disable this feature.
We get messages "GC cache entry '%s' (dev=%d ino=%d) was on gc-list for %d seconds" or "GC cache entry '%s'was on gc-list for %d seconds" in this condition:
(gc_sec > cache->gc_ttl) && (dead->value->ref_count > 0)
First condition means, item was deleted later then apc.gc_ttl seconds ago and its still in garbage collector list. Seconds condition means, item is still referenced.
e.g. when process unexpectedly died, reference is not decreased. First apc.ttl seconds is active in APC cache, then is deleted (there isn't next hit on this item). Now item is on garbage collector list (GC) and apc.gc_ttl timeout is running. When apc.gc_ttl is less then (now - item_deletion_time), warning is written and item is completely flushed.
Try to check your logs (web server, php, system/kernel) to critical errors, e.g. php, web server segfault.
I have a problem wtih session_start() on primary server. When I load page for the first time, it takes less than 1 second to complete request. If I'll wait for approximately 12-15 seconds and then reload page, time of loading will be the same. But when I'm trying to refresh page after, for example, 3 or 5 seconds after initial loading, time of the server's response equals 10 seconds.
I made some tests to define a bottleneck in my script and I found out, that function session_start() executes for 9.8 seconds. I'm using PEAR package HTTP_Session2. Here is code snippet:
HTTP_Session2::useCookies(SESSION_USE_COOKIE);
/* Next line was added to make logging of execution time possible. */
self::writeToFile('HTTP_useCookies(1) -> '.self::getWorkTime());
HTTP_Session2::start("SID");
self::writeToFile('HTTP_start(2) -> '.self::getWorkTime());
HTTP_Session2::setExpire(time() + SESSION_EXPIRE);
self::writeToFile('HTTP_setExpire(3) -> '.self::getWorkTime());
Text of the logs:
//First loading (13:34:35)
HTTP_useCookies(1) -> 0.00038
HTTP_start(2) -> 0.00077
HTTP_setExpire(3) -> 0.00090
// Second loading (13:34:39)(4 seconds later)
HTTP_useCookies(1) -> 0.00029
HTTP_start(2) -> <<<<<< 10.80752 >>>>>
HTTP_setExpire(3) -> <<<<<< 10.80780 >>>>>
//Third loading (13:34:56)
HTTP_useCookies(1) -> 0.00041
HTTP_start(2) -> 0.00071
HTTP_setExpire(3) -> 0.00083
So I found, that problem is in the HTTP_Session2::start() function. Code of the function HTTP_Session2::start():
public function start($name = 'SessionID', $id = null)
{
self::name($name);
if (is_null(self::detectID())) {
if ($id) {
self::id($id);
} else {
self::id(uniqid(dechex(rand())));
}
}
session_start();
if (!isset($_SESSION['__HTTP_Session2_Info'])) {
$_SESSION['__HTTP_Session2_Info'] = self::STARTED;
} else {
$_SESSION['__HTTP_Session2_Info'] = self::CONTINUED;
}
}
I can't understand what is the cause of time delay. Probably, there are wrong Apache settings on the server. Or, maybe, there are some 'locks' on files with session information.
Maybe, someone has already encountered such problems and can help to solve it.
The file containing the session's informations is locked during the time PHP serves a request.
So, if you have one PHP script (using a session) that's currently being executed, and the same user sends another request, the second request will wait until the first is finished.