This is the first time I've ever asked a question on here, but I've used stackoverflow many times in the past to find solutions for problems I'm having in my code.
I'm working on a database transfer page on a php site that uploads csv files and updates the database, and depending on the type of update the user selects this data can be updated/inserted by key. Because of this I want to run DBCC CHECKIDENT after the updates have been made to ensure that new entries will be incremented correctly after the largest key in the table.
This is the php code I'm running:
$getMaxID = new Query("SELECT MAX($tableKeys[$t]) as max from $t", __LINE__, __FILE__);
$maxID = $getMaxID->result(0,'max');
$result = new Query("DBCC CHECKIDENT ('$t', RESEED, $maxID)", __LINE__, __FILE__);
$t is the table name stored in an array of table names.
I get the following error from this code:
There has been a system error. We apologize for the inconvienience.
Error Details: [Microsoft][SQL Server Native Client 11.0][SQL Server]Checking identity information: current identity value '16', current column value '16'.
Line #: 615
File: C:\OCPOS\htdocs\OCPOS\menuTransfer\importMenu.php
Query: DBCC CHECKIDENT ('table', RESEED, 16)
What's confusing me is when I cut an paste DBCC CHECKIDENT ('table', RESEED, 16) into server management studio it works and i get:
Checking identity information: current identity value '16', current column value '16'.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
If anyone has any ideas what is causing this or if there's a post I missed that addresses this issue any help would be much appreciated.
Below is the query class. I didn't make it:
class Query{
var $stmt; //hold link to result
var $queryStr; //hold string
var $queryLine;
var $queryFile;
function Query($str, $line, $file)
{
global $conn;
$this->queryStr = $str;
$this->queryLine = $line;
$this->queryFile = $file;
$this->stmt = #sqlsrv_query($conn, $this->queryStr, array(), array("Scrollable" => SQLSRV_CURSOR_KEYSET)) or $this->error(sqlsrv_errors());
}
function error($var)
{
echo "
<p style='border: 1px solid #c00; margin: 5px; padding: 5px;'>
There has been a system error. We apologize for the inconvienience.<br/>
Error Details: ". $var[0]['message'] ."<br/>
Line #: $this->queryLine<br/>
File: $this->queryFile<br/>
Query: $this->queryStr<br/>
</p>
";
}
function fetch_array()
{
$array = sqlsrv_fetch_array($this->stmt);
if(is_array($array))
return $array;
else
{
return false;
}
}
function fetch_assoc_array()
{
$array = sqlsrv_fetch_array($this->stmt, SQLSRV_FETCH_ASSOC);
if(is_array($array))
return $array;
else
{
return false;
}
}
function num_rows()
{
return sqlsrv_num_rows($this->stmt);
}
function numrows()
{
return $this->num_rows();
}
function result($row, $var)
{
$array = sqlsrv_fetch_array($this->stmt, SQLSRV_FETCH_ASSOC, SQLSRV_SCROLL_FIRST);
return $array[$var];
}
function all() {
$return = array();
while ($tmp = $this->fetch_array()) {
$return[] = $tmp;
}
return $return;
}
function arr() {
return $this->fetch_array();
}
function getAll() {
$return = array();
while ($tmp = $this->fetch_array()) {
$return = array_merge($return, $tmp);
}
return $return;
}
function extract($var) {
$rv = array();
while ($tmp = $this->fetch_array()) {
$rv[] = $tmp[$var];
}
return $rv;
}
}
I believe what is happening is that the database driver in PHP is assuming that any output from the database connection (as opposed to Result Sets, which are returned separately) is some form of error. Note that the output you get from SQL Server is actually identical in the two cases, except for some extra context information. I have seen similar problems when the ROW_COUNT setting is left on, with the textual "rows affected/returned" message interpreted as an error.
Since DBCC commands are, to put it nicely, not very standardised, I doubt there is any way to suppress this message at the SQL Server end. However, since you know that it is not really an error, you should be able to simply ignore it on the PHP side. You are already suppressing PHP's own error mechanism with the # operator, so just need a version of your Query method without the or $this->error(sqlsrv_errors());.
Incidentally, this is a good example of the value of separating responsibilities carefully: since the database abstraction class has taken responsibility for formatting the error into HTML and displaying it to the user, you are more-or-less forced to alter or replace that logic. If, instead, it threw an Exception, your calling code could catch this particular case while leaving others to pass through to a global error-handler which formatted the message appropriately.
I used to set $expire = 0; in all cache.php files. Delete all from cache folder. Put $this->cache->delete(); in some random files. Use Ctrl+F5 in my brouser. But cache still alive.
Easiest way would be to just return false from the cache->get method:
system/library/cache.php:
public function get($key) {
return false;
Open up system/library/cache.php and comment these:
public function get($key) {
// return $this->adaptor->get($key);
}
public function set($key, $value) {
// return $this->adaptor->set($key, $value);
}
I know, it's very annoying. OpenCart really should implement an easier way to disable cache.
template_cache takes the value from developer_theme. Do the following:
Go to oc_setting table (database)
Find (developer_theme) of the key column
Then change the value column to 0
I found it!! This worked for me... find vqmod.php file and before $this->_filesModded[$sourcePath] = array('cached' => $changed); Assign $changed to false like $changed = false; save and reload page.
How should I multithread some php-cli code that needs a timeout?
I'm using PHP 5.6 on Centos 6.6 from the command line.
I'm not very familiar with multithreading terminology or code. I'll simplify the code here but it is 100% representative of what I want to do.
The non-threaded code currently looks something like this:
$datasets = MyLibrary::getAllRawDataFromDBasArrays();
foreach ($datasets as $dataset) {
MyLibrary::processRawDataAndStoreResultInDB($dataset);
}
exit; // just for clarity
I need to prefetch all my datasets, and each processRawDataAndStoreResultInDB() cannot fetch it's own dataset. Sometimes processRawDataAndStoreResultInDB() takes too long to process a dataset, so I want to limit the amount of time it has to process it.
So you can see that making it multithreaded would
Speed it up by allowing multiple processRawDataAndStoreResultInDB() to execute at the same time
Use set_time_limit() to limit the amount of time each one has to process each dataset
Notice that I don't need to come back to my main program. Since this is a simplification, you can trust that I don't want to collect all the processed datasets and do a single save into the DB after they are all done.
I'd like to do something like:
class MyWorkerThread extends SomeThreadType {
public function __construct($timeout, $dataset) {
$this->timeout = $timeout;
$this->dataset = $dataset;
}
public function run() {
set_time_limit($this->timeout);
MyLibrary::processRawDataAndStoreResultInDB($this->dataset);
}
}
$numberOfThreads = 4;
$pool = somePoolClass($numberOfThreads);
$pool->start();
$datasets = MyLibrary::getAllRawDataFromDBasArrays();
$timeoutForEachThread = 5; // seconds
foreach ($datasets as $dataset) {
$thread = new MyWorkerThread($timeoutForEachThread, $dataset);
$thread->addCallbackOnTerminated(function() {
if ($this->isTimeout()) {
MyLibrary::saveBadDatasetToDb($dataset);
}
}
$pool->addToQueue($thread);
}
$pool->waitUntilAllWorkersAreFinished();
exit; // for clarity
From my research online I've found the PHP extension pthreads which I can use with my thread-safe php CLI, or I could use the PCNTL extension or a wrapper library around it (say, Arara/Process)
https://github.com/krakjoe/pthreads (and the example directory)
https://github.com/Arara/Process (pcntl wrapper)
When I look at them and their examples though (especially the pthreads pool example) I get confused quickly by the terminology and which classes I should use to achieve the kind of multithreading I'm looking for.
I even wouldn't mind creating the pool class myself, if I had a isRunning(), isTerminated(), getTerminationStatus() and execute() function on a thread class, as it would be a simple queue.
Can someone with more experience please direct me to which library, classes and functions I should be using to map to my example above? Am I taking the wrong approach completely?
Thanks in advance.
Here comes an example using worker processes. I'm using the pcntl extension.
/**
* Spawns a worker process and returns it pid or -1
* if something goes wrong.
*
* #param callback function, closure or method to call
* #return integer
*/
function worker($callback) {
$pid = pcntl_fork();
if($pid === 0) {
// Child process
exit($callback());
} else {
// Main process or an error
return $pid;
}
}
$datasets = array(
array('test', '123'),
array('foo', 'bar')
);
$maxWorkers = 1;
$numWorkers = 0;
foreach($datasets as $dataset) {
$pid = worker(function () use ($dataset) {
// Do DB stuff here
var_dump($dataset);
return 0;
});
if($pid !== -1) {
$numWorkers++;
} else {
// Handle fork errors here
echo 'Failed to spawn worker';
}
// If $maxWorkers is reached we need to wait
// for at least one child to return
if($numWorkers === $maxWorkers) {
// $status is passed by reference
$pid = pcntl_wait($status);
echo "child process $pid returned $status\n";
$numWorkers--;
}
}
// (Non blocking) wait for the remaining childs
while(true) {
// $status is passed by reference
$pid = pcntl_wait($status, WNOHANG);
if(is_null($pid) || $pid === -1) {
break;
}
if($pid === 0) {
// Be patient ...
usleep(50000);
continue;
}
echo "child process $pid returned $status\n";
}
I'm trying to make an error logging class, I have some functions to set up various output methods like DB, file, return and screen. I want all errors to be stored into an array and when __destruct() is called, I want to stop the client from waiting for data and the log details about errors the user experienced. This way they don't have to report errors to me.
I have 2 modes, a simple GUI to test the functionality and the actual script generates responses in JSON, machine to machine. For GUI the final dump is fine but for JSON it destroys the response. So all error reporting is off and I have to handle any errors that would be dumped on screen myself, hence the $return in function flush_log($return) which make the function return a string if set to true.
After reporting flushing the errors I want to:unset($this->log_arrays)
or empty: $this->log_arrays=Array();, but it is out of scope - I understand why, my function uses a local copy - but how do I reset the array?
[EDIT]:
I tried:
$this->log_arrays = Array();
$this->log_arrays = null;
array popping:
for ($i = 1; count($this->log_arrays); $i++)
{
array_pop($this->log_arrays);
}
But I think none of it can work because within a class function you work with copies of variables, so they're basically out of scope.
[/EDIT]:
This is an already simplified class.. :
<?php
class log_strings extends mysqli
{
private $log_arrays = Array();
public function __construct($output_to_file=false, $output_to_db=true, $fall_back_file=true, $arguments, $ip=null)
{
// Setup mysqli connection, file handle or report error if one or all have failed.
// Also check wich outputs should be used and keep that info for later.
}
public function log($level, $string)
{
$log_arrays[] = Array('level' => $level, 'string' => $string);
}
public function __destruct()
{
$this->flush_log();
}
public function flush_log($return=false)
{
if (!isset($log_arrays) && count($log_arrays) == 0)
{
return true;
}
if ($return)
{
return $this->return_output();
}
else
{
$success = false;
// if enabled, output to db
if ($this->output_to_db)
{
$success = $success || $this->mysqli_output();
}
// if enabled or if db failed and fallback is enabled, output to file
if ($this->output_to_file || ($this->fall_back_file && !$success))
{
$success = $success || $this->file_output();
}
// if neither file or db succeeded, dump on screen
if ($success = false)
{
$this->screen_dump();
}
return true;
}
unset($this->log_arrays); // <= This is what it is all about!
}
private function screen_dump()
{
foreach($this->log_arrays as $array)
{
echo "<strong>{$array['level']}</strong>{$array['string']}<br/>\n";
}
}
private function mysqli_output()
{
// Output to db functionally equal to $this->screen_dump()
}
private function file_output()
{
// Output to file functionally equal to $this->screen_dump()
}
private function return_output()
{
// Return output functionally equal to $this->screen_dump()
}
}
?>
Resetting the array should work
$this->log_arrays = array();
Unset a class property is a very bad idea. Because it may be used in other methods or other classes using a potentionally getter of your class.
I'm trying to update a variable in APC, and will be many processes trying to do that.
APC doesn't provide locking functionality, so I'm considering using other mechanisms... what I've found so far is mysql's GET_LOCK(), and php's flock(). Anything else worth considering?
Update: I've found sem_acquire, but it seems to be a blocking lock.
/*
CLASS ExclusiveLock
Description
==================================================================
This is a pseudo implementation of mutex since php does not have
any thread synchronization objects
This class uses flock() as a base to provide locking functionality.
Lock will be released in following cases
1 - user calls unlock
2 - when this lock object gets deleted
3 - when request or script ends
==================================================================
Usage:
//get the lock
$lock = new ExclusiveLock( "mylock" );
//lock
if( $lock->lock( ) == FALSE )
error("Locking failed");
//--
//Do your work here
//--
//unlock
$lock->unlock();
===================================================================
*/
class ExclusiveLock
{
protected $key = null; //user given value
protected $file = null; //resource to lock
protected $own = FALSE; //have we locked resource
function __construct( $key )
{
$this->key = $key;
//create a new resource or get exisitng with same key
$this->file = fopen("$key.lockfile", 'w+');
}
function __destruct()
{
if( $this->own == TRUE )
$this->unlock( );
}
function lock( )
{
if( !flock($this->file, LOCK_EX | LOCK_NB))
{ //failed
$key = $this->key;
error_log("ExclusiveLock::acquire_lock FAILED to acquire lock [$key]");
return FALSE;
}
ftruncate($this->file, 0); // truncate file
//write something to just help debugging
fwrite( $this->file, "Locked\n");
fflush( $this->file );
$this->own = TRUE;
return TRUE; // success
}
function unlock( )
{
$key = $this->key;
if( $this->own == TRUE )
{
if( !flock($this->file, LOCK_UN) )
{ //failed
error_log("ExclusiveLock::lock FAILED to release lock [$key]");
return FALSE;
}
ftruncate($this->file, 0); // truncate file
//write something to just help debugging
fwrite( $this->file, "Unlocked\n");
fflush( $this->file );
$this->own = FALSE;
}
else
{
error_log("ExclusiveLock::unlock called on [$key] but its not acquired by caller");
}
return TRUE; // success
}
};
You can use the apc_add function to achieve this without resorting to file systems or mysql. apc_add only succeeds when the variable is not already stored; thus, providing a mechanism of locking. TTL can be used to ensure that falied lockholders won't keep on holding the lock forever.
The reason apc_add is the correct solution is because it avoids the race condition that would otherwise exist between checking the lock and setting it to 'locked by you'. Since apc_add only sets the value if it's not already set ( "adds" it to the cache ), it ensures that the lock can't be aquired by two calls at once, regardless of their proximity in time. No solution that doesn't check and set the lock at the same time will inherently suffer from this race condition; one atomic operation is required to successfully lock without race condition.
Since APC locks will only exist in the context of that php execution, it's probably not the best solution for general locking, as it doesn't support locks between hosts. Memcache also provides an atomic add function and thus can also be used with this technique - which is one method of locking between hosts. Redis also supports atomic 'SETNX' functions and TTL, and is a very common method of locking and synchronization between hosts. Howerver, the OP requests a solution for APC in particular.
If the point of the lock is to prevent multiple processes from trying to populate an empty cache key, why wouldn't you want to have a blocking lock?
$value = apc_fetch($KEY);
if ($value === FALSE) {
shm_acquire($SEMAPHORE);
$recheck_value = apc_fetch($KEY);
if ($recheck_value !== FALSE) {
$new_value = expensive_operation();
apc_store($KEY, $new_value);
$value = $new_value;
} else {
$value = $recheck_value;
}
shm_release($SEMAPHORE);
}
If the cache is good, you just roll with it. If there's nothing in the cache, you get a lock. Once you have the lock, you'll need to double-check the cache to make sure that, while you were waiting to get the lock, the cache wasn't repopulated. If the cache was repopulated, use that value & release the lock, otherwise, you do the computation, populate the cache & then release your lock.
Actually, check to see if this will work better then Peter's suggestion.
http://us2.php.net/flock
use an exclusive lock and if your comfortable with it, put everything else that attempted to lock the file in a 2-3 second sleep. If done right your site will experience a hang regarding the locked resource but not a horde of scripts fighting to cache the samething.
If you don't mind basing your lock on the filesystem, then you could use fopen() with mode 'x'. Here is an example:
$f = fopen("lockFile.txt", 'x');
if($f) {
$me = getmypid();
$now = date('Y-m-d H:i:s');
fwrite($f, "Locked by $me at $now\n");
fclose($f);
doStuffInLock();
unlink("lockFile.txt"); // unlock
}
else {
echo "File is locked: " . file_get_contents("lockFile.txt");
exit;
}
See www.php.net/fopen
I realize this is a year old, but I just stumbled upon the question while doing some research myself on locking in PHP.
It occurs to me that a solution might be possible using APC itself. Call me crazy, but this might be a workable approach:
function acquire_lock($key, $expire=60) {
if (is_locked($key)) {
return null;
}
return apc_store($key, true, $expire);
}
function release_lock($key) {
if (!is_locked($key)) {
return null;
}
return apc_delete($key);
}
function is_locked($key) {
return apc_fetch($key);
}
// example use
if (acquire_lock("foo")) {
do_something_that_requires_a_lock();
release_lock("foo");
}
In practice I might throw another function in there to generate a key to use here, just to prevent collision with an existing APC key, e.g.:
function key_for_lock($str) {
return md5($str."locked");
}
The $expire parameter is a nice feature of APC to use, since it prevents your lock from being held forever if your script dies or something like that.
Hopefully this answer is helpful for anyone else who stumbles here a year later.
EAccelerator has methods for it; eaccelerator_lock and eaccelerator_unlock.
Can't say if this is the best way to handle the job, but at least it is convenient.
function WhileLocked($pathname, callable $function, $proj = ' ')
{
// create a semaphore for a given pathname and optional project id
$semaphore = sem_get(ftok($pathname, $proj)); // see ftok for details
sem_acquire($semaphore);
try {
// capture result
$result = call_user_func($function);
} catch (Exception $e) {
// release lock and pass on all errors
sem_release($semaphore);
throw $e;
}
// also release lock if all is good
sem_release($semaphore);
return $result;
}
Usage is as simple as this.
$result = WhileLocked(__FILE__, function () use ($that) {
$this->doSomethingNonsimultaneously($that->getFoo());
});
Third optional argument can come handy if you use this function more than once per file.
Last but not least it isn't hard to modify this function (while keeping its signature) to use any other kind of locking mechanism at a later date, e.g. if you happen to find yourself working with multiple servers.
APC is now considered unmaintained and dead. It's successor APCu offers locking via apcu_entry. But be aware, that it also prohibits the concurrent execution of any other APCu functions. Depending on your use case, this might be OK for you.
From the manual:
Note: When control enters apcu_entry() the lock for the cache is acquired exclusively, it is released when control leaves apcu_entry(): In effect, this turns the body of generator into a critical section, disallowing two processes from executing the same code paths concurrently. In addition, it prohibits the concurrent execution of any other APCu functions, since they will acquire the same lock.
APCu has apcu_entry since 5.1.0, can implement a lock mechanism with it now:
/** get a lock, will wait until the lock is available,
* make sure handle deadlock yourself :p
*
* useage : $lock = lock('THE_LOCK_KEY', uniqid(), 50);
*
* #param $lock_key : the lock you want to get it
* #param $lock_value : the unique value to specify lock owner
* #param $retry_millis : wait befor retry
* #return ['lock_key'=>$lock_key, 'lock_value'=>$lock_value]
*/
function lock($lock_key, $lock_value, $retry_millis) {
$got_lock = false;
while (!$got_lock) {
$fetched_lock_value = apcu_entry($lock_key, function ($key) use ($lock_value) {
return $lock_value;
}, 100);
$got_lock = ($fetched_lock_value == $lock_value);
if (!$got_lock) usleep($retry_millis*1000);
}
return ['lock_key'=>$lock_key, 'lock_value'=>$lock_value];
}
/** release a lock
*
* usage : unlock($lock);
*
* #param $lock : return value of function lock
*/
function unlock($lock) {
apcu_delete($lock['lock_key']);
}
What I've found, actually, is that I don't need any locking at all... given what I'm trying to create is a map of all the class => path associations for autoload, it doesn't matter if one process overwrites what the other one has found (it's highly unlikely, if coded properly), because the data will get there eventually anyway. So, the solution turned out to be "no locks".