Implement atomic counter in Memcached without cas - php

We have a web page want to limit uo to 100 people can access concurrently, so we use a memcached to implement a global counter, e.g.
We are using http://www.php.net/manual/en/class.memcache.php so there is not cas, current code is something like
$count = $memcache_obj->get('count');
if ($count < 100) {
$memcache_obj->set('count', $count+1);
echo "Welcome";
} else {
echo "No luck";
}
As you can see there is race condition in the above code and but if we are not going to replace memcached extension which support cas, it is able to support it using PHP code only?

As answer to "emcconville". This is non-blocking even without CAS.
If your concerned about race conditions, and the count value is completely arbitrary, you can use Memcache::increment directly before any business logic.
The increment method will return the current value after the incrementation takes place; of which, you can compare results. Increment will also return false if the key has yet to be set; allowing for your application to deal with it as needed.
$current = $memcache_obj->increment('count');
if($current === false) {
// NOT_FOUND, so let's create it
// Will return false if has just been created by someone else.
$memcache_obj->add('count',0); // <-- no risk of race-condition
// At this point 'count' key is created by us or someone else (other server/process).
// "increment" will update 0 or whatever it is at the moment by 1.
$current = $memcache_obj->increment('count')
echo "You are the $current!";
}
if ($current < 100) {
echo "Hazah! You are under the limit. Congrats!";
} else {
echo "Ah Snap! No Luck - you reached the limit.";
// If your worried about the value growing _too_ big, just drop the value down.
// $memcache_obj->decrement('count');
}

If your concerned about race conditions, and the count value is completely arbitrary, you can use Memcache::increment directly before any business logic.
The increment method will return the current value after the incrementation takes place; of which, you can compare results. Increment will also return false if the key has yet to be set; allowing for your application to deal with it as needed.
$current = $memcache_obj->increment('count');
if($current === false) {
// NOT_FOUND, so let's create it
$memcache_obj->set('count',1); // <-- still risk of race-condition
echo "Your the first!";
} else if ($current < 100) {
echo "Hazah! Your under the limit.";
} else {
echo "Ah Snap! No Luck";
// If your worried about the value growing _too_ big, just drop the value down.
// $memcache_obj->decrement('count');
}

function memcache_atomic_increment($counter_name, $delta = 1) {
$mc = new Memcache;
$mc->connect('localhost', 11211) or die("Could not connect");
while (($mc->increment($counter_name, $delta) === false) &&
($mc->add($counter_name, ($delta<0)?0:$delta, 0, 0) === false)) {
// loop until one of them succeeds
}
return intval($mc->get($counter_name));
}

The comments in Memcache::add include an example locking function, have you tried it out?

Related

usleep() in while loop causing issues - whole process freezes

I am using php, Laravel, Redis, and SQL on an Ubuntu localhost server. I have made a bunch of methods that return results from API searches after some processing. I am calling 5 of these methods which will be very slow if done synchronously, so I've been experimenting with async approaches (which I know php isn't optimised for). After a few approaches I have found some success with pcntl_fork(), but I'm running into some nasty problems.
Edit: After some messing around I have found that if I remove the while loop then the code afterward executes properly, I have removed the while loop and placed it in the second 'search' method. However it still causes a freeze of the system. This makes no sense as there shouldn't be an infinite loop as if I manually query the Redis db, all 5 results are there.
This is my code: (I have a few custom classes for making and processing the API calls, fyi these methods work flawlessly)
//this caches the individual api results to a Redis list
public static function cacheAsyncApiSearch(string $searchQuery, int $maxResults = 20)
{
$key = "search:".$searchQuery; //for Redis
if(!Redis::client()->exists($key)) {
for ($i = 0; $i < 5; $i++) {
// Create a child process
$pid = pcntl_fork();
if ($pid == -1) {
// Fork failed
exit(1);
} elseif ($pid) {
// This is the parent process
// I have tried many versions of pcntl_wait, none work! They all still don't allow code to be ran afterwards (even within this elseif block), and the best it does is cache the 1st api case (YouTube)
// while (!pcntl_wait($status, WNOHANG)) {
// $exitStatus = pcntl_wexitstatus($status);
// // Do something with the exit status of the child process
// }
// dd($pid);
// pcntl_waitpid($pid, $status, WUNTRACED);
} else {
//child processes
switch ($i) {
case 0:
$results = YouTube::search($searchQuery, $maxResults)['results'];
Redis::client()->rPush($key,SearchResultDTO::jsonEncodeArray($results));
SearchResultDTO::convertResultDTOToModels($results);
break;
case 1:
$results = Dailymotion::search($searchQuery, $maxResults)['results'];
Redis::client()->rPush($key,SearchResultDTO::jsonEncodeArray($results));
SearchResultDTO::convertResultDTOToModels($results);
break;
case 2:
$results = Vimeo::search($searchQuery, $maxResults)['results'];
Redis::client()->rPush($key,SearchResultDTO::jsonEncodeArray($results));
SearchResultDTO::convertResultDTOToModels($results);
break;
case 3:
$results = Twitch::search($searchQuery, 2)['results'];
Redis::client()->rPush($key,SearchResultDTO::jsonEncodeArray($results));
SearchResultDTO::convertResultDTOToModels($results);
break;
case 4:
$results = Podcasts::getPodcastsFromItunesResults(Podcasts::search($searchQuery, 2)["response"]->results);
Redis::client()->rPush($key,SearchResultDTO::jsonEncodeArray($results));
SearchResultDTO::convertResultDTOToModels($results);
break;
}
$i = 10000;
exit(0);
}
}
// for noting the process id of the given process that gets to this point
Redis::client()->lPush("search_pid:".$searchQuery, $pid);
// sets a time out for the redis cache
Redis::client()->expire($key, 60*60*4);
while (is_numeric( Redis::client()->lLen($key)) && Redis::client()->lLen($key) < 5) {
usleep(500000); // 0.5 seconds
// pcntl_waitpid(-1, $status); //does this even do anything? not for me
}
return false; // not already cached
}
return true; // already cached
}
This code somewhat works, It performs the api calls and caches the Redis perfectly. However when the method is ran, no code will be ran after it (unless redis has found a cached version and the process is not forked).
This made me think that all processes are being exited (possibly true? if so i dont know why), so I tried writing a version without the exit(0) line. This works, I can then perform code after the method call, however I noticed (when getting SQL race conditions) that all 6 (5 child, 1 parent) processes continued to run their own version of the code after this method (e.g. some database writes)
public static function search(string $searchQuery, int $maxResults = 20): array
{
$key = "search:".$searchQuery;
$results = [];
// the quoted method above
self::cacheAsyncApiSearch($searchQuery, $maxResults);
foreach (Redis::client()->lRange($key,0,-1) as $result){
$results = array_merge($results, SearchResultDTO::jsonDecodeArray($result));
}
$creatorDTOs = [];
$videoDTOs = [];
$streamDTOs = [];
$playlistDTOs = [];
$podcastDTOs = [];
/** #var SearchResultDTO $result */
foreach ($results as $result) {
match ($result->kind) {
Kind::Creator => $creatorDTOs[] = $result,
Kind::Video => $videoDTOs[] = $result,
Kind::Stream => $streamDTOs[] = $result,
Kind::Playlist => $playlistDTOs[] = $result,
Kind::Podcast => $podcastDTOs[] = $result,
};
}
// did this to test how many times the code was being ran (the list has 6 1's in it)
Redis::client()->lPush("here", '1');
// I know this code isn't completely efficient since I already called these conversion methods before, however I am just trying to get the forking stuff to work right now.
return [
"creators" => SearchResultDTO::convertResultDTOToModels($creatorDTOs),
"videos" => SearchResultDTO::convertResultDTOToModels($videoDTOs),
"streams" => SearchResultDTO::convertResultDTOToModels($streamDTOs),
"playlists" => SearchResultDTO::convertResultDTOToModels($playlistDTOs),
"podcasts" => SearchResultDTO::convertResultDTOToModels($podcastDTOs)
];
}
These DTO's (Data Transfer Objects) are being used to populate a UI. So for example, when I make a search (that isn't cached), the page is blank forever. But if I refresh the page (after the search is cached) then the results show just fine.
This is the most bizarre problem I have ever ran into and I really appreciate any help.
Edit please read:
After some messing around I have found that if I remove the while loop then the code afterward executes properly, I have removed the while loop and placed it in the second 'search' method. However it still causes a freeze of the system. This makes no sense as there shouldn't be an infinite loop as if I manually query the Redis db, all 5 results are there. And the dd("two") can never be excecated unless the usleep() is removed. Hopefully this narrows the problem down.
Edit 2 please read:
I have figured out that I can get the dd("two") to work when usleep() is reduced to 0.05s from 0.5 seconds, but it still doesnt seem to run long enough for it to work.
if(!self::cacheAsyncApiSearch($searchQuery, $maxResults))
{
// make sure Redis is properly returning a number not object
$len = Redis::client()->lLen($key);
while(!is_numeric($len)){
usleep(500000); // 0.5 seconds
$len = Redis::client()->lLen($key);
}
//dd($len); //this dd() works
while ($len < 5) {
dd("one"); // this dd() works
usleep(500000); // 0.5 seconds
dd("two"); **//$this does not work, why?**
$len = Redis::client()->lLen($key);
}
}

My ELSE condition refuses to run in IF/ELSE Laravel

I have a basic if/else statement is Laravel.
$model = MyModel::where('row', 'abc123')->first();
// Returns 3 statuses - connected, inactive, disconnected
$status = getStatus();
if ($status === 'connected') {
if (!$model->is_active) {
$model->timed_at = now();
}
$model->is_active = true;
}
else {
if ($model->is_active) {
$model->last_timed_at = $model->updated_at;
}
$model->is_active = false;
}
$model->save();
NOTE: This function is called repeatedly at an interval of 10 seconds.
The ELSE functionality works flawlessly outside the ELSE statement.
But the moment it's in the ELSE, it doesn't run at all.
I have proven that it does reach the ELSE by adding an echo which works fine.
Another odd thing, when I place the ELSE in the IF side, it works perfectly fine.
Thanks everyone for the input.
Turns out the data I was tweaking during my testing gave me unexpected (they were actually expected) results.
Not sure how to fully close a question, but leaving this here to say it was a non-issue.

Why aren't my PHP error messages working correctly?

I'm creating a basic form to purchase adult & child tickets from. With the way the form is set up, the user must purchase an adult ticket, but they don't need to purchase a child ticket. I've added some error messages/validations to enforce rules within the form. All of the adult ticket error messages work correctly, but the child ones aren't working right.
I want the child ticket rules to check the following: that a valid number (aka not a letter) has been entered, the quantity is greater than 0, and a whole number has been entered. I thought I had the rules set so they only start validating if the child ticket input is not empty, but instead they're still trying to validate when it is empty, which I don't want it to do considering no child tickets need to be purchased. How do I get this to work correctly?
Here is my PHP code with the error messages.
<?php
$adult=$_POST['adult'];
$child=$_POST['child'];
$date=date('m/d/Y');
function isInteger($input) {
return(ctype_digit(strval($input)));
}
if (empty($adult)) {
$error_message='You must purchase at least 1 Adult ticket!';
}
else if (!is_numeric($adult)) {
$error_message="You must enter a valid number!";
}
else if ($adult <= 0) {
$error_message="You must enter a quantity greater than zero!";
}
else if (!isInteger($adult)) {
$error_message="You must enter a whole number for the quantity! (i.e. 1, 2, etc...)";
}
else if (!empty(!is_numeric($child))) {
$error_message="You must enter a valid number!";
}
else if (!empty($child <= 0)) {
$error_message="You must enter a quantity greater than zero!";
}
else if (!empty(!isInteger($child))) {
$error_message="You must enter a whole number for the quantity! (i.e. 1, 2, etc...)";
}
else if ($adult + $child > 5) {
$error_message="Sorry, there is a limit of 5 total tickets per customer!";
}
else {
$error_message='';
}
if($error_message !=""){
include('index.php');
exit();
}
?>
If $child = 1
if(!empty($child <= 0) ) is equivalent to if(!empty(false)) which makes no sense.
Same with (!empty(!is_numeric($child)))
Use if(isset($child) && $child <= 0) {} instead
You can also use $child = isset($_POST['child']) ? $_POST['child'] : 0
Not looking for an accepted answer, just wanted to suggest a more practical approach.
PHP has some very powerful built-in functions for validation. Two you can use are filter_var and filter_input, which can easily validate numeric values as integers, without needing to check each variation of an integer.
Additionally instead of chaining several elseif conditions, I suggest using throw to immediately raise an Exception that needs to be handled, otherwise halting execution at that point. Accompanied within a try/catch block to handle the exception as desired.
Example https://3v4l.org/PJfLv
$adult = filter_var($_POST['adult'] ?? null, FILTER_VALIDATE_INT);
$child = filter_var($_POST['child'] ?? null, FILTER_VALIDATE_INT);
try {
if (false === $adult || false === $child) {
throw new \InvalidArgumentException('You must enter a whole number (1, 2, etc...)');
}
if (0 >= $child || 0 >= $adult) {
throw new \InvalidArgumentException('You must enter a quantity greater than zero!');
}
// at this point child and adult must be >= 1
// ensure total is at least 1 and at most 5
$total_options = ['options' => ['min_range' => 1, 'max_range' => 5]];
$total = filter_var($adult + $child, FILTER_VALIDATE_INT, $total_options);
if (false === $total) {
throw new \InvalidArgumentException('Sorry, there is a limit of 5 total tickets per customer!');
}
// what should happen next...
} catch(\InvalidArgumentException $e) {
$error_message = $e->getMessage();
require __DIR__ . '/index.php';
## always use absolute paths to prevent include_path overrides/overhead
} finally {
// do something else no regardless of success or Exception...
}
For PHP < 7.0 replace
$_POST['child']) ?? null
with
isset($_POST['child']) ? $_POST['child'] : null

Any performance benefits of using apc_store vs apc_add (or vice versa)?

While I understand the differences between apc_store and apc_add, I was wondering if using one or the other has any performance benefits?
One would think apc_store COULD be a bit quicker since it does not need to do a check-if-exists before doing the insert.
Am I correct in my thinking?
Or would using apc_add in situations where we know FOR SURE that the entry does not exist prove to be a bit faster?
Short: apc_store() should be slightly slower than apc_add().
Longer: the only difference between the two is the exclusive flag passed to apc_store_helper() that in turns leads to the behavior difference in apc_cache_insert().
Here is what happens there:
if (((*slot)->key.h == key.h) && (!memcmp((*slot)->key.str, key.str, key.len))) {
if(exclusive) {
if (!(*slot)->value->ttl || (time_t) ((*slot)->ctime + (*slot)->value->ttl) >= t) {
goto nothing;
}
}
// THIS IS THE MAIN DIFFERENCE
apc_cache_remove_slot(cache, slot TSRMLS_CC);**
break;
} else
if((cache->ttl && (time_t)(*slot)->atime < (t - (time_t)cache->ttl)) ||
((*slot)->value->ttl && (time_t) ((*slot)->ctime + (*slot)->value->ttl) < t)) {
apc_cache_remove_slot(cache, slot TSRMLS_CC);
continue;
}
slot = &(*slot)->next;
}
if ((*slot = make_slot(cache, &key, value, *slot, t TSRMLS_CC)) != NULL) {
value->mem_size = ctxt->pool->size;
cache->header->mem_size += ctxt->pool->size;
cache->header->nentries++;
cache->header->ninserts++;
} else {
goto nothing;
}
The main difference is that apc_add() saves one slot removal if the value is already present. Real world benchmarks would obviously make a lot of sense to confirm that analysis.

in php i need one line if condition for time compare

i have to value
$mo=strtotime($input_array['MondayOpen']);
$mc=strtotime($input_array['MondayClose']);
now i need a if condition to display an error on below conditions
if one of them($mo or $mc) are empty, null or blank.
if close time($mc) is less than open time($mo)
means if both are empty(null) or $mc>$mo then go further
please suggest optimized one line if condition for this
i know it seems very basic question, but i m facing problem when both are null
either i was using simple
if(($mo==NULL && $mc!=NULL) || ( $mo>=$mc && ($mo!=NULL && $mc!=NULL)) )
Keep in mind that 0, null, and blank all mean completely different things here. As indicated previously, strtotime will never return NULL. However, 0 is a valid unix timestamp, whereas false means that the strtotime function was unable to process the value provided.
Also, you've requested that a single-line solution; however, in my opinion, it is much better in this case to write out each condition and display a different error message for each condition. That way, the user knows what actually went wrong. Perhaps this is a better way:
// Only check for errors if we have at least one value set
if (!empty($input['MondayOpen']) || !empty($input['MondayClosed']) {
$mo = strtotime($input['MondayOpen']);
$mc = strtotime($input['MondayClosed']);
$invalid = false;
if (false === $mo) {
echo "Invalid Opening Time\n";
$invalid = true;
}
if (false === $mc) {
echo "Invalid Closing Time\n";
$invalid = true;
}
if (!$invalid && $mc <= $mo) {
echo "Closing time must be After Opening Time\n";
$invalid = true;
}
if ($invalid) {
exit(); // Or handle errors more gracefully
}
}
// Do something useful
All right. How about this.
It checks whether $mo and $mc are valid dates using is_numeric. Any NULL or false values will be caught by that.
I haven't tested it but it should work.
I spread it into a huge block of code. In the beginning, when learning the language, this is the best way to make sense out of the code. It is not the most elegant, nor by far the shortest solution. Later, you can shorten it by removing whitespace, or by introducing or and stuff.
I'm not 100% sure about the number comparison part, and I don't have the time to check it right now. You'll have to try out whether it works.
You need to decide how you want to handle errors and insert the code to where my comments are. A simple echo might already do.
// If $mo or $mc are false, show error.
// Else, proceed to checking whether $mo is larger
// than $mc.
if ((!is_numeric($mo)) and (is_numeric($mc)))
{
// Error: $mo is either NULL, or false, or something else, but not a number.
// While $mc IS a number.
}
elseif ((!is_numeric($mc)) and (is_numeric($mo)))
{
// Error: $mc is either NULL, or false, or something else, but not a number.
// While $mo IS a number.
}
else
{
if (($mc <= $mo) and ((is_numeric($mc) or (is_numeric($mo)))))
{
// Error: closing time is before opening time.
}
else
{
// Success!!
}
}
in php, strotime will return a integer or false. Checking for null in this case will never bear fruit, but otherwise...
if((!$mo xor !$mc) || ($mc && $mc<=$mo)){
print('error');
}else{
print('no_error');
}
oops, edited for correctness. I transposed $mc and $mo. XOR should be correct though.
You can try:
print ((empty($mo) && empty($mc)) || ($mc > $mo)) ? 'case_true_message' : 'case_false_message';
But you should also check the manual :) - for basic control structures

Categories