I have a PHP API front end running on a webserver. This specific PHP program is subject to distribution, thus it should be as portable as possible.
The feature I want to implement is an IP cooldown period, meaning that the same IP can only request the API a maximum of two times per second, meaning at least a 500ms delay.
The approach I had in mind is storing the IP in an MySQL database, along with the latest request timestamp. I get the IP by:
if (getenv('REMOTE_ADDR'))
$ipaddress = getenv('REMOTE_ADDR');
But some servers might not have a MySQL database or the user installling this has no access. Another issue is the cleanup of the database.
Is there a more portable way of temporarily storing the IPs (keeping IPv6 in mind)?
and
How can I provide an automatic cleanup of IPs that are older than 500ms, with the least possible performance impact?
Also: I have no interest at looking at stored IPs, it is just about the delay.
This is how I solved it for now, using a file.
Procedure
Get client IP and hash it (to prevent file readout).
Open IP file and scan each line
Compare the time of the current record to the current time
If difference is greater than set timeout goto 5., else 7.
If IP matches client, create updated record, else
drop record.
If IP matches client, provide failure message, else copy record.
Example code
<?php
$sIPHash = md5($_SERVER[REMOTE_ADDR]);
$iSecDelay = 10;
$sPath = "bucket.cache";
$bReqAllow = false;
$iWait = -1;
$sContent = "";
if ($nFileHandle = fopen($sPath, "c+")) {
flock($nFileHandle, LOCK_EX);
$iCurLine = 0;
while (($sCurLine = fgets($nFileHandle, 4096)) !== FALSE) {
$iCurLine++;
$bIsIPRec = strpos($sCurLine, $sIPHash);
$iLastReq = strtok($sCurLine, '|');
// this record expired anyway:
if ( (time() - $iLastReq) > $iSecDelay ) {
// is it also our IP?
if ($bIsIPRec !== FALSE) {
$sContent .= time()."|".$sIPHash.PHP_EOL;
$bReqAllow = true;
}
} else {
if ($bIsIPRec !== FALSE) $iWait = ($iSecDelay-(time()-$iLastReq));
$sContent .= $sCurLine.PHP_EOL;
}
}
}
if ($iWait == -1 && $bReqAllow == false) {
// no record yet, create one
$sContent .= time()."|".$sIPHash.PHP_EOL;
echo "Request from new user successful!";
} elseif ($bReqAllow == true) {
echo "Request from old user successful!";
} else {
echo "Request failed! Wait " . $iWait . " seconds!";
}
ftruncate($nFileHandle, 0);
rewind($nFileHandle);
fwrite($nFileHandle, $sContent);
flock($nFileHandle, LOCK_UN);
fclose($nFileHandle);
?>
Remarks
New users
If the IP hash doesn't match any record, a new record is created. Attention: Access might fail if you do not have rights to do that.
Memory
If you expect much traffic, switch to a database solution like this all together.
Redundant code
"But minxomat", you might say, "now each client loops through the whole file!". Yes, indeed, and that is how I want it for my solution. This way, every client is responsible for the cleanup of the whole file. Even so, the performance impact is held low, because if every client is cleaning, file size will be kept at the absolute minimum. Change this, if this way doesn't work for you.
Related
I am making a Covid-19 statistics website - https://e-server24.eu/ . Every time somebody is entering the website, the PHP script is decoding JSON from 3 urls and storing data into some variables.
I want to make my website more optimized so my question is: Is there any script that can update the variables data one time per day, not every time someone accesses the website?
Thanks,
I suggest looking into memory object caching.
Many high-performance PHP web apps use caching extensions (e.g. Memcached, APCu, WinCache), accelerators (e.g. APC, varnish) and caching DBs like Redis. The setup can be a bit involved but you can get started with a simple role-your-own solution (inspired by this):
<?php
function cache_set($key, $val) {
$val = var_export($val, true);
// HHVM fails at __set_state, so just use object cast for now
$val = str_replace('stdClass::__set_state', '(object)', $val);
// Write to temp file first to ensure atomicity
$tmp = sys_get_temp_dir()."/$key." . uniqid('', true) . '.tmp';
file_put_contents($tmp, '<?php $val = ' . $val . ';', LOCK_EX);
rename($tmp, sys_get_temp_dir()."/$key");
}
function cache_get($key) {
//echo sys_get_temp_dir()."/$key";
#include sys_get_temp_dir()."/$key";
return isset($val) ? $val : false;
}
$ttl_hours = 24;
$now = new DateTime();
// Get results from cache if possible. Otherwise, retrieve it.
$data = cache_get('my_key');
$last_change = cache_get('my_key_last_mod');
if ($data === false || $last_change === false || $now->diff($last_change)->h >= $ttl_hours ) { // cached? h: Number of hours.
// expensive call to get the actual data; we simple create an object to demonstrate the concept
$myObj = new stdClass();
$myObj->name = "John";
$myObj->age = 30;
$myObj->city = "New York";
$data = json_encode($myObj);
// Add to user cache
cache_set('my_key', $data);
$last_change = new DateTime(); //now
// Add timestamp to user cache
cache_set('my_key_last_mod', $last_change);
}
echo $data;
Voila.
Furthermore; you could look into client-side caching and many other things. But this should give you an idea.
PS: Most memory cache systems allow to define a time-to-live (TTL) which makes this more concise. But I wanted to keep this example dependency-free. Cache cleaning was omitted here. Simply delete the temp file.
Simple way to do that
Create a script which will fetch , decode JSON data and store it to your database.
Then set a Cron jobs with time laps of 24 hours .
And when user visit your site fetch the data from your database instead of your api provider.
It works fine but later sometime the count just goes down to random
number. My guess is my code cannot process multiple visits at a time.
Where increment heppens
Where it displays the count
<?php
$args_loveteam = array('child_of' => 474);
$loveteam_children = get_categories($args_loveteam);
if(in_category('loveteams', $post->ID)){
foreach ($loveteam_children as $loveteam_child) {
$post_slug = $loveteam_child->slug;
echo "<script>console.log('".$post_slug."');</script>";
if(in_category($loveteam_child->name)){
/* counter */
// opens file to read saved hit number
if($loveteam_child->slug == "loveteam-mayward"){
$datei = fopen($_SERVER['DOCUMENT_ROOT']."/wp-content/themes/inside-showbiz-Vfeb13.ph-updated/countlog-".$post_slug."-2.txt","r");
}else{
$datei = fopen($_SERVER['DOCUMENT_ROOT']."/wp-content/themes/inside-showbiz-Vfeb13.ph-updated/countlog-".$post_slug.".txt","r");
}
$count = fgets($datei,1000);
fclose($datei);
$count=$count + 1 ;
// opens file to change new hit number
if($loveteam_child->slug == "loveteam-mayward"){
$datei = fopen($_SERVER['DOCUMENT_ROOT']."/wp-content/themes/inside-showbiz-Vfeb13.ph-updated/countlog-".$post_slug."-2.txt","w");
}else{
$datei = fopen($_SERVER['DOCUMENT_ROOT']."/wp-content/themes/inside-showbiz-Vfeb13.ph-updated/countlog-".$post_slug.".txt","w");
}
fwrite($datei, $count);
fclose($datei);
}
}
}
?>
I would at least change your code to this
foreach ($loveteam_children as $loveteam_child) {
$post_slug = $loveteam_child->slug;
echo "<script>console.log('".$post_slug."');</script>";
if($loveteam_child->slug == "loveteam-mayward"){
$filename = "{$_SERVER['DOCUMENT_ROOT']}/wp-content/themes/inside-showbiz-Vfeb13.ph-updated/countlog-{$post_slug}.txt";
}else{
$filename = "{$_SERVER['DOCUMENT_ROOT']}/wp-content/themes/inside-showbiz-Vfeb13.ph-updated/countlog-{$post_slug}-2.txt";
}
$count = file_get_contents($filename);
file_get_contents($filename, ++$count, LOCK_EX);
}
You could also try flock on the file to get a lock before modifying it. That way if another process comes along it has to wait on the first one. But file_put_contents works great for things like logging where you may have many processes competing for the same file.
Database should be ok, but even that may not be fast enough. It shouldn't mess up your data though.
Anyway hope it helps. This is kind of an odd question, concurrency can be a real pain if you have a high chance of process collisions and race conditions etc etc.
However as I mentioned (in the comments) using the filesystem is probably not going to provide the consistency you need. Probably the best for this may be some kind of in memory storage such as Redis. But that is hard to say without full knowing what you use it for. For example if it should persist on server reboot.
Hope it helps, good luck.
After almost 3 days of troubleshooting I gotta ask for advice.
A have a small imageboard with 4 images and 4 'like' buttons. Earlier I made it so the number of clicks with each button stores in a .txt file. Now I basically need to make it so a person can press a certain button only once.
This is ip.txt. The number to the left is button ID, to the right is IP of the person that clicked that button.
click-001||127.0.0.1
click-002||
This is very simple. I need to make sure it stores ip when I click on my PC, then stores another IP when I click on my pad - and stops whatever I do next. Now for the last few days it's been doing anything except that!
My current code with isset. That sees the first IP but doesn't add the second:
$file2 = 'ip.txt'; // path to text file that stores counts
$fh2 = fopen($file2, 'r+');
$ip_addr = $_SERVER['REMOTE_ADDR'];
$lines2 = '';
while(!feof($fh2)) {
$line2 = trim(fgets($fh2));
if ($line2) {
$line2 = explode('||', $line2);
if(isset($line2[0], $line2[1])) {
$item2 = trim($line2[0]);
if(!empty($item2)) {
if($item2 == $id) {
if(empty($line2[1])) {
$lines2 .= "$item2||$ip_addr\r\n";
file_put_contents($file2, $lines2);
} else {
// this is where it always fails
if (!isset($ip_addr)) { $ip_all = $line2[1] . " " . $ip_addr;
$lines2 .= "$item2||$ip_all\r\n";
file_put_contents($file2, $lines2);
} else {
echo "lul";
}
}
}
}
}
}
}
fclose($fh2);
I also tried this with in_array function:
$ip_all = array($line2[1]);
if (!in_array($ip_addr, $ip_all)) {
array_push($ip_all, ',' , $ip_addr);
$ip_fin = implode($ip_all);
$lines2 .= "$item2||$ip_fin\r\n";
file_put_contents($file2, $lines2);
^ This one also sees the first IP and adds the second, but then fails to find whether the IP is already there and just keeps adding copies when I click.
This is brutal. What am I doing wrong and is there an easier way?
Use MySQL database to accomplish this.
Using a text file is super inefficient and can cause conflicts when multiple users liked at the same time!
Insert the IP to database everytime a user clicked the 'Like' button and then use a select query to determine if this IP has liked the picture before.
I do not recommend using just IP tho as some ISP gives dynamic IP that changes the IP (Public IP) address every few seconds.
Use cookies to store a unique cookie for a user (if they are not logged in) or just ask the user to login first before voting!
Information about MySQL Insert and Select are everywhere on Google.
Here's one : https://www.w3schools.com/sql/
best method: use a database or use XML (XML have very useful library)
text file method:
get file and edit...
$id = "...";
$ip="...";
$file2 = file('ip.txt');
$file2 = array_map(
function($current_line) use ($id,$ip) {
$current_line = explode('||', $current_line);
if($current_line[0] == $id){
$current_line[]=$ip;
}
return join("||",$current_line);
},$file2
);
file_put_contents('ip.txt', implode('\n', $file2));
I am writing a PHP script which, upon a request, will make a call to a SOAP service with various parameters, some of which are taken from the request.
However, the particular SOAP service I am using requires that each request includes a unique ID, which in this case needs to increment for each request. It must not be based on time, and must be unique for each request, however it does not matter if values are skipped.
Using a MySQL data base to store a single value seems massively overkill. I have thought about storing and loading it into a file, but the issue of race conditions springs to mind.
I do have complete access to the server, which will be some kind of Linux flavour dedicated to this task.
Is there a simple way this can be achieved?
Before any new request get incremental value using PHP's time() function, since time will be unique for each request.
$increment_id = time();
If your application is single server you can try to store incremental ID in APC using:
$key = 'soap_service_name';
if (!apc_exists($key)) {
apc_store($key, 0);
}
$id = apc_inc($key);
You need to check if a key exists in APC cache and set 0, otherwise apc_inc fails and returns false
If you have multiserver application you can store incremental id in Memcache/Redis (that needs to run additional service):
$key = 'soap_service_name';
$memcache = memcache_connect('memcache_host', 11211);
if (!empty(memcache_exists($memcache, $key))) {
memcache_set($memcache, 0);
}
$id = memcache_increment($memcache, $key);
Same situation as APC if you call memcache_increment it will fail if key doesn't exists yet.
If that incremental ID should be stored persistently Redis would be more usefull because it has disk write of all data. It's kind of Memcache with disk write.
This is how I achieved this in the end. After considering the various options, databases and the various caching options seemed a bit overkill. In addition, caching, cookies and sessions seem to be designed to be relatively temporary, whereas I was really looking for a non-volatile solution.
This is what I came up with - a simple file locking solution. I hadn't realised PHP could deal with file locks but on discovering this, it seems the best way to go.
This example acquires an exclusive lock on the file, before reading and updating the value. If it hits max int, it resets. Then it waits for 5 seconds. If the script is called a few times in quick succession, observe that each request will wait for the lock to be release from the previous before continuing.
What's nice is, as this is PHP, non-existent file, invalid contents etc, will just cause the value to default to zero.
<?php
$f = fopen('sequence_num.txt', 'r+');
echo "Acquiring lock<br />\n";
flock($f, LOCK_EX);
echo "Lock acquired, updating value<br />\n";
$num = intval(fread($f, strlen(PHP_INT_MAX)));
echo "Old val = " . $num;
if ($num >= PHP_INT_MAX) {
$num = 0;
} else {
$num++;
}
echo " New val = " . $num;
echo "<br />Waiting 5 seconds<br />\n";
rewind($f);
ftruncate($f, 0);
fwrite($f, $num);
sleep(5);
echo "Releasing lock<br />\n";
flock($f, LOCK_UN);
fclose($f);
If you're happy to use a float as a unique value use:
$unique_id = microtime(true);
If you wish to simply increment, you may do so using a session var:
/**
* Get session increment.
*
* #param string $id
* #param int $default
* #return int
*/
function get_increment($id, $default = 0)
{
if (array_key_exists($id, $_SESSION)) $_SESSION[$id] += 1;
else $_SESSION[$id] = $default;
return $_SESSION[$id];
}
var_dump(get_increment('unique_id'));
Hello there so I just setup this basic poll, I inspired myself from something I found out there, and it's just a basic ajax poll that waves the results in a text file.
Although I was wondering, since I do not want the user to simply mass-click to advantage / disadvantage the results, i thought about adding a new text file that could simply save the IP, one on each line, and then checks if it's already logged, if yes, display the results, if not, show the poll.
My lines of code to save the result are:
<?php
$vote = $_REQUEST['vote'];
$filename = "votes.txt";
$content = file($filename);
$array = explode("-", $content[0]);
$yes = $array[0];
$no = $array[1];
if ($vote == 0)
{
$yes = $yes + 1;
}
if ($vote == 1)
{
$no = $no + 1;
}
$insert = $yes."-".$no;
$fp = fopen($filename,"w");
fputs($fp,$insert);
fclose($fp);
?>
So I'd like to know how I could check out the IPs, in the same way it does basically.
And I'm not interested in database, even for security measures, I'm alright with what Ive got.
Thanks to any help!
To stop multiple votes, I'd set a cookie once a user has voted. If the user reloads the page with the voting form on it and has a cookie, you could show just the results, or a "You have already voted." message. Note that this will not stop craftier people from double-voting - all they would have to do is remove the saved cookie, and they could re-vote.
Keep in mind though that IPs can be shared so your idea of storing IPs might backfire - people on a shared external-facing IP won't be able to vote, as your system will have registered a previous vote from someone at the same IP address.
easiest way is to write data to file is
file_put_contents($filename, $data)
and to read data from file
file_get_contents($filename);
To get IP Address of the user
$_SERVER['REMOTE_ADDR']
See php manual for file_put_contents for more information and file_get_contents
Here is sample code
<?php
// File path
$file = 'votedips.txt';
// Get User's IP Address
$ip = $_SERVER['REMOTE_ADDR'];
// Get data from file (if it exists) or initialize to empty string
$votedIps = file_exists($file) ? file_get_contents($file) : '';
//
$ips = explode("\n", $votedIps);
if (array_search($ip, $ips)) {
// USER VOTED
} else {
$ips[] = $ip;
}
// Write data to file
$data = implode("\n", $ips);
file_put_contents($file, $data);
?>
You can use file_get_contents to save the file's content into a variable and then use the strpos function to check if the IP exists in that variable.
For example:
$ipfile=file_get_contents('ip.txt');
if (strpos($ipfile, $_SERVER['REMOTE_ADDR'])!==FALSE) // show the results
else // show the poll
Be careful with storing IPs in a text file, and then using file_get_contents() and similar functions for loading the data/parseing. As an absolute worst case, assuming that every possible IP address used your system to vote, you'd end up with a text file in the many many gigabytes in size, and you'd exceed PHP's memory_limit very quickly.