I use yii2. And I need to find IP which is not used (is not in database) by method getFreeIPAddress. I have class like this:
class Radreply extends ActiveRecord {
const ATTRIBUTE_DEFAULT_IP_ADDRESS = 'Framed-IP-Address';
const IP_ADDRESS_MAX = '10.255.255.255'; // max value for IP
const IP_ADDRESS_MIN = '10.0.0.11'; // min value for IP
public function getIntegerIP(){ // converts IP from string to integer format
return ip2long($this->value);
}
public static function getFreeIPAddress(){
$records = self::findAll(['attribute'=>self::ATTRIBUTE_DEFAULT_IP_ADDRESS]); // get all record which contain IP address
$existIPs = ArrayHelper::getColumn($records,'integerIP'); // get array of IP which is converted to integer by method getIntegerIP
for ($integerIP = ip2long(self::IP_ADDRESS_MIN); $integerIP<=ip2long(self::IP_ADDRESS_MAX); $integerIP++){
// increasing one by one IP address in integer format from value IP_ADDRESS_MIN to value IP_ADDRESS_MAX
if (!in_array($integerIP, $existIPs)){
$stringIP = long2ip($integerIP);
$arrayDigits = explode('.', $stringIP);
$lastDigit = array_pop($arrayDigits);
if ($lastDigit!='0'){ // check if last digit of IP is not 0
return $stringIP;
}
}
}
return '';
}
}
Method getFreeIPAddress works find, but in db there are a lot of records with IP and increasing one by one IP and checking if this IP exist in db is very long way. How I can optimize this algorithm? Is there faster way to get unused IP?
I think, I've found better solution without extra table in database
class Radreply extends ActiveRecord {
const ATTRIBUTE_DEFAULT_IP_ADDRESS = 'Framed-IP-Address';
const IP_ADDRESS_MAX = '10.255.255.255'; // max value for IP
const IP_ADDRESS_MIN = '10.0.0.11'; // min value for IP
public function getIntegerIP(){ // converts IP from string to integer format
return ip2long($this->value);
}
public static function getFreeIPAddress(){
$records = self::findAll(['attribute'=>self::ATTRIBUTE_DEFAULT_IP_ADDRESS]); // gets all record which contain IP address
$existIPs = ArrayHelper::getColumn($records,'integerIP'); // gets array of IP which is converted to integer by method getIntegerIP
$intIpAddressMin = ip2long(self::IP_ADDRESS_MIN); // gets min IP in integer format
$endRange = empty($existIPs) ? $intIpAddressMin : max($existIPs); // checks if at least one IP is used
$availableIPs = range( $intIpAddressMin, $endRange + 2); // generates array with available IP addresses (+2 because next address can be with last digit 0)
$missingIPs = array_diff($availableIPs,$existIPs); // removes all used IP
foreach ($missingIPs as $value){
$lastDigit = $value % 256;
if ($lastDigit != 0){
return long2ip($value);
}
}
return '';
}
}
bool in_array ( mixed $needle , array $haystack [, bool $strict = FALSE ] )
In my opinion,you can set strict true .
my php code with strict = false
<?php
$y="1800";
$x = array();
for($j=0;$j<50000;$j++){
$x[]= "{$j}";
}
for($i=0;$i<30000;$i++){
if(in_array($y,$x)){
continue;
}
}
time php test.php
real 0m4.418s
user 0m4.404s
sys 0m0.012s
when strict is true
for($i=0;$i<30000;$i++){
if(in_array($y,$x ,true)){
continue;
}
}
time php test.php
real 0m1.548s
user 0m1.540s
sys 0m0.004s
what‘s more ,if you can get the used ip with ascending order . you can get the o(m+n) time complexity,which m is the length of all ip you should try , n is the length of all ip in db with merge algorithm .
if you can get the used ip with ascending order
in Pseudocode .
tmpIp = minIp;
while(temIp <= maxIp){
if( dbIsEmpty){
break;
}
dbIp =getNextFromDb();
while(temIp < dbIp){
printf temIp ;
temIp ++;
}
temIp ++;
}
while(temIp <= maxIp){
printf temIp ;
temIp++;
}
here is my php code, where i repalce echo ip by $count++;
In this demo there is about 80000 ip with type of long
<?php
function mergeSort( $result){
$minIp = ip2long('10.0.0.11') ;
$maxIp = ip2long('10.255.255.255');
$count =0;
$tmpIp = $minIp;
while($temIp <= $maxIp){
if( empty($result)){
break ;
}
$tmp = array_pop($result);
$dbIp =$tmp['ip'];
while($temIp < $dbIp){
// echo temIp ;
// i repalce it by count ++ , i don't want it
//full my teminal .
$count ++;
$temIp ++;
}
$temIp ++;
}
while($temIp <= $maxIp){
//echo $temIp ; replace by $count++
$count ++;
$temIp++;
}
return $count -1;
}
$servername = "localhost";
$username = "root";
$password = "aaaaa";
$dbname = "IP";
$conn = new PDO('mysql:host=' . $servername . ';dbname=' . $dbname , $username, $password);
$conn->setAttribute(PDO::ATTR_AUTOCOMMIT , true);
$stmt = $conn->prepare("select * from ipTable order by ip desc");
$stmt->execute();
$result = $stmt->fetchAll();
$count = mergeSort($result);
echo $count ;
?>
it take about 10s ;
time php test.php
184460881
real 0m10.626s
user 0m10.416s
sys 0m0.168s
Related
I made this code with:
https://github.com/influxdata/influxdb-php
https://www.highcharts.com/docs/working-with-data/live-data
<?php
include('/opt/lampp/htdocs/example/vendor/autoload.php');
$host = '127.0.0.1';
$port = 8086;
$dbname = 'aTimeSeries';
/*
$client = new \InfluxDB\Client($host, $port);
$database = $client->selectDB('aTimeSeries');
*/
//the exact same thing
$database = \InfluxDB\Client::fromDSN(sprintf('influxdb://user:pass#%s:%s/%s', $host, $port, $dbname));
//query of the last value
$result = $database->query('select * from valeurs group by * order by desc limit 1');
//recup le point
$points = $result->getPoints();
// Set the JSON header
header("Content-type: application/json");
// The x value is the current JavaScript time, which is the Unix time multiplied by 1000.
$x = time() * 1000;
// The y value is a random number
//$y = rand(0,100);
$y = $points;
// Create a PHP array and echo it as JSON
$ret = array($x, $y);
echo json_encode($ret);
?>
Output:
[1523603506000,[{"time":"2018-04-13T07:11:45.208943754Z","value":48}]]
But I would like this:
[1523603506000,48]
Or this:
[2018-04-13T07:11:45.208943754Z,48]
If I try to output $points only, I got only the last part of the array, but it is not what I would like either.
PS: if you have a better solution to do the same thing, maybe with nodeJS, I will surely listen
Thanks for your help,
I had another problem, but I solved it.
But still have 2 hours of lag, but it display right in highcharts now.
<?php
include('/opt/lampp/htdocs/example/vendor/autoload.php');
$host = '127.0.0.1';
$port = 8086;
$dbname = 'aTimeSeries';
//directly get the database object
$database = \InfluxDB\Client::fromDSN(sprintf('influxdb://user:pass#%s:%s/%s', $host, $port, $dbname));
//query of the last value
$result = $database->query('select * from valeurs group by * order by desc limit 1');
//get the point
$points = $result->getPoints();
//make the array right
$points = json_encode($points);
$points = json_decode($points);
//take only the seconds of the time
$points[0]->time = substr($points[0]->time, 0,20);
// Set the JSON header
header("Content-type: text/json");
// The x value is the current JavaScript time, which is the Unix time multiplied by 1000 and take the time
$x = strtotime($points[0]->time) * 1000;
// The y value take the value which is a random value
$y = $points[0]->value;
// Create a PHP array and echo it as JSON
$ret = array($x, $y);
echo json_encode($ret);
?>
I want to resolve weighted an Adapter from an factory which could be configured by user (enable/disable and weight %).
Example:
AdapterW ≃ 20% of transaction
AdapterX ≃ 30% of transaction
AdapterY ≃ 40% of transaction
AdapterZ ≃ 10% of transaction
I can grant that all items will never sum more than one hundred (100%), but sometimes any adapter could be deactivated.
I have the following parameters:
public function handleAdapter()
{
$isWActive = (boolean)$this->_config[self::W];
$isXActive = (boolean)$this->_config[self::X];
$isYActive = (boolean)$this->_config[self::Y];
$isZActive = (boolean)$this->_config[self::Z];
$WPercentage = (int)$this->_config[self::LOAD_BALANCE_W];
$XPercentage = (int)$this->_config[self::LOAD_BALANCE_X];
$YPercentage = (int)$this->_config[self::LOAD_BALANCE_Y];
$ZPercentage = (int)$this->_config[self::LOAD_BALANCE_Z];
.
.
.
return (self::W | self::X | self::Y | self::Z);
}
How can i balance weighted between this adapters dynamically?
Edit
created a gist to a executable code: https://gist.github.com/markomafs/5d892d06d6670909f9b4
This may not be the best approach, but you can try something like this:
public function handleAdapter()
{
//an array to return the balanced entries
$balancedEntries[] = false;
//verifies which of the options are active
$isWActive = (boolean)$this->_config[self::W];
$isXActive = (boolean)$this->_config[self::X];
$isYActive = (boolean)$this->_config[self::Y];
$isZActive = (boolean)$this->_config[self::Z];
//get configured percentage of each
$WPercentage = (int)$this->_config[self::LOAD_BALANCE_W];
$XPercentage = (int)$this->_config[self::LOAD_BALANCE_X];
$YPercentage = (int)$this->_config[self::LOAD_BALANCE_Y];
$ZPercentage = (int)$this->_config[self::LOAD_BALANCE_Z];
//here you fill the array according to the proportion defined by the percentages
if ($isWActive) {
for ($i = 0; $i < $WPercentage; $i++) {
$balancedEntries[] = self::W;
}
}
if ($isXActive) {
for ($i = 0; $i < $XPercentage; $i++) {
$balancedEntries[] = self::X;
}
}
if ($isYActive) {
for ($i = 0; $i < $YPercentage; $i++) {
$balancedEntries[] = self::Y;
}
}
if ($isZActive) {
for ($i = 0; $i < $ZPercentage; $i++) {
$balancedEntries[] = self::Z;
}
}
return $balancedEntries;
}
And then, in case you want a proportion of 1 to 100 (as in percentages):
$balancedResult = $balancedEntries[array_rand($balancedEntries, 1)];
Since array_rand will return 1 key from the original array, you use it to get it's value.
Another try, this should work for your case - But it only work if you have an adapter as a single char string, this is not visible by your question.
public function handleAdapter()
{
# a map with all adapters
$map = array(
self::W => self::LOAD_BALANCE_W,
self::X => self::LOAD_BALANCE_X,
self::Y => self::LOAD_BALANCE_Y,
self::Z => self::LOAD_BALANCE_Z
);
# generate a string map with one char per percentage point
$stringMap = "";
foreach($map as $key => $value){
# skip if disabled
if(!$this->_config[$key]) continue;
# repeat the key for each percentage point
$stringMap .= str_repeat($key, (int)$this->_config[$value]);
}
# return a random string char from the map
return $stringMap[rand(0, strlen($stringMap) - 1)];
}
Edit: I've misunderstood the question, the answer is wrong.
I understand your question so that you always want to return the adapter with the lowest load to force traffic to this adapter.
public function handleAdapter()
{
$isWActive = (boolean)$this->_config[self::W];
$isXActive = (boolean)$this->_config[self::X];
$isYActive = (boolean)$this->_config[self::Y];
$isZActive = (boolean)$this->_config[self::Z];
$WPercentage = (int)$this->_config[self::LOAD_BALANCE_W];
$XPercentage = (int)$this->_config[self::LOAD_BALANCE_X];
$YPercentage = (int)$this->_config[self::LOAD_BALANCE_Y];
$ZPercentage = (int)$this->_config[self::LOAD_BALANCE_Z];
$map = array();
if($isWActive) $map[self::W] = $WPercentage;
if($isXActive) $map[self::X] = $XPercentage;
if($isYActive) $map[self::Y] = $YPercentage;
if($isZActive) $map[self::Z] = $ZPercentage;
asort($map);
return key($map);
}
Edit: Fixed wrong sort(), you need asort() to maintain the index.
I'm working on a class whose purpose is to restrict users to making only 10 requests within any 30 second period. It utilizes a file to maintain IP addresses, last request time. and the number of tries they've made. The problem is that, no matter what I try, I can't get the filesize. I've tried using clearstatcache(), and I've tried using a function I found in the comments on the filesize() page of the PHP manual.
Here's the code, in it's current debugging state.
// Makes sure user can only try to generate a coupon x number of times over x amount of seconds
class IpChecker{
const WAIT_TIME = 30; //seconds until user can try again
const MAX_TRIES = 10; // maximum tries
const COUPON_IP = 0;
const COUPON_TIME = 1;
const COUPON_TRIES = 2;
private $ip_data;
private $path;
private $fh;
private $safe;
public function __construct(){
clearstatcache();
$this->path = realpath(dirname(__FILE__))."/ips/.ips";
$this->fh = fopen($this->path,'w+');
$this->filesize = $this->realfilesize($this->fh);
echo "fs: ".$this->filesize; exit;
$this->getIPs();
$this->checkIP();
$this->logRequest();
fclose($this->fh);
$this->safe || die(json_encode("You have exhausted all available tries. Please try again later."));
}
private function logRequest(){
$str = "";
foreach($this->ip_data as $data){
foreach($data as $col){
if(self::WAIT_TIME < (time() - $col[self::COUPON_TIME])) $str .= $col."\t";
}
$str = rtrim($str, '\t');
$str .= "\n";
}
$str = rtrim($str, '\n');
try{
$fw = fwrite($this->fh, $str) || die(json_encode("Unable to check IP"));
}catch(Exception $e){
die(json_encode($e));
}
}
private function checkIP(){
$IP = $_SERVER['REMOTE_ADDR'];
$TIME = time();
$safe = true;
$user_logged = false;
echo "<pre>"; var_dump($this->ip_data); exit;
foreach($this->ip_data as $key=>$data){
echo "<prE>"; var_dump($data); exit;
// if($data[$key][self::COUPON_IP] == $IP){
// $user_logged = true;
// if(
// (($TIME - $data[$key][self::COUPON_TIME]) < self::WAIT_TIME) ||
// (self::MAX_TRIES >= $data[$key][self::COUPON_TRIES])
// ) $safe = false;
// $this->ip_data[$key][self::COUPON_TRIES] = $this->ip_data[$key][self::COUPON_TRIES]+1;
// $this->ip_data[$key][self::COUPON_TIME] = $TIME;
// }
}
if(!$user_logged){
die("user not logged");
$this->ip_data[] = array(
self::COUPON_IP => $IP,
self::COUPON_TIME => $TIME,
self::COUPON_TRIES => 1
);
}
$this->safe = $safe;
}
private function getIPs(){
$IP_DATA = array();
echo file_get_contents($this->path); exit;
// this always returns 0.
$size = filesize($this->path);
echo "filesize: ".$size; exit;
if($size){
$IPs = fread($this->fh,$size);
$IP_ARR = explode("\n",$IPs);
foreach($IP_ARR as $line) $IP_DATA[] = explode("\t",$line);
}
$this->ip_data = $IP_DATA;
}
// Copied from the comments in the PHP Manual for filesize()
public function realfilesize($fp) {
$return = false;
if (is_resource($fp)) {
if (PHP_INT_SIZE < 8) {
// 32bit
if (0 === fseek($fp, 0, SEEK_END)) {
$return = 0.0;
$step = 0x7FFFFFFF;
while ($step > 0) {
if (0 === fseek($fp, - $step, SEEK_CUR)) {
$return += floatval($step);
} else {
$step >>= 1;
}
}
}
} elseif (0 === fseek($fp, 0, SEEK_END)) {
// 64bit
$return = ftell($fp);
}
}
return $return;
}
}
How can I get the real filesize? I'm on PHP 5.2.
I wasn;t able to figure out what was wrong with my code, if anythign, but I worked around it like this:
/**
* Makes sure user can only access a given page
* x number of times over x amount of seconds.
* If multiple instances of this class are used, the $namespace
* properties for each should be unique.
* Default wait time is 90 seconds while default request limit
* is 10 tries.
*
* -+ Usage +-
* Just create the object with any parameters, no need to assign,
* just make sure it's initialized at the top of the page:
* new RequestThrottler;
*
* -+- Parameters -+-
* null RequestThrottler ( [ string $namespace [, int $WaitTime [, int $MaxTries ] ] ] )
*/
class RequestThrottler{
// settings
private static $WAIT_TIME; // seconds until count expires
private static $MAX_TRIES; // maximum tries
// used to keep session variables unique
// in the event that this class is used in multiple places.
private $namespace;
// for debugging
const DBG = false;
// array index constants
const _TIME = 0;
const _TRIES = 1;
// defines whether or not access is permitted
private $safe;
// seconds until reset
private $secs;
/**
* -+- Constructor -+-
* #param String $namespace - A unique prefix for SESSION data
* #param Int $WaitTime - Total seconds before user can try again
* #param Int $MaxTries - Total tries user can make until their request is denied
*/
public function __construct($namespace='Throttler', $WaitTime=90, $MaxTries=10){
// make sure a session is available
if(!headers_sent() && !isset($_SESSION)) session_start();
if(!isset($_SESSION)) die(json_encode("No session available"));
// save settings
$this->namespace = $namespace;
self::$MAX_TRIES = $MaxTries;
self::$WAIT_TIME = $WaitTime;
// do the footwork
$this->checkHistory();
// if set to debug mode, print a short helpful string
if(self::DBG) die(json_encode(
"You are ".($this->safe ? 'SAFE' : 'NOT SAFE')."! "
. "This is try number {$_SESSION[$this->namespace.'_ATTEMPTS'][self::_TRIES]} of ".self::$MAX_TRIES.". "
. $this->secs." seconds since last attempt. "
. (self::$WAIT_TIME - $this->secs)." seconds until reset."
));
// if not safe, kill the script, show a message
$this->safe || die(json_encode("You're going too fast. Please try again in ".(self::$WAIT_TIME - $this->secs)." seconds."));
}
/**
* -+- checkHistory -+-
* Does the footwork to determine whether
* or not to throttle the current user/request.
*/
private function checkHistory(){
$TIME = time();
$safe = true;
// make sure session is iniitialized
if( !isset($_SESSION[$this->namespace.'_ATTEMPTS']) ||
!isset($_SESSION[$this->namespace.'_ATTEMPTS'][self::_TRIES]) ||
!isset($_SESSION[$this->namespace.'_ATTEMPTS'][self::_TIME]) )
$_SESSION[$this->namespace.'_ATTEMPTS'] = array(
self::_TIME =>$TIME,
self::_TRIES => 1
);
else $_SESSION[$this->namespace.'_ATTEMPTS'][self::_TRIES] =
$_SESSION[$this->namespace.'_ATTEMPTS'][self::_TRIES]+1;
// get seconds since last attempt
$secondSinceLastAttempt = $TIME - $_SESSION[$this->namespace.'_ATTEMPTS'][self::_TIME];
// reset the counter if the wait time has expired
if($secondSinceLastAttempt > self::$WAIT_TIME){
$_SESSION[$this->namespace.'_ATTEMPTS'][self::_TIME] = $TIME;
$_SESSION[$this->namespace.'_ATTEMPTS'][self::_TRIES] = 1;
$secondSinceLastAttempt = 0;
}
// finally, determine if we're safe
if($_SESSION[$this->namespace.'_ATTEMPTS'][self::_TRIES] >= self::$MAX_TRIES) $safe=false;
// log this for debugging
$this->secs = $secondSinceLastAttempt;
// save the "safe" flag
$this->safe = $safe;
}
}
I'm developping a php application which is used to process IP adresses. Thus, I'm juggling with mysql tables containing up to 4 billion rows.
I have a script that currently needs to fetch 65536 adresses from this table and the mysql query fails to give a response via PHP or even via phpMyAdmin when I try to extract these 65K lines.
The table containing the IP Adresses has 3 indexes ( 1 unique, 2 primary ) which are supposed to help it go faster but I simply cannot get past having mysql give an associative array back to PHP in order to continue my data processing.
Any tips as to how to circumvent this problem ?
Thx in advance !
$request = new Request(DB_NAME);
$request->select = '*';
$request->from = Etherwan_Adressage_Ip::TABLE_NAME;
$request->where = Etherwan_Adressage_Ip_Ressource::PRIMARY_KEY." = '".$options->Ressource_ID."'";
$request->order = " inet_aton(IP) ";
$Compteur = 0;
$Liste = array();
$result = $request->exec('select');
for ($i=0 ; $i<$result['length'] ; $i+=$CIDR->Adresses_Totales){
$Possible = TRUE;
$Selected = FALSE;
for ($j=$i ; $j<$i+$CIDR->Adresses_Totales ; $j++){
if ($result[$j]['Date_Affectation'] != '0000-00-00 00:00:00'){
if (isset($options->Include)){
if ($options->Include->Type != $result[$j]['Type_Liaison'] || $options->Include->Liaison_ID != $result[$j]['Liaison_ID'] || str_replace('/', '', $options->CIDR) != $result[$j]['Bits']){
$Possible = FALSE;
} else {
$Selected = TRUE;
}
} else {
$Possible = FALSE;
}
break;
}
}
if ($Possible){
$Liste[$Compteur]['text'] = $result[$i]['IP'] . " / " . $result[$i+$CIDR->Adresses_Totales-1]['IP'];
$Liste[$Compteur]['value'] = $result[$i]['Ressource_ID'];
$Liste[$Compteur]['selected'] = $Selected;
$Liste_IP = array();
for ($j=$i ; $j<$i+$CIDR->Adresses_Totales ; $j++){
if ($result[$j]['Nom'] != ''){
$result[$j]['Dispo'] = 0;
} else {
$result[$j]['Dispo'] = 1;
}
$Liste_IP[] = $result[$j];
}
$Liste[$Compteur]['Liste_IP'] = $Liste_IP;
$Compteur++;
}
}
$Liste['maxLength'] = $Liste['length'] = $Compteur;
return $Liste;
Link to table indexes ( JPG )
Link to data sample ( JPG )
I have about 1000 IP address. I calculate IP ranges for them using the bellow code:
public function ipRange($mainIp, $mask)
{
$mainIpLong = ip2long($mainIp);
$maskLong = ip2long($mask);
$netid = long2ip($mainIpLong & $maskLong);
$broadcast = long2ip($mainIpLong | ~$maskLong);
//here I insert $netid and $broadcast to a MySQL table
//so I have abount 1000 records
}
It calculates IP range correctly, for example if I call like this:
ipRange('91.99.98.243', '255.255.255.240');
Result will be:
$netid -> 91.99.98.240
$broadcast -> 91.99.98.255
Now I need to have a search function. It should find the sub-range for the given IP address, so if I call search('91.99.98.249'), the search() function should show the record that netid is 91.99.98.240 and broadcast field is 91.99.98.255.
How can I do that?
function find($ip){
//get $netid and $bcast from db one by one
//loop through all records
if(ip2long($ip)>=ip2long($netid) && ip2long($ip)<=ip2long($bcast) ){
echo $netid;
echo $bcast;
}
}
I solved it.
Note: _getConnection() function connects to database.
ip_address fields includes: IP, netid and broadcast.
public function search($ip)
{
$ranges = $this->_getConnection()->fetchAll("SELECT netid, broadcast FROM ip_address");
foreach($ranges as $range) {
$min = ip2long($range['netid']);
$max = ip2long($range['broadcast']);
if(ip2long($ip) > $min && ip2long($ip) < $max) {
$result = $this->_getConnection()->fetchAll("SELECT * FROM ip_address WHERE netid = ? AND broadcast = ?", array($range['netid'], $range['broadcast']));
}
}
return $result;
}