I have PHP array which I use to draw a graph
Json format:
{"y":24.1,"x":"2017-12-04 11:21:25"},
{"y":24.1,"x":"2017-12-04 11:32:25"},
{"y":24.3,"x":"2017-12-04 11:33:30"},
{"y":24.1,"x":"2017-12-04 11:34:25"},
{"y":24.2,"x":"2017-12-04 11:35:35"},.........
{"y":26.2,"x":"2017-12-04 11:36:35"}, ->goes up for about a minute
{"y":26.3,"x":"2017-12-04 11:37:35"},.........
{"y":24.1,"x":"2017-12-04 11:38:25"},
{"y":24.3,"x":"2017-12-04 11:39:30"}
y=is temperature and x value is date time,
as you can see temperature doesn't change so often even if, it change only for max 0.4. But sometimes after a long period of similar values it change for more than 0.4.
I would like to join those similar values, so graph would not have 200k of similar values but only those that are "important".
I would need an advice, how to make or which algorithm would be perfect to create optimized array like i would like.
perfect output:
{"y":24.1,"x":"2017-12-04 11:21:25"},.........
{"y":24.1,"x":"2017-12-04 11:34:25"},
{"y":24.2,"x":"2017-12-04 11:35:35"},.........
{"y":26.2,"x":"2017-12-04 11:36:35"}, ->goes up for about a minute
{"y":26.3,"x":"2017-12-04 11:37:35"},.........
{"y":24.1,"x":"2017-12-04 11:38:25"}
Any help?
As you specified php I'm going to assume you can handle this on the output side.
Basically, you want logic like "if the absolute value of the temperature exceeds the last temperature by so much, or the time is greater than the last time by x minutes, then let's output a point on the graph". If that's the case you can get the result by the following:
$temps = array(); //your data in the question
$temp = 0;
$time = 0;
$time_max = 120; //two minutes
$temp_important = .4; //max you'll tolerate
$output = [];
foreach($temps as $point){
if(strtotime($point['x']) - $time > $time_max || abs($point['y'] - $temp) >= $temp_important){
// add it to output
$output[] = $point;
}
//update our data points
if(strtotime($point['x']) - $time > $time_max){
$time = strtotime($point['x']);
}
if(abs($point['y'] - $temp) >= $temp_important){
$temp = $point['y'];
}
}
// and out we go..
echo json_encode($output);
Hmm, that's not exactly what you're asking for, as if the temp spiked in a short time and then went down immediately, you'd need to change your logic - but think of it in terms of requirements.
If you're RECEIVING data on the output side I'd write something in javascript to store these points in/out and use the same logic. You might need to buffer 2-3 points to make your decision. Your logic here is performing an important task so you'd want to encapsulate it and make sure you could specify the parameters easily.
Related
I need to calculate the difference, "for example in minutes", of a series of intervals, however there is no limit, because there may be N time records, between "tipo_2_pause" and "tipo_3_return".
Excluding "tipo_1_start" and "tipo_4_end", both "tipo_2_pause" and "tipo_3_return" is an array with N records.
As an example, I need to calculate the difference between the "tipo_4_end" and the "tipo_1_start".... ok... this is easy... but I'm not able to find a way in which I can calculate always considering the respective pairs between "tipo_2_pause" and "tipo_3_return".
In the example, the first pair between "tipo_2_pause" and "tipo_3_return", I hope to get the difference... which in this case is equal to 10min....in the second pair, by chance, it could also be 10min... but it could be any time .. but if there is no pair, the result would be 0.
Objective:
Being able to calculate the difference in minutes between each item in the array "tipo_2_pause" that pairs with the array "tipo_3_return".
You're right that the key to getting pairs is to use the length of the shorter array.
Then you just apply a for-loop.
Somewhat verbose (for clarity) first pass:
$numPairs = min(count($tipo2), count($tipo3));
$duration = 0;
for($i=0; $i<$numPairs; $i++){
$start = $tipo2[$i];
$end = $tipo3[$i];
$duration += minutes_between($start, $end); // implementing minutes_between is left as an exercise.
}
You have a function that always inputs an interval (natural numbers in this case), this function returns a result, but is quite expensive on the processor, simulated by sleep in this example:
function calculate($start, $end) {
$result = 0;
for($x=$start;$x<=$end;$x++) {
$result++;
usleep(250000);
}
return $result;
}
In order to be more efficient there is an array of old results, that contains the interval used an the result of the function for that interval:
$oldResults = [
['s'=>1, 'e'=>2, 'r' => 1],
['s'=>2, 'e'=>6, 'r' => 4],
['s'=>4, 'e'=>7, 'r' => 3]
];
If I call calculate(1,10) the function should be able to calculate new intervals based on old results and accumulate them, In this particular case it should take the old result from 1 to 2 add that to the old result from 2 to 6 and do a new calculate(6,10) and add that too. Take in consideration that the function ignores the old saved interval from 4 to 7 since it was more convenient to use 2-6.
This is a visual representation of the problem:
Of course in this example, calculate() is quite simple and you can just find particular ways to solve this problem around it, but in the real code calculate() is complex and the only thing I know is that calculate(n0,n3)==calculate(n0,n1)+calculate(n1,n2)+calculate(n2,n3).
I cannot find a way to solve the reuse of the old data without using a bunch of IF and foreach, I'm sure there is a more elegant approach to solve this.
You can play with the code here.
Note: I'm using PHP but I can read JS, Pyton, C and similar languages.
if you are certain that calculate(n0,n3)==calculate(n0,n1)+calculate(n1,n2)+calculate(n2,n3), then it seems to me that one approach might simply be to establish a database cache.
you can pre-calculate each discrete interval, and store its result in a record.
$start = 0;
$end = 1000;
for($i=1;$i<=$end;$i++) {
$result = calculate($start, $i);
$sql = "INSERT INTO calculated_cache (start, end, result) VALUES ($start,$i,$result)";
// execute statement via whatever dbms api
$start++;
}
now whenever new requests come in, a database lookup should be significantly faster. note you may need to tinker with my boundary cases in this rough example.
function fetch_calculated_cache($start, $end) {
$sql = "
SELECT SUM(result)
FROM calculated_cache
WHERE (start BETWEEN $start AND $end)
AND (end BETWEEN $start AND $end)
";
$result = // whatever dbms api you chose
return $result;
}
there are a couple obvious considerations such as:
cache invalidation. how often will the results of your calculate function change? you'll need to repopulate the database then.
how many intervals do you want to store? in my example, I arbitrarily picked 1000
will you ever need to retrieve non-sequential interval results? you'll need to apply the above procedure in chunks.
i wrote this:
function findFittingFromCache($from, $to, $cache){
//length for measuring usefulnes of chunk from cache (now 0.1 means 10% percent of total length)
$totalLength = abs($to - $from);
$candidates = array_filter($cache, function($val) use ($from, $to, $totalLength){
$chunkLength = abs($val['e'] - $val['s']);
if($from <= $val['s'] && $to >= $val['e'] && ($chunkLength/$totalLength > 0.1)){
return true;
}
return false;
});
//sorting to have non-decremental values of $x['s']
usort($candidates, function($a, $b){ return $a['s'] - $b['s']; });
$flowCheck = $from;
$needToCompute = array();
foreach($candidates as $key => $val){
if($val['s'] < $flowCheck){
//already using something with this interval
unset($candidates[$key]);
} else {
if($val['s'] > $flowCheck){
//save what will be needed to compute
$needToCompute[] = array('s'=>$flowCheck, 'e'=>$val['s']);
}
//increase starting position for next loop
$flowCheck = $val['e'];
}
}
//rest needs to be computed as well
if($flowCheck < $to){
$needToCompute[] = array('s'=>$flowCheck, 'e'=>$to);
}
return array("computed"=>$candidates, "missing"=>$needToCompute);
}
It is function which returns you two arrays, one "computed" holds found already computed pieces, second "missing" holds gaps between them which must be computed yet.
inside function there is 0.1 threshold, which disqualifies chunks shorter than 10% of total searched length, you can rewrite function to send threshold as parameter, or ommit it completely.
i presume results will be stored and after computing added into cache ($oldResults), which might be of any form (for example database as Jeff Puckett suggested). Do not forget to add all computed chunks and whole seeked length into cache.
I am sorry but i can't find a way without cycles and ifs
Working demo:
link
I have a piece of code which will select x number of days into the future and print it out.
I'm then trying to select data from those timestamps, and print out accordingly:
Below I am selecting the number of days in the future it should loop ($max) and how many rows/data there is ($data["cnt"])
$stmt=$dbh->prepare("select round((expire - unix_timestamp()) / 86400) as days, count(*) as cnt from xeon_users_rented WHERE user_by=:username group by days;");
$stmt->bindParam(":username",$userdata['username']);
$stmt->execute();
$data=$stmt->fetchAll();
$max = max(array_map(function($d){
return $d['days'];
}, $data));
$expireData = array();
I then loop through x number of days in the future and print it out: (Let's say that $x is = 10)
for($x = 0; $x <= $max; $x++){
if ($data[$x]["cnt"] > 0){
$expireData[] = $data[$x]["cnt"];
}else{
$expireData[] = 0;
}
$stamp = strtotime('+'.$x.' day', time());
$expireDays[] = date("Y/m/d", $stamp);
}
I then print out the days data and the data:
<?php echo implode("', '", $expireDays); ?>
<?php echo implode("', '", $expireData); ?>
Which gives me:
'2014/11/05', '2014/11/06', '2014/11/07', '2014/11/08', '2014/11/09', '2014/11/10', '2014/11/11', '2014/11/12', '2014/11/13', '2014/11/14', '2014/11/15'
and (where each digit represent a value for that specific date)
2,8,0,0,0,0,0,0,0,0
So far so good. The only problem here is, that the data (2,8,0,0etc.) is not correct. It should for example be:
0,0,2,0,0,0,8,0,0,0
My question is: How can I print out the data, where it matches the timestamp (xeon_users_rented.expire)?
To simplify my answer from before and to answer your question directly "How can I print out the data, where it matches the timestamp"?
You need to first put the unix timestamp of "strtotime('+'.$x.' day', time());" into an array from your original loop. Remove the expiredays[] stuff from that loop.
Then loop through that array and then use array_search for finding any matching indexes in the $data array.
if (found in $data array)
$expireDays[] = $data[array_search position]['cnt'];
else
$expireDays[] = 0;
From what I have gathered in what you are trying to establish, the sql query (for example) returns an array such as:
$data = array(
array("days"=>232975857, "cnt"=> 4),
array("days"=>232975867, "cnt"=> 10),
array("days"=>232976689, "cnt"=> 0),
array("days"=>232976688, "cnt"=> 2)
);
The max in your case is 10. However, please note that your code (below):
$max = max(array_map(function($d){
return $d['days'];
}, $data));
could return a lot of PHP E_NOTICE errors and be slow because you are working out a maximum from the unix_timestamp at that stage which is for example 232975867 (far too many loops I suspect that you need). The max should be worked out in the following way I suspect:
$max = count($data);
In my case (from my data example example) this will return something like 4 for which your for loop code will need to reference "<" not "<=". To optimise this I would actually put straight into the for loop "...; $x < count($data); ...;" unless of course you need the $max later.
Here is a big issue for me. I don't see where currently you have any correlation between the $stamp variable and the "days" column from your sql statement. Perhaps I have not seen enough information from you to fully understand or I am interpreting your question incorrectly but your sql for one will not necessarily return the same dates as the stamp variable will calculate and will not certainly return a cnt of 0 for any dates that do not exist in that table. This is why:
if ($data[$x]["cnt"] > 0){
part of the section is unnecessary and possibly incorrect.
To answer your question why do you get "2,8,0,0,0,0...." instead of the order you expected is because the sql does not return 0 cnt values as quite simply the date does not exist in it's table and your implode still returns '0's appended as you forcibly added the 0's afterwords with the line:
$expireData[] = 0;
First of all, you need to fill your data (or re-write your data to contain cnt of '0's in the correct array places). You can achieve this from the sql level with a subquery that ensures missing dates are contained and then a cnt value of 0 is enforced. I'll leave this to you to work out however another way (via PHP) is to do the following (pseudo code):
place each 'stamps' (in unix format) in array from previous loop
forloop $i through $stamps
array_search $stamps against $data['days']
if (found)
$expireDays[] = $data[array_search position]['cnt'];
else
$expireDays[] = 0;
This means that you remove the $expireDays from your first loop.
Finally, perhaps I have misunderstood and that your 'days' column shouldn't match your $stamp dates as those dates are in the future and your 'days' columns are in the past. In this case, your only option is to adjust your sql statement to forcibly include dates that are missing (between a certain date range)
Good luck.
I am implementing a system at the moment where it needs to allocate a number in a certain range to a person, but not use any number that has been used before.
Keep in mind, both the number range and exclusion list are both going to be quite large.
Initially, I thought doing something like this would be best:
<?php
$start = 1;
$end = 199999;
$excluded = array(4,6,7,8,9,34);
$found = FALSE;
while (!$found) {
$rand = mt_rand($start,$end);
if (!in_array($rand,$excluded)) {
$found = TRUE;
}
}
?>
But I don't think this is ideal, there is the possibility of an infinite loop (or it taking a very long time / timing out the script).
I also thought about generating an array of all the numbers I needed, but surely a massive array would be worse? Also doing an array diff on 2 massive arrays would surely take a long time too?
Something like this:
<?php
$start = 1;
$end = 199999;
$allnums = range($start,$end);
$excluded = array(4,6,7,8,9,34);
$searcharray = array_diff($allnums,$excluded);
$rand = array_rand($searcharray);
?>
So, my question would be which would be a better option? And is there another (better) way of doing this that someone has used before?
Array's holding large amounts of data will use up a lot of memory, can you not use a database to hold these numbers in? That's generally what they are designed for.
rand(1,N) but excluding array(a,b,c,..),
is there already a built-in function that I don't know or do I have to implement it myself(how?) ?
UPDATE
The qualified solution should have gold performance whether the size of the excluded array is big or not.
No built-in function, but you could do this:
function randWithout($from, $to, array $exceptions) {
sort($exceptions); // lets us use break; in the foreach reliably
$number = rand($from, $to - count($exceptions)); // or mt_rand()
foreach ($exceptions as $exception) {
if ($number >= $exception) {
$number++; // make up for the gap
} else /*if ($number < $exception)*/ {
break;
}
}
return $number;
}
That's off the top of my head, so it could use polishing - but at least you can't end up in an infinite-loop scenario, even hypothetically.
Note: The function breaks if $exceptions exhausts your range - e.g. calling randWithout(1, 2, array(1,2)) or randWithout(1, 2, array(0,1,2,3)) will not yield anything sensible (obviously), but in that case, the returned number will be outside the $from-$to range, so it's easy to catch.
If $exceptions is guaranteed to be sorted already, sort($exceptions); can be removed.
Eye-candy: Somewhat minimalistic visualisation of the algorithm.
I don't think there's such a function built-in ; you'll probably have to code it yourself.
To code this, you have two solutions :
Use a loop, to call rand() or mt_rand() until it returns a correct value
which means calling rand() several times, in the worst case
but this should work OK if N is big, and you don't have many forbidden values.
Build an array that contains only legal values
And use array_rand to pick one value from it
which will work fine if N is small
Depending on exactly what you need, and why, this approach might be an interesting alternative.
$numbers = array_diff(range(1, N), array(a, b, c));
// Either (not a real answer, but could be useful, depending on your circumstances)
shuffle($numbers); // $numbers is now a randomly-sorted array containing all the numbers that interest you
// Or:
$x = $numbers[array_rand($numbers)]; // $x is now a random number selected from the set of numbers you're interested in
So, if you don't need to generate the set of potential numbers each time, but are generating the set once and then picking a bunch of random number from the same set, this could be a good way to go.
The simplest way...
<?php
function rand_except($min, $max, $excepting = array()) {
$num = mt_rand($min, $max);
return in_array($num, $excepting) ? rand_except($min, $max, $excepting) : $num;
}
?>
What you need to do is calculate an array of skipped locations so you can pick a random position in a continuous array of length M = N - #of exceptions and easily map it back to the original array with holes. This will require time and space equal to the skipped array. I don't know php from a hole in the ground so forgive the textual semi-psudo code example.
Make a new array Offset[] the same length as the Exceptions array.
in Offset[i] store the first index in the imagined non-holey array that would have skipped i elements in the original array.
Now to pick a random element. Select a random number, r, in 0..M the number of remaining elements.
Find i such that Offset[i] <= r < Offest[i+i] this is easy with a binary search
Return r + i
Now, that is just a sketch you will need to deal with the ends of the arrays and if things are indexed form 0 or 1 and all that jazz. If you are clever you can actually compute the Offset array on the fly from the original, it is a bit less clear that way though.
Maybe its too late for answer, but I found this piece of code somewhere in my mind when trying to get random data from Database based on random ID excluding some number.
$excludedData = array(); // This is your excluded number
$maxVal = $this->db->count_all_results("game_pertanyaan"); // Get the maximum number based on my database
$randomNum = rand(1, $maxVal); // Make first initiation, I think you can put this directly in the while > in_array paramater, seems working as well, it's up to you
while (in_array($randomNum, $excludedData)) {
$randomNum = rand(1, $maxVal);
}
$randomNum; //Your random number excluding some number you choose
This is the fastest & best performance way to do it :
$all = range($Min,$Max);
$diff = array_diff($all,$Exclude);
shuffle($diff );
$data = array_slice($diff,0,$quantity);