I have a 1D array (XYData), e.g.
$TE = array(
"1"=>"20",
"2"=>"30",
"5"=>"50",
"10"=>"90"
)
I would like to create a memory effective PHP function which do the Linear interpolation of the passed X value and return the corresponding Y value. e.g.
calling function interpolate($TE,9.5)
then it should return 86
Is there any way to avoid the array search as the XYData set may be very long, say more then 100 points.
Thank you in advance!
No, you cannot avoid looking at your array. To make it more efficient you have to restructure your data. Do this by recursively looking for the middle, and then split it at that point into two parts. For your short example you would get this:
$TER = array("2 and lower" => array("1" => "20",
"2" => "30"),
"5 and higher" => array("5" => "50",
"8" => "100"));
No recursion is shown, and it really doesn't make any sense for such a small set of data, but when the dataset becomes large there's a clear advantage. It's basically a simple binary search tree.
But I have my doubts implementing it would be useful in this case. I'm not going to work it all out, you really should have 100.000 items or more to make this useful. If not, then just work through the array.
Related
I have a very complex array that I need to loop through.
Array(
[1] => Array(
[1] => ""
[2] => Array(
[1] => ""
[2] => Array(
[1] => ""
)
)
)
)
I can't use nested loops because this array could contain hundreds of nested arrays. Also, the nested ones could contain nested arrays too.
This array presents comments and replies, Where replies could contain more replies.
Any thoughts?
You could use a \RecursiveArrayIterator, which is part of the PHP SPL, shipped non-optional, with the PHP core.
<?php
$arr = [
'lvl1-A' => [
'lvl2' => [
'lvl3' => 'done'
],
],
'lvl1-B' => 'done',
];
function traverse( \Traversable $it ): void {
while ( $it->valid() ) {
$it->hasChildren()
? print "{$it->key()} => \n" and traverse( $it->getChildren() )
: print "{$it->key()} => {$it->current()}\n";
$it->next();
}
}
$it = new \RecursiveArrayIterator( $arr );
$it->rewind();
traverse( $it );
print 'Done.';
Run and play this example in the REPL here: https://3v4l.org/cGtoi
The code is just meant to verbosely explain what you can expect to see. The Iterator walks each level. How you actually code it is up to you. Keep in mind that filtering or flattening the array (read: transforming it up front) might be another option. You could as well use a generator and emit each level and maybe go with Cooperative Multitasking/ Coroutines as PHP core maintainer nikic explained in his blog post.
ProTip: Monitor your RAM consumption with different variants in case your nested Array really is large and maybe requested often or should deliver results fast.
In case you really need to be fast, consider streaming the result, so you can process the output while you are still working on processing the input array.
A last option might be to split the actual array in chunks (like when you are streaming them), therefore processing smaller parts.
The case is quite complex, as you have to loop, but you can't or don't want to for some reasons:
... that I need to loop through
and
I can't use nested loops because this array could contain hundreds of nested arrays
It means you have to either handle your data differently, as you can pack that huge amount of data to be processed later.
If for some reasons it's not an option, you can consider to:
split somehow this big array into smaller arrays
check how does it work with json_encode and parsing string with str_* functions and regex
Your question contains too many things we can't be sure e.g. what exactly these subarrays contain, can you ignore some parts of them, can you change the code that creates huge array in first place etc.
Assuming on the other hand that you could loop. What could bother you? The memory usage, how long it will take etc.?
You can always use cron to run it daily etc. but the most important is to find the cause why you ended up with huge array in the first place.
I am working on a application that has the following the following:
Each month, an API returns a series of values depending on the data, so something like (These are updated everyday so the results are cached)
$data = array(
array("2016-02-03" => 3", "2016-02-04" => 4", "2016-02-05" => 1"),
array("2016-02-03" => 1", "2016-02-04" => 2", "2016-02-05" => 3"),
array("2016-02-03" => 60", "2016-02-04" => 18", "2016-02-05" => 3"),
);
What I am therefore trying to achieve is that the algorithm will take the first key ("2016-02-03") then iterate through all of the sub arrays and find the values for this key and then sums them up and calculates the average, finally add this to another array. This will continue until there are no more keys left.
The problem is, I could have a huge foreach loop and do it that way, but the problem is, there are over 40 values and all contain around 30 days worth of data so this would be inefficient.
Is there an alternative to solving this problem? One that won't be intensive and slow?
I can only imagine the sulotion is to run the server for as long as it takes. I also suggest that after each date match you add the value to your new array and unset the index to reduce memory and the time needed to loop through everything.
In your new array you can have the syntax:
[ "Year-month-day" => [...] ]
Where the dots will be all the values.
I have a file that contains a two denominational array:
<?php
return [
'key1' => 'value1' ,
'key2' => 'value2' ,
'key3' => 'value3'
];
?>
I need to test for the presence of an array key. I am wondering what is the most efficient way of doing so? What is going to give me the fastest response time? At present this file is 1.2 megs. It is going to grow to the range of 10 megs.
Ron
I think usually built-in functions is more efficient and fast. So, try to use php array_key_exist function in this way:
if (array_key_exists("key",$array))
return true;
Most efficient would be to use some better key-value store than array (SQLite, Redis, etc...). In all other ways you'll end up consuming memory, since array is initiated and memory is consumed in all cases.
If you don't mind consuming memory and are trying to be efficient on CPU side and each array member has value, then probably...
$array = include('myfile.php');
if (isset($array[$mykey])) {
echo "is set\n";
}
... is most efficient.
In the language of Perl, I define a hash as a mapping between one thing and another or an essential list of elements. As stated in the documentation..
A hash is a basic data type. It uses keys to access its contents.
So basically a hash is close to an array. Their initializations even look very similar.
If I were to create a mapping in Perl, I could do something like below for comparing.
my %map = (
A => [qw(a b c d)],
B => [qw(c d f a)],
C => [qw(b d a e)],
);
my #keys = keys %map;
my %matches;
for my $k ( 1 .. #keys ) {
$matches{$_} |= 2**$k for #{$map{ $keys[$k-1] }};
}
for ( sort keys %matches ) {
my #found;
for my $k ( 1 .. #keys ) {
push #found, $keys[$k-1] if $matches{$_} & 2**$k;
}
print "$_ found in ", (#found? join(',', #found) : 0 ), "\n";
}
Output:
a found in A,C,B
b found in A,C
c found in A,B
d found in A,C,B
e found in C
f found in B
I would like to find out the best method of doing this for performance and efficiency in php
If I understand correctly, you are looking to apply your knowledge of Perl hashes to PHP. If I'm correct, then...
In PHP a "Perl hash" is generally called an "associative array", and PHP implements this as an array that happens to have keys as indexes and its values are just like a regular array. Check out the PHP Array docs for lots of examples about how PHP lets you work with arrays of this (and other) types.
The nice thing about PHP is it is very flexible as to how you can deal with arrays. You can define an array as having key-value pairs then treat it like a regular array and ignore the keys, and that works just fine. You can mix and match...it doesn't complain much.
Philosophically, a hash or map is just a way to keep discrete pieces of related information together. That's all most non-primitive data structures are, and PHP is not very opinionated about how you go about things; it has lots of built-in optimizations, and does a pretty solid job of doing these types of things efficiently.
To answer your questions related to your example:
1) As for simplicity (I think you mean) and maintainability, I don't think there's anything wrong with your use of an associative array. If a data set is in pairs, then key-value pairs is a natural way to express this type of data.
2) As for most efficient, as far as lines of code and script execution overhead goes...well, the use of such a mapping is a vanishingly small task for PHP. I don't think any other way of handling it would matter much, PHP can handle it by the thousands without complaint. Now if you could avoid the use of a regular expression, on the other hand...
3) You're using it, really. Don't over think it - in PHP this is just an "array", and that's it. It's a variable that holds an arbitrary amount of elements, and PHP handles multiple-dimensions or associativity pretty darn well. Well enough that it's almost never going to be the cause of any problem you have.
PHP will handle things like hash/maps behind the scenes very logically and efficiently, to the point that part of the whole point of the language is for you not to bother to try to think about such things. If you have relates pieces of data in chunks, use an array; if the pieces of data comes in pairs, use key-value pairs; if it comes by the dozen, use an "array of arrays" (a multidimensional array where some - or all - of it's elements are arrays).
PHP doesn't do anything stupid like create a massive overhead just because you wanted to use key-value pairs, and it has lots of built-in features like foreach $yourArray as $key => $value and the functions you used like array_keys() and array_values(). Feel free to use them - as core features they are generally pretty darn well optimized!
For what you are doing I would rather use sprintf:
$format = 'Hello %s how are you. Hey %s, hi %s!';
printf($format, 'foo', 'bar', 'baz');
I have code that takes a massive string from a SQL database and parses it into individual words and puts them into an array to be counted, with the goal of making a graph of the must used words, but I need to find a means of removing commonly used words. I made a very basic array of words to compare to but it's not very effective. Is their some means of a dictionary file i can compare it to? any ideas would be fantastic.
I am currently editing an existing "Data representation algorithm" at an internship and i really don't know where to start. It has been suggested I use a dictionary file but not only do I not have have one, I wouldn't know how to compare it.
You can do this using the in_array function:
<?php
$whitelist = array('a', 'the');
function whitelisted($var)
{
global $whitelist;
return (!in_array($var, $whitelist));
}
$str = "a lazy fox jumped over the lazy farmer";
print_r(array_count_values(array_filter(explode(" ", $str), "whitelisted")));
?>
//produces:
Array
(
[lazy] => 2
[fox] => 1
[jumped] => 1
[over] => 1
[farmer] => 1
)
Of course, you could and should re-arrange this to work with your own scope (global is probably not ideal), but it should get you started on pruning out common words you don't care to count.
http://ideone.com/kfNzM