I am trying to run the following code in PHP to query a MongoDB:
<?
$m = new Mongo(); // connect
$dogs = $m->dogs;
$races = $dogs->newdogs;
$js = "function() {
return this.location == 'SHEFFIELD'
}";
$dataSet = $races->find(array('$where' => $js));
foreach ($dataSet as $r){
}
?>
When I run this and watch the console, I see the query being run once.
When I change the foreach loop to be nested within another one like this:
foreach(range(1,5) as $test){
foreach ($dataSet as $r){
}
}
I see the query being run 7 times in the console?
Is this something stupid I am doing? A scoping issue? Or am I just misunderstanding how MongoDB is supposed to work?
Thanks
AH
This happens because $dataSet is a MongoCursor, not an array. A MongoCursor is a representation of a query. It will be turned into an array "on-demand", that means that when you use foreach on it, $dataSet is converted into an array by simply querying.
Since you do it within another loop, the MongoCursor is executed every time it encounters the foreach. If you don't want that behaviour, you can use iterator_to_array, since a MongoCursor is just an iterator:
$executed = iterator_to_array($dataSet); // Actual query execution
foreach($executed as $r) { // Iterate the array, not the iterator
// Hic sunt ponies
}
EDIT: Keep in mind that iterator_to_array converts the entire result set into an array in memory. If you have a very big result set, this can cause huge and unnecessary memory consumption. It's advisable to stick with a single foreach call, since it will only load one single row into memory at once.
Related
I'm executing a cursor. I have cutted off the code how the procedure is called and executed. This part is efficient.
At last I have a a not big cursor. I'm calling the procedure, which returns this cursor many times on the page and I need to create a multidimensional array from it. This array should look like like the following:
$ret = oci_execute($outrefc) ;
while ($row = #oci_fetch_array($outrefc))
{
foreach (array_keys($row) as $key)
{
$res[$i][$key] = $row[$key];
}
$i++;
}
Is there any way to make the upper snippet faster?
The multidimensional array should stay as it is. I only wonder if I could create it in any more efficient way.
Thank you!
I've got a foreach statement that on an item that has both objects and arrays in it.
foreach($result as $data)
that contains both arrays and objects. how do i specify the foreach to only select to loop through one or the other? when it loops through them all it takes forever
I had tried foreach($result->data as $data) but then it errors on the arrays telling me it is trying to get property of an object, which is understandable. once I add an if statement to check if the first result is an object it almost triples the script run time since there are so many results.
Well you could just use is_object() and is_array() (both return a boolean):
if (is_object($var)) {
// do something
} else if (is_array($var)) {
// well then, do something else
}
Let's say I'm trying to combine items from two lists, and I want to get this result:
A7
A8
B7
B8
This is my code:
<?php
$list1_array = array('A', 'B');
$list2_array = array('7', '8');
while(list( , $item1) = each($list1_array)) {
while(list( , $item2) = each($list2_array)) {
echo $item1.$item2."<br />";
}
}
?>
I get this result:
A7
A8
I seems like outside 'while' doesn't make the second loop?
What am I doing wrong?
While it might be better to use a slightly more common (perhaps more readable) approach (e.g. by using foreach loops as shown by GolezTrol,) in answer to your original questions:
The problem is most likely happening because the internal "cursor" (or "pointer") for your array is not being reset... so it never gets back to the start of the original array.
Instead, what if you try something like this:
<?php
$list1_array = array('A', 'B');
$list2_array = array('7', '8');
while(list(,$item1) = each($list1_array)) {
while(list(,$item2) = each($list2_array)) {
echo $item1.$item2."<br />";
}
reset($list2_array);
}
?>
Why not use foreach?
foreach ($list1_array as $item1)
{
foreach ($list2_array as $item2)
{
echo $item1.$item2."<br />";
}
}
Using a while loop with each makes the loop depend on the array pointer. An array has a pointer that tells you which item is the 'current' one. You can use functions like current to get the current item in the array. each is also such a function. It returns the current item (or actually an array with the key and value of the current item).
And therein lies the problem. The inner while loop stops when you are at the end of the array. So for the first item of the outer array (array1), the inner while loop (array2) runs fine. But the second time, the pointer is still at the end of the array and each returns false right away.
So, the solution could be to reset the array pointer, using the reset function as sharply pointed out by #summea. Or you can use a foreach loop, which is not affected by this fenomenon, because it resets the array pointer itself when it starts. Also, I this it's more readable, especially due to the weird list construct. Nevertheless, it might be good to know how the internals work, and your while loop works more low-level than foreach.
this is the PHP code i have:
while($row1 = mysql_fetch_object($query1)) {
echo "*".$row1->id."*";
while ($row2 = mysql_fetch_object($query2)) {
echo "%".$row2->id."%";
}
}
I want it to output for example: *1*%1%%2%%3%%4%*2*%1%%2%%3%%4%*3*%1%%2%%3%%4%
But what this loop outputs is: *1*%1%%2%%3%%4%*2**3*
(It only outputs the $row2 values in the first loop of $row1.
How can I fix this? Thanks.
A few things to note about mysql_fetch_object(). From the PHP document:
Warning: This extension is deprecated as of PHP 5.5.0
Please note above.
returns an object with properties that correspond to the fetched row and moves the internal data pointer ahead.
Please note what I bolded.
The $result object (in your code this would be $query2) is an iterative object that has a pointer pointing to the current item.
Current Item
|
v
[1][2][3][4]
When your first loop hits your second loop, it iterates over the whole thing, such that at the end, the object now looks something like this:
Current Item
|
v
[1][2][3][4]
For each iteration of the first loop, after the first time, the mysql_fetch_object() function basically goes like this:
mysql_fetch_object() ~
1. Get the next time
2. Uh, there are no more objects because we're at item [4]. Return done.
So, how do you get it to work? You could simply save the results into an array and then iterate over that array or you can reset the pointer with mysql_data_seek() (which is also deprecated as of 5.5).
To reset the data pointer, it would be something like this:
while($row1 = mysql_fetch_object($query1)) {
echo "*".$row1->id."*";
while ($row2 = mysql_fetch_object($query2)) {
echo "%".$row2->id."%";
}
// put the result pointer back to the front
mysql_data_seek($query2, 0)
}
Note1, This SO question/answer helped me find the function to use to reset the pointer.
Note, the downside is that you're calling a function that's creating an object, which creates processing overhead every time it runs.
The other option would be save the results into an array and just loop through the array every time:
$secondary_result = array();
while ($row2 = mysql_fetch_object($query2)) {
$secondary_result[] = $row2;
}
while($row1 = mysql_fetch_object($query1)) {
echo "*".$row1->id."*";
foreach($secondary_result as $row2) {
echo "%".$row2->id."%";
}
}
Note, this method will be creating extra memory usage of storing the objects in an array, but it would save on CPU processing as you're not re-creating the objects over and over again, as well as calling a function.
If you just print output, you can consider just saving the result first. No matter now many times you loop over $secondary_result, the final result will always be the same (as per your code, the first loop shows no signs of being directly influencing the second result).
In that case, this makes much more sense
$buffer = '';
while ($row2 = mysql_fetch_object($query2)) {
$buffer .= "%".$row2->id."%";
}
while($row1 = mysql_fetch_object($query1)) {
echo "*".$row1->id."*";
echo $buffer;
}
but I really don't know why you'd do that. If you're doing a nested loop, usually it's because the result of the first loop is affecting the second loop.
But I hope that helps!
Cheers!
EDIT
Per #Blazemonger's comment about looking ahead, the PDO equivalent would be: MySqli:Fetch-Object
When you have a result object from using the PDO function, you would loop like this:
while($row1 = $query1->fetch_object()) {
echo "*".$row1->id."*";
while ($row2 = $query1->fetch_object()) {
echo "%".$row2->id."%";
}
// put the result pointer back to the front
$query2->data_seek(0);
}
The above example shows both the fetch Object and pointer reset versions of MySqli.
The problem is that once you iterate fully through $query2, that's the end of your results. The next time through your $row1 loop, you're still at the end of $query2 and have no results left. Try using mysql_data_seek to go back to the start of your results:
while($row1 = mysql_fetch_object($query1)) {
echo "*".$row1->id."*";
mysql_data_seek($query2, 0);
while ($row2 = mysql_fetch_object($query2)) {
echo "%".$row2->id."%";
}
}
if you really need to repeat your second query data many times, get it into array first, and loop over this array as many times as you need.
First of all: The mysql_* functions are deprecated and will be removed in PHP 5.5. Consider using mysqli or PDO instead.
That being said; back to your question:
Each result set contains an internal pointer to one of the records in the result set. Initially, this pointer points to the first record, and is advanced with each call to mysql_fetch_object.
After your first inner loop, the internal pointer of the $query2 result set will already be at the end of the list, so subsequent calls to mysql_fetch_object will only return FALSE.
If your inner query depends on values from $row1, you will need to re-execute the second query within your outer loop. Otherwise, you can reset the result pointer with mysql_data_seek.
Perhaps you're not getting anything when you call $row2 = mysql_fetch_object($query2) which would give you the output you're getting.
After the first loop there are no more results for query2 to return. So the while is false in each additional loop. You would want to reset the query with mysqli_data_seek or storing all the data in a separate array and looping through that instead.
Please also note:
Per the documentation http://us2.php.net/manual/en/function.mysql-fetch-object.php
The mysql extension is deprecated as of PHP 5.5.0, and will be removed
in the future. Instead, the MySQLi or PDO_MySQL extension should be
used
I have several thousand records (stored in in a table in a MYSQL table) that I need to "batch process." All of the records contain a large JSON. In some cases, the JSON is over 1MB (yes, my DB is well over 1GB).
I have a function that grabs a record, decodes the JSON, changes some data, re-encodes the PHP array back to a JSON, and saves it back to the db. Pretty simple. FWIW, this is within the context of a CakePHP app.
Given an array of ID's, I'm attempting to do something like this (very simple mock code):
foreach ($ids as $id) {
$this->Model->id = $id;
$data = $this->Model->read();
$newData = processData($data);
$this->Model->save($newData);
}
The issue is that, very quickly, PHP runs out of memory. When running a foreach like this, it's almost as if PHP moves from one record to the next, without releasing the memory required for the preceding operations.
Is there anyway to run a loop in such a way that memory is freed before moving on to the next iteration of the loop, so that I can actually process the massive amount of data?
Edit: Adding more code. This function takes my JSON, converts it to a PHP array, does some manipulation (namely, reconfiguring data based on what's present in another array), and replacing values in the the original array. The JSON is many layers deep, hence the extremely long foreach loops.
function processData($theData) {
$toConvert = json_decode($theData['Program']['data'], $assoc = true);
foreach($toConvert['cycles'] as $cycle => $val) {
foreach($toConvert['cycles'][$cycle]['days'] as $day => $val) {
foreach($toConvert['cycles'][$cycle]['days'][$day]['sections'] as $section => $val) {
foreach($toConvert['cycles'][$cycle]['days'][$day]['sections'] as $section => $val) {
foreach($toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'] as $exercise => $val) {
if (isset($toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'][$exercise]['selectedFolder'])) {
$folderName = $toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'][$exercise]['selectedFolder']['folderName'];
if ( isset($newFolderList['Folders'][$folderName]) ) {
$toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'][$exercise]['selectedFolder'] = $newFolderList['Folders'][$folderName]['id'];
}
}
if (isset($toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'][$exercise]['selectedFile'])) {
$fileName = basename($toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'][$exercise]['selectedFile']['fileURL']);
if ( isset($newFolderList['Exercises'][$fileName]) ) {
$toConvert['cycles'][$cycle]['days'][$day]['sections'][$section]['exercises'][$exercise]['selectedFile'] = $newFolderList['Exercises'][$fileName]['id'];
}
}
}
}
}
}
}
return $toConvert;
}
Model->read() essentially just tells Cake to pull a record from the db, and returns it in an array. There's plenty of stuff that's happening behind the scenes, someone more knowledgable would have to explain that.
The first step I would do is make sure everything is passed by reference.
Eg,
foreach ($ids as $id) {
processData($data);
}
function processData(&$d){}
http://php.net/manual/en/language.references.pass.php