Querying inside of a for loop - php

I am wondering if there is a way to do what I am doing more efficiently. Right now, I have a class that retrives statuses from the database. It's pretty simple and shouldn't really effect performance all that much.
public function get ($var1, $var2, $var3)
{
$feed = array(); //Initialize an empty array
//Query the database
$Statement = $this->Database->prepare("SELECT id, name, excerpt, post, timestamp, tags, title FROM posts WHERE col1 = ? AND col2 = ? AND col3 = ? ORDER BY id DESC LIMIT 15");
$Statement->execute(array($var1, $var2, $var3));
while($row = $Statement->fetch(PDO::FETCH_ASSOC))
{
$posts[] = array( "id" => $row["id"], /*etc...*/ );
}
return $posts;
} //end get
And then my page set up something like this which I know is not efficient at all:
<?php for ($count = 1; $count <= $total; $count++): //Display the calendar
echo $count;
$feed = $Feed->get($count, $total, $var3);
foreach ($feed as $post):
echo $post["id"];
endforeach;
endfor; ?>
I hope that makes sense. There's a lot more html thrown in there and everything. Right now there are only 18 rows in my database, and it takes 10 seconds to load the page. Which is really bad. I have to set it up this way because of the design of the site. So the foreach loop has to be within the for loop because the whole thing is set up as a calendar.
My question is whether it would be more efficient to select all of the rows, save them outside of the for loop and then work with that array, or whether it's better to run each query inside the foreach loop the way I'm doing it now (i've read a lot, and know that most people say this is a huge no no). And what kind of issues would I run into if I used the former option and there were say a million rows in the database.
I hope that makes sense. I'll update the question if it doesn't. Right now though about 30 queries are being made to only access 1 or 2 rows. But the only other option I could come up with is selecting all of the rows in the table, and then working with that array, but if there are pretend 1 million rows in the db, I feel like that would affect performance a lot more.
Am I right, and what are some solutions? Thanks

I just want to point out that I did resolve the issue. If anyone is wondering why the foreach loop was querying so sow it was because I accidentally deleted a line where I connected to the Facebook api within the foreach loop every time to gather the poster's information. So if anyone ever stumbles upon this question, just to be sure I want to clarify that making many facebook->api calls is a bad thing. save the info in your database and query that instead.

Related

PHP Foreach loop too slow when applied on large arrays

So, basically, i have to loop thought an array of 25000 items, then compare each item with another array's ID and if the ID's from the first array and the second match then create another array of the matched items. That looks something like this.
foreach ($all_games as $game) {
foreach ($user_games_ids as $user_game_id) {
if ($user_game_id == $game["appid"]) {
$game_details['title'][] = $game["title"];
$game_details['price'][] = $game["price"];
$game_details['image'][] = $game["image_url"];
$game_details['appid'][] = $game["appid"];
}
}
}
I tested this loop with only 2500 records from the first array ($all_games) and about 2000 records from the second array ($user_games_ids) and as far as i figured, it takes about 10 seconds for the execution of that chunk of code, only the loops execution. Is that normal? Should that take that long or I'm I approaching the issue from the wrong side? Is there a way to reduce that time? Because when i apply that code to 25000 records that time will significantly increase.
Any help is appreciated,
Thanks.
EDIT: So there is no confusion, I can't use the DB query to improve the performance, although, i have added all 25000 games to the database, i can't do the same for the user games ids. There is no way that i know to get all of the users through that API i'm accessing, and even there is, that would be really a lot of users. I get user games ids on the fly when a user enters it's ID in the form and based on that i use file_get_contents to obtain those ids and then cross reference them with the database that stores all games. Again, that might not be the best way, but only one i could think of at this point.
If you re-index the $game array by appid using array_column(), then you can reduce it to one loop and just check if the data isset...
$game = array_column($game,null,"appid");
foreach ($user_games_ids as $user_game_id) {
if (isset( $game[$user_game_id])) {
$game_details['title'][] = $game[$user_game_id]["title"];
$game_details['price'][] = $game[$user_game_id]["price"];
$game_details['image'][] = $game[$user_game_id]["image_url"];
$game_details['appid'][] = $game[$user_game_id]["appid"];
}
}

mysql | PHP | Join within own table

i dont know if i am doing right or wrong, please dont judge me...
what i am trying to do is that if a record belongs to parent then it will have parent id assosiated with it.. let me show you my table schema below.
i have two columns
ItemCategoryID &
ItemParentCategoryID
Let Suppose a record on ItemCategoryID =4 belongs to ItemCategoryID =2 then the column ItemParentCategoryID on ID 4 will have the ID of ItemCategoryID.
I mean a loop with in its own table..
but problem is how to run the select query :P
I mean show all the parents and childs respective to their parents..
This is often a lazy design choise. Ideally you want a table for these relations or/and a set number of depths. If a parent_id's parent can have it's own parent_id, this means a potential infinite depth.
MySQL isn't a big fan of infinite nesting depths. But php don't mind. Either run multiple queryies in a loop such as Nil'z's1, or consider fetching all rows and sorting them out in arrays in php. Last solution is nice if you pretty much always get all rows, thus making MySQL filtering obsolete.
Lastly, consider if you could have a more ideal approach to this in your database structure. Don't be afraid to use more than one table for this.
This can be a strong performance thief in the future. An uncontrollable amount of mysql queries each time the page loads can easily get out of hands.
Try this:
function all_categories(){
$data = array();
$first = $this->db->select('itemParentCategoryId')->group_by('itemParentCategoryId')->get('table')->result_array();
if( isset( $first ) && is_array( $first ) && count( $first ) > 0 ){
foreach( $first as $key => $each ){
$second = $this->db->select('itemCategoryId, categoryName')->where_in('itemParentCategoryId', $each['itemParentCategoryId'])->get('table')->result_array();
$data[$key]['itemParentCategoryId'] = $each['itemParentCategoryId'];
$data[$key]['subs'] = $second;
}
}
print_r( $data );
}
I don't think you want/can to do this in your query since you can nest a long way.
You should make a getChilds function that calls itself when you retrieve a category. This way you can nest more than 2 levels.
function getCategory()
{
// Retrieve the category
// Get childs
$childs = $this->getCategoryByParent($categoryId);
}
function getCategorysByParent($parentId)
{
// Get category
// Get childs again.
}
MySQL does not support recursive queries. It is possible to emulate recursive queries through recursive calls to a stored procedure, but this is hackish and sub-optimal.
There are other ways to organise your data, these structures allow very efficient querying.
This question comes up so often I can't even be bothered to complain about your inability to use Google or SO search, or to offer a wordy explanation.
Here - use this library I made: http://codebyjeff.com/blog/2012/10/nested-data-with-mahana-hierarchy-library so you don't bring down your database

Reversing the order of displayed data from a SELECT query

My website (thanks to help from many of you here on SO) is set up to display a fixed number of updates (52) where each Friday, one rotates out, and a new one rotates in. The 52 updates are displayed 4 per page across 13 pages. It works perfectly.
However, the new update appears at the END of the displayed updates. I would like the new update to be the first one people will see.
Here is my query:
$conn = dbConnect('query');
$updates = "SELECT update_id, update_title, update_desc, path
FROM updates
WHERE flag_live = ?
ORDER BY update_id DESC
LIMIT ?, ?";
$stmt = $conn->prepare($updates);
$stmt->bind_param('sii', $uLive, $offset, $limit);
$stmt->bind_result($uId, $uTitle, $uDesc, $uPath);
$stmt->execute();
$stmt->store_result();
$stmt->fetch();
I want the items to display in reverse order as they currently do. If I change the ORDER BY from DESC to ASC it picks a different set of data which I do not want. Thus, from my research I have seen that either array_reverse() or rsort would do the trick. I need to know what syntax is correct to use array_reverse() properly here.
I don't get along well with PHP, but something like this?
$results = $stmt->fetchAll();
$results = array_reverse( $results );
foreach ($results as $row)
{
/* $row will contain each row in the reversed array one at a time. do your usual process-one-row logic here */
}
Btw, when you don't know what a PHP function returns when you call it, there is a very useful debugging function, var_dump. You can call it on anything and it'll print what it's composed of, recursively.
http://au1.php.net/manual/en/function.var-dump.php
Maybe try something like this:
$results = $stmt->fetchAll();
$results = array_reverse( $results );
The next part can be done a few different ways so it really depends on how you are going to use the results and paginate them.
You could use a for loop: http://us3.php.net/manual/en/control-structures.for.php
Or foreach: http://us3.php.net/manual/en/control-structures.foreach.php
Okay, after a lot of headbanging, I was able to accomplish this task, though not in a manner that I had considered when I posted this question. What I ended up doing was writing a subquery which allowed me to reverse the order of the updates displayed on each page, and as you'll know, this only reverses the order of the four updates visible on the page, NOT the overall order of the 52 chosen (see original question).
The next step was to 'reverse' my pagination system...no small task...rewriting logic, testing code, etc. So by reversing things with a subquery and reversing my pagination, I achieved my goal.
All of your suggestions her helped with the final outcome, though not in a way expected. I apologize in advance if this is not as concise as answering my own question should be.

Names for PHP query results and PHP loops.

This might be a simple question, but I can't find a definitive answer I can understand. I use PHP loops alot, I'm fairly new to PHP so they are usually simple like so:
<?php
$result = mssql_query("SELECT Price FROM Window_Extras WHERE ExtraID = '4' ");
while ($row = mssql_fetch_array($result)) {
?>
<a title="<?php echo $row['Colour']; ?>"></a>
<?php }?>
Is a really simple example, that doesn't make much sense, but I hope it shows how I use them. The question I wanted to ask was if $row and $result have to be named that for it to work, could they for example be named $priceresult and $pricerow?
This is because sometimes I would like to use multiple queries for a single loop, for example:
<?php
$result = mssql_query("SELECT Price FROM Extras WHERE ExtraID = '4' ");
$colourresult = mssql_query("SELECT ColourID FROM Colours WHERE Type = '8' ");
while ($row = mssql_fetch_array($result, $colourresult)) {
?>
This however didn't work, when I tried to echo out:
<?php echo $row['ColourID']; ?>
Can anyone tell me how I should be approaching this, and if I am at all on the correct track. Sorry if I havn't explained it very well.
To answer your first question:
Yes, you can use any variable name you like for the result and row variables. PHP doesn't care what you call them, and in fact it's perfectly possible to have several of them in use at any given time, in which case they obviously need to have different names.
You then followed up that question by asking why the following code doesn't work:
$result = mssql_query("SELECT Price FROM Extras WHERE ExtraID = '4' ");
$colourresult = mssql_query("SELECT ColourID FROM Colours WHERE Type = '8' ");
while ($row = mssql_fetch_array($result, $colourresult)) {
....
}
The reason for this is that the _fetch_array() function can only work with one set of results at a time. You would need to fetch a separate row array for each of them.
It's not clear what you're trying to do with these two queries, and why you would want to put them into the same loop together in the way you've shown.
I'm going to assume that the two queries are linked in some way that makes it logical for you to use them together like this? Perhaps the Extra item you're loading has a known Colour; ie you know that the Extra item numbered 4 is coloured with the Colour numbered 8?
Typically a program wouldn't be written with this knowledge; it would be part of the data. So in the Extras table, you would have a ColourID field, which would contain the value 8. The program would load the Extras record, see that the ColourID was set, and then load the matching Colours record according to what it saw.
Thus, your code could look something like this:
$result = mssql_query("SELECT Price FROM Extras WHERE ExtraID = '4' ");
while ($row = mssql_fetch_array($result)) {
$colourresult = mssql_query("SELECT ColourID FROM Colours WHERE Type = '".$row['colourID']."' ");
while ($row2 = mssql_fetch_array($result)) {
....
}
}
Inside the inner while loop, you could then access fields from either query, using $row or $row2 respectively (again, you can name these as you see fit).
However, that's not the end of the story, because SQL actually has the ability to merge these two queries into one without needing all that PHP code, using a thing call a SQL JOIN.
Now we can write a more complex query, but go back to having simpler PHP code:
$result = mssql_query("SELECT Extras.Price, Colours.ColourName FROM Extras WHERE ExtraID = '4' INNER JOIN Colours ON Colours.ColourID = Extras.ColourID");
while ($row = mssql_fetch_array($result)) {
....
}
If you're a beginner in PHP and SQL, these concepts are all probably new to you, so I advise trying them out, experimenting with them, and most importantly, reading a few (good quality) tutorials about them before proceeding much further.
Hope that helps. :)
(PS: as I said above, make sure you're reading good tutorials; beware of bad PHP examples and teaching sites -- there's a lot of them out there, teaching poor code and obsolete techniques; make sure you're reading something worthwhile. A good place to start might be http://phpmaster.com/)
This is because mssql_fetch_array can only take one result set. So removing $result and leaving $colourresult should work for you.
See: http://php.net/manual/en/function.mssql-fetch-array.php
Your variables ($...) can be called whatever you want, it's generally better to name them in a way that you can understand, hence most of the examples in the PHP Manual contain variables like $row, $result, $query, etc.
In terms of your database query, you can only pass one query to the mssql_query method. If you have data from different tables that you need to display, you should try and join the tables if possible using SQL rather than looping through multiple result sets.

PHP and MySQL - efficiently handling multiple one to many relationships

I am seeking some advice on the best way to retrieve and display my data using MySQL and PHP.
I have 3 tables, all 1 to many relationships as follows:
Each SCHEDULE has many OVERRIDES and each override has many LOCATIONS. I would like to retrieve this data so that it can all be displayed on a single PHP page e.g. list out my SCHEDULES. Within each schedule list the OVERRIDES, and within each override list the LOCATIONS.
Option1 - Is the best way to do this make 3 separate SQL queries and then write these to a PHP object? I could then iterate through each array and check for a match on the parent array.
Option 2 - I have thought quite a bit about joins however doing two right joins will return me a row for every entrance in all 3 tables.
Any thoughts and comments would be appreciated.
Best regards, Ben.
If you really want every piece of data, you're going to be retrieving the same number of rows, no matter how you do it. Best to get it all in one query.
SELECT schedule.id, overrides.id, locations.id, locations.name
FROM schedule
JOIN overrides ON overrides.schedule_id = schedule.id
JOIN locations ON locations.override_id = overrides.id
ORDER BY schedule.id, overrides.id, locations.id
By ordering the results like this, you can iterate through the result set and move on to the next schedule whenever the scheduleid changes, and the next location when the locationid changes.
Edit: a possible example of how to turn this data into a 3-dimensional array -
$last_schedule = 0;
$last_override = 0;
$schedules = array();
while ($row = mysql_fetch_array($query_result))
{
$schedule_id = $row[0];
$override_id = $row[1];
$location_id = $row[2];
$location_name = $row[3];
if ($schedule_id != $last_schedule)
{
$schedules[$schedule_id] = array();
}
if ($override_id != $last_override)
{
$schedules[$schedule_id][$override_id] = array();
}
$schedules[$schedule_id][$override_id][$location_id] = $location_name;
$last_schedule = $schedule_id;
$last_override = $override_id;
}
Quite primitive, I imagine your code will look different, but hopefully it makes some sense.

Categories