I have a friend who's using FluxBB and he has just installed a modification for this forum software. The viewtopic.php file has a while-loop which does a mysql_fetch_assoc() on a result object. Inside of this loop there's a second mysql_fetch_assoc() on a second result object.
The reason for the nested while loop is that there are many posts, and each posts HasMany thank-you's.
Here's some code to better illustrate what I mean:
$result = mysql_query("some-query");
while($cur_post = mysql_fetch_assoc($result)) {
// For every post, execute this second query
$secondResult = mysql_query("some-query that uses the $cur_post id");
while($thank_you = mysql_fetch_assoc($secondResult)) {
// Display the thank_you
}
}
The problem is that the outer-most while loop stops after just one iteration when using the second mysql_query. It does work if the first query is run again, which is a ridiculously dirty hack that works fine with OFFSET.
To me it seems that when the second mysql query is run, whatever $result is pointing at is invalidated. However, I barely know PHP so that's why I've come to SO with this problem.
How would you go about solving this issue? Is it correct that the $result pointer is affected by the second mysql query?
mysql_query() sends a unique query
(multiple queries are not supported)
to the currently active database on
the server that's associated with the
specified link_identifier.
Youll have to grab all your posts (or a batch of them) and store the ids or whatever other data you need into an array or object and then iterate over it making the appropriate query and displaying output.
Or you could use some sql like this:
SELECT thank_you.* FROM thank_you, post WHERE post.id = thank_you.post_id ORDER BY thank_you.post_id;
Of course with this you lose grouping youd have to add some extra logic to build the proper presentation structures based on if the post_id has changed within the loop.
Related
I'm using CakePHP 2.1.3. I have a performance problem of looping a large set of array data returned from find('all'). I want to retrieve a query result row by row to eliminate this expensive array processing. I don't want the result set of array returning from find() or query(). What I'm trying to do is like below:
$db = $this->Model->getDataSource();
$sql = 'SELECT * FROM my_table';
if($result = $db->execute($sql){
$db->resultSet($result);
while($row = $db->fetchResult()){
// do something with $row
}
}
However, I don't want to write the raw query. Is there any Cake function that just builds the query according to the association set and executes it without returning the result set?
[Edit]
I'm currently implementing the above script in controller. My model has no associations and so I don't need to use recursive = -1. It is the whole table fetching for the purpose of CSV export.
The Cake's find() has an internal array processing and the returned result set has to be looped again explicitly. I want to optimize the code by avoiding the array processing of large data twice.
Related issue: https://github.com/cakephp/cakephp/issues/6426
At first be sure, that you only fetch the data you really yreally need. Ideally you get everything you need with $this->YourModel->recursive = -1
Often performance problems arise due to many connected data.
When you have checked this I think a loop would be the best solution where you fetch the desired data in chunks via an between condition. Although I am not sure if this will help you.
Why do you want to go through the whole table? Do you perform some maintenance like e.g. filling a new field or updating a counter? Maybe you can achieve the goal better than by trying to fetch a whole table.
I'm not sure that I have the terminology correct but basically I have a website where members can subscribe to topics that they like and their details go into a 'favorites' table. Then when there is an update made to that topic I want each member to be sent a notification.
What I have so far is:
$update_topic_sql = "UPDATE topics SET ...My Code Here...";
$update_topic_res = mysqli_query($con, $update_topic_sql)or die(mysqli_error());
$get_favs_sql = "SELECT member FROM favourites WHERE topic = '$topic'";
$get_favs_res = mysqli_query($con, $get_favs_sql);
//Here I Need to update the Members selected above and enter them into a notes table
$insert_note_sql = "INSERT INTO notes ....";
Does anyone know how this can be achieved?
Ok, so we've got our result set of users. Now, I'm going to assume from the nature of the question that you may be a bit of a newcomer to either PHP, SQL(MySQL in this case), web development, or all of the above.
Your question part 1:
I have no idea how to create an array
This is easier than what you may think, and if you've already tried this then I apologize, I don't want to insult your intelligence. :)
Getting an array from a mysqli query is just a matter of a function call and a loop. When you ran your select query and saved the return value to a variable, you stored a mysqli result set. The mysqli library supports both procedural and object oriented styles, so since you're using the procedural method, so will I.
You've got your result set
$get_favs_res = mysqli_query($con, $get_favs_sql);
Now we need an array! At this point we need to think about exactly what our array should be of, and what we need to do with the contents of the request. You've stated that you want to make an array out of the results of the SELECT query
For the purposes of example, I'm going to assume that the "member" field you've returned is an ID of some sort, and therefore a numeric type, probably of type integer. I also don't know what your tables look like, so I'll be making some assumptions there too.
Method 1
//perform the operations needed on a per row basis
while($row = mysqli_fetch_assoc($get_favs_res)){
echo $row['member'];
}
Method 2
//instead of having to do all operations inside the loop, just make one big array out of the result set
$memberArr = array();
while($row = mysqli_fetch_assoc($get_favs_res)){
$memberArr[] = $row;
}
So what did we do there? Let's start from the beginning to give you an idea of how the array is actually being generated. First, the conditional in the while loop. We're setting a variable as the loop condition? Yup! And why is that? Because when PHP (and a lot of other languages) sets that variable, the conditional will check against the value of the variable for true or false.
Ok, so how does it get set to false? Remember, any non boolean false, non null, non 0 (assuming no type checking) resolves to true when it's assigned to something (again, no type checking).
The function returns one row at a time in the format of an associative array (hence the _assoc suffix). The keys to that associative array are simply the names of the columns from the row. So, in your case, there will be one value in the row array with the name "member". Each time mysqli_fetch_assoc() is called with your result set, a pointer is pointed to the next result in the set (it's an ordered set) and the process repeats itself. You essentially get a new array each time the loop iterates, and the pointer goes to the next result too. Eventually, the pointer will hit the end of the result set, in which case the function will return a NULL. Once the conditional picks up on that NULL, it'll exit.
In the second example, we're doing the exact same thing as the first. Grabbing an associative array for each row, but we're doing something a little differently. We're constructing a two dimensional array, or nested array, of rows. In this way, we can create a numerically indexed array of associative arrays. What have we done? Stored all the rows in one big array! So doing things like
$memberArr[0]['member'];//will return the id of the first member returned
$memberArr[1]['member'];//will return the id of the second member returned
$lastIndex = sizeof($memberArr-1);
$memberArr[$lastIndex]['member'];//will return the id of the last member returned.
Nifty!!!
That's all it takes to make your array. If you choose either method and do a print_r($row) (method 1) or print_r($memberArr) (method 2) you'll see what I'm talking about.
You question part 2:
Here I Need to update the Members selected above and enter them into a notes table
This is where things can get kind of murky and crazy. If you followed method 1 above, you'd pretty much have to call
mysqli_query("INSERT INTO notes VALUES($row['member']);
for each iteration of the loop. That'll work, but if you've got 10000 members, that's 10000 inserts into your table, kinda crap if you ask me!
If you follow method two above, you have an advantage. You have a useful data structure (that two dim array) that you can then further process to get better performance and make fewer queries. However, even from that point you've got some challenges, even with our new processing friendly array.
The first thing you can do, and this is fine for a small set of users, is use a multi-insert. This just involves some simple string building (which in and of itself can pose some issues, so don't rely on this all the time) We're gonna build a SQL query string to insert everything using our results. A multi insert query in MySQL is just like a normal INSERT except for one different: INSERT INTO notes VALUES (1),(2),(x)
Basically, for each row you are inserted, you separate the value set, that set delineated by (), with a comma.
$query = 'INSERT INTO notes VALUES ';
//now we need to iterate over our array. You have your choice of loops, but since it's numerically indexed, just go with a simple for loop
$numMembers = sizeof($memberArr);
for($i = 0; $i < $numMembers; $i++){
if($i > 0){
$query .= ",({$membersArr[$i]['member']})";//all but the first row need a comma
}
else {
$query .= "({$membersArr[$i]['member']})";//first row does not need a comma
}
}
mysqli_query($query);//add all the rows.
Doesn't matter how many rows you need to add, this will add them. However, this is still going to be a costly way to do things, and if you think your sets are going to be huge, don't use it. You're going to end up with a huge string, TONS of string concats, and an enormous query to run.
However, given the above, you can do what you're trying to do.
NOTE: These are grossly simplified ways of doing things, but I do it this way because I want you to get the fundamentals down before trying something that's going to be way more advanced. Someone is probably going to comment on this answer without reading this note telling me I don't know what I'm doing for going about this so dumbly. Remember, these are just the basics, and in no way reflect industrial strength techniques.
If you're curious about other ways of generating arrays from a mysqli result set:
The one I used above
An even easier way to make your big array but I wanted to show you the basic way of doing things before giving you the shortcuts. This is also one of those functions you shouldn't use much anyway.
Single row as associative(as bove), numeric, or both.
Some folks recommend using loadfiles for SQL as they are faster than inserts (meaning you would dump out your data to a file, and use a load query instead of running inserts)
Another method you can use with MySQL is as mentioned above by using INSERT ... SELECT
But that's a bit more of an advanced topic, since it's not the kind of query you'd see someone making a lot. Feel free to read the docs and give it a try!
I hope this at least begins to solve your problem. If I didn't cover something, something didn't make sense, or I didn't your question fully, please drop me a line and I'll do whatever I can to fix it for you.
In my program I launch an SQL query and get back a result resource. I then iterate through the rows of this result resource using the mysql_fetch_array() function and use the contents of the fields of each row to construct a further SQL query.
The result of launching this second query is the first set of results that I want. However, because the number of results produced by doing this is not many I want to make the search less specific by dropping the last record used to make the query.
e.g. the query which produces the first set of results I want could be:
SELECT uid FROM users WHERE (gender=male AND relationship_status=single
AND shoe_size=10)
I would then want to drop the last record so that my query became:
SELECT uid FROM users WHERE (gender=male AND relationship_status=single)
I have already written code to produce the first query but as I mentioned above I use the mysql_fetch_array function to iterate through ALL of the records. In subsequent "rounds" I only want to iterate through successively less records so that my query is less specific. How can I do this?
This seems like an very inefficient method too - so I'm welcome to any simple ideas which might make it more efficient.
EDIT: Thanks for the reply - Yeah I am actually doing this in my program. I am basically trying to implement a basic search algorithm by taking all the preferences a user has specified in the DB and using it to form a query to look for people with those preferences. So the first time search using all the criteria, then on successive attempts search using one less criteria and negate the user ids which were previously returned. At the moment I am constructing the query from scratch for each "round", but I want to find a way I can do this using the last query
Using the queries above, you could do:
SELECT uid
FROM users
WHERE uid NOT IN (
SELECT uid
FROM users
WHERE
(gender=male
AND relationship_status=single
AND shoe_size=10)
)
This will essentially turn your first query into a sub-query, and use that to negate the results returned. Ie, it will return all the rows, NOT IN the first query.
I'm learning currently php/mysql and I'm confused about this bit.
After some heads scratching I have figured out that mysql_fetch_array remembers which row it last accessed and accesses the next one. (I was originally trying to work out how the code was communicating this to it in example code)
so for database:
parent | job
-------+-------------
mom | receptionist
-------+-------------
dad | taxi driver
the code
mysql_fetch_array($result)[job]
returns 'receptionist' the first time and 'taxi driver' the second.
Where/how is it keeping track of this?
What happens if I don't want to access them in order?
thanks
internal implementation in PHP. Don't try to figure it out ;)
if you want a different order, then specify it in your database query.
Where/how is it keeping track of this?
The mySQL server has an internal result pointer. In some databases / wrappers / libraries you can rewind that pointer, but as far as I know, this is not possible in the mysql_* library of functions.
What happens if I don't want to access them in order?
You have to access them in some order. The only alternative to that is to fetch all rows into an array: Then you can access each row randomly.
If you want to change the order of records, do that in the query using the ORDER clause.
Some database wrappers like PDO have a fetchAll() method that does exactly that. For large result sets, this can be memory intensive and break the script's memory limit, which is why it's usually not done this way.
There is another way to attack this question.
If you want to know how YOU TOO can make functions that do what this one does. Here is how:
<?php
function count_off()
{
static $count = 1;
echo $count++;
}
count_off();
count_off();
count_off();
count_off();
count_off();
?>
the above will output 12345
I should mention. You shouldn't do this without a very good reason. It is SUPER hard to trace when debugging.
If you want to access them in a different order, use an ORDER BY clause in your SQL to change the order that the results are retrieved from the database.
The result of mysql_fetch_array is always the next result/row of the query (first the first row off course)
Intern it will keep a pointer how for it has fetched.
If you want to get them in an alternate order, you have to define it in the query.
Like said the result will always be in the order specified by the query (implicit or explicit)
If you wrote typical looking code like this:
$result = mysql_query("SELECT parent, job FROM table");
while ($row = mysql_fetch_array($result, MYSQL_ASSOC)) {
echo $row['parent'] . ' - ' . $row['job'];
}
Each time mysql_fetch_array() is called, if there is another row in the result, it will return the row as an associative array into the $row variable. Otherwise, it will return false, and the execution of the while loop will end.
Also, because you didn't specify an ORDER BY clause, it defaults to returning rows in the order they were inserted into the table.
The mysql_fetch_array() function grabs one row from the database, in the order that MySQL returns it from the query you gave.
To obtain all the rows, you can put the function in a loop.
while($row = mysql_fetch_array($result)) {
echo $row["job"];
}
This will output:
receptionist
taxi driver
You can change the order by using the sql term order by, which can alphabetically or numerically order your results by a certain column
select * from parent order by job
The above query will order the results alphabetically by the parent job field (results closer to A will come first in the mysql_fetch_*
I have used MySQL a lot, but I always wondered exactly how does it work - when I get a positive result, where is the data stored exactly? For example, I write like this:
$sql = "SELECT * FROM TABLE";
$result = mysql_query($sql);
while ($row = mysql_fetch_object($result)) {
echo $row->column_name;
}
When a result is returned, I am assuming it's holding all the data results or does it return in a fragment and only returns where it is asked for, like $row->column_name?
Or does it really return every single row of data even if you only wanted one column in $result?
Also, if I paginate using LIMIT, does it hold THAT original (old) result even if the database is updated?
The details are implementation dependent but generally speaking, results are buffered. Executing a query against a database will return some result set. If it's sufficiently small all the results may be returned with the initial call or some might be and more results are returned as you iterate over the result object.
Think of the sequence this way:
You open a connection to the database;
There is possibly a second call to select a database or it might be done as part of (1);
That authentication and connection step is (at least) one round trip to the server (ignoring persistent connections);
You execute a query on the client;
That query is sent to the server;
The server has to determine how to execute the query;
If the server has previously executed the query the execution plan may still be in the query cache. If not a new plan must be created;
The server executes the query as given and returns a result to the client;
That result will contain some buffer of rows that is implementation dependent. It might be 100 rows or more or less. All columns are returned for each row;
As you fetch more rows eventually the client will ask the server for more rows. This may be when the client runs out or it may be done preemptively. Again this is implementation dependent.
The idea of all this is to minimize roundtrips to the server without sending back too much unnecessary data, which is why if you ask for a million rows you won't get them all back at once.
LIMIT clauses--or any clause in fact--will modify the result set.
Lastly, (7) is important because SELECT * FROM table WHERE a = 'foo' and SELECT * FROM table WHERE a = 'bar' are two different queries as far as the database optimizer is concerned so an execution plan must be determined for each separately. But a parameterized query (SELECT * FROM table WHERE a = :param) with different parameters is one query and only needs to be planned once (at least until it falls out of the query cache).
I think you are confusing the two types of variables you're dealing with, and neither answer really clarifies that so far.
$result is a MySQL result object. It does not "contain any rows." When you say $result = mysql_query($sql), MySQL executes the query, and knows what rows will match, but the data has not been transferred over to the PHP side. $result can be thought of as a pointer to a query that you asked MySQL to execute.
When you say $row = mysql_fetch_object($result), that's when PHP's MySQL interface retrieves a row for you. Only that row is put into $row (as a plain old PHP object, but you can use a different fetch function to ask for an associative array, or specific column(s) from each row.)
Rows may be buffered with the expectation that you will be retrieving all of the rows in a tight loop (which is usually the case), but in general, rows are retrieved when you ask for them with one of the mysql_fetch_* functions.
If you only want one column from the database, then you should SELECT that_column FROM .... Using a LIMIT clause is also a good idea whenever possible, because MySQL can usually perform significant optimizations if it knows that you only want a certain group of rows.
The first question can be answered by reading up on resources
Since you are SELECTing "*", every column is returned for each mysql_fetch_object call. Just look at print_r($row) to see.
In simple words the resource returned it like an ID that the MySQL library associate with other data. I think it is like the identification card in your wallet, it's just a number and some information but asociated with a lot of more information if you give it to the goverment, or your cell-phone company, etc.