im currently doing a query that pulls a string from my db. but it has to check for every new row i import. and a file of 100k takes almost 4 hours to import. thats way too long. im assuming that my sql code to check if he exist is the thing slowing it down.
ive heard about indexing but i have no clue what it is or how to use it.
this is the current code im using:
$sql2 = $pdo->prepare('SELECT * FROM prospects WHERE :ssn = ssn');
$sql2->execute(array(':ssn' => $ssn));
if($sql2->fetch(PDO::FETCH_NUM) > 0){
so everytime the phpscript reads a new row, it does this check. problem is, that i cant put it in "on duplicate key" in the sql code. it has to check before going to any sql, because if this is empty, then it should continue doing its thing.
what could i do to make this more efficient regarding time? and also, if index is the way to go, could someone enlighten me how this would be done by either posting examples, linking a guide or php.net page. and how i could read from that index to do what i am in my code
So you have 100k records and you don't have any index? Start then with creating one
CREATE INDEX ssn_index ON prospects (ssn)
Now, each time when you try to select something from prospects table with where condition on ssn column MySQL is going to check where it should look for the records by the index. If this column is strongly selective (there are many different values) the query is going to be performed fast.
You can check your execution plan by querying
EXPLAIN SELECT * FROM prospects WHERE :ssn = ssn
First of, I'm pretty new to this site and coding in general so please explain in simple terms as I'm still learning! Thanks
Ok, so I've got a database of results. These are 1-6 ratings. I've already created the ability to retrieve certain results (user, group, all).
But now I'm wanting to alongside retrieving the group and all results to display at the top of the results a mean for each question.
So to start I'm wanting something like this I believe.
SELECT sum(r1), sum(r2), sum(r3) so on,
FROM table
This is where I get confused.
I think I'd need a variable to contain these and then another that counts the amount of entries to divide the total of r1 hence the mean.
Any ideas?..
To calculate a mean, use the AVG function, e.g.
SELECT AVG(r1), AVG(r2)
FROM table
See the MySQL docs.
I'm not sure that I have the terminology correct but basically I have a website where members can subscribe to topics that they like and their details go into a 'favorites' table. Then when there is an update made to that topic I want each member to be sent a notification.
What I have so far is:
$update_topic_sql = "UPDATE topics SET ...My Code Here...";
$update_topic_res = mysqli_query($con, $update_topic_sql)or die(mysqli_error());
$get_favs_sql = "SELECT member FROM favourites WHERE topic = '$topic'";
$get_favs_res = mysqli_query($con, $get_favs_sql);
//Here I Need to update the Members selected above and enter them into a notes table
$insert_note_sql = "INSERT INTO notes ....";
Does anyone know how this can be achieved?
Ok, so we've got our result set of users. Now, I'm going to assume from the nature of the question that you may be a bit of a newcomer to either PHP, SQL(MySQL in this case), web development, or all of the above.
Your question part 1:
I have no idea how to create an array
This is easier than what you may think, and if you've already tried this then I apologize, I don't want to insult your intelligence. :)
Getting an array from a mysqli query is just a matter of a function call and a loop. When you ran your select query and saved the return value to a variable, you stored a mysqli result set. The mysqli library supports both procedural and object oriented styles, so since you're using the procedural method, so will I.
You've got your result set
$get_favs_res = mysqli_query($con, $get_favs_sql);
Now we need an array! At this point we need to think about exactly what our array should be of, and what we need to do with the contents of the request. You've stated that you want to make an array out of the results of the SELECT query
For the purposes of example, I'm going to assume that the "member" field you've returned is an ID of some sort, and therefore a numeric type, probably of type integer. I also don't know what your tables look like, so I'll be making some assumptions there too.
Method 1
//perform the operations needed on a per row basis
while($row = mysqli_fetch_assoc($get_favs_res)){
echo $row['member'];
}
Method 2
//instead of having to do all operations inside the loop, just make one big array out of the result set
$memberArr = array();
while($row = mysqli_fetch_assoc($get_favs_res)){
$memberArr[] = $row;
}
So what did we do there? Let's start from the beginning to give you an idea of how the array is actually being generated. First, the conditional in the while loop. We're setting a variable as the loop condition? Yup! And why is that? Because when PHP (and a lot of other languages) sets that variable, the conditional will check against the value of the variable for true or false.
Ok, so how does it get set to false? Remember, any non boolean false, non null, non 0 (assuming no type checking) resolves to true when it's assigned to something (again, no type checking).
The function returns one row at a time in the format of an associative array (hence the _assoc suffix). The keys to that associative array are simply the names of the columns from the row. So, in your case, there will be one value in the row array with the name "member". Each time mysqli_fetch_assoc() is called with your result set, a pointer is pointed to the next result in the set (it's an ordered set) and the process repeats itself. You essentially get a new array each time the loop iterates, and the pointer goes to the next result too. Eventually, the pointer will hit the end of the result set, in which case the function will return a NULL. Once the conditional picks up on that NULL, it'll exit.
In the second example, we're doing the exact same thing as the first. Grabbing an associative array for each row, but we're doing something a little differently. We're constructing a two dimensional array, or nested array, of rows. In this way, we can create a numerically indexed array of associative arrays. What have we done? Stored all the rows in one big array! So doing things like
$memberArr[0]['member'];//will return the id of the first member returned
$memberArr[1]['member'];//will return the id of the second member returned
$lastIndex = sizeof($memberArr-1);
$memberArr[$lastIndex]['member'];//will return the id of the last member returned.
Nifty!!!
That's all it takes to make your array. If you choose either method and do a print_r($row) (method 1) or print_r($memberArr) (method 2) you'll see what I'm talking about.
You question part 2:
Here I Need to update the Members selected above and enter them into a notes table
This is where things can get kind of murky and crazy. If you followed method 1 above, you'd pretty much have to call
mysqli_query("INSERT INTO notes VALUES($row['member']);
for each iteration of the loop. That'll work, but if you've got 10000 members, that's 10000 inserts into your table, kinda crap if you ask me!
If you follow method two above, you have an advantage. You have a useful data structure (that two dim array) that you can then further process to get better performance and make fewer queries. However, even from that point you've got some challenges, even with our new processing friendly array.
The first thing you can do, and this is fine for a small set of users, is use a multi-insert. This just involves some simple string building (which in and of itself can pose some issues, so don't rely on this all the time) We're gonna build a SQL query string to insert everything using our results. A multi insert query in MySQL is just like a normal INSERT except for one different: INSERT INTO notes VALUES (1),(2),(x)
Basically, for each row you are inserted, you separate the value set, that set delineated by (), with a comma.
$query = 'INSERT INTO notes VALUES ';
//now we need to iterate over our array. You have your choice of loops, but since it's numerically indexed, just go with a simple for loop
$numMembers = sizeof($memberArr);
for($i = 0; $i < $numMembers; $i++){
if($i > 0){
$query .= ",({$membersArr[$i]['member']})";//all but the first row need a comma
}
else {
$query .= "({$membersArr[$i]['member']})";//first row does not need a comma
}
}
mysqli_query($query);//add all the rows.
Doesn't matter how many rows you need to add, this will add them. However, this is still going to be a costly way to do things, and if you think your sets are going to be huge, don't use it. You're going to end up with a huge string, TONS of string concats, and an enormous query to run.
However, given the above, you can do what you're trying to do.
NOTE: These are grossly simplified ways of doing things, but I do it this way because I want you to get the fundamentals down before trying something that's going to be way more advanced. Someone is probably going to comment on this answer without reading this note telling me I don't know what I'm doing for going about this so dumbly. Remember, these are just the basics, and in no way reflect industrial strength techniques.
If you're curious about other ways of generating arrays from a mysqli result set:
The one I used above
An even easier way to make your big array but I wanted to show you the basic way of doing things before giving you the shortcuts. This is also one of those functions you shouldn't use much anyway.
Single row as associative(as bove), numeric, or both.
Some folks recommend using loadfiles for SQL as they are faster than inserts (meaning you would dump out your data to a file, and use a load query instead of running inserts)
Another method you can use with MySQL is as mentioned above by using INSERT ... SELECT
But that's a bit more of an advanced topic, since it's not the kind of query you'd see someone making a lot. Feel free to read the docs and give it a try!
I hope this at least begins to solve your problem. If I didn't cover something, something didn't make sense, or I didn't your question fully, please drop me a line and I'll do whatever I can to fix it for you.
This is how the query looks like: (simplified)
SELECT * FROM posts WHERE user='37' ORDER BY date DESC
I currently only ave one row in that table, but still for some reason it returns two rows that are exactly the same. At first i thought i messed up with the loop, but i tried printing the returned array out with print_r() and it actually returns two rows.
I tried searching, but i didn't find any similar issues. I do however remember that a friend of mine had the same issue at school, so i'm sure we aint the only ones. I probably just didn't use the right search terms, heh.
If you have only one record (verify this), it has to be application logic that is duplicating the returned values.
Are you sure you only have one row in the table? If so, it seems like the problem must be happening outside of SQL.
What are you doing outside of this query? That seems like the likely source of the issue. You mention a loop: could you be adding the query result to your array twice? Or is the array persisting between calls without being reinitialized (in other words, the result of a previous query is remaining in the array when you don't expect it to)?
limit 1 is your friend :)
Try adding it to the end of your query.
I have a friend who's using FluxBB and he has just installed a modification for this forum software. The viewtopic.php file has a while-loop which does a mysql_fetch_assoc() on a result object. Inside of this loop there's a second mysql_fetch_assoc() on a second result object.
The reason for the nested while loop is that there are many posts, and each posts HasMany thank-you's.
Here's some code to better illustrate what I mean:
$result = mysql_query("some-query");
while($cur_post = mysql_fetch_assoc($result)) {
// For every post, execute this second query
$secondResult = mysql_query("some-query that uses the $cur_post id");
while($thank_you = mysql_fetch_assoc($secondResult)) {
// Display the thank_you
}
}
The problem is that the outer-most while loop stops after just one iteration when using the second mysql_query. It does work if the first query is run again, which is a ridiculously dirty hack that works fine with OFFSET.
To me it seems that when the second mysql query is run, whatever $result is pointing at is invalidated. However, I barely know PHP so that's why I've come to SO with this problem.
How would you go about solving this issue? Is it correct that the $result pointer is affected by the second mysql query?
mysql_query() sends a unique query
(multiple queries are not supported)
to the currently active database on
the server that's associated with the
specified link_identifier.
Youll have to grab all your posts (or a batch of them) and store the ids or whatever other data you need into an array or object and then iterate over it making the appropriate query and displaying output.
Or you could use some sql like this:
SELECT thank_you.* FROM thank_you, post WHERE post.id = thank_you.post_id ORDER BY thank_you.post_id;
Of course with this you lose grouping youd have to add some extra logic to build the proper presentation structures based on if the post_id has changed within the loop.