This is what i am doing now : - in PHP
foreach($array as $value)
{
$query = select abc from tblname where colname =" .$value.
// fire query
}
then i create array of these values and display accordingly.
The PROBLEM: -
I have applied foreach, which fires the query every time it encounters a value in the array.
result, if i have 10 values in my array it fires 10 queries. and uses network 10 times, result slow output.
What i want -
I want to give the array to a stored procedure which shall give me a resultset which will have the outputs corresponding to all the elements in the array.
I know this can be done but do not know how.
the mysql doesnot take arrays as datatype.
the result shall be that network shall be used only once, despit of any number of values in the array.
LIKE -
StoredProcedure(inputMysqlARRAY) // not possible, need a workaroung
{
// fire simple select(same) query for each value.
}
then call this stored procedure from PHP and input array. // need workaround.
You just have to be smarter about your calls. For instance, keeping cached DB objects around and that sort of thing.
Without knowing more about your code (your question is fairly garbled), it seems that if your query is something like this:
$query = "select abc from tblname where colname =" .$value; // run 10 times.
You really just need to write smarter code:
$values = array(); // Now, populate this array.
// When you're done, run the query:
$query = 'select abc from tblname where colname IN (\''.implode('\',\'', $values).'\')';
Generally, we refer to this as Dynamic SQL and is the underpinning for how things are typically done today. A stored procedure (or, based on how I read your question, stored function) is useful at times, but is somewhat antiquated as a first-order methodology for interfacing with SQL. The DB guys still sometimes swear by it, but I think that even they are fairly well in consensus that smarter queries are always better.
Related
I wrote a website using Google App Engine intended for the 'Datastore' database but now the website is already complete and because of Google's resources limit the website goes down too frequently (and I don't have any money to spend on it, even if I did I wouldn't :/)
Question:
When ever I fetch a table, I want to simply return it like this:
Each row is in it's own array to make it easy to iterate through
Instead of the standard number, each key would be the name of column is from
Problem:
No matter if I use fetch_all, fetch_assoc, or fetch_objectI never get anything that looks like what I am trying to get.
I am pretty bad with iterations and for loops so I couldn't figure how to do it manually.
The simplest way to build an associative array of database rows with mysqli_* is to do this:
$mysqli_query = mysqli_query($db_link, "SELECT * FROM tablename");
$result = array();
while($result[] = $mysqli_query->mysqli_fetch_assoc());
and $result will hold the array you want
I'm not sure that I have the terminology correct but basically I have a website where members can subscribe to topics that they like and their details go into a 'favorites' table. Then when there is an update made to that topic I want each member to be sent a notification.
What I have so far is:
$update_topic_sql = "UPDATE topics SET ...My Code Here...";
$update_topic_res = mysqli_query($con, $update_topic_sql)or die(mysqli_error());
$get_favs_sql = "SELECT member FROM favourites WHERE topic = '$topic'";
$get_favs_res = mysqli_query($con, $get_favs_sql);
//Here I Need to update the Members selected above and enter them into a notes table
$insert_note_sql = "INSERT INTO notes ....";
Does anyone know how this can be achieved?
Ok, so we've got our result set of users. Now, I'm going to assume from the nature of the question that you may be a bit of a newcomer to either PHP, SQL(MySQL in this case), web development, or all of the above.
Your question part 1:
I have no idea how to create an array
This is easier than what you may think, and if you've already tried this then I apologize, I don't want to insult your intelligence. :)
Getting an array from a mysqli query is just a matter of a function call and a loop. When you ran your select query and saved the return value to a variable, you stored a mysqli result set. The mysqli library supports both procedural and object oriented styles, so since you're using the procedural method, so will I.
You've got your result set
$get_favs_res = mysqli_query($con, $get_favs_sql);
Now we need an array! At this point we need to think about exactly what our array should be of, and what we need to do with the contents of the request. You've stated that you want to make an array out of the results of the SELECT query
For the purposes of example, I'm going to assume that the "member" field you've returned is an ID of some sort, and therefore a numeric type, probably of type integer. I also don't know what your tables look like, so I'll be making some assumptions there too.
Method 1
//perform the operations needed on a per row basis
while($row = mysqli_fetch_assoc($get_favs_res)){
echo $row['member'];
}
Method 2
//instead of having to do all operations inside the loop, just make one big array out of the result set
$memberArr = array();
while($row = mysqli_fetch_assoc($get_favs_res)){
$memberArr[] = $row;
}
So what did we do there? Let's start from the beginning to give you an idea of how the array is actually being generated. First, the conditional in the while loop. We're setting a variable as the loop condition? Yup! And why is that? Because when PHP (and a lot of other languages) sets that variable, the conditional will check against the value of the variable for true or false.
Ok, so how does it get set to false? Remember, any non boolean false, non null, non 0 (assuming no type checking) resolves to true when it's assigned to something (again, no type checking).
The function returns one row at a time in the format of an associative array (hence the _assoc suffix). The keys to that associative array are simply the names of the columns from the row. So, in your case, there will be one value in the row array with the name "member". Each time mysqli_fetch_assoc() is called with your result set, a pointer is pointed to the next result in the set (it's an ordered set) and the process repeats itself. You essentially get a new array each time the loop iterates, and the pointer goes to the next result too. Eventually, the pointer will hit the end of the result set, in which case the function will return a NULL. Once the conditional picks up on that NULL, it'll exit.
In the second example, we're doing the exact same thing as the first. Grabbing an associative array for each row, but we're doing something a little differently. We're constructing a two dimensional array, or nested array, of rows. In this way, we can create a numerically indexed array of associative arrays. What have we done? Stored all the rows in one big array! So doing things like
$memberArr[0]['member'];//will return the id of the first member returned
$memberArr[1]['member'];//will return the id of the second member returned
$lastIndex = sizeof($memberArr-1);
$memberArr[$lastIndex]['member'];//will return the id of the last member returned.
Nifty!!!
That's all it takes to make your array. If you choose either method and do a print_r($row) (method 1) or print_r($memberArr) (method 2) you'll see what I'm talking about.
You question part 2:
Here I Need to update the Members selected above and enter them into a notes table
This is where things can get kind of murky and crazy. If you followed method 1 above, you'd pretty much have to call
mysqli_query("INSERT INTO notes VALUES($row['member']);
for each iteration of the loop. That'll work, but if you've got 10000 members, that's 10000 inserts into your table, kinda crap if you ask me!
If you follow method two above, you have an advantage. You have a useful data structure (that two dim array) that you can then further process to get better performance and make fewer queries. However, even from that point you've got some challenges, even with our new processing friendly array.
The first thing you can do, and this is fine for a small set of users, is use a multi-insert. This just involves some simple string building (which in and of itself can pose some issues, so don't rely on this all the time) We're gonna build a SQL query string to insert everything using our results. A multi insert query in MySQL is just like a normal INSERT except for one different: INSERT INTO notes VALUES (1),(2),(x)
Basically, for each row you are inserted, you separate the value set, that set delineated by (), with a comma.
$query = 'INSERT INTO notes VALUES ';
//now we need to iterate over our array. You have your choice of loops, but since it's numerically indexed, just go with a simple for loop
$numMembers = sizeof($memberArr);
for($i = 0; $i < $numMembers; $i++){
if($i > 0){
$query .= ",({$membersArr[$i]['member']})";//all but the first row need a comma
}
else {
$query .= "({$membersArr[$i]['member']})";//first row does not need a comma
}
}
mysqli_query($query);//add all the rows.
Doesn't matter how many rows you need to add, this will add them. However, this is still going to be a costly way to do things, and if you think your sets are going to be huge, don't use it. You're going to end up with a huge string, TONS of string concats, and an enormous query to run.
However, given the above, you can do what you're trying to do.
NOTE: These are grossly simplified ways of doing things, but I do it this way because I want you to get the fundamentals down before trying something that's going to be way more advanced. Someone is probably going to comment on this answer without reading this note telling me I don't know what I'm doing for going about this so dumbly. Remember, these are just the basics, and in no way reflect industrial strength techniques.
If you're curious about other ways of generating arrays from a mysqli result set:
The one I used above
An even easier way to make your big array but I wanted to show you the basic way of doing things before giving you the shortcuts. This is also one of those functions you shouldn't use much anyway.
Single row as associative(as bove), numeric, or both.
Some folks recommend using loadfiles for SQL as they are faster than inserts (meaning you would dump out your data to a file, and use a load query instead of running inserts)
Another method you can use with MySQL is as mentioned above by using INSERT ... SELECT
But that's a bit more of an advanced topic, since it's not the kind of query you'd see someone making a lot. Feel free to read the docs and give it a try!
I hope this at least begins to solve your problem. If I didn't cover something, something didn't make sense, or I didn't your question fully, please drop me a line and I'll do whatever I can to fix it for you.
I have a database design here that looks this in simplified version:
Table building:
id
attribute1
attribute2
Data in there is like:
(1, 1, 1)
(2, 1, 2)
(3, 5, 4)
And the tables, attribute1_values and attribute2_values, structured as:
id
value
Which contains information like:
(1, "Textual description of option 1")
(2, "Textual description of option 2")
...
(6, "Textual description of option 6")
I am unsure whether this is the best setup or not, but it is done as such per requirements of my project manager. It definitely has some truth in it as you can modify the text easily now without messing op the id's.
However now I have come to a page where I need to list the attributes, so how do I go about there? I see two major options:
1) Make one big query which gathers all values from building and at the same time picks the correct textual representation from the attribute{x}_values table.
2) Make a small query that gathers all values from the building table. Then after that get the textual representation of each attribute one at a time.
What is the best option to pick? Is option 1 even faster as option 2 at all? If so, is it worth the extra trouble concerning maintenance?
Another suggestion would be to create a view on the server with only the data you need and query from that. That would keep the work on the server end, and you can pull just what you need each time.
If you have a small number of rows in attributes table, then I suggest to fetch them first, fetch all of them! store them into some array using id as index key in array.
Then you can proceed with building data, now you just have to use respective array to look for attribute value
I would recommend something in-between. Parse the result from the first table in php, and figure out how many attributes you need to select from each attribute[x]_values table.
You can then select attributes in bulk using one query per table, rather than one query per attribute, or one query per building.
Here is a PHP solution:
$query = "SELECT * FROM building";
$result = mysqli_query(connection,$query);
$query = "SELECT * FROM attribute1_values";
$result2 = mysqli_query(connection,$query);
$query = "SELECT * FROM attribute2_values";
$result3 = mysqli_query(connection,$query);
$n = mysqli_num_rows($result);
for($i = 1; $n <= $i; $i++) {
$row = mysqli_fetch_array($result);
mysqli_data_seek($result2,$row['attribute1']-1);
$row2 = mysqli_fetch_array($result2);
$row2['value'] //Use this as the value for attribute one of this object.
mysqli_data_seek($result3,$row['attribute2']-1);
$row3 = mysqli_fetch_array($result3);
$row3['value'] //Use this as the value for attribute one of this object.
}
Keep in mind that this solution requires that the tables attribute1_values and attribute2_values start at 1 and increase by 1 every single row.
Oracle / Postgres / MySql DBA here:
Running a query many times has quite a bit of overhead. There are multiple round trips to the db, and if it's on a remote server, this can add up. The DB will likely have to parse the same query multiple times in MySql which will be terribly inefficient if there are tons of rows. Now, one thing that your PHP method (multiple queries) has as an advantage is that it'll use less memory as it'll release the results as they're no longer needed (if you run the query as a nested loop that is, but if you query all the results up front, you'll have a lot of memory overhead, depending on the table sizes).
The optimal result would be to run it as 1 query, and fetch the results 1 at a time, displaying each one as needed and discarding it, which can reek havoc with MVC frameworks unless you're either comfortable running model code in your view, or run small view fragments.
Your question is very generic and i think that to get an answer you should give more hints to how this page will look like and how big the dataset is.
You will get all the buildings with theyr attributes or just one at time?
Cause your data structure look like very simple and anything more than a raspberrypi can handle it very good.
If you need one record at time you don't need any special technique, just JOIN the tables.
If you need to list all buildings and you want to save db time you have to measure your data.
If you have more attribute than buildings you have to choose one way, if you have 8 attributes and 2000 buildings you can think of caching attributes in an array with a select for each table and then just print them using the array. I don't think you will see any speed drop or improvement with so simple tables on a modern computer.
$att1[1]='description1'
$att1[2]='description2'
....
Never do one at a time queries, try to combine them into a single one.
MySQL will cache your query and it will run much faster. PhP loops are faster than doing many requests to the database.
The query cache stores the text of a SELECT statement together with the corresponding result that was sent to the client. If an identical statement is received later, the server retrieves the results from the query cache rather than parsing and executing the statement again.
http://dev.mysql.com/doc/refman/5.1/en/query-cache.html
i was playing with PDO on PostgreSQL 9.2.4 and was trying to fetch data from a table having millions on rows. My query returns about 100.000 rows.
I do not use any of PDOStatements's fetch function, i simply use the result from the PDO Objecte itels and loop through it.
But its getting slower and slower by time. At the beginning it was fetching like 200 rows per second. But the close it comes to its end, it gets slower. Now being at row 30.000 it fetches only 1 row per second. Why is it getting slower.
I do this, its pretty simple:
$dbh = new PDO("pgsql...");
$sql = "SELECT x, y FROM point WHERE name is NOT NULL and place IN ('area1', 'area2')";
$res = $dbh->query($sql);
$ins_sql = "INSERT INTO mypoints (x, y) VALUES ";
$ins_vals = [];
$ins_placeholders = [];
foreach($res as $row) {
$ins_placeholders[] = "(?,?)";
$ins_vals = array_merge($ins_vals, [$row['x'], $row['y']]);
printCounter();
}
// now build up one insert query using placeholders and values,
// to insert all of them in one shot into table mypoints
Function printCounter simply increases an int var and prints it. So i can see how many rows it has put already in that array before i create my insert statement out of it. I use one shot inserts to speed things up, better than doing 100.000 inserts.
But that foreach loop is getting slower by time. How can i increase the speed.
Is there a difference between fetch() and the simple loop method using the pdostatement in foreach?
when i start this php script, it takes like 5-10 seconds for the query. So this has nothing to do with how the table is setup and if i need indexes.
I have other tables returning 1 million rows, im not sure what is the best way to fetch them. I can raise PHP's memory_limit if needed, so the most important thing for me is SPEED.
Appreciate any help.
It's not likely that the slowness is related to the database, because after the $dbh->query() call, the query is finished and the resulting rows are all in memory (they are not in PHP variables yet, but they're in memory accessible at the pgsql module level).
The more likely culprit is the array_merge operation. The array becomes larger at every loop iteration, and the operation recreates the entire array each time.
You may want to do instead:
$ins_vals[] = [$row['x'], $row['y']];
Although personally, when concerned with speed, I'd use an even simpler flat structure:
$ins_vals[] = $x;
$ins_vals[] = $y;
Another unrelated point is that it seems to build a query with a huge number of placeholders, which is not how placeholders are normally used. To send large numbers of values to the server, the efficient way is to use COPY, possibly into a temporary table followed by server-side merge operations if it's not a plain insertion.
I dont know why, but using fetch() method instead and doing the $ins_val filling like this:
$ins_vals[] = $x;
$ins_vals[] = $y;
and using beginTransaction and commit makes now my script unbelievable fast.
Now it takes only about 1 minute to add my 100.000 points.
i think both array_merge and that "ugly" looping through the PDOStatement slowed down my script.
And why the heck someone downvoted my question? Are you punishing me because of my missing knowledge? Thanks.
Ok i generated a class where i set the sql and then put the values for each row with a method call. Whenever it reaches a specific limit, it starts a transaction, prepares the statement with as many placeholders as i have put values, then executes it with the array having all the values, then commit.
This seems to be fast enough, at least it doesnt get slower anymore.
For some reason its faster to add values in a flat structure as Daniel suggested. Thats enough for me.
Sometimes its good to have a function doing one step of insertion, because when the function returns, all the memory used in the function will be freed, so your memory usage stays low.
I've got this HTML form where I have multiple input elements: text, radio, and so on. These are intended to be options, conditional items to apply in an SQL query in order to pull more specific data from a database.
For example, the first input field is a textbox, the second a radio button, and the third a SELECT with two options. With the first, I would add a LIKE %name% sentence in the SQL query. The second is a radio button group with, let's say, a color, so I would add a WHERE color = $item comparing the one chosen with the database column. The third would pretty much be just like the second.
Now, I guess I could just use if sentences comparing the three items, in order to know which one to add to the SQL query, but my question is: is there a smarter way to do it? Like an array checking if those variables are set (I'm still using an if here, so nevermind).
I program simple things from time to time, since I've never done anything significantly complex, but sometimes, even for simple things, no matter how hard I strive to design something, I just can't.
In this case, I really can't figure (imagine, visualize) how would I structure the PHP code along with the SQL query, in order to add the these three conditions to the WHERE clause.
I'd greatly appreciate if you can help me out here. If you need more details, just let me know.
You can just build your sql statement as you go.
A simple example:
$sql = "SELECT ... WHERE 1=1"; // 1=1 is just an example to get the WHERE out of the way
if (first condition / certain $_POST variable is set)
{
$sql .= " AND LIKE %...%";
}
if (second condition)
{
$sql .= " AND something=...";
}
// etc.
// run query
Just for adding another solution, I can suggest you using stored procedure.
http://www.mysqltutorial.org/mysql-stored-procedure-tutorial.aspx
http://www.brainbell.com/tutorials/MySQL/Using_Stored_Procedures.htm
You'll just need to pass values to a sp and generate where condition inside it.
There's another option to generate parameterized query in PHP to prevent SQL Injections.
Here are some links on this topic.
http://us2.php.net/manual/en/security.database.php
http://www.roscripts.com/PHP_MySQL_by_examples-193.html
http://www.webmasterworld.com/php/3110596.htm
http://am.php.net/mysql_real_escape_string