I feel a little embarrassed as there is probably an easy solution, but I don't know enough MySQL to do it. How do I use one query to get data from each of these tables, and then return an array as illustrated below? Every attempt I make ends up returning either one tag, or returning multiple arrays of the same task, each with a different tag.
What should my query structure look like?
Thanks!
http://i.stack.imgur.com/ViqEs.png
The image's array actually shows how the data would look like after two queries, not one. To be able to do it in a single query, and this is because the data is not too complex, you could use a GROUP_CONCAT() to get all of the tags for a task and then use post-query logic to split the data into separate arrays.
The SQL query to get all of the requested data would be:
SELECT
tasks.*, GROUP_CONCAT(tag_name) AS tags
FROM
tasks LEFT JOIN tags ON tags.task_id=tags.id
WHERE
id=2
This query will return a single record; in that record, the column tags will hold a comma-separated list of all of the tags that belong to the task. You can split the data in that column into an array to build your desired structure.
An example, with PHP:
$result = mysql_query("SELECT tasks.*, GROUP_CONCAT(tag_name) AS tags FROM tasks LEFT JOIN tags ON tags.task_id=tags.id WHERE id=2");
// create the "$task" array that has a "task" and "tags" index
$task = array('task' => array(), 'tags' => array());
$task['task'] = mysql_fetch_assoc($result);
// split the comma-separated list of tags into an array
$task['tags'] = explode(',', $task['task']['tags']);
// delete the original "tags" entry that's returned by the sql query
unset($task['task']['tags']);
Please note that this example is void of any data validation, connection information, or other logic and should just be used as a rough idea as how you could split the data into your desired structure.
Related
I have a database that contains 4 relationship tables used to construct the content of a page:
content rel theme theme_meta
The rel table matches the contentID from the content table to the corresponding rel field of the theme table. theme_meta has a field called themeID that links it to the theme table.
So
When constructing a page at the moment I JOIN the content table to the del table, joining that to the theme table and that to the theme_meta table.
It gives me around 24 rows for each matched row of the content table.
I then use a some php foreach loops to restructure the results into multidimensional arrays per content row.
Is that efficient? Would it be faster and more efficient to make 2 calls to the database, one for content and one for theme. This would produce far fewer rows and be easier to work with but require a second call to the database.
As mentioned above, an approach that uses a single query is usually the best way (since database queries incur a lot of overhead).
Indeed, it sounds as though your alternative approach would loop over the results of one query (on the content table) each time calling some other query (on the other tables) to fetch the joined data: such an approach will prove very costly in the long-term and will not scale well.
Therefore, to assemble a multi-dimensional array from the data, you merely need sort the joined results accordingly and keep track of the last seen identifier as you loop over the resultset (in order to detect when one needs to traverse up a level within the resulting array):
$qry = $dbh->query('
SELECT *
FROM content
JOIN rel USING (contentID)
JOIN theme USING (rel)
JOIN theme_meta USING (themeID)
ORDER BY contentID
');
$arr = array();
$row = $qry->fetch();
while ($row) {
array_push($arr, array());
$cid = $row['contentID'];
do {
array_push(end($arr), $row);
} while ($row = $qry->fetch() and $row['contentID'] == $cid);
}
echo var_export($arr);
I would however caution that it is often unnecessarily costly to build such a PHP data structure from the results of a database query, as one can might be able to build and dispatch the requisite output whilst reading the resultset.
I'm making a car part system, to store all the parts inside mysql and then search for them.
Part adding goes like this:
you select up to 280 parts and add all the car info, then all the parts are serialized and put into mysql along with all the car info in a single row.
(for this example I'll say that my current database has 1000 cars and all of those cars have 280 parts selected)
The problem is that when I have 1000 cars with each of them having 280 parts, php and mysql starts getting slow and takes a lot of time to load the data, because the number of parts is 1000*280=280 000.
I use foreach on all of the cars and then put each part into another array.
The final array has 280 000 items and then I filter it by the selected parts in the search, so out of 28 000 parts it may have only have to print like 12 500 parts (if someone is searching for 50 different parts at the same time and 250 cars have that part).
Example database: http://pastebin.com/aXrpgeBP
$q=mysql_query("SELECT `id`,`brand`,`model`,`specification`,`year`,`fueltype`,`capacity`,`parts`,`parts_num` FROM `warehouse`");
while($r=mysql_fetch_assoc($q)){
$partai=unserialize($r['parts']);
unset($r['parts']); //unsetting unserialized parts so the whole car parts won't be passed into the final parts-only array
foreach($partai as $part){
$r['part']=$parttree[$part]; //$parttree is an array with all the part names and $part is the part id - so this returns the part name by it's id.
$r['part_id']=$part; // saves the part id for later filtering selected by the search
$final[]=$r;
}
}
$selectedparts=explode('|', substr($_GET['selected'], 0,strlen($_GET['selected'])-1)); //exploding selected part ids from data sent by jquery into an array
foreach($final as $f){
if(in_array($f['part_id'], $selectedparts)){
$show[]=$f; //filtering only the parts that need to be shown
}
}
echo json_encode($show);
This is the code I use to all the cars parts into arrays and the send it as json to the browser.
I'm not working on the pagination at the moment, but I'll be adding it later to show only 10 parts.
Could solution be to index all the parts into a different table once 24h(because new parts will be added daily) and then just stressing mysql more than php? Because php is doing all the hard work now.
Or using something like memcached to store the final unfiltered array once 24h and then just filter the parts that need to be shown with php?
These are the options I considered, but I know there must be a better way to solve this.
Yes, you should definitely put more emphasis on MySQL. Don't serialize the parts for each car into a single row of a single column. That's terribly inefficient.
Instead, make yourself a parts table, with columns for the various data items that describe each part.
part_id an autoincrement item.
car_id which car is this a part of
partnumber the part's external part number (barcode number?)
etc
Then, use JOIN operations.
Also, why don't you use a WHERE clause in your SELECT statement, to retrieve just the car you want?
Edit
If you're looking for a part, you definitely want a separate parts table. Then you can do a SQL search something like this.
SELECT w.id, w.model, w.specification, w.year, w.fueltype,
p.partnumber
FROM warehouse w
JOIN parts p ON (w.id = p.car_id)
WHERE p.partnumber = 'whatever-part-number-you-want'
This will take milliseconds, even if you have 100K cars in your system, if you index it right.
Your query should be something like:
<?php
$selectedparts=explode('|', substr($_GET['selected'], 0,strlen($_GET['selected'])-1)); //exploding selected part ids from data sent by jquery into an array
$where = ' id < 0 ';
foreach ($selectedparts AS $a){
$where .= " OR `parts` like '%".$a."%'";
}
$query = "SELECT * FROM `warehouse` WHERE ".$where." ORDER BY `id` ASC";//this is your query
//.... rest of your code
?>
Yes, look into has many relationships a car has many parts.
http://net.tutsplus.com/tutorials/databases/sql-for-beginners-part-3-database-relationships/
Then you can use an inner join to get the specified parts. You can do a where clause to match the specific partIds to filter out unwanted parts or cars.
I am building a dynamic news application for a site. The news will be split up into categories which may then contain subcategories(which may contain subcategories) and news articles. I have created a recursive function to return all of the category IDs for all subcategories of a certain category. I would like to run a single query to grab all of the news posts whose category is in the set that was returned by the recursive function.
When I try to use this (I'M USING PEAR DB)
//$myReturnedIDs is a comma delimited list of ids. 5,6,7,8,10,12
$oPrep = $oConn->prepare("SELECT NewsID FROM SiteNews WHERE NewsCategory IN (?)");
$oRes = $oConn->execute($oPrep, array($myReturnedIDs));
The resulting query looks like this:
SELECT NewsID FROM SiteNews WHERE NewsCategory IN('5,6,7,8,10,12')
while I need it to look like:
SELECT NewsID FROM SiteNews WHERE NewsCategory IN(5,6,7,8,10,12)
Now, I realize I could populate that portion of the query in the prepare statement, but I feel that leads to some possible hole for an attack. Am I being over cautious, since the data is not user generated? Is there some way to make this work? My other thought was to add a ? for each ID, amount would be determined before the prepare statement (after getting all the IDs), but that feels clunky to me. Any suggestions?
I'm going to guess that $myReturnedIDs is a comma-delimited string. So this:
array($myReturnedIDs)
yields a one-element array:
["5,6,7,8,10,12"]
If that's the case, then you want to use explode to make the array you want:
$oRes = $oConn->execute($oPrep, explode(",", $myReturnedIDs));
Hope that helps!
There seems to be no shortage of hierarchical data questions in MySQL on SO, however it seems they are mostly talking about managing such data in the database or actually retrieving recursively hierarchical data. My situation is neither. I have a grid of items I need to display. Each item can also have 0 or more comments associated with it. Right now, both the item, along with its data, are displayed in the grid as well as any comments belonging to that item. Usually there is some sort of drill down, dialog, or other user action required to see child data for a grid item but in this case we display both parent and child data in the same grid. Might not fit the de facto standards but it is what it is.
Right now the comments are retrieved by a separate MySQL query for every single parent item in the grid. I immediately cringe at this being aware of all the completely separate database queries that have to be run for a single page load. I haven't profiled but I wouldn't be too surprised if this is part of the slow page loads we sometimes see. I'd like to ideally bring this down to a single query or perhaps 2. However, I'm having difficulty coming up with a solution that sounds any better than what is currently being done.
My first thought was to flatten the comment children for each row with some sort of separator like '|' and then explode them back apart in PHP when rendering the page. The issue with this is it gets increasingly complicated with having to separate each field in a comment, and then each comment, and then account for the possibility of separator characters in the data. Just feels like a mess to maintain and debug.
My next thought was to left outer join the comments to the items and just account for the item duplicates in PHP. I'm working with Codeigniter's database library that returns a PHP array for database data. This sounds like potentially a lot of duplicated data in the resulting array which could possibly be system taxing for larger result sets. I'm thinking in most cases it wouldn't be too bad though so this option is currently at the top of my possibilities list. Ideally, if I understand MVC correctly, I should keep my database, business logic, and view/display as separate as possible. So again, ideally, there should not be any database "quirks" (for lack of a better word) apparent in the data returned by the model. That is, whatever calls for data from this model method, shouldn't be concerned with duplicate data like this. So I'd have to add on an additional loop to somehow eliminate the duplicate item array entries but only after I have retrieved all the child comments and placed them into their own array.
Two queries is another idea but then I have to pass numerous item IDs in the SQL statement for the comments and then go through and zip all the data together manually in PHP.
My goal isn't to get out of doing work here but I am hoping there is some more optimal (less resource intensive and less confusing to the coder) method I haven't thought of yet.
As you state in your question, using a join will bring back a lot of duplicate information. It should be simple enough to remove in PHP, but why bring it back in the first place?
Compiling a SQL statement with a list of IDs retrieved from the query for your list of items shouldn't be a problem (see cwallenpoole's answer). Alternatively, you could create a sub-query so that MySQL recreates the list of IDs for you - it depends on how intensive the sub-query is.
Select your items:
SELECT * FROM item WHERE description = 'Item 1';
Then select the comments for those items:
SELECT * FROM comment WHERE item_id IN (
SELECT id FROM item WHERE description = 'Item 1'
);
For the most part, I solve this type of problem using some sort of ORM Lazy-Loading system but it does not look like you've that as an option.
Have you considered:
Select all top-level items.
Select all second-level items by the ID's in the top-level set.
Associate the objects retrieved in 2 with the items found in 1 in PHP.
Basically (in pseudo-code)
$stmt = $pdo->query("SELECT ID /*columns*/ FROM ENTRIES");
$entries = array();
foreach( $row as $stmt->fetchAll(PDO::FETCH_ASSOC) )
{
$row['child-entities'] = array();
$entries[$row['id']] = $row;
}
$ids = implode(',',array_keys($entries));
$stmt = $pdo->query("SELECT PARENT_ID /*columns*/ FROM children WHERE PARENT_ID IN ($ids)");
foreach( $row as $stmt->fetchAll(PDO::FETCH_ASSOC) )
{
$entries[$row['parent_pid']]['child-entities'][] = $row;
}
$entries will now be an associative array with parent items directly associated with child items. Unless recursion is needed, that should be everything in two queries.
Say I have an array of strings in a php array called $foo with a few hundred entries, and I have a MySQL table 'people' that has a field named 'name' with a few thousand entries. What is an efficient way to find out which strings in $foo aren't a 'name' in an entry in 'people' without submitting a query for every string in $foo?
So I want to find out what strings in $foo have not already been entered in 'people.'
Note that it is clear that all of the data will have to be on one box at one point. The goal would be doing this at the same time minimizing the number of queries and the amount of php processing.
I'd put your $foo data in another table and do a LEFT OUTER JOIN with your names table. Otherwise, there aren't a lot of great ways to do this that don't involve iteration at some point.
The best I can come up with without using a temporary table is:
$list = join(",", $foo);
// fetch all rows of the result of
// "SELECT name FROM people WHERE name IN($list)"
// into an array $result
$missing_names = array_diff($foo, $result);
Note that if $foo contains user input it would have to be escaped first.
What about the following:
Get the list of names that are already in the db, using something like:
SELECT name FROM people WHERE name IN (imploded list of names)
Insert each item from the return of array_diff()
If you want to do it completely in SQL:
Create a temp table with every name in the PHP array.
Perform a query to populate a second temp table that will only include the new names.
Do an INSERT ... SELECT from the second temp table into the people table.
Neither will be terribly fast, although the second option might be slightly faster.
CREATE TEMPORARY TABLE PhpArray (name varchar(50));
-- you can probably do this more efficiently
INSERT INTO PhpArray VALUES ($foo[0]), ($foo[1]), ...;
SELECT People.*
FROM People
LEFT OUTER JOIN PhpArray USING (name)
WHERE PhpArray.name IS NULL;
For a few hundred entries, just use array_diff() or array_diff_assoc()
$query = 'SELECT name FROM table WHERE name != '.implode(' OR name != '. $foo);
Yeash, that doesn't look like it would scale well at all.
I'm not sure there is a more efficient way to do this other than to submit all the strings to the database.
Basically there are two options: get a list of all the strings in MySQL and pull them into PHP and do the comparisons, or send the list of all the strings to the MySQL server and let it do the comparisons. MySQL is going to do the comparisons much faster than PHP, unless the list in the database is a great deal smaller than the list in PHP.
You can either create a temporary table, but either way your pushing all the data to the database.