MySQL table joins, nested select statements, or create a view? - php

Question

disclaimer: I posted this on another site first
I have a table (res_table) that is about 200 columns wide. One of these columns is named "feature_lk", and it consists of a string of numbers which are "|" delimited. The numbers stand for feature catagories which reside in another table named "features"
Thanks to this thread: http://www.kirupa.com/forum/showthread.php?t=224203 I figured out how to parse the features out!
Now my problem is how to look them up? I feel like I either need to join my two tables, but I'm not sure how, or I need to do a another select query for each of the features that I parse.. This is what I have to far (removed connection strings for posting purposes)
PHP Code:
<?php
$sql = ("SELECT * FROM res_table");
$result = mysql_query($sql);
while($row = mysql_fetch_array($result))
{
$feature_string = $row['features_lk'];
$features = explode( '|', $feature_string );
foreach( $features as $feature ) {
$feature = trim( $feature );
echo $feature.': ';
$sql2 = "SELECT * from features where features.feature_id like $feature";
$result2 = mysql_query($sql2);
while ($row2 = mysql_fetch_array($result2))
{
$feat_desc = $row2['feature_description']; //this is another column in the features table
echo $feat_desc . '<br>';
}
}
echo '<br>';
}
?>
SO that works OK because when I run it, i'll get about results that look like this:
13: None
62: Water Softener - Rented
71: Full
168: Barn
222: Storage Shed
226: Walkout
309: Detached
347: 2 Story
384: Attic Storage
439: Laundry Hook Up
466: Rural
476: Trees
512: School Bus
562: Mud Room
563: Pantry
2273: Septic Tank
643: Private Well
My question is: is there a better way to do this? There are about 10k rows in the main res_table with only a couple hundred hits, you can see that the number of select statements performed grows LARGE in no time at all.
I'm sure this is PHP + MySQL 101 stuff, but I'm just a beginner so any ideas? Thanks in advance.

Answer 1

When you are storing more than one piece of information in a column, your table is not normalized. Doing lookups on feature_lk will necessarily be slow and difficult. feature_lk should become its own table:
Table feature_lk:
res_table_id FK to res_table
feature_id FK to feature table
primary key(res_table_id,feature_id)
Then your query is:
SELECT f.* from features f
JOIN feature_lk lk ON (f.id=lk.feature_id)
JOIN res_table r ON (lk.res_table_id=r.id);
One query only. No loop. No parsing out the features.
ETA
stored procedure for splitting an arbitrary length string by an arbitrary character
DELIMITER $$
DROP PROCEDURE IF EXISTS `dorepeat` $$
CREATE PROCEDURE `dorepeat`(in ToBeSplit LONGTEXT , in Splitter CHAR)
Begin
DECLARE TotalLength INT;
DECLARE SplitterPosition INT;
DECLARE SubstringLength INT;
DECLARE SubstringStart INT;
DROP Table if exists Split_Values;
CREATE temporary TABLE Split_Values (split varchar(255));
SET TotalLength = LENGTH(ToBeSplit);
SET SplitterPosition = LOCATE(Splitter, ToBeSplit);
SET SubstringStart = 1;
ss: WHILE SplitterPosition < TotalLength DO
IF SplitterPosition!=0 THEN
SET SubstringLength = SplitterPosition - SubstringStart;
Insert into Split_Values VALUES (SUBSTRING(ToBeSplit,SubstringStart,SubstringLength));
SET SubstringStart = SplitterPosition+1;
SET SplitterPosition = LOCATE(Splitter, ToBeSplit, SplitterPosition+1);
ELSE
Insert into Split_Values VALUES (SUBSTRING(ToBeSplit,SubstringStart));
SET SplitterPosition=TotalLength;
END IF;
END WHILE ss;
End $$
DELIMITER ;
Using dorepeat in another procedure makes temp table with res_table_id and each feature:
DELIMITER $$
DROP PROCEDURE IF EXISTS `multido` $$
CREATE PROCEDURE `multido`()
Begin
DECLARE done INT default 0;
DECLARE rt_id INT (10);
DECLARE features LONGTEXT;
DECLARE mycur cursor for select distinct res_table_id, feature_lk from res_table WHERE feature_lk!='';
DECLARE continue handler for sqlstate '02000' set done=1;
drop table if exists tmpfeatures;
create temporary table tmpfeatures( res_table_id int(10), feature varchar(255));
open mycur;
repeat
fetch mycur into rt_id,features;
call dorepeat(features,'|');
insert into tmpfeatures select rt_id, trim(split) from Split_Values;
until done end repeat;
close mycur;
End $$
DELIMITER ;

Answer 2

You're feeling the pain of poor database modeling here. If you have any control over the database schema, then you should fix it so that this is properly normalized. Anytime you see a pipe (or comma, or tab, or whatever) delinated list in a database, you should be very suspicious of it.
You should have a join table between your table and categories, generally named something like RES_CATEGORIES that contains the ID from RES and the ID from CATEGORIES. This is the standard way to model a many-to-many relationship in a relational database.
If you can't control the schema, than your best bet is to just parse that out in code and execute a separate query (or queries) to get the category info. You can at least specify multiple category IDs in the where clause, to make it slightly less painful.

Answer 3

from what i understand in your question, you need an intermediate table. for example, you have the table tbl_user and tbl_features where users can subscribe to a number of features and each feature can be subscribed by a number of users.
your database would be more manageable with an extra table tbl_userfeatures {userFeatureID, userID, featureID}, which links the other two tables and allows you to add different combinations.

Answer 4

One simple optimisation step would be to fetch the features in one step, instead of looping over them. Something like this:
$result = mysql_query('SELECT * FROM res_table');
while ($row = mysql_fetch_array($result)) {
$features = str_replace('|', ',', $features);
$result2 = mysql_query("SELECT * FROM features WHERE feature_id IN $features");
while ($row2 = mysql_fetch_array($result2) {
printf('%d: %s', $row2['feature_id'], $row2['feature_description']);
}
}
That's one query for each row in res_table instead of one for each feature.
But before you do this, first listen to the other responses. If you are able to change the database schema to something saner, do so!