I have a table with products that fall under specific categories, but the products within each category can contain multiple meta data tracking field
Table: products
id name category metadata
1 something 1 blue,red,purple
2 something else 2 left,right,middle
I have been trying to contemplate the best method to have a single product table but can't seem to squeeze the metadata in conveniently. for now I have created a table with all the metadata and fields for tracking the related category (the sequence is so i can order them withing a dropdown etc..)
Updated table: products
id name category metadata
1 something 1 1,2,3
2 something else 2 4,5,6
Table: metadata
id category sequence option
1 1 1 blue
2 1 2 red
3 1 3 purple
4 2 1 left
5 2 2 right
6 2 3 middle
If this format makes sense .. I am trying to generate a query that will search for values in my product table and grab each and all of the related meta values. The issue I am having is trying to find a unique value in the products field. if I do a MySQL search for LIKE(%1%) I will get matches for 1, 11, 21, 31 etc ... I thought of adding a leading and trailing comma to the field by default and then search for ",1," which would be unique .. but there has to be a better way ...
Any recommendations (regarding format or query)?
It's not an ideal design to have comma-separated values within a single database field. Aside from the problem you mentioned (difficult to search), your queries will be less efficient, as the DB won't be able to use indices for the lookup.
I'd recommend making a separate table products_metadata with a many-to-one relationship to the products table. Have the *metadata_id*, and the *product_id*, which is a foreign key linking back to the products table. That will make your job much easier.
You want to add another table, which links products to their metadata. It will have two columns: productid and metadataid which refer to the relevant entries in the products and metadata tables respectively. Then you no longer keep metadata in the products table, but JOIN them together as required.
Related
I have been given a project where there are multiple categories stored in a row in a comma delimited list (e.g. 1,2,3). Then I have a categories table that has categories that looks like this:
ID ShowcaseCategory DisplayOrder
1 Member's Work 0
2 Eastern Resorts 1
3 WesternResorts 2
4 Products 4
5 Nordic Skiing 3
I want to be able to pull the ShowcaseCategory (from above) from the table based on more than one value in the comma delimited list. For example, if the category contained 1,5 I would get Member’s Work and Nordic Skiing.
I think I need to get it in an array, possibly explode it and be able to do some join on the ID to get the (literal) ShowcaseCategory. Or, is there an easier way to do this?
In the end I want to dynamically build a list of checkboxes of Categories so they can be updated.
All help is appreciated
One option uses find_in_set(). Assuming that parameter :categories_list holds the csv list of categories ids, you would go:
select ShowcaseCategory
from mytable
where find_in_set(id, :categories_list)
Given a string of text as search criteria, in every field in a (very large, 50+ columns) table.
there is one main table and many smaller ones that are connected to it, but have 'ids' in the main table, not actual searchable text values.
such as:
TABLE SALES LOCATIONS
id location_name customer_id ... other fields
2 normalville 4
where customer id is the primary key in another table:
TABLE CUSTOMER
id name industry
4 EXC Selling Things
is there any elegant way to accomplish this with SQL without using a whole slew of joins or subqueries specifically targeted at each of the 50+ fields?
there are probably 11 fields in the main table that are actually just ids pointing to other tables.
also of note, I am using the Yii framework.
Thanks!
I am working on a website which hold millions of records now (apologies cannot reveal which site) initially it had few hundred records so the query below was acceptable
Query: SELECT * FROM….WHERE category LIKE ‘%,3,%’;
But now it just kills the database as for each query it has to go through the entire 2Mil records with above query
Category table
ID NAME
1 Female
2 Fashion
3 Clothing
4 Accessories
5 Top
6 Dress
7 Earring
8 Short dress
9 Long dress
10 Male
Product table
ID…..Category….other bits
1 ,1,2,3,6,9, ……
2 ,1,2,4,7,
3 ,1,2,3,5,
4 ,10,2,3,4,
you have the picture as what is happening above. Now if I do FullText index on category row in product table it gives only 1 cardinality :(
How can I overcome this?
I have considered duplicating row with each category but the database is huge currently 2 GIG and with duplicates it will turn roughly 10 GIG… more like a problem then a solution
Keep in mind that storing numbers as strings takes about twice as many bytes per digit as storing numbers as integers. Plus all those commas.
So if you're concerned about space, it won't be as much expansion as you fear to store the data in a normalized fashion.
And it will allow you to write proper queries that take advantage of indexes. So if there is some expansion, you will have traded a little bit of storage space for a big improvement in speed.
Tip: if you're using InnoDB, the primary key doesn't cost any storage because the table itself is stored as the primary key index. You should define your normalized table with the category id first and then the product id second, if you need to optimize for searches by category.
CREATE TABLE CategoryProduct (
categoryid INT,
productid INT,
PRIMARY KEY (categoryid, productid)
);
See also my answer to Is storing a delimited list in a database column really that bad? for more disadvantages to using comma-separated lists.
I would consider a new table, say Product_Category (unimaginative I know) where each row contains a column for a Foreign Key (FK) relation to the Product.id and a column for the category.
The category column can probably be a TINYINT which would only require 1 byte to store while I guess the FK column would be the same as the Product.id column (probably INT - 4 bytes), you could then index both columns so you can either find out which categories a product belongs to as well as which products belong in a category. Also, this table wouldn't need to have a Primary Key (i.e. id), saving you an an extra 4 bytes.
(see MySQL Data Type Storage Requirements)
With this solution each row in this new database would take up about 5 bytes. Since each character in the sting takes up 1 byte (Assuming ASCII and latin1 encoding), you would be looking at an increase of 3 bytes (including comma) per category per product by removing Product.category and putting the items into Product_Category, however that's no where near as big a gain as duplicating entire product rows. However, there is the cost of changing your code (unless you're far better than I am at joins).
Does this help any?
One solution I've seen is to use three tables:
categories lists your categories
products lists your products, without any attached category information
category_map is a special table: each row links a product_id to a category_id
To look up products by category, you can then match rows in category_map against rows in products.
This is an imperfect example, but it gets the gist of it:
SELECT * FROM
(
SELECT * FROM category_map
WHERE category_id=1
) AS map
INNER JOIN products
ON products.id = map.product_id;
Table joins are a very powerful tool; you may want to spend some time reading up on them, if you're new to using them. Coding Horror has a visual explanation that skims over the details.
It would be a good idea to set up foreign key constraints or otherwise make sure that entries in category_map correspond to existing entries in products and categories.
I have a database that stores components in a component table.
There is a second table for component prices. This table has a field that maps to a component id. My price table needs to store information for a bunch of different prices (10, 100, 1000, 10K, 100K, 1M). The thing is, there is a possibility in the future to store other price types such as 25K or 50K.
As of right now, my price table looks like this:
id component_id type price
where the type can take values from 1-6 currently. This is good because it will allow me to add new price types in the future very easily.
The other option is a price table that looks like this:
id component_id price_10 price_100 price_1000 price_10K price_100K price_1M
But in this case, I would need to add a new field every time a new price type is added.
I hope that people here would agree with the first method.
But using the first method, I'm having trouble displaying a page that would display all my components in my database with the 6 prices it may or may not have (should show 0 in this case). Obviously this would be simple using the second method.
This is the query I have so far:
SELECT * FROM `component` LEFT JOIN `component_cost` ON `cmpcst_component` = `cmp_id`
EDIT:
I thought I would show some sample data from the component price table:
The prices are a unit price for an amount X. X ranges from 10 to 1 million. So I might have something like this in my component price table:
id component_id type price
1 1 1 0.50
2 1 2 0.45
3 1 3 0.40
4 1 4 0.35
5 1 5 0.32
6 1 6 0.30
The first option its much better.
For display the data, create a view with a pivot table.
You can found help here
http://en.wikibooks.org/wiki/MySQL/Pivot_table
Create two tables, one for components and one for compontent's prices (for a minimum count). Then you can add as many prices per component you would like. Now and in the future.
This is a variation of your first example, you just don't hardcode types but has the number that is related to a certain price.
Try something like:
select * from component c join compcst_component cc on true and c.id = cc.cmp_id;
I'm not sure I follow you.. Hope you can give us the table structures, but here's one try:
SELECT c.cmp_id, ISNULL(p.price, 0) AS Price
FROM component AS c INNER JOI component_cost AS p
ON p.cmpcst_component = c.cmp_id
Maybe I don't have the column-to-table matchup right, but try that.
The first time I tried to do this, I created a field in the category table called query. That contained strings like:
brand = "Burberry" AND type != "Watch"
Which I then inserted into the WHERE clause of a query to find a category's products.
That probably wasn't the best design.
My second attempt was to use a tagging system. I would create a tag table with tags like Burberry and Watch. I had a table tying the tags to the products (HABTM). I also had a table tying the tags to the categories.
The table tying tags to categories had an extra field called include which if it was a 1 then all products selected must also have that tag. Or if it was a 0 then all products selected must NOT have that tag.
This seemed to be a better design then my original, but it required some pretty complex joins.
Now I need to approach this problem once again.
One difference is I am now using the CakePHP (1.3) framework.
Before I try reinventing the wheel again. I was wondering if there are any known patterns/solutions I could use?
Probably you've already done that somehow by now, but here are my 2cents:
I'd drop Categories<->Tags, because I feel that you're unnecessarily duplicating data with it.
I.e. tables should be just categories, categories_products, products, products_tags and tags.
This way:
you wouldn't have to bother about changing category tags when products are added or removed from category
your searches would become more uniform (since there's only one tagging table)
and your tags still would be no more than 3 JOINS away - which is quite comfortable :)
From what I can understand you should have 5 tables:
Categories
Products
Tags
Categories_Tags
Products_Tags
UPDATE: When the user defines what should be selected, the HABTM tables are updated so that the tags/categories link to the products they should be linked to only.
So the query will look something like:
SELECT * FROM products WHERE ID in (SELECT product_id from tag list to include) AND ID NOT IN (select product_id FROM tag list to NOT include)
Maybe I'm missing what you're trying to accomplish here, but this sounds like you're making it more complicated than it needs to be.
Create three tables: Product, Category, and ProductCategory. Product and Category each have an id. Then ProductCategory includes ProductId / CategoryId pairs.
Like:
Product
ProductId Name
1 Lamp
2 Carpet
3 Drill
4 Power cord
5 3/8" bolt
Category
CategoryId Name
1 Electrical
2 Home decor
3 Hardware
ProductCategory
ProductId CategoryId
1 1
1 2
2 2
3 1
3 3
4 1
5 3
Then if you want, e.g., to know all the "Hardware" items:
select product.*
from category
join productcategory using (categoryid)
join product using (productid)
where category.name='Hardware'