What is the best way to create a unique identifier for an array of UUIDs?
I have a Product that consists of multiple components, each component has an UUID assigned to it.
When creating a new Product with it's components, I need to know if there is already another product having exactly the same components. I think going through MySQL, loading each product with it's components and checking them one by one will be time consuming.
This is my idea:
products table
--------------
id, name, components_uuid_hash
product_components table
------------------------
product_id, component_id, ...
components table
----------------
id, uuid, name, ...
I would calculate the components_uuid_hash:
$product_component_uuids = ['227A0140-F0FB-4FDA-B780-85152AB02927', 'FA0E6D52-F0E4-4F55-87F5-8D73625AEDA3'];
sort($product_component_uuids);
$component_uuid_hash = md5(serialize($product_component_uuids));
I would do this for each product, giving me a unique hash of the whole UUID array that I could simply lookup in the Products table to know if there has already been a different product with the same hash.
Can someone please confirm if this would work or if there is a different better approach?
Assuming you already have a list of uuids you can try to do it in MySQL with something like
select pc.product_id, count(c.id) as co from
product_components pc left join components c on (
pc.component_id=c.id and
c.uuid in ('227A0140-F0FB-4FDA-B780-85152AB02927', 'FA0E6D52-F0E4-4F55-87F5-8D73625AEDA3')
)
group by pc.product_id having co=2;
This will return all products that have exactly two components with UUIDs matching those in the list.
You need to adjust the count value at the end to match the number of uuids.
This will still do a full scan on the product_components table and will probably use a temporary table, so your plan to keep a hash of the UUIDs in the products table and search by it will probably be faster from a MySQL point of view, but you will have to update it every time a product or a component changes.
Related
We have a php/mysql system with about 5 core entities. We now need to add the ability for customers to create custom fields for some of these entities on a per project basis.
They would contain a label, key, type, default value, and possible allowed values.
This is so they could add a custom date field, or a custom dropdown to the UI and save this value against the specific entity.
What is the best approach for storing this kind of data in a mySQL database? I need to store both the config for the field, and then the current value for a specific entity.
I've had a look at various options here.. https://ayende.com/blog/3498/multi-tenancy-extensible-data-model
But this is not really at a tenancy level, more a project level.
I was thinking...
A CustomFields table to hold the configuration of a field against an entity type and project id.
A CustomFieldValues table to hold the value saved against the field - a row per field ( entity_id | field_id | field_value)
Then we create relationships between the entities and these custom values when retrieving the entities.
The issue with this is that there will be as many rows in the Values table as there are custom fields - so saving a entity will result in X extra rows. On top of that, these are versioned, so once a new version is created, there will be another X rows created for that new version.
Also, you can't index the fields on name, joins would become pretty complex i think as you have to join to the configuration and the values to build the key value pair to return against the entity, and how would you select based on a custom field name, when the filed name was actually a value?
I don't want to add dynamic columns to the table, as this will affect ALL the entites in the whole system - not just the ones in the current client / project.
The other option is to store the values in a JSON column.
This could be on the entity row itself customFields or similar. This would prevent the extra rows per field, but also has issues with lack of indexing etc, and still need to join to the config table. However, you could perform queries by the property name if the key=value was stored in the JSON... WHERE entity.customFields->"$.myCustomFieldName" > 1.
Storing the filed name in the json does mean you cannot change it once created, without a lot of pain.
If anyone has any advice on approaches for this, or articles to point me at that would be much appreciated - Im sure this has been solved many times before....
JSON records: No! A thousand times no! If you do that, just wait until somebody actually uses your system for a few tens of millions of records, then asks you to search on one of your extra fields. Your support people will curse your name.
Key-value store. Probably yes. There's a very widely deployed existence proof of this design: WordPress. It has a table called wp_postmeta, containing metadata fields applying to wp_posts (blog pages and posts). It's proven successful.
You will need to do some multiple joining to use this stuff. For example, to search on height and eye-color, you'd need
SELECT p.person_id, p.first, p.last, h.value height, e.value eye_color
FROM person p
LEFT JOIN attrib h ON p.person_id = h.person_id AND h.key='eye_color'
LEFT JOIN attrib e ON p.person_id = e.person_id AND e.key='height'
WHERE e.value='green' and CAST(h.value AS INT) < 160
As the CAST in that WHERE clause shows, you'll have some struggles with data type as well.
You'll need LEFT JOIN operations in this sort of attribute lookup; ordinary inner JOIN operations will suppress rows with missing attributes, and that might not work for you.
But, if you do a good job with indexes, you'll be able to get decent performance from this approach.
The table structure envisioned in my example doesn't have your table describing each additional field, but you know how to add that. It also doesn't have explicit support for multi-project / multitenant data separation. But you can add that as well.
I was about to create my tables when I noticed, sh!t. I have no field to group similar entries (that is the main purpose of this table lol). The idea is, there is a family that has signed up and I want to group them. So when I pull out data, I can assign prices to the family as a whole.
gp_ID customer leader_Of_Group
1 Turk yes
1 JD no
1 Sarah no
1 Felina no
2 John no
2 Manny no
2 Jaden yes
*note - simplified table for readability
My problem is, I don't know how i'm going to achieve the gp_ID. I am confident primary keys don't allow duplicate values so, i'm stumped at the moment.
The gp_ID will be entered automatically, I just don't know how to increment after each family has signed up. Furthermore, there is functionality where my client can select which passengers to group.
I'm not sure how to go about setting up this table or how to query it in a way that each family will increment appropriately. My only thought so far is finding out the gp_ID based on the latest entry and increment, and even then I don't know how to go about doing that or if i'm on the right track. Also, would an auto increment field be necessary too? Any help/guidance would be greatly appreciated.
You are facing an issue because you really have two entities. One of the those entities is the group and the other is the group members.
Your process for adding members to a group should be:
Add a row to the Groups table. This would have an auto-incrementing id.
Add rows to the GroupMembers table, identifying the group using the previous id.
Voila! The groups will be well identified. The Groups table itself could have columns such as:
GroupId (auto-incremented id)
Name for the group
Creation date
Leader_MemberId
The latter would be a way of ensuring that each group has exactly one leader, without having to write a trigger to enforce this constraint.
I'm making a php web page with Yii and MySQL. My problem is in database part.
I have two tables, "stock" and "given". They are like:
stock = id (primary key), name, amount (this is integer), details
and
given = id (from stock), name (from stock), amount (given amount, thus not same with stock), ...
I want to create a table "leftStock" which is the same format with "stock" but has different amount. It will be the number left in stock after given some. Since in different times, different amounts of the same item may be given, the given table's id and name is not unique. Thus i'll use "sum(given.amount)" but its not definite for me.
I thought i can use "create table...select" style coding but i could not construct its structure. Can anybody help me?
(Also can i use "view" in a Yii web page? Because i won't make any direct change on "leftStock" table, its being view may be feasible either.)
It should be done by creating a query too.
Try this, I dont test it yet:
SELECT
(SELECT SUM(a.amount - b.amount)
FROM stock AS a, given AS b
WHERE a.id = b.id AND a.id = id) AS `itemleft`, *
FROM stock
I have a MySQL database with a growing number of users and each user has a list of items they want and of items they have - and each user has a specific ID
The current database was created some time ago and it currently has each users with a specific row in a WANT or HAVE table with 50 columns per row with the user id as the primary key and each item WANT or HAVE has a specific id number.
this currently limits the addition of 50 items per user and greatly complicates searches and other functions with the databases
When redoing the database - would it be viable to instead simply create a 2 column WANT and HAVE table with each row having the user ID and the Item ID. That way there is no 'theoretical' limit to items per user.
Each time a member loads the profile page - a list of their want and have items will then be compiled using a simple SELECT WHERE ID = ##### statement from the have or want table
Furthermore i would need to make comparisons of user to user item lists, most common items, user with most items, complete user searches for items that one user wants and the other user has... - blah blah
The amount of users will range from 5000 - 20000
and each user averages about 15 - 20 items
will this be a viable MySQL structure or do i have to rethink my strategy?
Thanks alot for your help!
This will certainly be a viable structure in mysql. It can handle very large amounts of data. When you build it though, make sure that you put proper indexes on the user/item IDs so that the queries will return nice and quick.
This is called a one to many relationship in database terms.
Table1 holds:
userName | ID
Table2 holds:
userID | ItemID
You simply put as many rows into the second table as you want.
In your case, I would probably structure the tables as this:
users
id | userName | otherFieldsAsNeeded
items
userID | itemID | needWantID
This way, you can either have a simple lookup for needWantID - for example 1 for Need, 2 for Want. But later down the track, you can add 3 for wishlist for example.
Edit: just make sure that you aren't storing your item information in table items just store the user relationship to the item. Have all the item information in a table (itemDetails for example) which holds your descriptions, prices and whatever else you want.
I would recommend 2 tables, a Wants table and a Have table. Each table would have a user_id and product_id. I think this is the most normalized and gives you "unlimited" items per user.
Or, you could have one table with a user_id, product_id, and type ('WANT' or 'HAVE'). I would probably go with option 1.
As you mentioned in your question, yes, it would make much more sense to have a separate tables for WANTs and HAVEs. These tables could have an Id column which would relate the row to the user, and a column that actually dictates what the WANT or HAVE item is. This method would allow for much more room to expand.
It should be noted that if you have a lot of of these rows, you may need to increase the capacity of your server in order to maintain quick queries. If you have millions of rows, they will have a great deal of strain on the server (depending on your setup).
What you're theorizing is a very legitimate database structure. For a many to many relationship (which is what you want), the only way I've seen this done is to, like you say, have a relationships table with user_id and item_it as the columns. You could expand on it, but that's the basic idea.
This design is much more flexible and allows for the infinite items per user that you want.
In order to handle wants and have, you could create two tables or you could just use one and have a third column which would hold just one byte, indicating whether the user/item match is a want or a need. Depending on the specifics of your projects, either would be a viable option.
So, what you would end up with is at least the following tables:
Table: users
Cols:
user_id
any other user info
Table: relationships
Cols:
user_id
item_id
type (1 byte/boolean)
Table: items
Cols:
item_id
any other item info
Hope that helps!
I'm trying to create a web index. Every advertiser in my database will be able to appear on a few categories, so I've added a categorys column, and in that column I'll store the categories separated by "," so it will look like:
1,3,5
The problem is that I have no idea how I'm supposed to select all of the advertisers in a certain category, like: mysql_query("SELECT * FROM advertisers WHERE category = ??");
If categories is another database table, you shouldn't use a plain-text field like that. Create a "pivot table" for the purpose, something like advertisers_categories that links the two tables together. With setup, you could do a query like:
SELECT A.* FROM advertisers AS A
JOIN advertisers_categories AS AC ON AC.advertiser_id = A.id
WHERE AC.category_id = 12;
The schema of advertisers_categories would look something like this:
# advertisers_categories
# --> id INT
# --> advertiser_id INT
# --> category_id INT
You should design your database in another way. Take a look at Atomicity.
Short: You should not store your value in the form of 1,3,5.
I won't give you an answer because if you starting you use it this way now, you going to run into much more severe problems later. No offense :)
It's not possible having comma-separated values to do this strictly in an SQL query. You could return every row and have a PHP script which goes through each row, using explode($row,',') and then if(in_array($exploded_row,'CATEGORY')) to check for the existence of the category.
The more common solution is to restructure your database. You're thinking too two-dimensionally. You're looking for the Many to Many Data Model
advertisers
-----------
id
name
etc.
categories
----------
id
name
etc.
ad_cat
------
advertiser_id
category_id
So ad_cat will have at least one (usually more) entry per advertiser and at least one (usually more) entry per category, and every entry in ad_cat will link one advertiser to one category.
The SQL query then involves grabbing every line from ad_cat with the desired category_id(s) and searching for an advertiser whose id is in the resulting query's output.
Your implementation as-is will make it difficult and taxing on your server's resources to do what you want.
I'd recommend creating a table that relates advertisers to categories and then querying on that table given a category id value to obtain the advertisers that are in that category.
That is a very wrong way to define categories, because your array of values cannot be normalized.
Instead, define another table called CATEGORIES, and use a JOIN-table to match CATEGORIES with ADVERTIZERS.
Only then you will be able to properly select it.
Hope this helps!