I struggle to find an easy way to recreate a navigation structure with datas stored in a mysql database. I can't change the database structure, it's automatically generated.
I use Laravel 5.2 as the base framework for this project.
Here's the database structure:
Datas:
| id | origin | type | value |
|----|--------|------------|------------------------------------|
| 1 | 20 | menuLevel | 0 |
| 2 | 20 | menuTitle | The title of this menu item |
| 3 | 20 | menuParent | 0 |
| 4 | 20 | menuUrl | /url-of-the-menu |
| 5 | 21 | menuLevel | 1 |
| 6 | 21 | menuTitle | The title of this second menu item |
| 7 | 21 | menuParent | 20 |
| 8 | 21 | menuUrl | /url-of-the-submenu |
And much more data of this kind. But all menu items always contains four properties, the level, the title, the url and the parent item that I have to put together.
With this structure, I try to recreate the navigation but I can't find the proper way to query the database to obtain a result that I can then use with a jquery tree plugin to navigate through the generated menu.
Ideally, this structure would be:
{
'origin': {
'menuLevel': 0,
'menuTitle': 'The title',
'menuParent': 0,
'menuUrl': '/url'
}, and so on..
}
Does anyone already faced this kind of structure? What could be the best way to achieve this? Thanks in advance for pointing me in the right direction.
if by 'origin' you mean 20, you probably could query
SELECT * FROM [tablename] ORDER BY origin
and then put them into an array until the origin changes, then open a new array. This also fetches the unstructured data.
if you want exactly those four attributes, you could do a query like
SELECT t1.origin, t1.value as menuLevel, t2.value as menuTitle,
t3.value as menuParent, t4.value as menuUrl
FROM [table] t1
LEFT JOIN [table] t2 ON (t1.origin=t2.origin AND t2.type='menuTitle')
LEFT JOIN [table] t3 ON (t1.origin=t3.origin AND t3.type='menuParent')
LEFT JOIN [table] t4 ON (t1.origin=t4.origin AND t4.type='menuUrl')
WHERE t1.type='menuLevel'
This could be extended to all the unstructured data as well, you'd then have a lot of NULLs that you could filter out or not.
add: I hope laravel puts an index on (id, type) or this query might be quite slow.
Related
I'm writing a PHP package whereby I need to store a set of "documents" each with their own attributes which can vary in quantity, name and type, just like the attributes for different types of products could differ (E.g. a shoe may have a material, color and style but a smartphone may have an operating system, weight, size etc.)
| id | name |
|-----|------------|
| 1 | Acme Shoe |
| 2 | Acme Phone |
I want to be able to query all of my documents, or products by their attributes. The queries could range from a very simple WHERE attribute_a = value_a to a much more complicated nested set of clauses, like WHERE ((attribute_a = value_a OR attribute_a > value_b) AND attribute_b LIKE '%pattern%')
My ideal scenario would be to use the native JSON support afforded by MySQL 5.7+ and MariaDB 10.2+ to store the attributes against each document and use the handy JSON_EXTRACT function to extract any attribute that I want to query.
| id | name | attributes |
|-----|------------|----------------------------------------|
| 1 | Acme Shoe | {"material":"canvas","color":"black"} |
| 2 | Acme Phone | {"os":"android","weight":100} |
SELECT *
FROM documents
WHERE (
JSON_EXTRACT(attributes, "$.weight") = 1
OR JSON_EXTRACT(attributes, "$.weight") > 99
)
AND JSON_EXTRACT(attributes, "$.os") LIKE '%droid%'
Unfortunately, my package needs to be able to support older versions of MySQL and MariaDB. I had considered storing JSON in a TEXT or LONGTEXT field and using REGEX to parse out the values of the attributes I need when making comparisons but I can imagine that would be incredibly resource intensive and slow. Please correct me if I'm wrong.
So as it stands, I feel like I'm locked into going for an EAV type solution:
| id | name |
|-----|------------|
| 1 | Acme Shoe |
| 2 | Acme Phone |
| id | document_id | key | value |
|-----|-------------|----------|---------|
| 1 | 1 | material | canvas |
| 2 | 1 | color | black |
| 3 | 2 | os | android |
| 4 | 2 | weight | 100 |
Finding the documents with one WHERE clause is relatively trivial:
SELECT DISTINCT(document_id)
FROM document_attributes
WHERE key = 'material'
AND value = 'canvas'
However, I have no idea how I would implement more complicated WHERE clauses. Particularly, the problem being that the attributes are stored in separate rows. E.g.
Getting the documents that have canvas material AND are colored black.
Getting the documents that have android os AND have weight either, 1 or greater than 99.
Any advice or recommendations would be greatly appreciated.
Edit
After some consideration with the EAV approach, the best I have managed to come up with so far is repeatedly joining the attributes table to the documents table for each attribute involved in the query. From there, I'm able to use each attribute's value in the WHERE clause. For example, selecting all products where the attribute "material" is "canvas", OR the "weight" is greater than 99:
SELECT d.id AS id, a1.value AS material, a2.value AS weight
FROM documents AS d
LEFT JOIN attributes AS a1 ON a1.document_id = d.id AND a1.name = 'material'
LEFT JOIN attributes AS a2 ON a2.document_id = d.id AND a2.name = 'weight'
WHERE a1.value = 'canvas'
AND a2.value > 99
This appears to yield:
| id | material | weight |
|----|----------|--------|
| 1 | canvas | NULL |
| 2 | NULL | 100 |
Assuming the document_id/key/value combination is unique, you could do something like this:
SELECT document_id FROM example
WHERE `key`='material' AND `value`='canvas'
OR `key`='color' AND `value`='black'
GROUP BY document_id
HAVING COUNT(*) = 2;
SELECT document_id FROM example
WHERE `key`='os' AND `value`='android'
OR (`key`='weight' AND (`value` = 1) OR (`value` > 99))
GROUP BY document_id
HAVING COUNT(*) = 2;
Try this SQL:
select SUBSTRING_INDEX( SUBSTRING_INDEX(attributes,'"',4) ,'"',-1) from documents;
I have a page that displays a list of projects. With each project is displayed the following data retrieved from a mysqli database:
Title
Subtitle
Description
Part number (1 of x)
The total number of photos associated with that project
A randomly selected photo from the project
A list of tags
Projects are displayed 6 per page using a pagination system
As this is based on an old project of mine, it was originally done with sloppy code (I was just learning and did not know any better) using many queries. Three, in fact, just for items 5-7, and those were contained within a while loop that worked with the pagination system. I'm now quite aware that this is not even close to being the right way to do business.
I am familiar with INNER JOIN and the use of subqueries, but I'm concerned that I may not be able to get all of this data using just one select query for the following reasons:
Items 1-4 are easy enough with a basic SELECT query, BUT...
Item 5 needs a SELECT COUNT AND...
Item 6 needs a basic SELECT query with an ORDER by RAND LIMIT 1 to
select one random photo out of all those associated with each project
(using FilesystemIterator is out of the question, because the photos
table has a column indicating 0 if a photo is inactive and 1 if it is
active)
Item 7 is selected from a cross reference table for the tags and
projects and a table containing the tag ID and names
Given that, I'm not certain if all this can (r even should for that matter) be done with just one query or if it will need more than one query. I have read repeatedly how it is worth a swat on the nose with a newspaper to nest one or more queries inside a while loop. I've even read that multiple queries is, in general, a bad idea.
So I'm stuck. I realize this is likely to sound too general, but I don't have any code that works, just the old code that uses 4 queries to do the job, 3 of which are nested in a while loop.
Database structure below.
Projects table:
+-------------+---------+----------+---------------+------+
| project_id | title | subtitle | description | part |
|---------------------------------------------------------|
| 1 | Chevy | Engine | Modify | 1 |
| 2 | Ford | Trans | Rebuild | 1 |
| 3 | Mopar | Diff | Swap | 1 |
+-------------+---------+----------+---------------+------+
Photos table:
+----------+------------+--------+
| photo_id | project_id | active |
|--------------------------------|
| 1 | 1 | 1 |
| 2 | 1 | 1 |
| 3 | 1 | 1 |
| 4 | 2 | 1 |
| 5 | 2 | 1 |
| 6 | 2 | 1 |
| 7 | 3 | 1 |
| 8 | 3 | 1 |
| 9 | 3 | 1 |
+----------+------------+--------+
Tags table:
+--------+------------------+
| tag_id | tag |
|---------------------------|
| 1 | classic |
| 2 | new car |
| 3 | truck |
| 4 | performance |
| 5 | easy |
| 6 | difficult |
| 7 | hard |
| 8 | oem |
| 9 | aftermarket |
+--------+------------------+
Tag/Project cross-reference table:
+------------+-----------+
| project_id | tag_id |
|------------------------|
| 1 | 1 |
| 1 | 3 |
| 1 | 4 |
| 2 | 2 |
| 2 | 5 |
| 3 | 6 |
| 3 | 9 |
+------------+-----------+
I'm not asking for the code to be written for me, but if what I'm asking makes sense, I'd sincerely appreciate a shove in the right direction. Often times I struggle with both the PHP and MySQLi manuals online, so if there's any way to break this down, then fantastic.
Thank you all so much.
You're able to do subqueries inside your SELECT clause, like this:
SELECT
p.title, p.subtitle, p.description, p.part,
(SELECT COUNT(photo_id) FROM Photos where project_id = p.project_id) as total_photos,
(SELECT photo_id FROM Photos where project_id = p.project_id ORDER BY RAND LIMIT 1) as random_photo
FROM projects as p
Now, for the list of tags, as it returns more than one row, you can't do a subquery and you should do one query for every project. Well, in fact you can if you return all the tags in some kind of concatenation, like a comma separated list: tag1,tag2,tag3... but I don't recommend this one time that you will need to explode the column value. Do it only if you have many many projects and the performance to retrieve the list of tags for each individual project is fairly low. If you really want, you can:
SELECT
p.title, p.subtitle, p.description, p.part,
(SELECT COUNT(photo_id) FROM Photos where project_id = p.project_id) as total_photos,
(SELECT photo_id FROM Photos where project_id = p.project_id ORDER BY RAND LIMIT 1) as random_photo,
(SELECT GROUP_CONCAT(tag SEPARATOR ', ') FROM tags WHERE tag_id in (SELECT tag_id FROM tagproject WHERE project_id = p.project_id)) as tags
FROM projects as p
As you said from item 1 to 4 you already have the solution.
Add to the same query a SQL_CALC_FOUND_ROWS instead of a SELECT COUNT to solve the item 5.
For the item 6 you can use a subquery or maybe a LEFT JOIN limiting to one result.
For the latest item you can also use a subquery joining all the tags in a single result (separated by comma for instance).
I'm currently in the process of developing a site that amongst other things allows a user to filter a marketplace by showing or hiding items they have already purchased. This works on a basic AJAX call that passes through the current conditions of those filters available, and then using CodeIgniter's active record, it builds the appropriate query.
My issue is wrapping my head around the query so that if a user selects to hide purchased items the query omits / ignores any relevant records (i.e. if user_id = 5 and hide purchased is true, any scenes that user_id = 5 owns are not returned in the query).
Tbl: scenes
-------------------------------------------------------------------------
| design_id | scene_id | scene_name | ... [irrelevant columns to the Q] |
|-----------|----------|------------|-----------------------------------|
| 1 | 1 | welcome | |
| 1 | 2 | hello | |
| 2 | 3 | asd | |
-------------------------------------------------------------------------
The designs table is very similar to this and includes references to the game, game type, design name and so forth.
Tbl: user_scenes
----------------------------------------------------------------------
| design_id | scene_id | user_id | ... [irrelevant columns to the Q] |
|-----------|----------|---------|-----------------------------------|
| 1 | 1 | 5 | |
| 1 | 2 | 5 | |
| 1 | 1 | 9 | |
----------------------------------------------------------------------
Query
SELECT `designs`.`design_id`, `designs`.`design_name`, `scenes`.`scene_id`, `scenes`.`scene_name`, `scenes`.`scene_description`, `scenes`.`scene_unique_code`, `scenes`.`date_created`, `scenes`.`scene_cost`, `scenes`.`type`, `games`.`game_title`, `games`.`game_title_short`, `games_genres`.`genre`
FROM (`scenes`)
JOIN `designs` ON `designs`.`design_id` = `scenes`.`design_id`
JOIN `games` ON `designs`.`game_id` = `games`.`game_id`
JOIN `games_genres` ON `games`.`genre_id` = `games_genres`.`genre_id`
WHERE `scenes`.`private` = 0
ORDER BY `designs`.`design_name` asc, `scenes`.`scene_name` asc
LIMIT 6
The query uses CodeIgniter's active record ($this->db->select() / $this->db->where()) but that is somewhat irrelevant.
--
I've tried things like an INNER JOIN with user_scenes and then grouping by scene_id, but that presents an issue with only returning scenes that are present in user_scenes. I then made an attempt at a subquery but then questioned whether that was the correct route.
I understand there are other ways - looping through the returned data and querying whether that record exists for a specific user, but that I suspect would be highly inefficient. As such, I'm at a loss as to what to try and would appreciate any help.
I don't know if your setup permits it, but I would do a subselect:
Either via a NOT IN:
SELECT * FROM `scenes`
WHERE `scenes`.`scene_id` NOT IN (SELECT `scene_id` FROM `user_scenes` WHERE `user_id` = 5)
Or maybe via a LEFT JOIN:
SELECT * FROM `scenes`
LEFT JOIN (SELECT `scene_id`, `user_id` FROM `user_scenes` WHERE `user_id` = 5) AS `user_scenes`
ON `scenes`.`scene_id` = `user_scenes`.`scene_id`
WHERE `user_scenes`.`user_id` IS NULL
Bit I guess the first way is faster.
I have a scenario and i'm confused about how i can go about designing the database schema for it.
In my software (php)
there are companies and applications.
companies need to have licenses to access applications.
now the fields (for form while purchasing licenses) for each application is different.
for ex:
for application1:
fields are:
no of users
no of groups
for application2:
no of users
for application3:
number of hours of usage
Prices are based on these fields.
Now i need to design schema for this so that on one page company can manage licenses for all applications.
How can i make this schema generic?
Please help.
Thanks.
You can go with this type of structure
select * from applicationMaster
| APPID | APPNAME |
------------------------
| 1 | Application1 |
| 2 | Application2 |
ApplicationMaster will go with main Application related details which won't be repeated such Name, date etc.
Query 2:
select * from applicationField
| FIELDID | APPID | FIELDNAME |
---------------------------------
| 1 | 1 | NoOfUsers |
| 2 | 1 | NoOfGroups |
| 3 | 2 | NoHourusage |
ApplicationField can adjust any number of field for a particular appId.
So AppId 1 has 2 fields NoofUsers and NoOfGroups. It is also capable to adjust newer fields for a particular app if you want.
Query 3:
ApplicationValue will have the values for every license aplication so it will have compId which represents which company has applied using fieldId which refers to applicationField table we can get for which app values are stored.
select * from applicationValue
| ID | COMPID | FIELDID | FIELDVALUE |
--------------------------------------
| 1 | 1 | 1 | 50 |
| 2 | 1 | 2 | 150 |
| 3 | 2 | 3 | 350 |
| 4 | 3 | 1 | 450 |
| 5 | 3 | 2 | 50 |
applicationPriceMaster stores the price package for each application. There could be multiple package for a application.
select * from applicationPriceMaster
| APPPACKAGE | APPID | TOTALPRICE |
-----------------------------------
| 1 | 1 | 50 |
| 2 | 1 | 100 |
For each application package its details will posted in this table.
select * from applicationPriceDetail
| APPPACKAGE | FIELDID | QUANT |
--------------------------------
| 1 | 1 | 1 |
| 1 | 2 | 1 |
| 2 | 1 | 10 |
| 2 | 2 | 1 |
NOTE Please check the structure as it is now too complex and check what type of queries you would be running on these table and its performance.
select apm.APPPACKAGE, TOTALPRICE from
applicationPriceMaster apm
inner join
(select APPPACKAGE from applicationPriceDetail
where FIELDID=1 and QUANT=1)a
on apm.APPPACKAGE = a.APPPACKAGE
inner join
(select APPPACKAGE from applicationPriceDetail
where FIELDID=2 and QUANT=1)b
on
a.APPPACKAGE=b.APPPACKAGE
SQL FIDDLE:
| APPPACKAGE | TOTALPRICE |
---------------------------
| 1 | 50 |
For single filter you have to use this query, so you have to increase number of inner query with the number of inner filter.
select apm.APPPACKAGE, TOTALPRICE from
applicationPriceMaster apm
inner join
(select APPPACKAGE from applicationPriceDetail
where FIELDID=1 and QUANT=1)a
on apm.APPPACKAGE = a.APPPACKAGE
NOTE-This query is quite complex and will only work if the values are same as mentioned in the packagedetail table and will work only if the values are 2 filter you have to remove 1 inner join if there is only 1 filter. So I suggest you to reconsider before using this approach.
What you have there, could be easily mapped to Classes in an OO language (like PHP). You have an Abstract License, and then 3 Subclasses (ApplicationByUsersAndGroups, etc). Then, mapping to a Relational database is a very common problem, here is a nice article about it: http://www.ibm.com/developerworks/library/ws-mapping-to-rdb/
It has 3 options, it depends on the way you want to structure your application which one you should use. I recommend reading it, it is not that long.
One way is
Table LICENCES:
LICENSE_ID ==> UNIQUE IDENTIFIER
COMPANY_ID ==> references table COMPANIES
APPLICATION_ID ==> references table APPLICATIONS
LICENCE_TYPE ==> either of "BY_GROUPS_AND_USERS", "BY_USERS", "BY_HOURS"
LICENCE_BODY_ID ==> ID of specific body table
[...]
Table LIC_TYPE_BY_GROUPS_AND_USERS:
LICENCE_BODY_ID ==> body identifier
NO_GROUP
NO_USERS
[...]
Table LIC_TYPE_BY_USERS:
LICENCE_BODY_ID ==> body identifier
NO_USERS
[...]
This way, your intention is clear. Even after long time comming back, you will know in no time how things are organized, which fields are used in which case...
how about a table structured this way:
LicenseId int PK
CompanyId Int PK
AppId Int PK
LicenseType int
NumberOfUsers int
NumberOfGroups int
NumberOfHours int
Price Money
Depending on LicenseType, you will use different column in your business logic,
you might need to add CompanyID and/or AppID, that depends how you going to structure those tables as well as relation ships between company/app/license.
Some questions to think about:
Can one company have different License Types for same App?
Can one company have different Apps?
Dont complicate things, if the number of users is unlimited then set it to 999999 or some other max value.
This keeps the license check logic (which will run every time a user logs in ) simple and the same for all applications.
You will need extra logic in the licenses maintenance application, but this should also be pretty simple:
if the cost_per_user is = 0 then set no_of_users = 99999
Again you end up with the same licensing screen and logic for all your applications.
So let us say that I have a menu system with all the navigation items stored in a MySQL table like so:
Table: Menu
-------------------------------------------------------
| id | title | url | parent_id |
-------------------------------------------------------
| 1 | Home | /home | 0 |
| 2 | About | /about | 0 |
| 3 | History | /about/history | 2 |
| 4 | Location | /about/location | 2 |
| 5 | Staff | /about/staff | 2 |
| 6 | Articles | /blog | 0 |
| 7 | Archive | /blog/archive | 6 |
| 8 | Tags | /blog/tags | 6 |
| 9 | Tag Name 1 | /blog/tags/tag-name-1 | 8 |
| 10 | Tag Name 2 | /blog/tags/tag-name-2 | 8 |
-------------------------------------------------------
As you can see this table is quite simple with the only complication being the self referencing column parent_id, which defines how the menu should be nested.
So this would produce the following menu:
- Home
- About
- History
- Location
- Staff
- Articles
- Archive
- Tags
- Tag Name 1
- Tag Name 2
Is there a way to get this structure from the aforementioned table without making use of a recursive function in PHP (but it could be Python, Java or any other language) that queries the database with each iteration?
Ideally this could be handled with one MySQL query. Perhaps the table structure needs to be changed to accommodate this - if so how?
You could pull all of it out in one single pull, and then work with it recursively in PHP. That way you save some of the query time, but gain a little scripting time.
I would do something like this:
Get all data, ordered by parent id
Put row into $data[$parent_id][]
define function to build menu, takes one param which is id
get $data[$id] and work with that array, building the array.
while looping through the items, check if size of $data[current-item-id] > 0
if so, call above function with 0 as param
This way, you only query the database once, but use a little more of the servers ram.
If you're fetching the whole tree and you can't or don't want to change the table structure, take a look at https://stackoverflow.com/a/8325451/4833
This can be done in sql query, take a look at this resource which explains recursion in a query
http://www.artfulsoftware.com/mysqlbook/sampler/mysqled1ch20.html.
MySQL don't have an default function to do that.
You can make an procedure with loop to get the data result you want, or create an function and use in your sql select.
Anyway you will use loop.
Example:
DROP PROCEDURE IF EXISTS famsubtree;
DELIMITER go
CREATE PROCEDURE famsubtree( root INT )
BEGIN
DROP TABLE IF EXISTS famsubtree;
CREATE TABLE famsubtree
SELECT childID, parentID, 0 AS level
FROM familytree
WHERE parentID = root;
ALTER TABLE famsubtree ADD PRIMARY KEY(childID,parentID);
REPEAT
INSERT IGNORE INTO famsubtree
SELECT f.childID, f.parentID, s.level+1
FROM familytree AS f
JOIN famsubtree AS s ON f.parentID = s.childID;
UNTIL Row_Count() = 0 END REPEAT;
E ND ;
go
DELIMITER ;
And use to query:
call famsubtree(1); -- from the root you can see forever
SELECT Concat(Space(level),parentID) AS Parent, Group_Concat(childID ORDER BY childID) AS Child
FROM famsubtree
GROUP BY parentID;