I have two table
one table is alldata ( here info_id is a text field data inserted using php )
=================
id | info_id
=================
1 | 2, 3, 5, 9
2 |
=================
second table is info
=================
id | name
=================
1 | one
2 | two
3 | three
4 | four
5 | five
6 | six
7 | seven
9 | eight
9 | nine
=================
now I want to select list of data from table two where data id will be matched with table one first item info_id data
my query is
SELECT i.* FROM `info` as i,`alldata` as a where i.id IN(a.info_id) and a.id=1
my query works but select only one item from table two.But there are multiple matched.
You have a very poor database design. First, storing numeric ids as strings is a bad idea -- numbers should be stored as numbers. Second, SQL offers this great data structure for storing lists. It is called a table, not a string.
You should really have a junction table, one one row per id and info_id.
That said, sometimes we a struck with substandard data structure. MySQL offers support for this. You can use:
SELECT i.*
FROM `info` i JOIN
`alldata` a
ON FIND_IN_SET(i.id, REPLACE(a.info_id, ', ', ',') ) > 0
WHERE a.id = 1;
You should also learn to use proper, explicit join syntax. If you use this method, instead of fixing the database design, you are not allowed to complain about performance. MySQL cannot take advantage of things like indexes to improve the performance of this type of query.
Related
I have a table with all my invoice items as packages:
Table: invoice_items
invoice_item_id | package_id | addon_1 | addon_2 | addon_3 | ...
----------------|------------|---------|---------|
1 | 6 | 2 | 5 | 3 |
Then my other table:
Table: addons
addon_id | addon_name | addon_desc |
----------|--------------|--------------------------|
1 | Dance Lights | Brighten up the party... |
2 | Fog Machine | Add some fog for an e... |
Instead of taking up space storing the addon name in my invoice_items table, I'd like to just include the addon_id in the addon_1, addon_2, etc columns.
How do I then get the name of the addon when doing a query for invoice_item rows?
Right now I just have it programmed into the page that if addon_id == 1, echo "Dance Lights", etc but I'd like to do it in the query. Here is my current query:
$invoice_items_SQL = "
SELECT invoice_items.*, packages.*
FROM `invoice_items`
INNER JOIN packages ON invoice_items.invoice_item_id = packages.package_id
WHERE `event_id` = \"$event_id\"
";
So I'm able to do this with packages, but only because there's just one package_id per row, but there are up to 9 addons :(
The most direct way of doing it is to join onto the table multiple times. That's a bit naff though because you'll write almost the same thing 9 times.
Another, better way would be to restructure your tables - you need another table with 2 data columns: invoice_id and addon_id. You then need either an auto-inc primary column, or use both of those existing columns as a dual primary key. So this is a many-to-many junction table.
From there you can can query without having 9 repetitive joins, but you will get a row of each package for every addon it has (so if it has three addons it will appear three times in the results). And then from there you can use GROUP_CONCAT to concatenate the names of the addons into a single field so that you only get one row per invoice.
My table looks like this:
+------------------------+
| id | title | position |
+------------------------+
| 1 | test 2 | 3 |
+------------------------+
| 2 | test 3 | 1 |
+------------------------+
| 3 | test 1 | 0 |
+------------------------+
I found this query which retrieves the rows ordered based on the position field which holds the id of the predecessor.
SELECT
*
FROM
mytable AS t1
LEFT JOIN
mytable AS t2
ON t2.position = t1.id
I wonder why this is working because there is no order by clause and the database should't know that position 0 is the row to start at.
The result is dependent on the order you inserted the rows into the table. If, for example, you had inserted the row with id=3 before you inserted the row with id=2, then you would have got a non-sorted result.
As it stands, you are pulling the data out of t1 in the order of id because that is the order you put the elements into the table
See http://sqlfiddle.com/#!2/63a925/2 and try it for yourself.
N.B. Databases are not guaranteed to work as you state, it is simply that most databases work this way. You should not rely on this behaviour as a minor change to the schema or query could ruin your whole day! Note also that if id is a (primary?) key, the insert order will probably be overridden by the fact that the database will pull the rows out in the order of the index.
That query is joining in table 2 based on the ID in table 1 equaling the position in table 2. Since the IDs in table 1 are sequential, the output appears to be sorted
I have a table which contains a standard auto-incrementing ID, a type identifier, a number, and some other irrelevant fields. When I insert a new object into this table, the number should auto-increment based on the type identifier.
Here is an example of how the output should look:
id type_id number
1 1 1
2 1 2
3 2 1
4 1 3
5 3 1
6 3 2
7 1 4
8 2 2
As you can see, every time I insert a new object, the number increments according to the type_id (i.e. if I insert an object with type_id of 1 and there are 5 objects matching this type_id already, the number on the new object should be 6).
I'm trying to find a performant way of doing this with huge concurrency. For example, there might be 300 inserts within the same second for the same type_id and they need to be handled sequentially.
Methods I've tried already:
PHP
This was a bad idea but I've added it for completeness. A request was made to get the MAX() number for the item type and then add the number + 1 as part of an insert. This is quick but doesn't work concurrently as there could be 200 inserts between the request for MAX() and that particular insert leading to multiple objects with the same number and type_id.
Locking
Manually locking and unlocking the table before and after each insert in order to maintain the increment. This caused performance issues due to the number of concurrent inserts and because the table is constantly read from throughout the app.
Transaction with Subquery
This is how I'm currently doing it but it still causes massive performance issues:
START TRANSACTION;
INSERT INTO objects (type_id,number) VALUES ($type_id, (SELECT COALESCE(MAX(number),0)+1 FROM objects WHERE type_id = $type_id FOR UPDATE));
COMMIT;
Another negative thing about this approach is that I need to do a follow up query in order to get the number that was added (i.e. searching for an object with the $type_id ordered by number desc so I can see the number that was created - this is done based on a $user_id so it works but adds an extra query which I'd like to avoid)
Triggers
I looked into using a trigger in order to dynamically add the number upon insert but this wasn't performant as I need to perform a query on the table I'm inserting into (which isn't allowed so has to be within a subquery causing performance issues).
Grouped Auto-Increment
I've had a look at grouped auto-increment (so that the number would auto-increment based on type_id) but then I lose my auto-increment ID.
Does anybody have any ideas on how I can make this performant at the level of concurrent inserts that I need? My table is currently InnoDB on MySQL 5.5
Appreciate any help!
Update: Just in case it is relevant, the objects table has several million objects in it. Some of the type_id can have around 500,000 objects assigned to them.
Use transaction and select ... for update. This will solve concurrency conflicts.
In Transaction with Subquery
Try to make index on column type_id
I think by making index on column type_id it will speed up your subquery.
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
,type_id INT NOT NULL
);
INSERT INTO my_table VALUES
(1,1),(2,1),(3,2),(4,1),(5,3),(6,3),(7,1),(8,2);
SELECT x.*
, COUNT(*) rank
FROM my_table x
JOIN my_table y
ON y.type_id = x.type_id
AND y.id <= x.id
GROUP
BY id
ORDER
BY type_id
, rank;
+----+---------+------+
| id | type_id | rank |
+----+---------+------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 4 | 1 | 3 |
| 7 | 1 | 4 |
| 3 | 2 | 1 |
| 8 | 2 | 2 |
| 5 | 3 | 1 |
| 6 | 3 | 2 |
+----+---------+------+
or, if performance is an issue, just do the same thing with a couple of #variables.
Perhaps an idea to create a (temporary) table for all rows with a common "type_id".
In that table you can use auto-incrementing for your num colomn.
Then your num shoud be fully trustable.
Then you can select your data and update your first table.
I have a table witch has 45 columns but only a few of these are yet completed. This table is continuously updated and added etc. In my auto-complete function i want to select these records ordered by the most completed fields(i hope you understand)?
One of the solutions is to create another filed (the "rank" field) and create a php function that selects * the records and gives a rank for each record.
... but i was wondering if there is a more simple way of doing this only whit a single ORDER BY?
MySQL has no function to count the number of non-NULL fields on a row, as far as I know.
So the only way I can think of is to use an explicit condition:
SELECT * FROM mytable
ORDER BY (IF( column1 IS NULL, 0, 1)
+IF( column2 IS NULL, 0, 1)
...
+IF( column45 IS NULL, 0, 1)) DESC;
...it is ugly as sin, but should do the trick.
You could also devise a TRIGGER to increment an extra column "fields_filled". The trigger costs you on UPDATE, the 45 IFs hurt you on SELECT; you'll have to model what is more convenient.
Note that indexing all fields to speed up SELECT will cost you when updating (and 45 different indexes probably cost as much as a table scan on select, not to say that the indexed field is a VARCHAR). Run some tests, but I believe that the 45-IF solution is likely to be the best overall.
UPDATE:
If you can rework your table structure to normalize it somewhat, you could put the fields in a my_values table. Then you would have a "header table" (maybe with only a unique ID) and a "data table". Empty fields would not exist at all, and then you could sort by how many filled fields are there by using a RIGHT JOIN, counting the filled fields with COUNT(). This would also greatly speed up UPDATE operations, and would allow you to efficiently employ indexes.
EXAMPLE (from table setup to two normalized tables setup):
Let us say we have a set of Customer records. We will have a short subset of "mandatory" data such as ID, username, password, email, etc.; then we will have a maybe much larger subset of "optional" data such as nickname, avatar, date of birth, and so on. As a first step let us assume that all these data are varchar (this, at first sight, looks like a limitation when compared to the single table solution where each column may have its own datatype).
So we have a table like,
ID username ....
1 jdoe etc.
2 jqaverage etc.
3 jkilroy etc.
Then we have the optional-data table. Here John Doe has filled all fields, Joe Q. Average only two, and Kilroy none (even if he was here).
userid var val
1 name John
1 born Stratford-upon-Avon
1 when 11-07-1974
2 name Joe Quentin
2 when 09-04-1962
In order to reproduce the "single table" output in MySQL we have to create a quite complex VIEW with lots of LEFT JOINs. This view will nonetheless be very fast if we have an index based on (userid, var) (even better if we use a numeric constant or a SET instead of a varchar for the datatype of var:
CREATE OR REPLACE VIEW usertable AS SELECT users.*,
names.val AS name // (1)
FROM users
LEFT JOIN userdata AS names ON ( users.id = names.id AND names.var = 'name') // (2)
;
Each field in our logical model, e.g., "name", will be contained in a tuple ( id, 'name', value ) in the optional data table.
And it will yield a line of the form <FIELDNAME>s.val AS <FIELDNAME> in the section (1) of the above query, referring to a line of the form LEFT JOIN userdata AS <FIELDNAME>s ON ( users.id = <FIELDNAME>s.id AND <FIELDNAME>s.var = '<FIELDNAME>') in section (2). So we can construct the query dynamically by concatenating the first textline of the above query with a dynamic Section 1, the text 'FROM users ' and a dynamically-built Section 2.
Once we do this, SELECTs on the view are exactly identical to before -- but now they fetch data from two normalized tables via JOINs.
EXPLAIN SELECT * FROM usertable;
will tell us that adding columns to this setup does not slow down appreciably operations, i.e., this solution scales reasonably well.
INSERTs will have to be modified (we only insert mandatory data, and only in the first table) and UPDATEs as well: we either UPDATE the mandatory data table, or a single row of the optional data table. But if the target row isn't there, then it must be INSERTed.
So we have to replace
UPDATE usertable SET name = 'John Doe', born = 'New York' WHERE id = 1;
with an 'upsert', in this case
INSERT INTO userdata VALUES
( 1, 'name', 'John Doe' ),
( 1, 'born', 'New York' )
ON DUPLICATE KEY UPDATE val = VALUES(val);
(We need a UNIQUE INDEX on userdata(id, var) for ON DUPLICATE KEY to work).
Depending on row size and disk issues, this change might yield an appreciable performance gain.
Note that if this modification is not performed, the existing queries will not yield errors - they will silently fail.
Here for example we modify the names of two users; one does have a name on record, the other has NULL. The first is modified, the second is not.
mysql> SELECT * FROM usertable;
+------+-----------+-------------+------+------+
| id | username | name | born | age |
+------+-----------+-------------+------+------+
| 1 | jdoe | John Doe | NULL | NULL |
| 2 | jqaverage | NULL | NULL | NULL |
| 3 | jtkilroy | NULL | NULL | NULL |
+------+-----------+-------------+------+------+
3 rows in set (0.00 sec)
mysql> UPDATE usertable SET name = 'John Doe II' WHERE username = 'jdoe';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> UPDATE usertable SET name = 'James T. Kilroy' WHERE username = 'jtkilroy';
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
mysql> select * from usertable;
+------+-----------+-------------+------+------+
| id | username | name | born | age |
+------+-----------+-------------+------+------+
| 1 | jdoe | John Doe II | NULL | NULL |
| 2 | jqaverage | NULL | NULL | NULL |
| 3 | jtkilroy | NULL | NULL | NULL |
+------+-----------+-------------+------+------+
3 rows in set (0.00 sec)
To know the rank of each row, for those users that do have a rank, we simply retrieve the count of userdata rows per id:
SELECT id, COUNT(*) AS rank FROM userdata GROUP BY id
Now to extract rows in "filled status" order, we do:
SELECT usertable.* FROM usertable
LEFT JOIN ( SELECT id, COUNT(*) AS rank FROM userdata GROUP BY id ) AS ranking
ON (usertable.id = ranking.id)
ORDER BY rank DESC, id;
The LEFT JOIN ensures that rankless individuals get retrieved too, and the additional ordering by id ensures that people with identical rank always come out in the same order.
Ok, so I have an organization id column named org_id in several databases.
I am writing a search function that will query two of these databases, and look for all org_id's associated with a value in these two tables.
I ONLY want to ultimately have the org_id's that correspond with the values I am querying for in EACH table.
For example:
Say I have an org_id of 3 that is in the tables cult_xref and cat_xref, which are both associated with an organization table, which isn't really relevant for this. I want to ONLY pull those org_id's which are in BOTH tables cult_xref and cat_xref, based on values I put into those tables, say 2 and 6 respectively.
So:
cult_xref
org_id | cult_id
3 | 2
4 | 2
3 | 5
and
cat_xref
org_id | cat_id
3 | 6
3 | 1
7 | 6
I would only want to pull the org_id's that fulfill cult_id['2'] and cat_id['6'] at the SAME TIME.
In SQL, this is called a JOIN.
SELECT org_id
FROM cult_xref INNER JOIN cat_xref USING (org_id)
WHERE (cult_id,cat_id) = (2,6)
For a nice introduction to joins, see A Visual Explanation of SQL Joins