I have a database and an external file. What these two share are reference codes for products.
But in the external file I have all my reference codes saved down, whilst plenty are still missing in the database. Is there a way to make a query so that I can check what values are missing in my database, in a given table?
There's no need to worry about how the XML interfaces with the database. I already have that down through PHP and simplexml. I am mostly struggling with the query to use in this case.
Database
XML File
AJS2S
AJS2S
ABBB2
ABBB2
JJI90K
JJJJ92
If you have a list of valus at hand and you want to check which ones are missing in your table, you enumerate them in a union all subquery, then use not exists:
select x.product_code
from (
select 'AJS2S' as product_code
union all select 'ABBB2'
union all ...
) x
where not exists (select 1 from mytable t where t.product_code = x.product_code)
Or, in very recent versions of MySQL (8.0.19 or higher), you can use the values() row constructor:
select x.product_code
from (values row('AJS2S'), row('ABBB2'), ...) x(product_code)
where not exists (select 1 from mytable t where t.product_code = x.product_code)
Of course, if you have your xml data already loaded in a table, say xmltable, then you can use that instead of the subquery:
select x.product_code
from xmltable x(product_code)
where not exists (select 1 from mytable t where t.product_code = x.product_code)
You would use not exists:
select code
from xml
where not exists (select 1 from database d where d.code = xml.code);
This retrieves each code -- so it might have many duplicates. You can summarize using group by:
select code, count(*)
from xml
where not exists (select 1 from database d where d.code = xml.code)
group by code;
I am building a website as a diagnostic aid for neurological conditions. It is coded in html and communicates with a MySQL database via PHP. The primary table which feeds information to the website is structured as follows:
Image showing table structure with rows representing Neurological Conditions and columns providing information on symptoms associated with these conditions
The table above can be reproduced using the following MySQL code:
CREATE TABLE IF NOT EXISTS my_table (
`Condition` VARCHAR(22) CHARACTER SET utf8,
`Diarrhoea` INT,
`Headache` INT,
`Hyporeflexia` INT,
`Hypoaesthesia_Spinothalamic` INT
);
INSERT INTO my_table VALUES
('Abetalipoproteinaemia',1,NULL,1,NULL),
('Caffeine toxicity',1,1,NULL,NULL),
('Vitamin B12 deficiency',NULL,NULL,1,2);
SELECT * FROM my_table;
Cell values are as follows:
(m,n)=1 if condition and symptom are associated
(m,n)=2 if condition and symptom CANNOT be associated. The presence of this symptom excludes the condition as a possible diagnosis.
(m,n)=null if no information exists or if symptom and condition are not associated
I'm struggling to write an SQL query which will identify all the columns (n) for a specific condition (m) where the value of the cell (m,n) = 2.
So far my reading has highlighted ideas about pivot tables (I can't see how I would be able to use them for this problem) and database normalisation which I don't think is possible because of the other queries I am running on the same table.
An example based on the table above:
Patient presents with hyporeflexia
SQL query identifies this could be cause by either "abetalipoproteinaemia" or "vitamin B12 deficiency" - this all works fine already
I want to establish whether any of the conditions identified (abetalipoproteinaemia and vitamin B12 deficiency) have symptoms that would exclude the diagnosis (any cell in that row = 2) and return the name of any column (symptom) for which this is the case.
A query to the SQL database identifies vitamin B12 deficiency would be excluded as a possible diagnosis if spinothalamic hypoesthesia is present - this will be fed back to the html display.
Any help would be much appreciated - thanks for your time!
I think it would be more usual to arrange the data something like this - apologies for any spelling errors or poor terminology, but if you pay peanuts...
syptom condition exclusion
Abetalipoproteinaemia Diarrhoea 0
Abetalipoproteinaemia Hyporeflexia 0
Caffeine toxicity Diarrhoea 0
Caffeine toxicity Headache 0
Vitamin B12 deficiency Hyporeflexia 0
Vitamin B12 deficiency Hypoaesthesia Spinthalamic 1
You would then take this one or two steps further, and have a table for symptoms, a table for conditions, and a table which says which symptom relates to which condition, and how.
Query pattern would be much more straightforward if the table were designed following normative relational patterns.
Consider the resultset returned by a query of this form:
SELECT v.condition
, v.symptom
, v.associated_or_excluded
FROM ( SELECT t1.`Condition` AS `condition`
, 'Diarrhoea' AS `symptom`
, t1.`Diarrhoea` AS `associated_or_excluded`
FROM mytable t1
UNION ALL
SELECT t2.`Condition`
, 'Headache'
, t2.`Headache`
FROM mytable t2
UNION ALL
SELECT t3.`Condition`
, 'Hyporeflexia'
, t3.`Hyporeflexia`
FROM mytable t3
UNION ALL
SELECT t4.`Condition`
, 'Hypoaesthesia_Spinothalamic'
, t4.`Hypoaesthesia_Spinothalamic`
FROM mytable t4
) v
We could use that query as an inline view (a rowsource) for an outer query, or a new table could be populated with the result from this query INSERT ... SELECT to convert.
With that resultset, with the data in standard relational form, we avoid the struggle by writing a simple query like this:
SELECT t.symptom
FROM ( ... ) t
WHERE t.condition = 'Hyporeflexia'
AND t.associated_or_excluded = 2
that will return symptoms that are excluded from a particular condition.
(or, to put it in terms of the original question, where a value of 2 is found the intersection of m and n)
Note that ( ... ) is replaced with a table name or with an inline view returning the result from query above.
Note that the entirety of the "struggle" is inside the parens, with the inline view query that gets the data represented in a suitable form.
SELECT t.symptom
FROM ( -- inline view query
SELECT t1.`Condition` AS `condition`
, 'Diarrhoea' AS `symptom`
, t1.`Diarrhoea` AS `associated_or_excluded`
FROM mytable t1
UNION ALL
SELECT t2.`Condition`
, 'Headache'
, t2.`Headache`
FROM mytable t2
UNION ALL
SELECT t3.`Condition`
, 'Hyporeflexia'
, t3.`Hyporeflexia`
FROM mytable t3
UNION ALL
SELECT t4.`Condition`
, 'Hypoaesthesia_Spinothalamic'
, t4.`Hypoaesthesia_Spinothalamic`
FROM mytable t4
) t
WHERE t.condition = 'Hyporeflexia'
AND t.associated_or_excluded = 2
I have an sql query that simply adds a row to an existing database table. The first field is id and I assume this value has to be specified in the query.
So then id has to be the number of rows existing + 1. I'm determining the id like this:
SELECT COUNT(1) FROM testtable
The problem is that this returns the number of rows that have ever been added, including deleted ones. Because I have been adding and removing from this table, this number is greater than the number of EXISTING rows in the table which is what I want.
How can I count the existing rows in the table instead of the existing + deleted rows?
If possible switch to using an auto_increment column for your id and don't be concerned with gaps in the sequence of ids. Your own implementation of id generation may inflict more harm especially in a long run.
Now, back to your immediate question. You are probably looking for this
SELECT MAX(id) + 1 new_id
FROM Table1
Note: This query might fail under heavy load when several concurrent sessions issuing this
query might grab the same id and your subsequent INSERT will fail. Therefore again consider using an auto_increment for your id.
Here is SQLFiddle demo
That's a bad idea, because you can end up with duplicate IDs, especially if you delete rows in the middle. If you're not going to use an auto-increment field, you can add the ID to the insert. Just use this in place of the value:
((SELECT MAX(t.id) + 1 FROM table t)
The full query would then be:
INSERT INTO table_name (id, col1, col2, col3) VALUES ((SELECT MAX(t.id) + 1 FROM table_name t), :col1, :col2, :col3)
SELECT COUNT(*) from table
will return only the number of entries in it, not based on the ID
There is also possibility to find, and fill the gaps, by picking first free id
SELECT MIN(t1.id + 1) AS free_id
FROM t1
WHERE NOT EXISTS (SELECT t2.id FROM t2 WHERE t2.id = t1.id + 1)
I have a column in one of my table where I store multiple ids seperated by comma's.
Is there a way in which I can use this column's value in the "IN" clause of a query.
The column(city) has values like 6,7,8,16,21,2
I need to use as
select * from table where e_ID in (Select city from locations where e_Id=?)
I am satisfied with Crozin's answer, but I am open to suggestions, views and options.
Feel free to share your views.
Building on the FIND_IN_SET() example from #Jeremy Smith, you can do it with a join so you don't have to run a subquery.
SELECT * FROM table t
JOIN locations l ON FIND_IN_SET(t.e_ID, l.city) > 0
WHERE l.e_ID = ?
This is known to perform very poorly, since it has to do table-scans, evaluating the FIND_IN_SET() function for every combination of rows in table and locations. It cannot make use of an index, and there's no way to improve it.
I know you said you are trying to make the best of a bad database design, but you must understand just how drastically bad this is.
Explanation: Suppose I were to ask you to look up everyone in a telephone book whose first, middle, or last initial is "J." There's no way the sorted order of the book helps in this case, since you have to scan every single page anyway.
The LIKE solution given by #fthiella has a similar problem with regards to performance. It cannot be indexed.
Also see my answer to Is storing a delimited list in a database column really that bad? for other pitfalls of this way of storing denormalized data.
If you can create a supplementary table to store an index, you can map the locations to each entry in the city list:
CREATE TABLE location2city (
location INT,
city INT,
PRIMARY KEY (location, city)
);
Assuming you have a lookup table for all possible cities (not just those mentioned in the table) you can bear the inefficiency one time to produce the mapping:
INSERT INTO location2city (location, city)
SELECT l.e_ID, c.e_ID FROM cities c JOIN locations l
ON FIND_IN_SET(c.e_ID, l.city) > 0;
Now you can run a much more efficient query to find entries in your table:
SELECT * FROM location2city l
JOIN table t ON t.e_ID = l.city
WHERE l.e_ID = ?;
This can make use of an index. Now you just need to take care that any INSERT/UPDATE/DELETE of rows in locations also inserts the corresponding mapping rows in location2city.
From MySQL's point of view you're not storing multiple ids separated by comma - you're storing a text value, which has the exact same meaing as "Hello World" or "I like cakes!" - i.e. it doesn't have any meaing.
What you have to do is to create a separated table that will link two objects from the database together. Read more about many-to-many or one-to-many (depending on your requirements) relationships in SQL-based databases.
Rather than use IN on your query, use FIND_IN_SET (docs):
SELECT * FROM table
WHERE 0 < FIND_IN_SET(e_ID, (
SELECT city FROM locations WHERE e_ID=?))
The usual caveats about first form normalization apply (the database shouldn't store multiple values in a single column), but if you're stuck with it, then the above statement should help.
This does not use IN clause, but it should do what you need:
Select *
from table
where
CONCAT(',', (Select city from locations where e_Id=?), ',')
LIKE
CONCAT('%,', e_ID, ',%')
but you have to make sure that e_ID does not contain any commas or any jolly character.
e.g.
CONCAT(',', '6,7,8,16,21,2', ',') returns ',6,7,8,16,21,2,'
e_ID=1 --> ',6,7,8,16,21,2,' LIKE '%,1,%' ? FALSE
e_ID=6 --> ',6,7,8,16,21,2,' LIKE '%,6,%' ? TRUE
e_ID=21 --> ',6,7,8,16,21,2,' LIKE '%,21,%' ? TRUE
e_ID=2 --> ',6,7,8,16,21,2,' LIKE '%,2,%' ? TRUE
e_ID=3 --> ',6,7,8,16,21,2,' LIKE '%,3,%' ? FALSE
etc.
Don't know if this is what you want to accomplish. With MySQL there is feature to concatenate values from a group GROUP_CONCAT
You can try something like this:
select * from table where e_ID in (Select GROUP_CONCAT(city SEPARATOR ',') from locations where e_Id=?)
this one in for oracle ..here string concatenation is done by wm_concat
select * from table where e_ID in (Select wm_concat(city) from locations where e_Id=?)
yes i agree with raheel shan .. in order put this "in" clause we need to make that column into row below code one do that job.
select * from table where to_char(e_ID)
in (
select substr(city,instr(city,',',1,rownum)+1,instr(city,',',1,rownum+1)-instr(city,',',1,rownum)-1) from
(
select ','||WM_CONCAT(city)||',' city,length(WM_CONCAT(city))-length(replace(WM_CONCAT(city),','))+1 CNT from locations where e_Id=? ) TST
,ALL_OBJECTS OBJ where TST.CNT>=rownum
) ;
you should use
FIND_IN_SET Returns position of value in string of comma-separated values
mysql> SELECT FIND_IN_SET('b','a,b,c,d');
-> 2
You need to "SPLIT" the city column values. It will be like:
SELECT *
FROM table
WHERE e_ID IN (SELECT TO_NUMBER(
SPLIT_STR(city /*string*/
, ',' /*delimiter*/
, 1 /*start_position*/
)
)
FROM locations);
You can read more about the MySQL split_str function here: http://blog.fedecarg.com/2009/02/22/mysql-split-string-function/
Also, I have used the TO_NUMBER function of Oracle here. Please replace it with a proper MySQL function.
IN takes rows so taking comma seperated column for search will not do what you want but if you provide data like this ('1','2','3') this will work but you can not save data like this in your field whatever you insert in the column it will take the whole thing as a string.
You can create a prepared statement dynamically like this
set #sql = concat('select * from city where city_id in (',
(select cities from location where location_id = 3),
')');
prepare in_stmt from #sql;
execute in_stmt;
deallocate prepare in_stmt;
Ref: Use a comma-separated string in an IN () in MySQL
Recently I faced the same problem and this is how I resolved it.
It worked for me, hope this is what you were looking for.
select * from table_name t where (select (CONCAT(',',(Select city from locations l where l.e_Id=?),',')) as city_string) LIKE CONCAT('%,',t.e_ID,',%');
Example: It will look like this
select * from table_name t where ',6,7,8,16,21,2,' LIKE '%,2,%';
i want to sort my table in this manner.
This is the current state of my table:
And this would be the outcome:
So basically in pseudo code it's:
IF SAME STRING IN "tag" COLUMN THEN add both "power"
Can anyone suggest an efficient way to achieve this if any?
select id, sum(power), tag from YourTabeName group by tag
If you need to query from it I would just make it a view.
Create a new table called t1_temp with the exact same structure as your original table called t1. The following should get the job done:
TRUNCATE t1_temp; INSERT INTO t1_temp (id, power, tag) VALUES (SELECT id, SUM(power), tag FROM t1 GROUP BY tag); RENAME TABLE t1 TO t1_temp2, t1_temp TO t1, t1_temp2 TO t1_temp
However, I will recommend that you try to modify your insert statement. Create a unique index on tag and then use something like the following (assuming that id is auto-increment)
INSERT INTO t1 (power, tag) VALUES (3, 'option1') ON DUPLICATE KEY UPDATE power = power + 3
Even though you've already solved this is another approach
update your_table as t1
inner join
(select tag,sum(power) as power from your_table group by tag) as t2
set t1.power = t2.power
where t1.tag = t2.tag
alter ignore table your_table add unique index i (tag);