Select IDs not in database - php
I have an array of products IDs and I need to find the IDs that are not present in the database. Sure I can do it like this:
// IDs in database: 1, 2, 3, 4, 5.
$products = array(4, 5, 6, 7);
$in_db = $db->exec("SELECT `id` FROM `table` WHERE `id` IN (" . implode($products) . ");");
$in_db_ids = array();
foreach($in_db as $i){
$in_db_ids[] = $i['id'];
}
$missing = array_diff($products, $in_db_ids);
But this is long and boring. I thought also about something like this:
// this query would be generated with PHP using an implode or something
$query = "
SELECT `t`.`id` as `missing`
SELECT 4 as `id` UNION SELECT 5 UNION SELECT 6 UNION SELECT 7
LEFT JOIN `table` USING(`id`)
WHERE `missing` is NULL;
";
$missing = $db->exec($query);
But this is so inelegant.
I think there should be a proper way to write that SELECT x UNION SELECT y UNION SELECT z, or there may be another nice way to do this check.
What do you think, guys?
To do it in the MySQL database, you will need a table or a temporary table that has a number of rows equal to the highest index in the table you are scanning for missing blanks. There is no way around this -- MySQL always "loops" through tables, it isn't a procedural language where you can set up your own loop.
That being said, if you create or have a table of sufficient size, you can create the natural number sequence using a user variable. Let's call the big table bigtable (it doesn't matter what columns are in it - we just need a name of one of the columns - I use 'acolumn').
set #row = 0;
select n from
(select #row := #row + 1 as n from bigtable) as t
left join mytable on mytable.id = t.n
where mytable.acolumn is null;
Let me know if this needs a little more explanation.
Related
SELECT count rows from 2 tables and sum the results
I have the PHP code: $query = mysqli_query($mysqli, "SELECT * FROM `table_1`"); $result = mysqli_num_rows($query); $queryTwo = mysqli_query($mysqli, "SELECT * FROM `table_2`"); $resultTwo = mysqli_num_rows($queryTwo); $number = $result + $resultTwo; return $number; The point is that sometimes, the $number variable is returning NULL, when it should not supposed to do that. I have always rows in those 2 tables, and the returned result should not be NULL, ever. Is this a correct approach to sum the number of rows from 2 tables? I don`t understand why sometimes I get NULL instead of a number.
Well, I would suggest you to do it like select ( select count(*) from Table1 ) + ( select count(*) from Table2 ) as total_rows executing this query and getting the value of total_rows will return you true result Or you can create a stored procedure to do the same thing. as explained below CREATE PROCEDURE sp_Test AS -- Create two integer values DECLARE #tableOneCount int, #tableTwoCount int -- Get the number of rows from the first table SELECT #tableOneCount = (SELECT COUNT(*) FROM Table1 WHERE WhereClause) SELECT #tableTwoCount = (SELECT COUNT(*) FROM Table2 WHERE WhereClause) -- Return the sum of the two table sizes SELECT TotalCount = #tableOneCount + #tableTwoCount
Why don't you go with only one query like this: you will have the result directly in one step and it will avoid contacting the DB twice to fetch intermediate result and it will also simplify your program! SELECT (select count(*) from table_1) + (select count(*) from table_2)
Which Clause to use instead of IN Clause in MYSQL
My Question is Which Clause use instead of IN Clause in MYSQL with PHP Because IN Clause Limit 1024 character. My Character limit exceed to 1024 character. SELECT * FROM TblUser WHERE Status != 'Deleted' AND UserId IN (0,10,11,12,13,14,15,22,45,114,144,155,156,167,211,439,440,441,443,445,450,455,456,457,458,459,1111,1154,1156,1165,1451,1541,11111,11112,11113,11114,11115,11116,11117,11118,11119,11656,15451,16561,17671,18781,33131,33311,33411,54511,111110,111111,111112,111113,111114,111115,111116,111117,111118,111119,111120,111121,111122,111123,111124,111125,111126,111127,111128,111129,111130,111131,111132,111133,111134,111135,111136,111137,111138,111139,111140,111141,111142,111143,111144,111145,111146,111147,111148,111149,111150,111151,111152,111153,111154,111155,111156,111157,111158,111159,111160,111161,111162,111163,111164,111165,111166,111167,111168,111169,111170,111171,111172,111173,111174,111175,111176,111177,111178,111179,111180,111181,111182,111183,111184,111185,111186,111187,111188,111189,111190,111191,111192,111193,111194,111195,111196,111197,111198,111199,1111100,11112101,11112102,11112103,11112104,11112105,11112106,11112107,11112108,11112109,11112110,11112111,11112112,11112113,11112114,11112115,11112116,11112117,11112118,11112119,11112120,11112121,11112122,11112123,11112124,11112125,11112126,11112127,11112128,11112129,11112130,11112131,11112132,11112133,11112134,11112135,11112136,11112137,11112138,11112139,11112140,11112141,11112142,11112143,11112144,11112145,11112146,11112147,11112148,11112149,11112150,11112151,11112152,11112153,11112154,11112155,11112156,11112157,11112158,11112159,11112160,11112161,11112162,11112163,11112164,11112165,11112166,11112167,11112168,11112169,11112170,11112171,11112172,11112173,11112174,11112175,11112176,11112177,11112178,11112179,11112180,11112181,11112182,11112183,11112184,11112185,11112186,11112187,11112188,11112189,11112190,11112191,11112192,11112193,11112194,11112195,11112196,11112197,11112198,11112199,11112200); Please help Which Clause use instead of IN Clause in MYSQL with PHP?
Is the list of values that are used in the IN statement always the same, or is that list the result of some other query? If the latter is the case, you could use that query as a subquery: SELECT * FROM tbluser WHERE status != 'Deleted' AND userid IN ( SELECT userid FROM sometable WHERE ... )
Yes you are limited within an IN statement, one thing you could do is create a temporary table which stores the values. CREATE TEMPORARY TABLE IF NOT EXISTS temp AS (SELECT userid FROM tbluser); In PHP create your INSERT INTO script: $str = '0,10,11,12,13,14,15,22,45,114,144,155,156,167,211,439,440,441,443,445,450,455,456,457,458,459,1111,1154,1156,1165,1451,1541,11111,11112,11113,11114,11115,11116,11117,11118,11119,11656,15451,16561,17671,18781,33131,33311,33411,54511,111110,111111,111112,111113,111114,111115,111116,111117,111118,111119,111120,111121,111122,111123,111124,111125,111126,111127,111128,111129,111130,111131,111132,111133,111134,111135,111136,111137,111138,111139,111140,111141,111142,111143,111144,111145,111146,111147,111148,111149,111150,111151,111152,111153,111154,111155,111156,111157,111158,111159,111160,111161,111162,111163,111164,111165,111166,111167,111168,111169,111170,111171,111172,111173,111174,111175,111176,111177,111178,111179,111180,111181,111182,111183,111184,111185,111186,111187,111188,111189,111190,111191,111192,111193,111194,111195,111196,111197,111198,111199,1111100,11112101,11112102,11112103,11112104,11112105,11112106,11112107,11112108,11112109,11112110,11112111,11112112,11112113,11112114,11112115,11112116,11112117,11112118,11112119,11112120,11112121,11112122,11112123,11112124,11112125,11112126,11112127,11112128,11112129,11112130,11112131,11112132,11112133,11112134,11112135,11112136,11112137,11112138,11112139,11112140,11112141,11112142,11112143,11112144,11112145,11112146,11112147,11112148,11112149,11112150,11112151,11112152,11112153,11112154,11112155,11112156,11112157,11112158,11112159,11112160,11112161,11112162,11112163,11112164,11112165,11112166,11112167,11112168,11112169,11112170,11112171,11112172,11112173,11112174,11112175,11112176,11112177,11112178,11112179,11112180,11112181,11112182,11112183,11112184,11112185,11112186,11112187,11112188,11112189,11112190,11112191,11112192,11112193,11112194,11112195,11112196,11112197,11112198,11112199,11112200'; $ids = explode(',', $str); foreach ($ids as $value){ echo 'INSERT INTO temp VALUES(' . $value . '); </br>'; } But change the echo for the mysqli query. Then: SELECT * FROM tbluser u where status !='Deleted' And exists(select * from temp u1 where u1.userid = u.userid) Or you can do an inner join temp u1 on u1.userid = u.userid
#Abid as you mentioned your question in this case #Guss is right. What i am thinking if values mentioned IN clause comes from GROUP_CONCAT then might be possible duplicate values exist with comma seperated. So to remove duplicacy we can use DISTINCT. For example : SELECT GROUP_CONCAT(UserId) FROM sometable Above query can give duplicate values. SELECT GROUP_CONCAT(DISTINCT UserId) FROM sometable SELECT t1.UserRoleId, t1.EntityId, GROUP_CONCAT( DISTINCT t1.PermissionId ) AS Permissions FROM userpermission t1, bill_companies t2 WHERE t1.Status = 'Active' AND t2.status = 'Active' AND UserGroupId IN ( 84, 85, 86 ) LIMIT 0 , 30 Above query gives you userid's without duplicacy. This can also reduce the size of comma seperated ids. Hope this will help.
You could create groups using php if there are Ids which follow by increment e. g. 1,2,3 and replace them by a OR (id >= 1 AND id <= 3) Alternative approach: Search Max and Min ID in your List and query using an interval (greater than and less than) and use php to do the job.
If you really want to avoid using IN then you could use FIND_IN_SET():- SELECT * FROM TblUser WHERE Status != 'Deleted' AND FIND_IN_SET(UserId, '0,10,11,12,13,14,15,22,45,114,144,155,156,167,211,439,440,441,443,445,450,455,456,457,458,459,1111,1154,1156,1165,1451,1541,11111,11112,11113,11114,11115,11116,11117,11118,11119,11656,15451,16561,17671,18781,33131,33311,33411,54511,111110,111111,111112,111113,111114,111115,111116,111117,111118,111119,111120,111121,111122,111123,111124,111125,111126,111127,111128,111129,111130,111131,111132,111133,111134,111135,111136,111137,111138,111139,111140,111141,111142,111143,111144,111145,111146,111147,111148,111149,111150,111151,111152,111153,111154,111155,111156,111157,111158,111159,111160,111161,111162,111163,111164,111165,111166,111167,111168,111169,111170,111171,111172,111173,111174,111175,111176,111177,111178,111179,111180,111181,111182,111183,111184,111185,111186,111187,111188,111189,111190,111191,111192,111193,111194,111195,111196,111197,111198,111199,1111100,11112101,11112102,11112103,11112104,11112105,11112106,11112107,11112108,11112109,11112110,11112111,11112112,11112113,11112114,11112115,11112116,11112117,11112118,11112119,11112120,11112121,11112122,11112123,11112124,11112125,11112126,11112127,11112128,11112129,11112130,11112131,11112132,11112133,11112134,11112135,11112136,11112137,11112138,11112139,11112140,11112141,11112142,11112143,11112144,11112145,11112146,11112147,11112148,11112149,11112150,11112151,11112152,11112153,11112154,11112155,11112156,11112157,11112158,11112159,11112160,11112161,11112162,11112163,11112164,11112165,11112166,11112167,11112168,11112169,11112170,11112171,11112172,11112173,11112174,11112175,11112176,11112177,11112178,11112179,11112180,11112181,11112182,11112183,11112184,11112185,11112186,11112187,11112188,11112189,11112190,11112191,11112192,11112193,11112194,11112195,11112196,11112197,11112198,11112199,11112200') However the only limit for the number of entries in an IN clause is set by max_allowed_packet (as others have stated) and the same restriction would apply to this. max_allowed_packet
MySQL - query for duplicates of a column and return both original and duplicate rows
I have a table that I use to store some systematically chosen "serial numbers" for each product that is bought... The problem is, a CSV was uploaded that I believe contained some duplicate "serial numbers", which means that when the application tries to modify a row, it may not be modifying the correct one. I need to be able to query the database and get all rows that are a double of the serial_number column. It should look something like this: ID, serial_number, meta1, meta2, meta3 3, 123456, 0, 2, 4 55, 123456, 0, 0, 0 6, 345678, 0, 1, 2 99, 345678, 0, 1, 2 So as you can see, I need to be able to see both the original row and the duplicate row and all of it's columns of data ... this is so I can compare them and determine what data is now inconsistent.
Some versions of MySQL implement in with a subquery very inefficiently. A safe alternative is a join: SELECT t.* FROM t join (select serial_number, count(*) as cnt from t group by serial_number ) tsum on tsum.serial_number = t.serial_number and cnt > 1 order by t.serial_number; Another alternative is to use an exists clause: select t.* from t where exists (select * from t t2 where t2.serial_number = t.serial_number and t2.id <> t.id) order by t.serial_number; Both these queries (as well as the one proposed by #fthiella) are standard SQL. Both would benefit from an index on (serial_number, id).
SELECT * FROM yourtable WHERE serial_number IN (SELECT serial_number FROM yourtable GROUP BY serial_number HAVING COUNT(*)>1) ORDER BY serial_number, id
MYSQL working slowly as query with subquery than 2 queries (+php)
I have table (about 80'000 rows), looks like id, parentId, col1, col2, col3... 1, null, 'A', 'B', 'C' 2, 1, ... 3, 1, ... 4, null, ... 5, 4, ... (one level parent - child only) and I need get all dependent rows - SELECT ... FROM table WHERE id = :id OR parentId = :id OR id IN ( SELECT parentId FROM table WHERE id = :id ) but why this request working slowly instead 2 request - if I get parentId on php first? $t = executeQuery('SELECT parentId FROM table WHERE id = :Id;', $id); if ($t) { $id = $t; } $t = executeQuery('SELECT * FROM table WHERE id = :id OR parentId = :id ORDER BY id;', $id); PS: max depends rows < 70 PPS: id select_type table type possible_keys key key_len ref rows Extra 1 PRIMARY product ALL PRIMARY,parentId NULL NULL NULL 73415 Using where 2 DEPENDENT SUBQUERY product const PRIMARY,parentId PRIMARY 4 const 1
Change the IN for an equal = SELECT ... FROM table WHERE id = :id OR parentId = :id OR id = ( SELECT parentId FROM table WHERE id = :id ) or change it to a join: SELECT ... FROM table inner join ( SELECT parentId FROM table WHERE id = :id ) s on s.parentID = table.id or s.parentID = table.parentID
Well, in the first case, MySQL need to create an intermediate result, store it in memory and then iterate over it to find all the relevant id in the table. In the second way, assuming you correctly created an index on id and parent id, it simply go straigth to the index, find the relevant rows, and send you back the result immediately.
UNION works faster for this case this allows first query to user UNION INDEX and second just uses inner join, then merges results. SELECT * FROM `table` WHERE id = :id OR parentId = :id UNION SELECT t1.* FROM `table` t1 JOIN `table` t2 ON t2.parentId = t1.id AND t2.id = :id
An EXPLAIN might shed some more light on the problem for you. Look into EXISTS, or rewriting your query as a JOIN.
It's a long shot but in first case you have "IN" statement of the WHERE part of the query. Maybe MySQL tries to optimize the query as if there would be multiple options and in the second case there is no IN part, so the compiled query is more straight forward for the database - thus utilizing the indexes in better manner. Basically for 2 queries on the same connection the overhead of performing the queries should be minimal and irelevant in this case. Also subqueries in general are not very optimizable by the query parser. Try using JOIN instead (if possible).
SELECT from two tables WHERE different columns in each table equal $id ORDER BY common column (PHP/MySQL)
I'm trying to SELECT from two tables and ORDER BY date (a column they both have). One table (tableA) has a column called "A" and the other table (tableB) has a column called "B", I use array_key_exists() to differentiate between the two (If "A" key exists, I run the array through FunctionA(), if "B" key exists, I run the array through FunctionB()). I only need the 20 latest (date wise) entries. I need the SQL Query to accomplish this. I already know a reply will be "if they're similarly structured, then you should just use a single table", but I don't want to do that because tableA is drastically different from tableB (a lot more columns in tableA), and using a single table to store the data would result in a LOT of empty columns for entries formatted for tableB, not to mention it'd be a very ugly looking table format due to tableB not needing the majority of tableA's columns). I just want to display data from both tables in an ordered (by date) fashion, and in one single stream. I need to SELECT WHERE tableA.poster_id = $id and tableB.receiver_id = $id by the way. SOLUTION: I'm updating this just in case anyone else with the same dilemma comes along. After implementing the SQL query that #Erik A. Brandstadmoen had graciously given me, this is basically what my code ended up as: $MySQL->SQL("SELECT * FROM (SELECT A.id AS id, A.date AS date, 'tableA' AS source FROM tableA A WHERE A.poster_id = $id UNION SELECT B.id AS id, B.date AS date, 'tableB' AS source FROM tableB B WHERE B.receiver_id = $id) AS T ORDER BY T.date DESC LIMIT 0, 20"); $GetStream = array(); $i = 0; while ($row = mysql_fetch_array($MySQL->Result)) { $GetStream[$i]['id'] = $row['id']; $GetStream[$i]['date']=$row['date']; $GetStream[$i]['source'] = $row['source']; $i++; } *** later on down the code *** $i = 0; while ($i<count($GetStream)) { if ($GetStream[$i]['source'] == "tableA") { FunctionA($GetStream[$i]); } else { FunctionB($GetStream[$i]); } $i++; }
Try using UNION: SELECT * FROM ( SELECT A.col1 AS x, A.col2 As y, A.col3 AS date FROM tableA A WHERE tableA.poster_id = $id UNION SELECT B.colA AS x, B.colB AS y, B.colC AS date FROM tableB B WHERE tableB.receiver_id = $id ) ORDER BY date DESC LIMIT 0, 20 OR, IF you would like to keep duplicates between tableA and tableB, use UNION ALL instead. EDIT, according to your comments, I understand that you need a column indicating which table the row is from. You can just add a static column in the select, like this: SELECT * FROM ( SELECT A.col1 AS x, A.col2 As y, A.col3 AS date, 'A' as source FROM tableA A WHERE tableA.poster_id = $id UNION SELECT B.colA AS x, B.colB AS y, B.colC AS date, 'B' as source FROM tableB B WHERE tableB.receiver_id = $id ) ORDER BY date DESC LIMIT 0, 20 This gives you a nice table on the following form: x y date source ========================= (v1) (v2) (d1) 'A' (v3) (v4) (d2) 'B' (v1) (v2) (d3) 'B' (v3) (v4) (d4) 'A' That does what you want, doesn't it? It's a bit difficult understanding what you are really trying to achieve with this...