MySQL - selective removing duplicated records - php

Question

I have db with few 1000 of contacts and would like to delete all duplicated records. Sql query I have at the moment works well ( when in records - tel, email, name1 are duplicated). Query deletes duplicates with lower id then last occurring record. But in some cases another fields of the record a filled in already (important ones will by title and name2). What i would like to achieve is for mysql to check if these fields are filled in and keep only the record with most information filed in.
My Query
<?php
$del_duplicate_contacts = $mysqli->query("
DELETE ca
FROM contacts ca
LEFT JOIN
(
SELECT MAX(id) id, name1, tel, email
FROM contacts
GROUP BY name1, tel, email
) cb ON ca.id = cb.id AND
ca.name1 = cb.name1 AND
ca.tel = cb.tel AND
ca.email = cb.email
WHERE cb.id IS NULL
");
?>
Example of table:
ID title name1 name2 tel email
1 John 01234 1#1.com
2 Mr John Smith 01234 1#1.com
3 John 01234 1#1.com
My query will delete record 1 and 2. I would like to keep only nr 2 and delete 1 and 3.
How I can achieve that? Is is possible? Or maybe i should involve PHP, if so How?

Answer 1

Use order by in group_concat, you can try this:
DELETE c1 FROM contacts c1
JOIN (
SELECT
substring_index(group_concat(id ORDER BY ((title IS NULL OR title ='') AND (name2 IS NULL OR name2 = '')), id DESC), ',', 1) AS id,
name1, tel, email
FROM contacts
GROUP BY name1, tel, email
) c2
ON c1.name1 = c2.name1 AND c1.tel = c2.tel AND c1.email = c2.email AND c1.id <> c2.id;
Demo Here

Answer 2

I have got solution using NOT EXIST clause instead of NOT IN
DELETE FROM contacts
WHERE NOT EXISTS (
SELECT 1 FROM (
SELECT * FROM (
SELECT * FROM contact AS tmp ORDER BY title DESC, name1 DESC, name2 DESC, email DESC, tel DESC )
as tbl group by name1)
as test WHERE contact.id= test.id
)

Answer 3

OOPS - This is the worst answer I have produced so far -- warning the top bit is dangerous, not sure why I did not include any group -- please continue to the bottom part, which is working now:
DELETE FROM contacts WHERE ID IN (
SELECT ID FROM (
SELECT DISTINCT a.ID
FROM contacts AS a
JOIN contacts AS b
ON a.name1 = b.name1
AND a.tel = b.tel
AND a.email = b.email
ORDER BY a.name1 DESC, a.name2 DESC, a.title DESC
LIMIT 1,100000
) AS tmp
)
LIMIT Must be 1, xxxx -- not 0, xxxx to keep the first one undeleted
As you cannot delete directly from the same table found in sub-query, just add a mask layer, so it is now tested working
Before Deleting, always double check what is going to be deleted:
SELECT * FROM contacts WHERE ID IN (
SELECT ID FROM (
SELECT DISTINCT a.ID
...
LIMIT 1,100000
) AS tmp
)
Apologies for the damage, luckily you did on a testing db
=====================================
Now here is the correct solution:
Let's check what is in the testing table:
According to question, we noticed only #2 #4 #5 is good to keep. And here is the result:
And we want to delete any records not in above list, before delete, we double check what is going to be deleted:
And we are ready to delete:
And here is the SQL, make sure you do on testing db first:
DELETE FROM contacts WHERE ID NOT IN (
SELECT * FROM (
SELECT ID FROM (
SELECT * FROM contacts ORDER BY title DESC, name1 DESC, name2 DESC, tel DESC, email DESC
) AS tmp
GROUP BY name1, tel, email
) AS del
)

Answer 4

This query will work without any ordering options !
DELETE FROM contacts where ID NOT IN (
SELECT ID FROM ( Select A.ID from contacts as A
join contacts AS B
ON A.name1 = B.name1
AND A.name2 = B.name2
AND A.tel = B.tel
AND A.email = B.email) As mytry);