Ignore duplicate rows - php

Question

I am importing some data from a csv file into MySQL and trying to ignore duplicate rows.
mysql_query("INSERT IGNORE INTO products (parent_product_url, child_product_url, swatch) VALUES ('".$row[0]."', '".$row[1]."', '".$row[2]."')");
My csv file.
polo.htm,red.htm,red.jpg
polo.htm,green.htm,green.jpg
round-neck.htm,green.htm,green.jpg
Now if I run below csv file it should ignore first three rows as they already exists in the table. It should insert only fourth row.
polo.htm,red.htm,red.jpg
polo.htm,green.htm,green.jpg
round-neck.htm,green.htm,green.jpg
v-neck.htm,red.htm,red.jpg

Answer 1

I prefer on duplicate key update because insert ignore ignores all errors, not just duplication errors.
Regardless of which you use, your problem is probably the lack of unique constraint/index.
You don't specify what you mean by "duplicate". Assuming you mean all the columns:
create unique index unq_products_3 on products(parent_product_url, child_product_url, swatch);
Note: there is a maximum length to the keys used for indexes, depending on the storage engine. If your columns are too long, you may need to think about other approaches.

Answer 2

Records are inserted again when you re-execute insert statements because the inserts are not violating any unique or primary key index. Therefore MySQL doesn't have anything to ignore.
create table products (
parent_product_url varchar(100),
child_product_url varchar(100),
swatch varchar(100)
);
-- this will enter both records
insert ignore into products values ('polo.htm', 'red.htm', 'red.jpg');
insert ignore into products values ('polo.htm', 'green.htm', 'green.jpg');
-- this will enter both records **AGAIN**
insert ignore into products values ('polo.htm', 'red.htm', 'red.jpg');
insert ignore into products values ('polo.htm', 'green.htm', 'green.jpg');
Now let's add uniqueness to parent_product_url and try again:
truncate table products;
create unique index uk_products_parent_product_url on products(parent_product_url);
insert ignore into products values ('polo.htm', 'red.htm', 'red.jpg');
insert ignore into products values ('polo.htm', 'green.htm', 'green.jpg');
This will enter only the first record. 2nd record will be ignored and a warning will be thrown. No error will be thrown.
If you desire to have a combination of the 3 columns to be unique, then you would do this (This is what Gordon Linoff has mentioned also...I am just adding more context):
alter table products drop key uk_products_parent_product_url;
create unique index uk_products_parenturl_childurl_swatch on
products(parent_product_url, child_product_url, swatch);
insert ignore into products values ('polo.htm', 'red.htm', 'red.jpg');
insert ignore into products values ('polo.htm', 'green.htm', 'green.jpg');
Now you will see only two records inserted even when you re-execute the same 2 insert statements many times.
From https://dev.mysql.com/doc/refman/5.5/en/insert.html
If you use the IGNORE keyword, errors that occur while executing the
INSERT statement are ignored. For example, without IGNORE, a row that
duplicates an existing UNIQUE index or PRIMARY KEY value in the table
causes a duplicate-key error and the statement is aborted. With
IGNORE, the row is discarded and no error occurs. Ignored errors may
generate warnings instead, although duplicate-key errors do not.

Answer 3

I got it solved with the help of this Answer -> Insert query check if record exists - If not, Insert it
Below is my updated query
mysql_query("INSERT INTO products (parent_product_url, child_product_url, swatch)
SELECT * FROM (SELECT '".$row[0]."', '".$row[1]."', '".$row[2]."') AS tmp
WHERE NOT EXISTS (
SELECT * FROM products WHERE parent_product_url='".$row[0]."' AND child_product_url='".$row[1]."' AND swatch='".$row[2]."'
);");