Remove duplicates from PHP array (array_unique) - php

Question

I am getting a specific field from 4 different tables.
<?php
//I connect to the database and the 4 tables
// Location of CSV
$location = 'path.csv';
// List creation that will be updated with the fields and be put into my CSV file
$list = array();
// Read csv file to avoid adding duplicates
$file = fopen($location, 'r');
$data = array();
while($row = fgetcsv($file))
{
$data[] = $row;
}
// Query 1
$sql = ('select distinct(field) as field from '.$table1.'');
// Run the query
$query = $Db->query($sql);
// Check for SQL errors
if ($Db->error)
{
return ($Db->error);
}
// Put data in the list
while ($row = $query->fetch_assoc())
{
array_push($list,array($row['field'], ''));
}
// Query 2
$sql = ('select distinct(field) as field from '.$table2.'');
// Run the query
$query = $Db->query($sql);
// Check for SQL errors
if ($Db->error)
{
return ($Db->error);
}
// Put data in the list
while ($row = $query->fetch_assoc())
{
array_push($list,array($row['field'], ''));
}
// Query 3
$sql = ('select distinct(field) as field from '.$table3.'');
// Run the query
$query = $Db->query($sql);
// Check for SQL errors
if ($Db->error)
{
return ($Db->error);
}
// Put data in the list
while ($row = $query->fetch_assoc())
{
array_push($list,array($row['field'], ''));
}
// Query 4
$sql = ('select distinct(field) as field from '.$table4.'');
// Run the query
$query = $Db->query($sql);
// Check for SQL errors
if ($Db->error)
{
return ($Db->error);
}
// Put data in the list
while ($row = $query->fetch_assoc())
{
array_push($list,array($row['field'], ''));
}
// Save list in the csv file without overwriting
$fp = fopen($location, 'a');
foreach (array_unique($list) as $fields)
{
if (in_array($fields, $data))
{
echo "Duplicate found";
}
else
{
echo "Save to file";
fputcsv($fp, $fields);
}
}
fclose($fp);
?>
In the end I check if the fields are already in the file. The only problem is that I still have duplicates because some tables might have exactly the same field. So, I want to remove the duplicates from the PHP array "list".
I am using :
$cleanlist = array_unique($list);
but I am getting an error:
PHP Notice: Array to string conversion
More specifically the change in my code is :
$cleanlist = array_unique($list);
// Save list in the csv file without overwriting
$fp = fopen($location, 'a');
foreach ($cleanlist as $fields)
{
if (in_array($fields, $data))
{
echo "Duplicate found";
}
else
{
echo "Save to file";
fputcsv($fp, $fields);
}
}

Answer 1

As explain in the docs, array_unique compares elements as strings by default. You are getting this error because PHP is trying to convert an Array to string. You have a 2D array, an array of array.
You can use the flag SORT_REGULAR to compare the elements as they are.
But be careful, only same key/value pairs are considered identicals.

Answer 2

You could reduce amount of code a lot by using UNION in SELECT statement.
SELECT field FROM table1
UNION
SELECT field FROM table2
UNION
SELECT field FROM table3
UNION
SELECT field FROM table4
UNION returns distinct results by default.

Answer 3

That worked :
$list = array_map("unserialize", array_unique(array_map("serialize", $list)));
The $list is a 2d array.