Remove duplicates from CSV file using PHP - php

Question

First of all I load PHPExcel.php
Secondly, I am using this code:
$location = '/path/file.csv';
$inputFileType = 'CSV';
$objReader = PHPExcel_IOFactory::createReader($inputFileType);
$objPHPExcel = $objReader->load($location);
$worksheet = $objPHPExcel->getActiveSheet();
$list = array();
foreach ($worksheet->getRowIterator() as $row)
{
$rowIndex = $row->getRowIndex();
$cellValue = $worksheet->getCell('A'.$rowIndex)->getValue();
array_push($list, $cellValue);
}
$count = count($list);
for ($rowIndex = $count; $rowIndex != 1; $rowIndex--)
{
$cellValue = $worksheet->getCell('A'.$rowIndex)->getValue();
for ($i = $rowIndex - 2; $i != 0; $i--)
{
if ($list[$i] == $cellValue)
{
$worksheet->removeRow($rowIndex);
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'CSV');
$objWriter->save($location);
break;
}
}
}
So, I am trying to remove the rows when there are duplicate values in the first column. The code does not work. When I first run it in putty, I have to wait for ages. I interrupt the process and then I run it again. Then it runs, but in my csv file I have wrong results (duplicates are 300 but I am getting -600 rows).

Answer 1

In order to read a CSV file you dont have to use PHPExcel. Instead you can use a native php code like this one:
<?php
// Array which will hold all analyzed lines
$uniqueEntries = array();
$dublicatedEntries = array();
$delimiter = ',';
$file = 'test.csv';
//Open the file
if (($handle = fopen($file, "r")) !== false) {
// read each line into an array
while (($data = fgetcsv($handle, 8192, $delimiter)) !== false) {
// build a "line" from the parsed data
$line = join($delimiter, $data);
//If the line content has ben discovered before - save to duplicated and skip the rest..
if (isset($uniqueEntries[$line])){
dublicatedEntries[] = $line;
continue;
}
// save the line
$uniqueEntries[$line] = true;
}
fclose($handle);
}
// build the new content-data
$contents = '';
foreach ($uniqueEntries as $line => $bool) $contents .= $line . "\r\n";
// save it to a new file
file_put_contents("test_unique.csv", $contents);
?>
This code is untested but should work.
This will give you a .csv file with all unique entries.