I need to find and replace all the values of rows of a CSV using PHP;
I am trying this but its replacing even the headers to 0 and row values are not doubling as it suppose to.
public function checkForNumericValues()
{
// Read the columns and detect numeric values
if (($this->handle = fopen($this->csvFile, "r")) !== FALSE)
{
$fhandle = fopen($this->csvFile,"r");
$content = fread($fhandle,filesize($this->csvFile));
while (($this->data = fgetcsv($this->handle, 1000, ",")) !== FALSE)
{
$this->num = count($this->data);
// Skipping the header
if($this->row == 1)
{
$this->row++;
continue;
}
$this->row++;
// Check and replace the numeric values
for ($j=0; $j < $this->num; $j++)
{
if(is_numeric($this->data[$j]))
{
$content = str_replace($this->data[$j], $this->data[$j] * 2, $content);
}
else
{
$content = str_replace($this->data[$j], 0, $content);
}
}
break;
// print_r($content);
}
$fhandle = fopen($this->csvFile,"w");
fwrite($fhandle,$content);
fclose($this->handle);
}
echo "Numeric and String values been changed in rows of the CSV!";
}
CSV is like this:
You shouldn't update the entire $contents when you're processing each field in the CSV, just update that field. Your str_replace() will replace substrings elsewhere in the file; for instance, if the current field contains 5, you'll replace all the 5's in the file with 10, so 125 will become 1210.
You can do it correctly by replacing the element in the $this->data array. After you do that, you can then join them back into a string with implode(). Then you can keep all the updated lines in a string, which you write back to the file at the end.
You can skip the header line by calling fgets() before the while loop.
public function checkForNumericValues()
{
// Read the columns and detect numeric values
if (($this->handle = fopen($this->csvFile, "r")) !== FALSE)
{
$output = "";
$output .= fgets($this->csvFile); // Copy header line to output
while (($this->data = fgetcsv($this->handle, 1000, ",")) !== FALSE)
{
$this->num = count($this->data);
// Check and replace the numeric values
for ($j=0; $j < $this->num; $j++)
{
if(is_numeric($this->data[$j]))
{
$this->data[$j] *= 2;
}
else
{
$this->data[$j] = 0;
}
}
$output .= implode(',', $this->data) . "\n";
}
fclose($this->handle);
$fhandle = fopen($this->csvFile,"w");
fwrite($fhandle,$output);
fclose($fhandle);
}
echo "Numeric and String values been changed in rows of the CSV!";
}
Related
I have a CSV file containing two rows of data, one which the user will input and the other will be returned. e.g. inputting a post/zipcode which will be found in the CSV file, the data in the next cell should be returned.
<?php
function csv_to_array($filename='bn.csv', $delimiter=',')
{
if(!file_exists($filename) || !is_readable($filename))
return FALSE;
$header = NULL;
$data = array();
if (($handle = fopen($filename, 'r')) !== FALSE)
{
while (($row = fgetcsv($handle, 1000, $delimiter)) !== FALSE)
{
if(!$header)
$header = $row;
else
$data[] = array_combine($header, $row);
}
fclose($handle);
}
return $data;
}
function search($data, $x) {
for($i = 0; $i < sizeof($data); $i++) {
if($data[$i] == $x) return $i;
}
return -1;
}
$data = array("bn.csv");
echo search($data, "BN1 1AA");
print_r(csv_to_array('bn.csv'));
?>
Currently I am just getting -1 returned, what should I do?
Thanks in advance.
$data = array("bn.csv"); creates an array with the literal string bv.csv in it. I'm guessing you want $data = csv_to_array("bn.csv"); here instead. I would build an associative array like this:
$data = [
'BN1 1AA' => 'E05002432',
'...' => '...',
'...' => '...',
];
Via something like:
while (($row = fgetcsv($handle, 1000, $delimiter)) !== FALSE) {
$data[$row[0]] = $row[1];
}
Thus, you can refer to an entry directly by its key, you don't need to iterate over every item to search for a match:
$search = 'BN1 1AA';
if (array_key_exists($search, $data)) {
echo $data[$search];
} else {
echo "not found";
}
There are several problems towards the end of your code.
In search() - you are trying to match the postcode with the entire row in $data[$i] == $x, $data[$i] is actually a list of the fields in the row, so perhaps matching it with ($data[$i][$field] == $x) and $field is the field name of the postcode.
You were calling search() with an array of the file names for the data and not the data from the file, so this code calls csv_to_array() first and passes the result to search()...
Lastly - the sample data didn't have BN1 1AA in it anyway, so should return -1.
function search($data, $x, $field = 'Postcode') {
for($i = 0; $i < sizeof($data); $i++) {
if ($data[$i][$field] == $x)
return $i;
}
return -1;
}
$data = csv_to_array('a.csv');
print_r($data);
echo search($data, "BN1 1AD");
I would like to convert a CSV to Json, use the header row as a key, and each line as object. How do I go about doing this?
----------------------------------CSV---------------------------------
InvKey,DocNum,CardCode
11704,1611704,BENV1072
11703,1611703,BENV1073
---------------------------------PHP-----------------------------------
if (($handle = fopen('upload/BEN-new.csv'. '', "r")) !== FALSE) {
while (($row_array = fgetcsv($handle, 1024, ","))) {
while ($val != '') {
foreach ($row_array as $key => $val) {
$row_array[] = $val;
}
}
$complete[] = $row_array;
}
fclose($handle);
}
echo json_encode($complete);
Just read the first line separately and merge it into every row:
if (($handle = fopen('upload/BEN-new.csv', 'r')) === false) {
die('Error opening file');
}
$headers = fgetcsv($handle, 1024, ',');
$complete = array();
while ($row = fgetcsv($handle, 1024, ',')) {
$complete[] = array_combine($headers, $row);
}
fclose($handle);
echo json_encode($complete);
I find myself converting csv strings to arrays or objects every few months.
I created a class because I'm lazy and dont like copy/pasting code.
This class will convert a csv string to custom class objects:
Convert csv string to arrays or objects in PHP
$feed="https://gist.githubusercontent.com/devfaysal/9143ca22afcbf252d521f5bf2bdc6194/raw/ec46f6c2017325345e7df2483d8829231049bce8/data.csv";
//Read the csv and return as array
$data = array_map('str_getcsv', file($feed));
//Get the first raw as the key
$keys = array_shift($data);
//Add label to each value
$newArray = array_map(function($values) use ($keys){
return array_combine($keys, $values);
}, $data);
// Print it out as JSON
header('Content-Type: application/json');
echo json_encode($newArray);
Main gist:
https://gist.github.com/devfaysal/9143ca22afcbf252d521f5bf2bdc6194
For those who'd like things spelled out a little more + some room to further parse any row / column without additional loops:
function csv_to_json_byheader($filename){
$json = array();
if (($handle = fopen($filename, "r")) !== FALSE) {
$rownum = 0;
$header = array();
while (($row = fgetcsv($handle, 1024, ",")) !== FALSE) {
if ($rownum === 0) {
for($i=0; $i < count($row); $i++){
// maybe you want to strip special characters or merge duplicate columns here?
$header[$i] = trim($row[$i]);
}
} else {
if (count($row) === count($header)) {
$rowJson = array();
foreach($header as $i=>$head) {
// maybe handle special row/cell parsing here, per column header
$rowJson[$head] = $row[$i];
}
array_push($json, $rowJson);
}
}
$rownum++;
}
fclose($handle);
}
return $json;
}
I have a CSV file which contains a mixture of English and Chinese characters (it is a list of contacts exported from the Mozilla Thunderbird email program). I am trying to create a function which can extract the information from this file. It appears that function fgetcsv() does not support multibyte characters. Since I am running PHP5.2, I do not have access to str_getcsv().
Although the situation above refers to English and Chinese, I am looking for a solution which will work with any language.
Right now I have the function namecards_import_str_getcsv() as my CSV parsing function, which tries to mimic str_getcsv().
function namecards_import_str_getcsv($input, $delimiter = ',', $enclosure = '"', $escape = '\\', $eol = '\n') {
if (!function_exists('str_getcsv')) {
if (is_string($input) && !empty($input)) {
$output = array();
$tmp = preg_split("/".$eol."/",$input);
if (is_array($tmp) && !empty($tmp)) {
while (list($line_num, $line) = each($tmp)) {
if (preg_match("/" . $escape . $enclosure . "/", $line)) {
while ($strlen = strlen($line)) {
$pos_delimiter = strpos($line, $delimiter);
$pos_enclosure_start = strpos($line, $enclosure);
if (is_int($pos_delimiter) && is_int($pos_enclosure_start) && ($pos_enclosure_start < $pos_delimiter)) {
$enclosed_str = substr($line, 1);
$pos_enclosure_end = strpos($enclosed_str, $enclosure);
$enclosed_str = substr($enclosed_str, 0, $pos_enclosure_end);
$output[$line_num][] = $enclosed_str;
$offset = $pos_enclosure_end + 3;
}
else {
if (empty($pos_delimiter) && empty($pos_enclosure_start)) {
$output[$line_num][] = substr($line, 0);
$offset = strlen($line);
}
else {
$output[$line_num][] = substr($line,0,$pos_delimiter);
$offset = (!empty($pos_enclosure_start) && ($pos_enclosure_start < $pos_delimiter))? $pos_enclosure_start : $pos_delimiter + 1;
}
}
$line = substr($line,$offset);
}
}
else {
$line = preg_split("/" . $delimiter . "/", $line);
/*
* Validating against pesky extra line breaks creating false rows.
*/
if (is_array($line) && !empty($line[0])) {
$output[$line_num] = $line;
}
}
}
return $output;
}
else {
return false;
}
}
else {
return false;
}
}
else {
return str_getcsv($input);
}
}
This function is called by the following line of code:
$file = $_SESSION['namecards_csv_file'];
if (file_exists($file->uri)) {
// Load raw csv content into a handler variable.
$handle = fopen($file->uri, "r");
$cardinfo = array();
while (($data = fgets($handle)) !== FALSE) {
$data = namecards_import_str_getcsv($data);
dsm($data);
$cardinfo[] = $data[0];
}
fclose($handle);
}
else {
drupal_set_message(t('CSV file doesn\'t exist'), 'error');
}
In the array of results the strings of Chinese characters are in the correct place in the array by they appear as symbols e.g. "��".
Another method I had tried before this was to simply use fgetcsv() (See below example). But in this case the elements of the returned array were empty.
$file = $_SESSION['namecards_csv_file'];
if (file_exists($file->uri)) {
// Load raw csv content into a handler variable.
$handle = fopen($file->uri, "r");
$cardinfo = array();
while (($data = fgetcsv($handle, 5000, ",")) !== FALSE) {
dsm($data);
$cardinfo[] = $data;
}
fclose($handle);
}
else {
drupal_set_message(t('CSV file doesn\'t exist'), 'error');
}
In case you are interested here is the contents of the CSV file:
First Name,Last Name,Display Name,Nickname,Primary Email,Secondary Email,Screen Name,Work Phone,Home Phone,Fax Number,Pager Number,Mobile Number,Home Address,Home Address 2,Home City,Home State,Home ZipCode,Home Country,Work Address,Work Address 2,Work City,Work State,Work ZipCode,Work Country,Job Title,Department,Organization,Web Page 1,Web Page 2,Birth Year,Birth Month,Birth Day,Custom 1,Custom 2,Custom 3,Custom 4,Notes,
Ben,Gunn,Ben Gunn,Benny,ben1#asdf.com,ben2#asdf.com,,+94 (10) 11111111,+94 (10) 22222222,+94 (10) 33333333,,+94 44444444444,12 Benny Lane,,Beijing,Beijing,100028,China,13 asdfsdfs,,sdfsf,sdfsdf,134323,China,Manager,Sales,Benny Inc,,,,,,,,,,,
乔,康,乔 康,小康,,,,,,,,,,,,,,,北京市朝阳区,,,,,,,,,,,,,,,,,,,
Just writing up as an answer what was figured out in the comments:
fgetcsv is locale sensitive, so make sure to setlocale to a UTF-8 locale.
I need to print out csv file into html or put a numeric data into database:
But I need to start a loop at specific position and break it at another specific position (regex).
So I need to reprint only rows with numerical data and all columns from them.
Following is pseudo-code - not working properly:
<?php
$row = 1;
$handle = fopen("test.csv", "r");
while ($data = fgetcsv($handle, 1000, ","))
{
if (preg_match('/[Morning]/', $data[0]) === 1 // start at this rwo plus two lines down )
{
$num = count($data);
$row++;
for ($c=0; $c < $num; $c++)
{
for ($c=0; $c < $num; $c++)
{
echo $data[$c] . " ";
}
if (preg_match('/[Total Cash:]/', $data[0]) === 1)
{ break; row -1 }
}
echo "<br>";
}
}
fclose($handle); ?>
So csv goes like this:
/--some lines--/
Date: 3/3/11,
Morning,
--blank line---
Customer No,Time,CheckNo,Total,
1234,12-45,01,20.00,
1236,1-00,03,30.00,
1240,2-00,06,30.00,
--more numerical rows of data at variable length that I need to loop over--
1500,4-00,07,22.00,
----,----,---,----,
Total Cash, , , ,120.00,
/--some other lines--and it goes/
Lunch Time,
---similar like Morning above ---
Any info how to properly addrres this issue is appreciated, I can now do so many loops and regex but with this I need some more time and help. Thanks.
$lines = file('test.csv'); //read file into an array, one entry per line
$active = false; //keep track of what rows to parse
//loop one line at a time
for ($i = 0; $i < count($lines); $i++) {
$line = $lines[$i];
if (strpos($line, 'Morning') !== false) { //start parsing on the next row
$active = true;
$i += 2; //skip the blank line and header
continue;
}
if (strpos($line, '----,') !== false) { //stop parsing rows
$active = false;
}
if ($active) { //if parsing enabled, split the line on commas and do something with the values
$values = str_getcsv(trim($line));
foreach ($values as $value) {
echo $value . " "; //these are the numbers
}
}
}
$lines = file('test.csv');
$parsing = false;
foreach ($lines as $line)
{
$parsing = ((strpos($line, 'Morning') !== false) || $parsing)
&& ((strpos($line, 'Total Cash') === false);
if (!$parsing)
continue;
$values = strgetcsv($line);
echo implode(' ', $values);
}
Edit: Basically, it does the same as Dan Grossmans solution, but shorter ;-)
$lines = file('test.csv');
// Skip the unwanted lines
// Means: Every line until the line containing "Morning,"
do {
$line = array_shift($lines);
} while(trim($line) !== 'Morning,');
$lines = array_slice($lines, 2); // Mentioned something about "2 lines below" or such" ^^
// Do something with the remaining lines, until
// Line _begins_ with "Total Cash"
while(strncmp('Total Cash', trim($line = array_shift($lines)), 10) !== 0) {
echo implode(' ', str_getcsv($line)) . PHP_EOL;
}
I am curious if you have a string how would you detect the delimiter?
We know php can split a string up with explode() which requires a delimiter parameter.
But what about a method to detect the delimiter before sending it to explode function?
Right now I am just outputting the string to the user and they enter the delimiter. That's fine -- but I am looking for the application to pattern recognize for me.
Should I look to regular expressions for this type of pattern recognition in a string?
EDIT: I have failed to initially specify that there is a likely expected set of delimiters. Any delimiter that is probably used in a CSV. So technically anyone could use any character to delimit a CSV file but it is more probable to use one of the following characters: comma, semicolon, vertical bar and a space.
EDIT 2: Here is the workable solution I came up with for a "determined delimiter".
$get_images = "86236058.jpg 86236134.jpg 86236134.jpg";
//Detection of delimiter of image filenames.
$probable_delimiters = array(",", " ", "|", ";");
$delimiter_count_array = array();
foreach ($probable_delimiters as $probable_delimiter) {
$probable_delimiter_count = substr_count($get_images, $probable_delimiter);
$delimiter_count_array[$probable_delimiter] = $probable_delimiter_count;
}
$max_value = max($delimiter_count_array);
$determined_delimiter_array = array_keys($delimiter_count_array, max($delimiter_count_array));
while( $element = each( $determined_delimiter_array ) ){
$determined_delimiter_count = $element['key'];
$determined_delimiter = $element['value'];
}
$images = explode("{$determined_delimiter}", $get_images);
Determine which delimiters you consider probable (like ,, ; and |) and for each search how often they occur in the string (substr_count). Then choose the one with most occurrences as the delimiter and explode.
Even though that might not be fail-safe it should work in most cases ;)
I would say this works 99.99% of the cases :)
The basic idea is, that number of valid delimiters should be the same line by line.
This script calculates delimiter count discrepancies between all lines.
Less discrepancy means more likely valid delimiter.
Putting it all together this function read rows and return it back as an array:
function readCSV($fileName)
{
//detect these delimeters
$delA = array(";", ",", "|", "\t");
$linesA = array();
$resultA = array();
$maxLines = 20; //maximum lines to parse for detection, this can be higher for more precision
$lines = count(file($fileName));
if ($lines < $maxLines) {//if lines are less than the given maximum
$maxLines = $lines;
}
//load lines
foreach ($delA as $key => $del) {
$rowNum = 0;
if (($handle = fopen($fileName, "r")) !== false) {
$linesA[$key] = array();
while ((($data = fgetcsv($handle, 1000, $del)) !== false) && ($rowNum < $maxLines)) {
$linesA[$key][] = count($data);
$rowNum++;
}
fclose($handle);
}
}
//count rows delimiter number discrepancy from each other
foreach ($delA as $key => $del) {
echo 'try for key=' . $key . ' delimeter=' . $del;
$discr = 0;
foreach ($linesA[$key] as $actNum) {
if ($actNum == 1) {
$resultA[$key] = 65535; //there is only one column with this delimeter in this line, so this is not our delimiter, set this discrepancy to high
break;
}
foreach ($linesA[$key] as $actNum2) {
$discr += abs($actNum - $actNum2);
}
//if its the real delimeter this result should the nearest to 0
//because in the ideal (errorless) case all lines have same column number
$resultA[$key] = $discr;
}
}
var_dump($resultA);
//select the discrepancy nearest to 0, this would be our delimiter
$delRes = 65535;
foreach ($resultA as $key => $res) {
if ($res < $delRes) {
$delRes = $res;
$delKey = $key;
}
}
$delimeter = $delA[$delKey];
echo '$delimeter=' . $delimeter;
//get rows
$row = 0;
$rowsA = array();
if (($handle = fopen($fileName, "r")) !== false) {
while (($data = fgetcsv($handle, 1000, $delimeter)) !== false) {
$rowsA[$row] = Array();
$num = count($data);
for ($c = 0; $c < $num; $c++) {
$rowsA[$row][] = trim($data[$c]);
}
$row++;
}
fclose($handle);
}
return $rowsA;
}
I have the same problem, I am dealing with a lot of CSV's from various databases, which various people extract to CSV in various ways, sometimes different each time for the same dataset ... Have simply implemented a function like this in my convert base class
protected function detectDelimiter() {
$handle = #fopen($this->CSVFile, "r");
if ($handle) {
$line=fgets($handle, 4096);
fclose($handle);
$test=explode(',', $line);
if (count($test)>1) return ',';
$test=explode(';', $line);
if (count($test)>1) return ';';
//.. and so on
}
//return default delimiter
return $this->delimiter;
}
I made something like this:
$line = fgetcsv($handle, 1000, "|");
if (isset($line[1]))
{
echo "delimiter is: |";
$delimiter="|";
}
else
{
$line1 = fgetcsv($handle, 1000, ";");
if (isset($line1[1]))
{
echo "delimiter is: ;";
$delimiter=";";
}
else
{
echo "delimiter is: ,";
$delimiter=",";
}
}
This simply checks whether there is a second column after a line is read.
I am having the same issue. My system will recieve CSV files from the client but it could use ";", "," or " " as delimiter and I wnat to improve the system so the client dont have to know which is (They never do).
I search and found this library:
https://github.com/parsecsv/parsecsv-for-php
Very good and easy to use.