reading and formatting csv data using explode and arrays in php - php

The Objective
To read the csv file, and separate each line into an array. The first line (field names) displayed once, and then loop through the remaining data.
I have this function to open and explode the csv file
$myFile = "csv.csv";
$fh = fopen($myFile, 'r');
$theData = fread($fh, filesize($myFile));
fclose($fh);
$csv = explode(",", $theData);
This is the CSV file in question
id,sub,type,regprice,natprice
1,4,Team,40,75
2,4,Individual,15,35
3,4,Stunt Group,50,150
4,4,Coed Partner Stunt,50,150
What i need to know how to do, is load the first line into an array separately, then loop through the remaining arrays in the following manner.
Array[0][0] - Array[0][1] - Array[0][2] - Array[0][3] - Array[0][4]
-------------------------------------------------------------------
Array[1][0] - Array[1][1] - Array[1][2] - Array[1][3] - Array[1][4]
Array[2][0] - Array[2][1] - Array[2][2] - Array[2][3] - Array[2][4]
Array[3][0] - Array[3][1] - Array[3][2] - Array[3][3] - Array[3][4]
Array[4][0] - Array[4][1] - Array[4][2] - Array[4][3] - Array[4][4]

fgetcsv() will probably do all this for you.

Try this:
foreach ($csv as $i=>$row) {
$rowStr = implode(' - ',$row)."\n";
print($rowStr);
if ($i == 0) {
print(str_repeat('-',strlen($rowStr))."\n");
}
}
Edit: fixed syntax error.

<?php
$myFile = "csv.csv";
$fh = fopen($myFile, 'r');
$headers = fgetcsv($fh);
$data = array();
while (! feof($fh))
{
$row = fgetcsv($fh);
if (!empty($row))
{
$obj = new stdClass;
foreach ($row as $i => $value)
{
$key = $headers[$i];
$obj->$key = $value;
}
$data[] = $obj;
}
}
fclose($fh);
print_r($data);
?>
This will output:
Array
(
[0] => stdClass Object
(
[id] => 1
[sub] => 4
[type] => Team
[regprice] => 40
[natprice] => 75
)
[1] => stdClass Object
(
[id] => 2
[sub] => 4
[type] => Individual
[regprice] => 15
[natprice] => 35
)
[2] => stdClass Object
(
[id] => 3
[sub] => 4
[type] => Stunt Group
[regprice] => 50
[natprice] => 150
)
[3] => stdClass Object
(
[id] => 4
[sub] => 4
[type] => Coed Partner Stunt
[regprice] => 50
[natprice] => 150
)
)

I found this to work
I added a / at the end of the CSV lines.
$myFile = "csv.csv";
$fh = fopen($myFile, 'r');
$theData = fread($fh, filesize($myFile));
fclose($fh);
$csvpre = explode("/", $theData);
$i = 1;
foreach ( $csvpre AS $key => $value){
$info = explode(",", $value);
if($i == "1"){
echo "$info[0] - $info[1] - $info[2] - $info[3] - $info[4]<br>";
$i++;
} else {
echo "<span style=\"color:#ff0000;\">$info[0] - $info[1] - $info[2] - $info[3] - $info[4]</span><br>";
}
}

$file_array=file('csv.csv');
$lines=count($file_array);
$first_line=explode(',',$file_array[0]);
$fl_text=implode(' - ',$first_line);
echo $fl_text.'<br>';
for($i=1;$i<$lines;$i++)
{
$line_text=str_replace(',',' - ',$file_array[$i]);
echo '<span style="color:#ff0000;">'.$line_text.'</span><br>';
}
This includes a couple of ways to print out the lines without needing to explode each one. You can replace the ',' with a space using str_replace, or you can explode and then implode. Probably str_replace is more efficient.
Also, the file() command reads each line of a file into values in an array.

Related

How to separate csv array by column?

I have a giant list in excel, they are just two columns
name type
The file is currently being read:
$lines = array_map('str_getcsv', file('file.csv', FILE_IGNORE_NEW_LINES));
print_r($lines); returns:
name1;type
name2;type2
...
Array ( [0] => Array ( [0] => name1;type1) [1] => Array ( [0] => name2;type2)...
I would like to access separate name and type in an foreach
How can I do this?
Thanks
str_getcsv default delimiter is ',' so you need call it somehow with explicitly specifying ';' as delimeter
for example like this
$myGetCsv = function ($str) {
return str_getcsv($str, ';');
};
$lines = array_map($myGetCsv, file('file.csv', FILE_IGNORE_NEW_LINES));
Use this code for CSV file reading. it ease to use and understand how its work.
if (($handle = fopen('./suppression_invalid_emails.csv', "r")) !== false) {
//getting column name
$column_headers = fgetcsv($handle);
foreach ($column_headers as $header) {
//column as array;
$result[$header] = array();
}
// get data for column;
while (($data = fgetcsv($handle)) !== false) {
$i = 0;
foreach ($result as &$column) {
$column[] = $data[$i++];
}
}
fclose($handle);
echo '<pre>'; print_r($result); echo '</pre>';
}
i use two basic function for this 1st one is fopen for reading file and other is fgetcsv for getting data for column.
and result is:
Array
(
[reason] => Array
(
[0] => Mail domain mentioned in email address is unknown
[1] => Known bad domain
[2] => Known bad domain
)
[email] => Array
(
[0] => muneque#collegeclub.om
[1] => saharfa2000#hatmail.com
[2] => tawheeda81#yahoo.co
)
[created] => Array
(
[0] => 1502644294
[1] => 1502480171
[2] => 1502344320
)
)

PHP - Fastest way to convert a 2d array into a 3d array that is grouped by a specific value

I would like to convert this two dimensional array of records:
[records] => Array
(
[0] => Array
(
[0] => Pears
[1] => Green
[2] => Box
[3] => 20
)
[1] => Array
(
[0] => Pears
[1] => Yellow
[2] => Packet
[3] => 4
)
[2] => Array
(
[0] => Peaches
[1] => Orange
[2] => Packet
[3] => 4
)
[3] => Array
(
[0] => Apples
[1] => Red
[2] => Box
[3] => 20
)
)
Into this three dimensional array where each array key is grouped by a certain value from the original array:
[converted_records] => Array
(
[Pears] => Array
(
[0] => Array
(
[0] => Green
[1] => Box
[2] => 20
)
[1] => Array
(
[0] => Yellow
[1] => Packet
[2] => 4
)
)
[Peaches] => Array
(
[0] => Array
(
[0] => Orange
[1] => Packet
[2] => 4
)
)
[Apples] => Array
(
[0] => Array
(
[0] => Red
[1] => Box
[2] => 20
)
)
)
I can do this like so:
$array = // Sample data like the first array above
$storage = array();
$cnt = 0;
foreach ($array as $key=>$values) {
$storage[$values[0]][$cnt] = array (
0 => $values[1],
1 => $values[2],
2 => $values[3]
);
$cnt ++;
}
I wanted to know if there is a more optimal way to do this. I am not aware of any functions within PHP that are capable of this so I can only assume that this is basically how it would be done.
The problem is though, this is going to be repeated so so many times and every little millisecond is going to count so I really want to know what is the best way to accomplish this task?
EDIT
The records array is created by parsing a .CSV file as follows:
$records = array_map('str_getcsv', file('file.csv'));
EDIT #2
I did a simple benchmark test on a set of 10 results (5k records each) to get an average runtime of 0.645478 seconds. Granted there is a few other things going on before this so this is not a true indication of actual performance but a good indication for comparison to other methods.
EDIT #3
I did a test with about 20x the records. The average of my routine was 14.91971.
At some point the answer below by #num8er had $records[$key][] = array_shift($data); before updating the answer as it is now.
When I tried testing with the larger set of results this it ran out of memory as its generating an error for each record.
This being said, once i did $records[$key][] = $data; the routine completed with an average of 18.03699 seconds with gc_collect_cycles() commented out.
I've reached the conclusion that although #num8ers method is faster for smaller files, for larger ones my method works out quicker.
If you're only looking for some clean code:
$array = array_map('str_getcsv', file('file.csv'));
$storage = array();
foreach ($array as $values) {
$key = array_shift($values);
$storage[$key][] = $values;
}
Unless you have hundreds of thousands of array entries, speed shouldnt be a concern either.
reading big file to memory using file() (1st iteration when it reads file)
and then iterating lines using array_map (2nd iteration after each line of file is read to array)
doing foreach on array (3rd iteration)
it is bad idea when You're looking for performance.
You're iterating 3 times. so what about 100K records? it will iterate 300K times?
most performant way is to do it while reading file. there is only 1 iteration - reading lines (100K records == 100K iteration):
ini_set('memory_limit', '1024M');
set_time_limit(0);
$file = 'file.csv';
$file = fopen($file, 'r');
$records = array();
while($data = fgetcsv($file)) {
$key = $data[0];
if(!isset($records[$key])) {
$records[$key] = array();
}
$records[$key][] = array(0 => $data[1],
1 => $data[2],
2 => $data[3]);
gc_collect_cycles();
}
fclose($file);
and here is parent -> children processing for huge files:
<?php
ini_set('memory_limit', '1024M');
set_time_limit(0);
function child_main($file)
{
$my_pid = getmypid();
print "Starting child pid: $my_pid\n";
/**
* OUR ROUTINE
*/
$file = fopen($file, 'r');
$records = array();
while($data = fgetcsv($file)) {
$key = $data[0];
if(!isset($records[$key])) {
$records[$key] = array();
}
$records[$key][] = array(0 => $data[1],
1 => $data[2],
2 => $data[3]);
gc_collect_cycles();
}
fclose($file);
unlink($file);
return 1;
}
$file = __DIR__."/file.csv";
$files = glob(__DIR__.'/part_*');
if(sizeof($files)==0) {
exec('split -l 1000 '.$file.' part_');
$files = glob(__DIR__.'/part_*');
}
$children = array();
foreach($files AS $file) {
if(($pid = pcntl_fork()) == 0) {
exit(child_main($file));
}
else {
$children[] = $pid;
}
}
foreach($children as $pid) {
$pid = pcntl_wait($status);
if(pcntl_wifexited($status)) {
$code = pcntl_wexitstatus($status);
print "pid $pid returned exit code: $code\n";
}
else {
print "$pid was unnaturally terminated\n";
}
}
?>

Calculating average value for array with PHP?

My head is exploding with this. I can't seem to create a working solution.
I have a file in this format:
99895|35378|0.01
99895|813|-0.97
99895|771|0.29
442|833|-1.06
442|485|-0.61
442|367|-0.14
442|478|0.77
442|947|-0.07
7977|987|0.76
7977|819|0.37
7977|819|0.36
7977|653|1.16
7977|1653|1.15
I want to calculate average values from third column for each id from the first column.
This seems so easy but I can't get this to work. How do you get averages for any said id from first column?
EDIT:
Some sample code I've written before:
$file = file_get_contents("results.txt");
$file = explode("
", $file);
$howMany = count($file);
for($a = 0;$a<$howMany;$a++)
{
$row = explode("|", $file[$a]);
$id = $row[0];
$howManyAlready = count($bigTable[$id]);
$bigTable[$id][$howManyAlready] = $row[2];
}
I've added the code. So far it gets the results into an array ( with offset corresponding to its id ) But I am having trouble with how to get those results and calculate average for each id and then present it on the screen.
Something like this should definitely work..
<?php
$arr=array();
$arrv=array_filter(file('myfile.txt'),'strlen');
foreach($arrv as $v)
{
array_push($arr,#array_pop(explode('|',$v)));
}
echo $avg = array_sum($arr)/count($arr);
You can try doing this :
$file = file_get_contents("results.txt");
$file = explode("
", $file);
$valuesPerID = Array();
foreach($file as $row)
{
$row_array = explode("|", $row);
$id = $row_array[0];
if(!array_key_exists($id, $valuesPerID))
{
$valuesPerID[$id] = Array();
}
$valuesPerID[$id][] = $row_array[2];
}
Now in the $valuesPerID array, you'll have all your ID as keys, and for each ID, all the values associated with the said ID. They you can easily calculate the average of these values !
You can use array mapping to assign value and id.
For example (assume that you have handled the text):
<?php
$data = array("99895|35378|0.01",
"99895|813|-0.97",
"99895|771|0.29",
"442|833|-1.06",
"442|485|-0.61",
"442|367|-0.14",
"442|478|0.77",
"442|947|-0.07",
"7977|987|0.76",
"7977|819|0.37",
"7977|819|0.36",
"7977|653|1.16",
"7977|1653|1.15");
$bucket = array();
$count = array();
foreach($data as $line) {
list($id, $what_s_this, $number) = explode("|", $line);
$count[$id]++;
$bucket[$id]+= (float)$number;
}
foreach($bucket as $id => $sum) {
echo "id:". $id. ", average". $sum / $count[$id]. "\n";
}
This should put you on the right track.
<?php
$values = array();
foreach (file('file.txt') as $line) {
list($id, $thingymabob, $value) = explode('|', $line);
if ( ! array_key_exists($id, $values)) {
$values[ $id ] = array();
}
array_push($values[ $id ], $value);
}
foreach ($values as $id => $value) {
printf(
"%d has an average of %f\n",
$id,
array_sum($value) / count($value)
);
}
Here some something I've just written.
In my example it takes it from a string.
<?php
$test1 = "";
$test2 = "";
$count1 = 0;
$count2 = 0;
$string = "99895|35378|0.01
99895|813|-0.97
99895|771|0.29
442|833|-1.06
442|485|-0.61
442|367|-0.14
442|478|0.77
442|947|-0.07
7977|987|0.76
7977|819|0.37
7977|819|0.36
7977|653|1.16
7977|1653|1.15";
$d = explode("\n", $string);
foreach ($d as $k => $v)
{
$d2 = explode("|", $v);
if ($d2[0] == '99895'){
$count1++;
$test1 += $d2[2];
}
if ($d2[0] == '442'){
$count2++;
$test2 += $d2[2];
}
}
$result1 = $test1 / $count1;
$result2 = $test2 / $count2;
echo $result1. " <br> ". $result2;
I don't know how well this will work as I don't know if the values are set or not.
If You try below code
<?php
$file_content = array();
$handle = fopen("test.txt", "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
// process the line read.
$line_ex = explode("|",$line);
array_push($file_content,$line_ex);
}
} else {
// error opening the file.
}
echo "<pre>";
print_r($file_content);
echo "<pre>";
?>
then You will get below output
Array
(
[0] => Array
(
[0] => 99895
[1] => 35378
[2] => 0.01
)
[1] => Array
(
[0] => 99895
[1] => 813
[2] => -0.97
)
[2] => Array
(
[0] => 99895
[1] => 771
[2] => 0.29
)
[3] => Array
(
[0] => 442
[1] => 833
[2] => -1.06
)
[4] => Array
(
[0] => 442
[1] => 485
[2] => -0.61
)
[5] => Array
(
[0] => 442
[1] => 367
[2] => -0.14
)
[6] => Array
(
[0] => 442
[1] => 478
[2] => 0.77
)
[7] => Array
(
[0] => 442
[1] => 947
[2] => -0.07
)
[8] => Array
(
[0] => 7977
[1] => 987
[2] => 0.76
)
[9] => Array
(
[0] => 7977
[1] => 819
[2] => 0.37
)
[10] => Array
(
[0] => 7977
[1] => 819
[2] => 0.36
)
[11] => Array
(
[0] => 7977
[1] => 653
[2] => 1.16
)
[12] => Array
(
[0] => 7977
[1] => 1653
[2] => 1.15
)
)
For determining average value from third column of corresponding first column - I am researching on it. When I will be done I'll put it here.
Try this:
$filename = 'results.txt';
$result = $counter = $values = array();
$file = fopen($filename, 'r') or die("Couldn't open $filename");
while ($line = fgets($file)) {
$content = explode('|', $line);
if (empty($content[0]) or empty($content[2]))
continue;
$values[$content[0]][] = (float) $content[2];
++$counter[$content[0]];
}
foreach ($values as $key => $value) {
$result[$key] = array_sum($value) / $counter[$key];
}
fclose($file);

Reading 2D Array from a text file, HELP!

UPDATE: I realized that the issue was I was using print instead of echo to print the data, so it was showing the array instead of the data within it. Thanks a ton guys!
I currently have a text file that looks like this:
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
And I'm using this function:
function rFile($fileName){
$resultF = fopen($fileName, "r") or die("can't open file");
$array = array(); //Create the first dimension of a 2D array
$i=0;
while(!feof($resultF)){
$line = fgets($resultF);
$line = trim($line, "\n");
$tokens = explode(",",$line);
$array[$i]=array(); //Create the second dimension of the 2D array
$tokenCount = sizeof($tokens);
for($j=0; $j<$tokenCount; $j++){
$array[$i][$j] = $tokens[$j];
}
$i++;
}
return $array;
}
Essentially, it's supposed to read through the file, explode each "0" and store it in a 2D array, $array. For some reason it returns this:
Array[0]
Array[1]
Array[2]
....etc etc
Anyone know what I did wrong?
PHP multi-dimensional arrays are just arrays of arrays. There's no need for the inner loop. You can just do
while(...) {
... fgets stuff
$array[$i] = explode(',', $line);
$i++;
}
and get the same effect.
You're going about it the hard way, by using for loops and counters. By using PHP's $array[] = $val append syntax, you can save a lot of effort here.
// all the file handling code...
while(!feof($resultF)){
$line = fgets($resultF);
$line = trim($line, "\n");
$tokens = explode(",",$line);
// Append the line array.
$array[] = $tokens; //Create the second dimension of the 2D array
}
return $array;
Or to be even more concise:
$array[] = explode(",",$line);
try this :
function rFile($filename) {
$lines = file($filename);
if ($lines !== false) {
foreach ($lines as & $line) {
$line = explode(',', trim($line, '\n\r'));
}
}
return $lines;
}
$f = file($fileName);
$results = array();
foreach ($f as $line) {
$line = preg_replace("\n", "", $line);
$results[] = explode(",", $line);
}
Hmm, since this is comma delimitated, we can use fgetcsv to make this short and simple:
$file = fopen('test.txt', 'r');
$matrix = array();
while($entries = fgetcsv($file)) {
$matrix[] = $entries;
}
fclose($file);
Resulting array:
Array
(
[0] => Array
(
[0] => 0
[1] => 0
[2] => 0
[3] => 0
[4] => 0
)
[1] => Array
(
[0] => 0
[1] => 0
[2] => 0
[3] => 0
[4] => 0
)
[2] => Array
(
[0] => 0
[1] => 0
[2] => 0
[3] => 0
[4] => 0
)
[3] => Array
(
[0] => 0
[1] => 0
[2] => 0
[3] => 0
[4] => 0
)
[4] => Array
(
[0] => 0
[1] => 0
[2] => 0
[3] => 0
[4] => 0
)
)

PHP, Building ID'd array from non-incremental ID'd CSV

Basically, here is my CSV File:
1,"Gold"
2,"English Version"
10,"Sword+0"
11,"Sword+1"
12,"Sword+2"
And so on, you get the idea. There are other parts where the ID is not incremental, perhaps one is 2899 and then the next one is 3020. I'm trying to build an array from this with fgetcsv();. I can do it fine, but I've failed so far to match up my array IDs with the ID from the CSV.
Here's a simple one that simply builds an incremental array from the file:
$file = fopen("item_proto.csv", "r");
$i = 1;
while(! feof($file)){
$gvar['item'][$i] = (fgetcsv($file));
$i++;
}
fclose($file);
This of course results in:
Array
(
[item] => Array
(
[1] => Array
(
[0] => 1
[1] => Gold
)
[2] => Array
(
[0] => 2
[1] => English Version
)
[3] => Array
(
[0] => 10
[1] => Sword+0
But I'd like [item][x] to match up with [item][x][y].
Try this:
$file = fopen("item_proto.csv", "r");
$i = 1;
while(! feof($file)){
$line = fgetcsv($file);
$gvar['item'][$line[0]] = $line;
$i++;
}
fclose($file);

Categories