I am attempting to insert the data from an uploaded file into a single dimension array.
The file is as such that there is one student number to a line like so:
392232,392231,etc
this is the most common way I've found online:
while (($line = fgetcsv($file, 25, ',')) !== FALSE) {
//$line is an array of the csv elements
print_r($line);
}
However form what I understand this will create an array ($line) for each row. Which is not what I want.
that aside I tried this to see if it is working and my code is not printing out the array after using ftgetcsv(). The file is successfully uploading.
here is my code:
if(isset($_FILES['csv_file']) && is_uploaded_file($_FILES['csv_file']['tmp_name'])){
//create file name
$file_path = "csv_files/" . $_FILES['csv_file']['name'];
//move uploaded file to upload dir
if (!move_uploaded_file($_FILES['csv_file']['tmp_name'], $file_path)) {
//error moving upload file
echo "Error moving uploaded file";
}
print_r($_FILES['csv_file']);
$file = fopen('$file_path', 'r');
while (($line = fgetcsv($file, 25, ',')) !== FALSE) {
//$line is an array of the csv elements
print_r($line);
}
//delete csv file
unlink($file_path);
}
First off, can anyone obviously see why it wouldnt work to at least print them as seperate arrays of data (each row).
Second, is it possible to set it so that it creates a 1d array of all rows in the file?
Many thanks,
Question 1 is because of
print_r($_FILES['csv_file']);
$file = fopen('$file_path', 'r');
should be:
$file = fopen($file_path, 'r');
and for Question 2, check out the array_push
1st Question:
This line will actually try to open a file called '$file_path' because you're using single quotes (so it doesn't expand to the value of the variable). You can just remove the quotes.
$file = fopen('$file_path', 'r');
$file is null after this.
2nd Question:
If all you want to do is convert a file into an array by lines you can use one of these instead:
file() - get whole file into a 1D array of lines of the file (closest to what you want)
fgets() - get a string per line per call; keep calling this until it returns false to get each line one at a time
file_get_contents() - get the whole file into a string and process as you like
According to PHP.net $line has to return as array.
"returns an array containing the fields read."
But if you are sure it's contains only one student number you can use $line[0] to get the first line value (Ignoring the ",")
Here are some general comments on your code:
You are passing the file path into the fopen() function incorrectly. The variable should not be surrounded with single quotes.
Since you are deleting the CSV file after processing it, moving it is unnecessary. Simply use $_FILES['csv_file']['tmp_name'] as the path to the file.
Since there is only one entry per row in your CSV file, simply access the first element of the array that is returned from fgetcsv(): $numbers[] = $line[0];
Related
I need help processing files holding about 46k lines or more than 30MB of data.
My original idea was to open the file and turn each line into an array element. This worked the first time as the array held about 32k values total.
The second time, the process was repeated, the array only held 1011 elements, and finally, the third time it could only hold 100.
I'm confused and don't know much about the backend array processes. Can someone explain what is happening and fix the code?
function file_to_array($cvsFile){
$handle = fopen($cvsFile, "r");
$path = fread($handle, filesize($cvsFile));
fclose($handle);
//Turn the file into an array and separate lines to elements
$csv = explode(",", $path);
//Remove common double spaces
foreach ($csv as $key => $line){
$csv[$key] = str_replace(' ', '', str_getcsv($line));
}
array_filter($csv);
//get the row count for the file and array
$rows = count($csv);
$filerows = count(file($cvsFile)); //this no longer works
echo "File has $filerows and array has $rows";
return $csv;
}
The approach here can be split in 2.
Optimized file reading and processing
Proper storage solution
Optimized file processing can be done like so:
$handle = fopen($cvsFile, "r");
$rowsSucceed = 0;
$rowsFailed = 0;
if ($handle) {
while (($line = fgets($handle)) !== false) { // Reading file by line
// Process CSV line and check if it was parsed correctly
// And count as you go
if (!empty($parsedLine)) {
$csv[$key] = ... ;
$rowsSucceed++;
} else {
$rowsFailed++;
}
}
fclose($handle);
} else {
// Error handling
}
$totalLines = $rowsSucceed + $rowsFailed;
Also you can avoid array_filter() simply by not adding processed line if its empty.
It will allow to optimize memory usage during script execution.
Proper storage
Proper storage here is needed for performing operations on certain amount of data. File reading are ineffective and expensive. Using simple file based database like sqlite can help you a lot and increase overall performance of your script.
For this purpose you probably should process your CSV directly to database and than perform count operation on parsed data avoiding excessive file line counts etc.
Also it gives you further advantage on working with data not keeping it all in memory.
Your question says you want to "turn each line into an array element" but that is definitely not what you are doing. The code is quite clear; it reads the entire file into $path and then uses explode() to make one massive flat array of every element on every line. Then later you're trying to run str_getcsv() on each item, which of course isn't going to work; you've already exploded all the commas away.
Looping over the file using fgetcsv() makes more sense:
function file_to_array($cvsFile) {
$filerows = 0;
$handle = fopen($cvsFile, "r");
while ($line = fgetcsv($handle)) {
$filerows++;
// skip empty lines
if ($line[0] === null) {
continue;
}
//Remove common double spaces
$csv[] = str_replace(' ', '', $line);
}
//get the row count for the file and array
$rows = count($csv);
echo "File has $filerows and array has $rows";
fclose($handle);
return $csv;
}
My text file sample.txt. I want to exclude the first row from the text file and store the other rows into mysql database.
ID Name EMail
1 Siva xyz#gmail.com
2 vinoth xxx#gmail.com
3 ashwin yyy#gmail.com
Now I want to read this data from the text file except the first row(ID,name,email) and store into the MYsql db.Because already I have created a filed in database with the same name.
I have tried
$handle = #fopen($filename, "r"); //read line one by one
while (!feof($handle)) // Loop till end of file.
{
$buffer = fgets($handle, 4096); // Read a line.
}
print_r($buffer); // It shows all the text.
Please let me know how to do this?
Thanks.
Regards,
Siva R
It's easier if you use file() since it will get all rows in an array instead:
// Get all rows in an array (and tell file not to include the trailing new lines
$rows = file($filename, FILE_IGNORE_NEW_LINES);
// Remove the first element (first row) from the array
array_shift($rows);
// Now do what you want with the rest
foreach ($rows as $lineNumber => $row) {
// do something cool with the row data
}
If you want to get it all as a string again, without the first row, just implode it with a new line as glue:
// The rows still contain the line break, since we only trimmed the copy
$content = implode("\n", $rows);
Note: As #Don'tPanic pointed out in his comment, using file() is simple and easy but not advisable if the original file is large, since it will read the whole thing into memory as an array (and arrays take more memory than strings). He also correctly recommended the FILE_IGNORE_NEW_LINES-flag, just so you know :-)
You can just call fgets once before your while loop to get the header row out of the way.
$firstline = fgets($handle, 4096);
while (!feof($handle)) // Loop till end of file.
{ ...
I have a CSV that is downloaded from the wholesaler everynight with updated prices.
What I need to do is edit the price column (2nd column) and multiply the current value by 1.3 (30%).
My code to read the provided CSV and take just the columns I need is below, however I can't seem to figure out how to edit the price column.
<?php
// open the csv file in write mode
$fp = fopen('var/import/tb_prices.csv', 'w');
// read csv file
if (($handle = fopen("var/import/Cbl_4036_2408.csv", "r")) !== FALSE) {
$targetColumns = array(1, 2, 3); // get data from the 1st, 4th and 15th column
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$targetData = array(); // array that hold target data
foreach($targetColumns as $column){ // loop throught the targeted columns array
if($column[2]){
$data[$column] = $data[0] * 1.3;
}
$targetData[] = $data[$column]; // get the data from the column
}
# Populate the multidimensional array.
$csvarray[$nn] = $targetData; // add target data to csvarray
// write csv file
fputcsv($fp, $targetData);
}
fclose($handle);
fclose($fp);
echo "CSV File Written Successfully!";
}
?>
Could somebody point me in the right direction please, explaining how you've worked out the function too so I can learn at the same time.
You are multiplying your price column always as - $data[0] * 1.3.
It may be wrong here.
Other views:
If you are doing it once in a lifetime of this data(csv) handling, try to solve it using mysql itself only. Create the table similar to the database, import the .csv data into that mysql table. And then, SQL operate as you want.
No loops; no coding, no file read/write, and precise control over what you want to do with UPDATE. You just need to be aware of the delimiters (line separators eg. \r\n, column separators (eg. comma or tab or semicolon) and data encoding in double/single-quotes or not)
Once you modify your data, you can export it back to csv again.
If you want to handle the .csv file itself, open it in one connection (read only mode), and write to another file - saving the original data.
you say that the column that contains the price is the second but then use that index with zero. anyway the whole thing can be easier
$handle = fopen("test.csv", "r");
if ( $handle !== FALSE) {
$out = "";
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$data[1] = ((float)$data[1] * 1.3);
$out .= implode(";",$data) . "\n";
}
fclose($handle);
file_put_contents("test2.csv", $out);
}
this code open a csv file with comma as separator.
than read every line and for every line it's multiplies the second coloumn (index 1) for 1.3
this line
$out .= implode(";",$data) . "\n";
generate a line for new csb file. see implode on the officile documentation ...
after I close the connection to the file. and 'useless to have a connection with two files when you can do the writing of the second file in one fell swoop. the thing is true for small files
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
how to extract data from csv file in php
I have some data in XLS, I save them as CSV, the delimiter is **comma*.
Then I am trying to load the data from this CSV file:
$input = explode("\r\n", fread($file, filesize("my_data.csv")));
print_r($input);
The output:
Array ( [0] => data from the CSV file)
This is the problem - in the array is always just one item, where are printed out all data from the CSV file. How is that possible? Why isn't in the array as much items as is rows in the CSV file?
Also, I've tried to change "\r\n" for "\n", but it's the same.
What I am trying to do - load each line from the CSV file and this each line to process.
EXAMPLE OF THE FILE:
a,b,c,d
e,f,g,h
OUTPUT:
a,b,c,d e,f,g,h
I'd recommend using php's built in csv file reading function fgetcsv() and not create your own: http://php.net/manual/en/function.fgetcsv.php
if (($handle = fopen($file, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$row = implode (",",$data); //puts back together the row from the csv
echo $row. "\n"; //assuming you want a visual linebreak on console, add the \n
}
fclose($handle);
}
How I solved this issue:
In the XLS file, before exporting the file into CSV, I added in the end of sheet one more column with the char '.
I'm creating a script that will read a csv file and display it on a textarea using fgetcsv.
$handle = #fopen($filePath, "r");
if ($handle)
{
while (($buffer = fgetcsv($handle, 1000,",")) !== false)
{
foreach($buffer as $buff){
echo $buff."\n";
}
}
}
The format of the csv is
"line1-content1","line1-content2"
"line2-content1","line2-content2"
Using fgetcsv, the content will display inside the textarea without double-quote and comma. Can I format it so that it will also display the duoble quotes and comma?
Then upon saving it using fputcsv
$file_to_load = $_GET['filepath'];
$filePath = $dir.$file_to_load;
$trans = trim($_POST['txtarea']);
$keyarr = split("\n",$trans);
$fp = fopen($filePath, 'w');
foreach (array ($keyarr) as $fields) {
fputcsv($fp, $fields);
}
fclose($fp);
Looking on the csv file, it saved the csv but displays it like this
"line1-content1
","line1-content2
","line2-content1
","line2-content2"
It separates the "line1-content1" and "line1-content2" into two lines and put a comma after the end of every line.
Now I want to keep the formatting of #2. How will I code it?
Can you guide me into the right direction? Thanks!
Sounds like you want to display the actual raw CSV text, not the parsed data within the CSV. Instead of using fgetcsv(), just use fgets() and you'll get the text line without any parsing, preserving the quotes and commas.
As for fputcsv, it's going to write out what you pass into it, so make sure that whatever's coming back from the form is cleaned up (e.g. extra line breaks stripped out).