merge csv files using PHP - php

I have a .csv file which I can use with Google maps API to successfully create map data.
What I'm looking to do is merge 2 (or more) .csv files and display the TOTAL data on the Google map in the same way. They are all in the same format.
I have the paths to the 2 csv files and if need be, a blank .csv file in the same directory where the files could be merged to...
Unfortuantely, the .csv files all have an initial 'header row' which would be awesome to omit when merging...
If anyone can point me in the right direction, I'd be very happy. Thanks
edit: I've tried:
$data1 = file_get_contents('google_map_data.csv');
$data2 = file_get_contents('google_map_data2.csv');
$TOTALdata = "google_map_dataALL.csv";
function joinFiles(array $files, $result)
{
if(!is_array($files)) {
throw new Exception('`$files` must be an array');
}
$wH = fopen($result, "w+");
foreach($files as $file)
{
$fh = fopen($file, "r");
while(!feof($fh))
{
fwrite($wH, fgets($fh));
}
fclose($fh);
unset($fh);
fwrite($wH, "\n"); //usually last line doesn't have a newline
}
fclose($wH);
unset($wH);
joinFiles(array($data1, $data2), $TOTALdata);

I'm assuming both files are small, so loading them all in one go should be OK.
The code loads both files then removes the first line off the second one. It also removes any end of line from the first file, but adds it's own to ensure it always has a new line...
$data1 = file_get_contents('google_map_data.csv');
$data2 = file_get_contents('google_map_data2.csv');
$TOTALdata = "google_map_dataALL.csv";
$data2 = ltrim(strstr($data2, PHP_EOL));
file_put_contents($TOTALdata, rtrim($data1).PHP_EOL.$data2);

Related

fputcsv - Combining multiple CSVs into one; empty enclosures "" only output with delimiter

I'm coding a plugin that runs everyday at 5am. It combines multiple csv files (That have a txt extension).
Currently, it is working... HOWEVER, the output format is incorrect.
The input will look like this:
"","","","","email#gmail.com","PARK PLACE 109 AVE","SOME RANDOM DATA","","","",""
And so on. this is only a partial row.
The ouput of this code does not retun the same format. It produces something like this without the " in columns without data
,,,,email#gmail.com,"PARK PLACE 109 AVE","SOME RANDOM DATA",,,,
Here is the part of the function that combines everything:
function combine_and_email_csv_files() {
// Get the current time and date
$now = new DateTime();
$date_string = $now->format('Y-m-d_H-i-s');
// Get the specified directories
$source_directory = get_option('csv_file_combiner_source_directory');
$destination_directory = get_option('csv_file_combiner_destination_directory');
// Load the CSV files from the source directory
$csv_files = glob("$source_directory/*.txt");
// Create an empty array to store the combined CSV data
$combined_csv_data = array();
// Loop through the CSV files
foreach ($csv_files as $file) {
// Load the CSV data from the file
$csv_data = array_map('str_getcsv', file($file));
// Add the CSV data to the combined CSV data array
$combined_csv_data = array_merge($combined_csv_data, $csv_data);
}
// Create the combined CSV file
$combined_csv_file = fopen("$destination_directory/$date_string.txt", 'w');
// Write the combined CSV data to the file
foreach ($combined_csv_data as $line) {
fputcsv($combined_csv_file, $line);
}
// Close the combined CSV file
fclose($combined_csv_file);
}
No matter, what I've tried... it's not working. I'm missing something simple I know.
Thank you Nigel!
So this thread, Forcing fputcsv to Use Enclosure For *all* Fields helped me get there....
Using fputs instead of fputscsv and force "" on null values is the short answer for me. Works beautifully... code is below:
function combine_and_email_csv_files() {
// Get the current time and date
$now = new DateTime();
$date_string = $now->format('Y-m-d_H-i-s');
// Get the specified directories
$source_directory = get_option('csv_file_combiner_source_directory');
$destination_directory = get_option('csv_file_combiner_destination_directory');
// Load the CSV files from the source directory
$csv_files = glob("$source_directory/*.txt");
// Create an empty array to store the combined CSV data
$combined_csv_data = array();
// Loop through the CSV files
foreach ($csv_files as $file) {
// Load the CSV data from the file
$csv_data = array_map('str_getcsv', file($file));
// Add the CSV data to the combined CSV data array
$combined_csv_data = array_merge($combined_csv_data, $csv_data);
}
// Create the combined CSV file
$combined_csv_file = fopen("$destination_directory/$date_string.txt", 'w');
// Write the combined CSV data to the file
foreach ($combined_csv_data as $line) {
// Enclose each value in double quotes
$line = array_map(function($val) {
if (empty($val)) {
return "\"\"";
}
return "\"$val\"";
}, $line);
// Convert the line array to a CSV formatted string
$line_string = implode(',', $line) . "\n";
// Write the string to the file
fputs($combined_csv_file, $line_string);
}
Thank you Sammitch
After much haggling with this problem... Sammitch pointed out why not just concat the files... Simplicity is the ultimate sophistication... right?
*Note: this will only work for my specific circumstance. All I'm doing now is concating the files and checking each file ends with a new line and just plain skipping the csv manipulation.
Code below:
function combine_and_email_csv_files() {
// Get the current time and date
$now = new DateTime();
$date_string = $now->format('Y-m-d_H-i-s');
// Get the specified directories
$source_directory = get_option('csv_file_combiner_source_directory');
$destination_directory = get_option('csv_file_combiner_destination_directory');
// Load the files from the source directory
$files = glob("$source_directory/*.txt");
// Create the combined file
$combined_file = fopen("$destination_directory/$date_string.txt", 'w');
// Loop through the files
foreach ($files as $file) {
// Read the contents of the file
$contents = file_get_contents($file);
// Ensure that the file ends with a newline character
if (substr($contents, -1) != "\n") {
$contents .= "\n";
}
// Write the contents of the file to the combined file
fwrite($combined_file, $contents);
}
// Close the combined file
fclose($combined_file);

Move first row from a csv to another using php

What I need to do is to be able to move the first row from a testdata.csv every time I run the .php to another .csv with the name testdata_new.csv(appending data).
This is an example of data that includes Name, Age, Job
Example data testdata.csv:
John,32,Scientist
Mary,25,Employer
Nick,36,Designer
Miky,46,Sales
Alex,29,Logistics
This is what the .php will do running it:
Cut the first row from testdata.csv(john,32,scientist) and paste it to the new testdata_new.csv under the first row(header) that will always be "Name Age Job".
Save testdata_new.csv and testdata.csv with the remaining rows.
I did some tests but I'm still far away from the solution.
<?php
$file = "testdata.csv";
$f = fopen($file, "r");
$i = 0;
$file2 = str_replace(".csv", "_new.csv", $file);
$f2 = fopen($file2,"a");
while ($i<2) {
$record = fgetcsv($f);
foreach($record as $field) {
echo $field . "<br>";
}
$i++;
}
fwrite($f2,fread($f, filesize($file)));
fclose($f);
fclose($f2);
?>
Executing the script will display the first row of the template.csv file
and will produce another file with the name template_new.csv with the following rows:
Mary,25,Employer
Nick,36,Designer
Miky,46,Sales
Alex,29,Logistics
What I really need to have in the template_new.csv file is only the first row displayed:
John,32,Scientist
And save again the template.csv without the first row as the idea is to cut and paste the rows, as following:
Mary,25,Employer
Nick,36,Designer
Miky,46,Sales
Alex,29,Logistics
Thank you all in advance for your help!
As easy as this ;-)
$old_file = 'testdata.csv';
$new_file = 'testdata_new.csv';
$file_to_read = file_get_contents($old_file); // Reading entire file
$lines_to_read = explode("\n", $file_to_read); // Creating array of lines
if ( $lines_to_read == '' ) die('EOF'); // No data
$line_to_append = array_shift( $lines_to_read ); // Extracting first line
$file_to_append = file_get_contents($new_file); // Reading entire file
if ( substr($file_to_append, -1, 1) != "\n" ) $file_to_append.="\n"; // If new file doesn't ends in new line I add it
// Writing files
file_put_contents($new_file, $file_to_append . $line_to_append . "\n");
file_put_contents($old_file, implode("\n", $lines_to_read));

php compare two file sizes then change file name if one is larger

I am new here and need a bit of help. I have a php script which is pulling data out of a database and creating .csv files. I need to add some logic to the script which can compare two files and then rename the file if the files size is equal to or greater than a specific (TBD) size.
Basically this script runs twice a hour and I would only like the .csv files rewritten if the file size is large enough. THis is all in hopes that It will prevent .csv files being created which are incomplete or too small.
This a bit of the code which is creating the .csv documents. Any help would be much appreciated.
$course_csv = fopen('/Course.csv','w');
$courses_u = array_unique($courses, SORT_REGULAR);
foreach($courses_u as $course){
fputcsv($course_csv, $course, '|');
}
fclose($course_csv);
$data = file('/Course.csv');
$handle = fopen("/Course.csv", "w");
foreach ($data as $line) {
$line = str_replace(array("\r\n", ',','"'), "", $line);
fwrite($handle, "{$line}");
$maxfilesize = 2048;
$myfilesize = filesize('/Course.csv');
if ($myfilesize > $maxfilesize) {
rename('/Course.csv', '/CourseToBig.csv');
}

Append at the beginning of the file in PHP [duplicate]

This question already has answers here:
Need to write at beginning of file with PHP
(10 answers)
Closed 9 years ago.
Hi I want to append a row at the beginning of the file using php.
Lets say for example the file is containing the following contnet:
Hello Stack Overflow, you are really helping me a lot.
And now i Want to add a row on top of the repvious one like this:
www.stackoverflow.com
Hello Stack Overflow, you are really helping me a lot.
This is the code that I am having at the moment in a script.
$fp = fopen($file, 'a+') or die("can't open file");
$theOldData = fread($fp, filesize($file));
fclose($fp);
$fp = fopen($file, 'w+') or die("can't open file");
$toBeWriteToFile = $insertNewRow.$theOldData;
fwrite($fp, $toBeWriteToFile);
fclose($fp);
I want some optimal solution for it, as I am using it in a php script. Here are some solutions i found on here:
Need to write at beginning of file with PHP
which says the following to append at the beginning:
<?php
$file_data = "Stuff you want to add\n";
$file_data .= file_get_contents('database.txt');
file_put_contents('database.txt', $file_data);
?>
And other one here:
Using php, how to insert text without overwriting to the beginning of a text file
says the following:
$old_content = file_get_contents($file);
fwrite($file, $new_content."\n".$old_content);
So my final question is, which is the best method to use (I mean optimal) among all the above methods. Is there any better possibly than above?
Looking for your thoughts on this!!!.
function file_prepend ($string, $filename) {
$fileContent = file_get_contents ($filename);
file_put_contents ($filename, $string . "\n" . $fileContent);
}
usage :
file_prepend("couldn't connect to the database", 'database.logs');
My personal preference when writing to a file is to use file_put_contents
From the manual:
This function is identical to calling fopen(), fwrite() and fclose()
successively to write data to a file.
Because the function automatically handles those three functions for me I do not have to remember to close the resource after I'm done with it.
There is no really efficient way to write before the first line in a file. Both solutions mentioned in your questions create a new file from copying everything from the old one then write new data (and there is no much difference between the two methods).
If you are really after efficiency, ie avoiding the whole copy of the existing file, and you need to have the last inserted line being the first in the file, it all depends how you plan on using the file after it is created.
three files
Per you comment, you could create three files header, content and footer and output each of them in sequence ; that would avoid the copy even if header is created after content.
work reverse in one file
This method puts the file in memory (array).
Since you know you create the content before the header, always write lines in reverse order, footer, content, then header:
function write_reverse($lines, $file) { // $lines is an array
for($i=count($lines)-1 ; $i>=0 ; $i--) fwrite($file, $lines[$i]);
}
then you call write_reverse() first with footer, then content and finally header. Each time you want to add something at the beginning of the file, just write at the end...
Then to read the file for output
$lines = array();
while (($line = fgets($file)) !== false) $lines[] = $line;
// then print from last one
for ($i=count($lines)-1 ; $i>=0 ; $i--) echo $lines[$i];
Then there is another consideration: could you avoid using files at all - eg via PHP APC
You mean prepending. I suggest you read the line and replace it with next line without losing data.
<?php
$dataToBeAdded = "www.stackoverflow.com";
$file = "database.txt";
$handle = fopen($file, "r+");
$final_length = filesize($file) + strlen($dataToBeAdded );
$existingData = fread($handle, strlen($dataToBeAdded ));
rewind($handle);
$i = 1;
while (ftell($handle) < $final_length)
{
fwrite($handle, $dataToBeAdded );
$dataToBeAdded = $existingData ;
$existingData = fread($handle, strlen($dataToBeAdded ));
fseek($handle, $i * strlen($dataToBeAdded ));
$i++;
}
?>

Separating txt file to many csv files with PHP

So I was able to parse the txt file into a csv file
$data = array();
while ($line= fgets ($fh)) {
$stack = array($LAUS,$FIPS,$CountyName,$Date,$_CLF,$_EMP,$_UNEMP,$RATE);
array_push($data, $stack);
}
$file = fopen('file.csv','w');
foreach ($data as $fields) {
fputcsv($file, $fields,',','"');
}
fclose($file);
My question is, what is the best way to create multiple csv files that are seperated by Month (also the year, like Jan01.csv, Jan02.csv).
I'm taking a bit of a guess at the formatting of your date
while ($line = fgets ($fh)) {
// Not sure where you're getting these values, but I'm assuming it's correct
$stack = array($LAUS,$FIPS,$CountyName,$Date,$_CLF,$_EMP,$_UNEMP,$RATE);
// Assuming $Date looks like this '2011-10-04 15:00:00'
$filename = date('My', strtotime($Date)) . '.csv';
$file = fopen($filename,'a+');
fputcsv($file, $stack,',','"');
fclose($file);
}
This will be a little slow since you're opening and closing files constantly, but since I don't know the size of your original data set I don't want to use up all the memory caching the result before I write it.
Be aware that running this multiple times will end up with duplicate data being inserted into your CSV files. You may want to add some code to remove/clear out any currently existing CSV files before you run this bit of code.

Categories