I'm using a simple function to write write arrays to a CSV-file, which look like this:
function writeToCSV($array) {
$fp = fopen('programmes.csv', 'a');
fputcsv($fp, $array);
fclose($fp);
}
Simple as a pie. However, is there anyway to know what line-number the pointer is at? Because I want to be able to after 1000 lines to begin writing to a new file. Why? Because I need to be able to import them to a database later with some memory constraints, and to parse a CSV-file with 15000 lines is a no-no.
function writeToCSV($array) {
$i = 1;
$j = 1;
$fp = fopen('programmes' . $j . '.csv', 'a');
foreach($array as $fields) {
if ($i % 1000 == 0) {
fclose($fp);
$fp = fopen('programmes' . $j . '.csv', 'a');
$j = $j + 1;
}
fputcsv($fp, $fields);
$i = $i + 1;
}
fclose($fp);
}
Try this:
count(file('programmes.csv'));
This will give you the number of lines in a file.
I haven't tried if this works, but i would do something like this:
<?php
function writeToCSV($array) {
// count lines in the current file
$linecount = 0;
$fh = fopen('programmes.csv','rb') or die("ERROR OPENING DATA");
while (fgets($fh) !== false) $linecount++;
fclose($fh);
$aSize = sizeof($array);
if (($linecount + $aSize) > 1000) {
// split array
$limit = 1000 - $linecount;
$a = array_slice($array, 0, $limit);
$b = array_slice($array, $limit);
// write into first file
$fp = fopen('programmes.csv', 'a');
foreach($a as $field) fputcsv($fp, $field);
fclose($fp);
// write into second file
$fp = fopen('programmes2.csv', 'a');
foreach($b as $field) fputcsv($fp, $field);
fclose($fp);
} else {
$fp = fopen('programmes.csv', 'a');
$idx = 0;
while ($linecount < 1000) {
// fill the file to the 1000 lines
fputcsv($fp, $array[$idx]);
++$linecount;
++$idx;
}
fclose($fp);
if ($idx != $aSize) {
// create new file
$fp = fopen('programmes.csv', 'a');
while ($idx< $aSize) {
// fill the file to the 1000 lines
fputcsv($fp, $array[$idx]);
++$idx;
}
fclose($fp);
}
}
}
?>
Related
I would like to split a very large CSV file (20 000 lines) to be able to put everything in my MySQL database (after I use cronjob for load the php file every hours).
So I have found code to split my file :
$inputFile = 'Shipping.csv';
$outputFile = 'output';
$splitSize = 1000;
$in = fopen($inputFile, 'r');
$rowCount = 0;
$fileCount = 1;
while (!feof($in)) {
if (($rowCount % $splitSize) == 0) {
if ($rowCount > 0) {
fclose($out);
}
$out = fopen($outputFile . $fileCount++ . '.csv', 'w');
}
$data = fgetcsv($in);
if ($data)
fputcsv($out, $data);
$rowCount++;
}
fclose($out);
I have tried to do this but they don't work :
require_once dirname(__DIR__).'/pdo.php';
$inputFile = 'Shipping.csv';
$outputFile = 'output';
$splitSize = 1000;
//open uploaded csv file with read only mode
$in = fopen($inputFile, 'r');
//skip first line
fgetcsv($in);
$rowCount = 0;
$fileCount = 1;
while (!feof($in)) {
if (($rowCount % $splitSize) == 0) {
if ($rowCount > 0) {
fclose($out);
}
$out = fopen($outputFile . $fileCount++ . '.csv', 'w');
}
$data = fgetcsv($in);
if ($data){
fputcsv($out, $data);
}
while(($line = fgetcsv($out)) !== FALSE){
//check whether member already exists in database with same email
$prevQuery = "SELECT id FROM shipbp WHERE license = '".$line[1]."'";
$prevResult = $bdd->query($prevQuery);
if($prevResult->rowCount() > 0)
{
$bdd->query("UPDATE shipbp SET license = '".$line[0]."'");
}
else{
$bdd->query("INSERT INTO shipbp (license) VALUES ('".$line[0]."')");
}
}
$rowCount++;
}
fclose($out);
CSV file look like :
P135460,002,00003,250,AS44563,0.35,,Blabla,17/3/2017 00:00:00,SB,Blabla
How I can do ?
Thank you for your help
help collect from 2 files csv - one. To merge similar cell. An example is shown below, and part of my code.
I would be very grateful for the help and thanks in advance
csv1.csv =
name;price
Tom;1000
Anna;2000
Aleks;3000
Maikal;2000
Piter;5000
csv2.csv =
name;price
Tom;1200
Anna;2300
Andre;2000
Maikal2;2400
all.csv - The resulting file.
name;price;price
Aleks;3000;
Andre;;2000
Anna;2000;2300
Piter;5000;
Maikal;2000;
Maikal2;;2400
Tom;1000;1200
I have a script that simply glued file based on 1 line. But it gives a different result.
if(#$_FILES["DATAF_1"]["size"]>0 AND #$_FILES["DATAF_2"]["size"]>0)
{
$data=array();
function read_csv($file)
{
$out=array();
$lines = file($file);
foreach ($lines as $line_num => $line)
{
$out[]=explode(";",$line);
}
return $out;
}
function action_csv(&$array,$input)
{
for ($i=0; $i<count($input); $i++)
{
for ($i2=1; $i2<count($input[$i]); $i2++) {
if(trim($input[$i][0])!=""){$array[trim($input[$i][0])][]=trim($input[$i][$i2]);}
}
}
}
function form_csv($data)
{
$out=array();
foreach ($data as $line_num => $line)
{
$out[]=$line_num.";".implode(";",$line);
}
return implode("\r\n",$out);
}
$data1=read_csv($_FILES["DATAF_1"]["tmp_name"]);
$data2=read_csv($_FILES["DATAF_2"]["tmp_name"]);
action_csv($data,$data1);
action_csv($data,$data2);
#unlink("out.csv");
$fp = fopen ("out.csv", "w+");
fwrite ($fp, form_csv($data));
fclose ($fp);
I tried your code without the $_Files (just from the filesystem) and it works. So the Problem might be in the file uploade and not in the code you posted.
EDIT:
So now it should work as you wish:
function genAllIndexes(&$array,&$maxIndex){
foreach($array as &$line){
for($i = 0; $i < $maxIndex; $i++){
if(!isset($line[$i])){
$line[$i] = "";
}
}
ksort($line);
}
}
function action_csv(&$array,$input, &$counter){
for ($i=0; $i<count($input); $i++){
for ($i2=1; $i2<count($input[$i]); $i2++) {
if(trim($input[$i][0])!=""){
$array[trim($input[$i][0])][$counter]=trim($input[$i][$i2]);
}
}
}
$counter++;
}
$data1=read_csv('data1.csv');
$data2=read_csv('data2.csv');
$counter = 0;
action_csv($data,$data1, $counter);
action_csv($data,$data2, $counter);
genAllIndexes($data, $counter);
ksort($data);
#unlink("out.csv");
$fp = fopen ("out.csv", "w+");
fwrite ($fp, form_csv($data));
fclose ($fp);
I want to export a row in CSV file to JSON file with require : 1 line in CSV export 1 file JSON. I success to export file,but can't filter the row that i want to export. Can anyone help me?
<?php
header('Content-type: application/json');
$feed = 'jsondata_palpad.csv';
$keys = array();
$newArray = array();
function csvToArray($file, $delimiter) {
if (($handle = fopen($file, 'r')) !== FALSE) {
$i = 0;
while (($lineArray = fgetcsv($handle, 4000, $delimiter, '"')) !== FALSE) {
for ($j = 0; $j < count($lineArray); $j++) {
$arr[$i][$j] = $lineArray[$j];
}
$i++;
}
fclose($handle);
}
return $arr;
}
$data = csvToArray($feed, ',');
$count = count($data) - 1;
$labels = array_shift($data);
foreach ($labels as $label) {
$keys[] = $label;
}
$keys[] = 'id';
for ($i = 0; $i < $count; $i++) {
$data[$i][] = $i;
}
for ($j = 0; $j < $count; $j++) {
$d = array_combine($keys, $data[$j]);
$newArray[$j] = $d;
}
//Xuat file JSON
for ($i = 0; $i < 5; $i++) {
$json = json_encode($newArray[$i]);
echo $json . "<br />\n";
$filename = $data[$i][1] . ".json";
echo $filename . "<br />\n";
//echo "title[" . $data[$i][4] . "]";
$handle = fopen("./" . $filename, "w");
fwrite($handle, $json);
fclose($handle);
}
?>
If recommend = 2,it will export line whre id=2.
I want to export id,product_id,title,outline to JSON file.This is sample JSON file:
http://img839.imageshack.us/img839/5277/21527799.png
This is CSV file :
http://img690.imageshack.us/img690/1526/43004273.png
You are looping through all 6 lines of the csv file and saving them to the JSON file:
for ($i = 0; $i < 5; $i++) {
...
}
If you know what row number it is you want, don't loop through every row - just call the code inside the loop for that row. For example, if you wanted the 2nd row:
$row_we_want = 1;
$json = json_encode($newArray[$row_we_want]);
$filename = $data[$row_we_want][1] . ".json";
$handle = fopen("./" . $filename, "w");
fwrite($handle, $json);
fclose($handle);
If you are unsure about which row it is and need to check each one, you could do so in the loop:
for ($i = 0; $i < 5; $i++) {
if (some_condition) {
$json = json_encode($newArray[$i]);
$filename = $i][1] . ".json";
$handle = fopen("./" . $filename, "w");
fwrite($handle, $json);
fclose($handle);
}
}
Also it might be worth replacing 5 in your loop with the length of the array using the count function if it is not always a fixed length.
I have a problem and need a help from you!
I want to convert CSV to JSON,and 1 line in CSV with export 1 file JSON (JSON's content is CSV line's content,and JSON's name is id of this line).
I try to do,but it's only display and download JSON.
Please give me a suggestion for this problem.
Thanks for your help!
<?php
header('Content-type: application/json');
$feed = 'jsondata_palpad.csv';
$keys = array();
$newArray = array();
function csvToArray($file, $delimiter) {
if (($handle = fopen($file, 'r')) !== FALSE) {
$i = 0;
while (($lineArray = fgetcsv($handle, 4000, $delimiter, '"')) !== FALSE) {
for ($j = 0; $j < count($lineArray); $j++) {
$arr[$i][$j] = $lineArray[$j];
}
$i++;
}
fclose($handle);
}
return $arr;
}
$data = csvToArray($feed, ',');
$count = count($data) - 1;
$labels = array_shift($data);
foreach ($labels as $label) {
$keys[] = $label;
}
$keys[] = 'id';
for ($i = 0; $i < $count; $i++) {
$data[$i][] = $i;
}
for ($j = 0; $j < $count; $j++) {
$d = array_combine($keys, $data[$j]);
$newArray[$j] = $d;
}
header("Content-type: application/json");
header("Content-Disposition: attachment; filename=test.json");
header("Expires: 0");
header("Pragma: no-cache");
echo json_encode($newArray);
?>
So you have a titled CSV file.
To read it, first convert the rows into an array:
$csv = array_map("str_getcsv", file($filename));
$head = array_shift($csv);
For conversion into an associative array:
$csv = array_map("array_combine", array_fill(0, count($csv), $head), $csv);
To generate the files:
foreach ($csv as $index => $row) {
file_put_contents("$index.json", json_encode($row));
}
Here the loop counter $index is used for generating the filenames.
See the manual for explanations on:
array_map
str_getcsv
file_put_contents
If you did want something else, write a more precise question next time.
how to remove every line except the first 20 using php from a text file?
If loading the entire file in memory is feasible you can do:
// read the file in an array.
$file = file($filename);
// slice first 20 elements.
$file = array_slice($file,0,20);
// write back to file after joining.
file_put_contents($filename,implode("",$file));
A better solution would be to use the function ftruncate which takes the file handle and the new size of the file in bytes as follows:
// open the file in read-write mode.
$handle = fopen($filename, 'r+');
if(!$handle) {
// die here.
}
// new length of the file.
$length = 0;
// line count.
$count = 0;
// read line by line.
while (($buffer = fgets($handle)) !== false) {
// increment line count.
++$count;
// if count exceeds limit..break.
if($count > 20) {
break;
}
// add the current line length to final length.
$length += strlen($buffer);
}
// truncate the file to new file length.
ftruncate($handle, $length);
// close the file.
fclose($handle);
For a memory efficient solution you can use
$file = new SplFileObject('/path/to/file.txt', 'a+');
$file->seek(19); // zero-based, hence 19 is line 20
$file->ftruncate($file->ftell());
Apologies, mis-read the question...
$filename = "blah.txt";
$lines = file($filename);
$data = "";
for ($i = 0; $i < 20; $i++) {
$data .= $lines[$i] . PHP_EOL;
}
file_put_contents($filename, $data);
Something like:
$lines_array = file("yourFile.txt");
$new_output = "";
for ($i=0; $i<20; $i++){
$new_output .= $lines_array[$i];
}
file_put_contents("yourFile.txt", $new_output);
This should work as well without huge memory usage
$result = '';
$file = fopen('/path/to/file.txt', 'r');
for ($i = 0; $i < 20; $i++)
{
$result .= fgets($file);
}
fclose($file);
file_put_contents('/path/to/file.txt', $result);