PHP Writing Complex Order Info to CSV - php

I am writing an order exporter in PHP and am having difficulty with php checking a csv header and then writing to a file.
I have my code which opens the file, then writes to the csv
$fh = fopen($file_compile, 'w');
I then write my header:
$header = '"OrderId", "Customer";
foreach ($products as $product) {
$header .= ', "' . $product . '"';
}
fputs($fh, $header . "\n");
which gives me an output
I now generate my $line array
which outputs:
array(3) {
["OrderId"]=>
string(9) "100000033"
["Customer"]=>
string(14) "Graeme Houston"
["total"]=>
array(3) {
["Socks"]=>
int(12)
["Books"]=>
int(23)
["Wallets"]=>
int(12)
}
}
Now as you can see, my array doesn't exactly match the CSV above it, which is fine, what I would like to achieve is, if there is a value in the total that matches a value in the header, to but the integer value in the cell. (scratches head)
To illustrate:
I must add I have been trying to figure this out for a few days, I cant get my head round it, hence the post on here. I would be extremely grateful if someone could point me in the right direction.

Not that complex...
$header = array('OrderId','Customer');
foreach ($products as $product) {
$header[] = $product;
}
$tmp = '"' .implode('","',$header) . '"';
fputs($fh, $tmp . "\n");
For your header construction... Then for you line output something similar to:
foreach($header as $key){
if(empty($line[$key])) $line[$key] = NULL;
if(empty($line[$key]) && !empty($line['total'][$key])){
$line[$key] = $line['total'][$key];
}
}
unset($line['total']);
Now you've filled the gaps for non-existent columns, and given values to the existent ones.

Related

How to remove unwanted characters/text - php

When I retrieve a file, it outputs:
...22nlarray(3) { [0]=> string(62) "/public_html/wp/wp-content/plugins/AbonneerProgrammas/Albums/." [1]=> string(63) "/public_html/wp/wp-content/plugins/AbonneerProgrammas/Albums/.." [2]=> string(69) "/public_html/wp/wp-content/plugins/AbonneerProgrammas/Albums/22nl.mp3" }
I, however, only want the 22nl displayed. I do not want the rest over there. How can I do that? Is there a function that deletes the rest of the output except 22nl (which is a filename)?
My PHP code:
// get contents of the current directory
$contents = ftp_nlist($conn_id, $destination_folder);
foreach ($contents as $mp3_url) {
$filename = basename($mp3_url, ".mp3");
echo "<a href='$mp3_url'>$filename</a>";
}
var_dump($contents);
There are similar questions to mine, however, they did not provide a good answer for me.
Greetings,
Rezoo Aftib
We can say that your value from the array will be
$value='string(69) "/public_html/wp/wp-content/plugins/AbonneerProgrammas/Albums/22nl.mp3"';
So, you can make that to export your file name:
$value = explode('"',$value);
$exploded = explode('/', $value[1]);
$full_mp3_name = end($exploded);
$just_name = explode(".",$full_mp3_name);
$just_name = $just_name[0];
When you print $full_mp3_name you will need to have 22nl.mp3
When you print $just_name you will need to have 22nl
It will better with functions but its example if it will be option for you.

PHP CSV-Upload UTF-8 (with and without BOM)

Can someone perhaps explain me the difference - and how to recognize or change the format?
I've a simple HTML-Upload-Form and after uploading I parse the file contents with fgetcsv(). After parsing I've an array like this
array(2) {
[0]=>
array(9) {
["OrderId"]=>
string(13) "FG-456887"
["Product"]=>
string(7) "B9876"
}
[1]=>
array(9) {
["OrderId"]=>
string(13) "FG-852562"
["Product"]=>
string(7) "B9877"
}
}
var_dump() shows me (apparently) exactly the same dump, when using files with or without BOM, but when I make a simple loop over this array and check if the OrderId (first field in the CSV) is empty - this always fails, when the CSV is encoded without BOM. When I save the same file with BOM - everything works fine.
foreach ($data as $position) {
$orderid = $position["OrderId"];
if (empty($orderid)) die('No orderid found');
}
And it is only the first field - the other fields are ok.
Found it myself. Don't know, if it's elegant - but it works...
function remove_utf8_bom($text) {
$bom = pack('H*','EFBBBF');
$text = preg_replace("/^$bom/", '', $text);
return $text;
}
function csv_to_array($filename='', $delimiter=';', $seperator = '"') {
if(!file_exists($filename) || !is_readable($filename))
return FALSE;
$csvdata = file($filename);
$header = NULL;
$data = array();
foreach ($csvdata as $line) {
$row = remove_utf8_bom($line);
$row = str_getcsv($row,$delimiter,$seperator);
if(!$header)
$header = $row;
else
$data[] = array_combine($header, $row);
}
return $data;
}
Background:
Unbeknownst to me I was in the same situation. I only realized it when I could not use the data that I imported from csv files.
Problem:
While importing two columns from a CSV file I could not access the data in the first column in the array:
array() => ['project_nr' => '0000000', 'project_name']
I tried:
array_keys($myArray);
And it worked as expected, but not until further analysis did I see that the first column 'project_nr' was 13 characters and not 10 characters. Which I later realized was BOM being read in.
Solution:
$str = file_get_contents('yourfile.utf8.csv');
$bom = pack("CCC", 0xef, 0xbb, 0xbf);
if (0 === strncmp($str, $bom, 3)) {
echo "BOM detected - file is UTF-8\n";
$str = substr($str, 3);
}
Reference:
Here is where I found the solution
Anecdote:
I placed this solution here in hopes of connecting google searches for not being able to access specific keys in an array to BOM UTF8 CSV upload.(which is what I needed and was not able to find) I hope that perhaps it may be of help to some desperately searching soul.

Incrementally read a file and put in DB. Doesn't give errors, but does not insert data completely or correctly

I am trying to write a file to a database 500 lines at a time so I do not run low on memory by avoiding dealing with very large arrays. For some reason, I am not getting any errors, but I am seeing a very, very small fraction entered into my table.
$ln = intval(shell_exec("wc -l $text_filename_with_path"));
echo "FILENAME WITH PATH: " . $text_filename_with_path ."\n\n";
echo "ARRAY LENGTH: " . $ln . "\n\n";
//pointer is initialized at zero
$fp = fopen($text_filename_with_path, "r");
$offset = 0;
$c = 0;
while($offset < $ln){
$row_limit = 500;
//get a 500 row section of the file
$chunk = fgets($fp, $row_limit);
//prepare for `pg_copy_from` by exploding to array
$chunk = explode("\n", $chunk);
//each record from the file being read is just one element
//prepare for three column DB table by adding columns (one
//unique PK built from UNIX time concat with counter, the
//other from a non-unique batch ID)
array_walk($chunk,
function (&$item, $key) use ($datetime, $c) {
$item = time() . $c . $key . "\t" . $datetime . "\t" . $item;
}
);
//increase offset to in order to move pointer forward
$offset += $row_limit;
//set pointer ahead to new position
fseek($fp, $offset);
echo "CURRENT POINTER: " . ftell($fp) . "\n"; //prints out 500, 1000, 1500 as expected
//insert array directly into DB from array
pg_copy_from($con, "ops.log_cache_test", $chunk, "\t", "\\NULL");
//increment to keep PK column unique
$c++;
}
I am getting as I say a fraction of the contents of the file, and lots of the data looks a bit messed up, eg about have the entries are blank in the part of the array element that gets assigned by $item within my array_walk() callback. Further it seems that exploding on \n is not working properly as lines seem exploded at ununiform positions (ie, log records don't look symmetrical). Have I just made a total mess out of this
You are not using fgets properly (2nd parameter isn't the number of rows);
There are two ways I can think of at the moment to solve it:
1. A loop getting one line at a time, until you've reached your row limit.
code should look something like this (not tested, assuming the end of line char is "\n" and no "\r")
<?php
/**Your code and initialization here*/
while (!feof($file)){
$counter = 0;
$buffer = array();
while (($line = fgets($file)) !== false && $counter < $row_limit) {
$line = str_replace("\n", "", $line); // fgets gets the line with the newline char at the end of line.
$buffer[] = $line;
$counter++;
}
insertRows($rows);
}
function insertRows($rows){
/** your code here */
}?>
Assuming the file isn't too big- using file_get_contents();
code should look something like this (same assumptions)
<?php
/**Your code and initialization here*/
$data = file_get_contents($filename);
if ($data === FALSE )
echo "Could not get content for file $filename\n";
$data = explode("\n",$data);
for ($offset=0;$offset<count($data);$offset+=$row_limit){
insertRows(array_slice ($rows,$offset,$row_limit));
}
function insertRows($rows){
/** your code here */
}
I didn't test it, so I hope it's ok.

Variable contents from a file disappears and loop doesnt enter

I have the following code to read from a file, and write back to it after some computation.
if(file_exists(CACHE_FILE_PATH)) {
//read the cache and delete that line!
$inp = array();
$cache = fopen(CACHE_FILE_PATH, 'r');
if($cache) {
while(!feof($cache)) {
$tmp = fgets($cache);
//some logic with $tmp
$inp[] = $tmp;
}
fclose($cache);
}
var_dump($inp);
$cache = fopen(CACHE_FILE_PATH, 'w');
var_dump($inp);
if($cache) {
var_dump($inp);
foreach ($inp as $val) {
echo "\nIN THE LOOP";
fwrite($val."\n");
}
fclose($cache);
}
}
The output of the var_dumps is:
array(3) {
[0]=>
string(13) "bedupako|714
"
[1]=>
string(16) "newBedupako|624
"
[2]=>
string(19) "radioExtension|128
"
}
array(3) {
[0]=>
string(13) "bedupako|714
"
[1]=>
string(16) "newBedupako|624
"
[2]=>
string(19) "radioExtension|128
"
}
array(3) {
[0]=>
string(13) "bedupako|714
"
[1]=>
string(16) "newBedupako|624
"
[2]=>
string(19) "radioExtension|128
"
}
Even though its an array, it is not going in the loop and printing IN THE LOOP! Why?
This part of your code:
fwrite($val."\n");
Should be:
fwrite($cache, $val); // the "\n" is only required if it was stripped off after fgets()
The first argument to fwrite() must be a file descriptor opened with fopen().
Of course, if you had turned on error_reporting(-1) and ini_set('display_errors', 'On') during development you would have spotted this immediately :)
As suggested in the comments, you should try to simplify your code by using constructs like file() to read the whole file into an array of lines and then use join() and file_put_contents() to write the whole thing back.
If you just want a cache of key/value pairs, you could look into something like this:
// to read, assuming the cache file exists
$cache = include CACHE_FILE_PATH;
// write back cache
file_put_contents(CACHE_FILE_PATH, '<?php return ' . var_export($cache, true) . ';');
It reads and writes files containing data structures that PHP itself can read (a lot faster than you can).

Opening and reading a 2GB csv

I have a been having problems opening and reading the contents of a 2gb csv file. Everytime I run the script it exhausts the servers memory (10GB VPS Cloud Server) and then gets killed. I have made a test script and was wondering if anyone could have a look and confirm that I am not doing anything silly (php wise) here that would cause what seems and unsually high amount of memory usage. I have spoken to my hosting company but they seem to be of the opinion that it is a code problem. So just wondering if anyone can look over this and confirm there is nothing in the code that would cause this kind of problem.
Also if you deal with 2GB csvs, have you encounted anything like this before ?
Thanks
Tim
<?php
ini_set("memory_limit", "10240M");
$start = time();
echo date("Y-m-d H:i:s", $start)."\n";
$file = 'myfile.csv';
$lines = $keys = array();
$line_count = 0;
$csv = fopen($file, "r");
if(!empty($csv))
{
echo "file open \n";
while(($csv_line = fgetcsv($csv, null, ',', '"')) !== false)
{
if($line_count==0) {
foreach($csv_line as $item) {
$keys[] = preg_replace("/[^a-zA-Z0-9]/", "", $item);
}
} else {
$array = array();
for ($i = 0; $i <count($csv_line); $i++) {
$array[$keys[$i]] = $csv_line[$i];
}
$lines[] = (object) $array;
//print_r($array);
//echo "<br/><br/>";
}
$line_count++;
}
if ($line_count == 0) {
echo "invalid csv or wrong delimiter / enclosure ".$file;
}
} else {
echo "cannot open ".$file;
}
fclose ($csv);
echo $line_count . " rows \n";
$end = time();
echo date("Y-m-d H:i:s", $end)."\n";
$time = number_format((($end - $start)/60), 2);
echo $time."\n";
echo "peak memory usages ".memory_get_peak_usage(true)."\n";
it is not actually an "opening" problem but rather processing problem
I am sure you don't need to keep all the parsed lines in the memory like you currently do.
Why not just put the parsed line wherever it belongs to - a database or another file or anything?
It will make your code to keep in the memory as little as just one line at a time.
As others have already pointed out, you're loading the whole 2 GB file into memory. You do this while creating an array with multiple strings out of each line, so factually the resulting memory needed is more than the plain file-size.
You might want to process each row of the CSV file separately, ideally with an iterator, for example one that returns each line as a keyed array:
$csv = new CSVFile('../data/test.csv');
foreach ($csv as $line) {
var_dump($line);
}
Exemplary output here:
array(3) {
["Make"]=> string(5) "Chevy"
["Model"]=> string(4) "1500"
["Note"]=> string(6) "loaded"
}
array(3) {
["Make"]=> string(5) "Chevy"
["Model"]=> string(4) "2500"
["Note"]=> string(0) ""
}
array(3) {
["Make"]=> string(5) "Chevy"
["Model"]=> string(0) ""
["Note"]=> string(6) "loaded"
}
This iterator is inspired by one that's build in in PHP called SPLFileObject. As this is an iterator, you decide what you do with each line's/row's data. See the related question: Process CSV Into Array With Column Headings For Key
class CSVFile extends SplFileObject
{
private $keys;
public function __construct($file)
{
parent::__construct($file);
$this->setFlags(SplFileObject::READ_CSV);
}
public function rewind()
{
parent::rewind();
$this->keys = parent::current();
parent::next();
}
public function current()
{
return array_combine($this->keys, parent::current());
}
public function getKeys()
{
return $this->keys;
}
}
PHP is really the wrong language for this. String manipulation usually results in copies of strings being allocated in memory, and garbage collection will only occur when the script ends, when it is really no longer needed. If you know how to do it, and it fits the execution environment, you'd be better with perl or sed/awk.
Having said this, there are two memory hogs on the script. The first is the foreach, which copies the array. Do a foreach on the array_keys, and refer back to the string entry in the array to get at the lines. The second, is the one referred by #YourCommonSense: you should design your algorithm so it works in streaming mode (i.e. not requiring the storage of the full dataset in memory). At a cursory glance, it seems feasible.

Categories