How can I trim these 7 columns down to 5 columns, by running some script of sorts.
I recall you can do this using regex / php but buggered if I can recall how we did it.
Example code ( of a GEO IP db ( 115,000 lines )
"3231296768","3231297023","ripencc","702518400","EU","EU","European Union"
"3231297024","3231297279","ripencc","441763200","EU","EU","European Union"
"3231297280","3231297535","ripencc","702518400","EU","EU","European Union"
"3231297536","3231297791","ripencc","702518400","EU","EU","European Union"
"3231297792","3231298047","ripencc","702518400","EU","EU","European Union"
"3231298048","3231298303","ripencc","702518400","EU","EU","European Union"
"3231298304","3231298559","ripencc","702518400","EU","EU","European Union"
I need to remove columns 3 and 4 from every line.
Any help appreciated.
While jimw's answer is the best answer in general, if you want a pure PHP solution I would suggest the following:
$input = 'input.txt';
$output = 'output.txt';
if (false !== ($ih = fopen($input, 'r'))) {
$oh = fopen($output, 'w');
while (false !== ($data = fgetcsv($ih))) {
// this is where you build your new row
$outputData = array($data[0], $data[1], $data[4], $data[5], $data[6]);
fputcsv($oh, $outputData);
}
fclose($ih);
fclose($oh);
}
From 'a script of sorts' and 'regex/PHP', I infer that you just want this done, and don't care what language is used. If you're on *nix:
cut -d, -f1,2,5,6,7 file.csv
'cut' is a standard unix command-line utility, found on everything from OS X to AIX. The arguments I've used are:
-d, # this sets the 'delimiter' to a comma, for CSV
-f1,2,5,6,7 # this selects which fields to print
So together, it takes a file where each line consists of fields separated by commas and prints out fields one to five of it.
The same effect can be achieved in any programming language. I don't know PHP very well, so I won't attempt to produce it in PHP.
Edit: From the PHP docs, adapted slightly:
function apply_quotes($string) {
return '"'.$string.'"';
}
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$data = array_map("apply_quotes", $data);
echo join(",", array($data[0], $data[1], $data[4], $data[5], $data[6]))."\n";
}
fclose($handle);
}
Related
I have a CSV file that contains around 8500 lines but I'm getting a really weird "bug".
I'm validating the data inside the CSV to make sure the data is cool to import into the database. I currently just log the data errors to a log file, but when I open it I see error reports for rows upto 8800 (give or take).
I did some basic debugging to see what's what and did this to begin with:
foreach ($csv as $key => $row)
{
if ($key > 8500) {
echo '<pre>';
print_r($row);
echo '</pre>';
}
}
and that only returned about 50/60 more which is fine as the total rows is around that number.
I then tried doing this to get the end array result:
$last = end($csv);
print_r($last);
and that showed an array with data as expected. However when I do this:
var_dump(array_keys($csv));
then it shows 8800 (give or take) values. Doing count($csv) returns the same number.
I've tried going into the actual CSV and highlighting everything below the last row and hitting clear but it still has the same affect..
Here's how I build my $csv array:
$skus = $csv = [];
if (($handle = fopen($fileTmp, 'r')) !== false) {
set_time_limit(0);
$i = 0;
while (($csvData = fgetcsv($handle, 1000, ',')) !== false)
{
$colCount = count($csvData);
$csv[$i]['sku'] = $csvData[0];
$csv[$i]['desc'] = $csvData[1];
$csv[$i]['ean'] = $csvData[2];
$csv[$i]['rrp_less_vat'] = $csvData[3];
$csv[$i]['rrp_inc_vat'] = $csvData[4];
$csv[$i]['stock'] = $csvData[5];
$csv[$i]['est_delivery'] = $csvData[6];
$csv[$i]['img_name'] = $csvData[7];
$csv[$i]['vatable'] = $csvData[8];
$csv[$i]['obsolete'] = $csvData[9];
$csv[$i]['dead'] = $csvData[10];
$csv[$i]['replacement_product'] = $csvData[11];
$csv[$i]['brand'] = $csvData[12];
$csv[$i]['ext_desc'] = $csvData[13];
$i++;
}
fclose($handle);
}
Am I doing something wrong that I can't see in building the array or is this unexpected behaviour?
PHP version: 7.1
OS: Linux Mint
You have lines that are longer than the $length argument you are passing to fgetcsv(). From the documentation, emphasis mine:
Must be greater than the longest line (in characters) to be found in the CSV file (allowing for trailing line-end characters). Otherwise the line is split in chunks of length characters, unless the split would occur inside an enclosure.
The easiest fix is to stop limiting the length of the line to 1000:
while (($csvData = fgetcsv($handle)) !== false)
I use the php script below to remove some columns from a csv, order them new and save it as new file.
And it works for the file i made it for.
Now i need to do the same with another csv but i don't know whats wrong. I always get a comma befor the data in the first column.
This is what i have, but it doesn't really work.
<?php
$input = 'http://***/original.csv';
$output = 'new.csv';
if (false !== ($ih = fopen($input, 'r'))) {
$oh = fopen($output, 'w');
while (false !== ($data = fgetcsv($ih))) {
// this is where you build your new row
$outputData = array($data[4], $data[0]);
fputcsv($oh, $outputData);
}
fclose($ih);
fclose($oh);
}
The original.csv looks that:
subproduct_number barcode stock week_number qty_estimate productid variantid
05096470000 4024144513543 J 3 6 35016
ae214 848518017215 N 23 0 7 35015
05097280000 4024144513727 J 1 32 34990
The seperator is ';'. The same seperator is used in the file that is working
But here it will be go wrong because my saved new.csv looks like this:
subproduct_number barcode stock week_number qty_estimate productid variantid
,05096470000 4024144513543 J 3 6 35016
,ae214 848518017215 N 23 0 7 35015
,05097280000 4024144513727 J 1 32 34990
But what i need is a new csv that looks like this:
qty_estimate subproduct_number
3 05096470000
0 ae214
1 05097280000
As you can see, i need only the 5. column ($data[4]) as first and the first column ($data[0]) as the second one.
I hope someone can point me in the reight direction.
Thanks
You can do so:
while (false !== ($data = fgetcsv($ih))) {
$data = explode(';', $data[0]);
$outputData = array($data[4], $data[0]);
fputcsv($oh, $outputData, ';');
}
I have a csv file with the following columns:
date, time, info1, info 2, info 3, info 4, info 5,
The file can be 1000 lines long or just have 3 lines. Every 15 seconds a new line is made with the above columns. But only when the devide is on.
A user should trim the csv file to the the specific start and end date/time he has choosen from a calendar or dropdownboxes. so let's say from 3 march 2016 10:00 till 7 may 2016 17:30.
is this possible with a csv file?
I know you can read and write to a csv with fgetcsv, fopen and fwrite.
But how to get the specific lines within the date range.. I have no clue.
Can someone point me in the right direction?
Any help or advice is appreciated :-)
<?php
$file = '/path/to/file/csv';
$separator = ';';
$length = 2048;
$newFile = [];
$i = 0;
if(($handle = fopen($file, 'r')) !== false){
while(($data = fgetcsv($handle, $length, $separator)) !== false) {
if($i == 0){
$newFile[] = implode($separator, $data);
$i++;
}else{
$fileDate = $data[0].' '.$data[1];
if(check_in_range('03-03-2016 10:00:00', '07-05-2016 17:30', $fileDate)){
$newFile[] = implode($separator, $data);
}
}
}
}
$newCsvFileData = implode(PHP_EOL, $newFile);
?>
Function check_in_rage() comes from How to check if a date is in a given range? posted by #ConroyP
I'm importing data from CSV file to database by using php
$i = 0;
while(($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
if($i!=0) {
if(trim($data[0])!=' ') {
$branddata['brand'] = $data[0];
$branddata['create_date'] = date('Y-m-d H:i:s');
print_r($branddata); die;
$res = $this->mdl_brands->insertIntoBrand($branddata);
}
}
$i++;
}
and My csv file has following data
After the Brand Name in the 2 row I have added more than one blank space which is inserting entry into the database.
However I have used the trim function to remove the blank spaces but it not working.
Please guide how to avoid this.
As you said it:
I have used the trim function to remove the blank spaces
You removed them! So you don't have to check for 1 space, just do:
if(trim($data[0]) != '') {
//^^ See here
Hello everyone and I immediately apologize, as
I have seen various threads on the site, but unfortunately my knowledge is still insufficient to complete my project.
I have a text file and I have to do the sum of each column (just need the total):
1003|name1|1208.00|2.00 |96.00 |0.00|0.00|0.00|0.00|98.00 |90.95 |7.05 |8516.40
1011|name2|1450.00|2.00 |49.00 |0.00|0.00|0.00|0.00|51.00 |44.62 |6.38 |9243.7
1004|name3|1450.00|25.00|170.00|0.00|0.00|0.00|0.00|195.00|175.75|19.25|27912.5 <br>
1002|name4|765.00 |1.00 |17.00 |0.00|0.00|0.00|0.00|18.00 |15.13 |2.87 |2193.26
I need to get this(I have this file on linux then we can use Bash, PHP, Mysql... ):
1003|name1|1208.00|2.00 |96.00 |0.00|0.00|0.00|0.00|98.00 |90.95 |7.05 |8516.40
1011|name2|1450.00|2.00 |49.00 |0.00|0.00|0.00|0.00|51.00 |44.62 |6.38 |9243.7
1004|name3|1450.00|25.00|170.00|0.00|0.00|0.00|0.00|195.00|175.75|19.25|27912.5 <br>
1002|name4|765.00 |1.00 |17.00 |0.00|0.00|0.00|0.00|18.00 |15.13 |2.87 |2193.26 <br>
xxxx|Total |4873.00|30.00|332.00|0.00|0.00|0.00|0.00|362.00 |326.45|35.55|47865.86
Where xxxx is the Id number (No sum here).
I've been trying to do this in PHP and MySQL -- No luck so far.
try something like:
$file = '/path/to/your_file.txt';
if ( ($file = fopen($file, "r")) !== FALSE) {
$total = 0;
$row_1 = 0;
while (($line = fgetcsv($file, 1000, "|")) !== FALSE) {
// brutal dirt sanitization
foreach ( $line as $k => $v ) {
$line[$k] = (float) preg_replace('#[^0-9\.]#','', $v);
}
$total = $total + array_sum(array_slice($line, 2));
$row_1 = $row_1 + array_sum(array_slice($line, 2, 1));
//...
}
echo $total.' | '.$row_1; //...
}
else echo 'error ...';
also, you can sanitize each row by replacing array_sum() by array_map() wih a callback function
Psuedocode:
open source file for reading
open destination file for writing
initialise totaling array to zero values
while not EOF
read in line from file
explode line into working array
for x=2 ; x<14; x++
add totalling array with floatval( working array )
write line out to destination file
close read file
write out totals array to destination file
close destingation file
Try to get the text file data into an excel spreadsheet and then add up the columns.
You can use VB to get the text into excel and then continue adding up the values of each column.
1) replace all | chars with , using str_replace
2) Use str_getcsv to create array out of the above resulting csv string
3) use foreach and loop through each row and calculate total
some PHP code
$str = file_get_contents('myfile.txt');
$str = str_replace('|', ',', $str);
$csv = str_getcsv($str);
$totals = array(0,0,0,0);
foreach ($csv as $row) {
$totals[0] += trim($row[0]);
$totals[1] += trim($row[2]);
$totals[2] += trim($row[3]);
$totals[3] += trim($row[4]);
}
the $totals array contains all totals!