I created a CSV parser that works fine for some CSV files I've found online, but one that I converted from XLS to CSV via Microsoft Excel 2011 does not work.
The ones that work are formatted as such:
"Sort Order","Common Name","Formal Name","Type","Sub Type","Sovereignty","Capital","ISO 4217 Currency Code","ISO 4217 Currency Name","ITU-T Telephone Code","ISO 3166-1 2 Letter Code","ISO 3166-1 3 Letter Code","ISO 3166-1 Number","IANA Country Code TLD"
"1","Afghanistan","Islamic State of Afghanistan","Independent State",,,"Kabul","AFN","Afghani","+93","AF","AFG","004",".af".........................etc...
The one that doesn't work is formatted like this:
Order Id,Date Ordered,Date Returned,Product Id,Description,Order Reason Code,Return Qty,Order Return Comment,Ship To Name,Ship To Address1,Ship To Address2,Ship To Address3,Ship To City,Ship To State,Ship To Zipcode,Ship To Country,Disposition,Ship To Email,ShipVia
5555555,2013-07-05 13:58:36.000,2013-08-16 00:00:00.000,5555-55,0555 - Some Test Thing,Refund,2,,jeric beatty,123 fake st,,,burke,NJ,55055,US,Discard,test#test.com,Super Fast Shipping
Is there anyway to get excel to export in the format as the first one? I would like to avoid doing this manually as the file is huge and I would have to manually edit lots of parts of it where I couldn't do a "replace all". Another issue could be that there are double and sometimes triple commas in some places. Though this does appear in both files.
Here is the parser:
function ingest_csv() {
$file_url = 'http://www.path.to/csv/file.csv';
$record_num = 0;
$records = array();
$header = array();
if (($handle = fopen($file_url, "r")) !== FALSE) {
$records['id'] = '';
while (($data = fgetcsv($handle)) !== FALSE) {
$records['id'][$record_num] = '';
$cell_num = 0;
foreach ($data as $cell) {
if($record_num == 0) {
$header = $data;
} else {
$current_key = $header[$cell_num];
$records['id'][$record_num][$current_key] = $cell;
}
$cell_num++;
}
$record_num++;
}
fclose($handle);
}
else {
echo 'could not open file.';
}
return array($record_num, $records);
}
function batch_csv() {
list($num_rows, $rows) = ingest_csv
print_r($num_rows);
print_r($rows);
}
As mentioned in the comments though you may be trying to reinvent the wheel here, though personally I've asked questions where I didn't want to give long rambling explanations of why I was forced to use unconventional approaches so should this be one of those situations here's an answer.
In OpenOffice Calculator (for example) and when you go to save as CSV you get a number of further options including the decision to double quote all fields.
Unfortunately Excel doesn't give you the choice, but Microsoft do offer up a workaround using a macro - http://support.microsoft.com/kb/291296/en-us
Related
I am reading a CSV file to import data. I am having a column with some auto-generated numbers(text & Numbers). The problem is in some of the rows my script reads the value as exponential number.
Example: 58597E68 is considered 5.86E+72
I need it to read as String as not number. The issue occurs only if I am having the character (E) in middle of the auto-generated number.
$feed = 'path-to-csv/import.csv';
if (!file_exists($feed)) {
//$feed = 'import.csv';
exit('Cannot find the CSV file: ' . $feed);
}
$row=0;
if (($handle = fopen($feed, 'r')) !== FALSE) {
while (($data_csv_rows = fgetcsv($handle, 1000000, ',')) !== FALSE) {
$row++;
if ($row == 1) {
continue;
} // skipping header row
echo "Row " . ($row-1) . "<br>";print_r($data_csv_rows);echo "<br><br>";
}
}
The problem is not your CSV but the original software (probably Excel) that produced the CSV.
CSV is a simple data format when you find something like 5.86E+72 it's like that in the CSV data and it's too late to fix it.
To avoid this make sure you export the data correct into CSV.
Some PHP code to find this kind of bad data in a field:
if (strpos($value, 'E+') !== FALSE) {
preg_match('~E\+[0-9]+$~', $value, $preg_result);
if (isset($preg_result[0])) {
die('Probably wrong data found within "'.$value.'".');
}
}
}
In your case it seems that 58597E68 is converted to float(5.8597E+72).
At least with str_getcsv() I can not recreate the problem, see https://3v4l.org/RZ1eA.
By definition it would be correct, since there are no " around this data, so PHP tries to determinate the type of this data and if it is potentally a numeric value... So be sure add " around strings. PHP String to Numeric Conversion documentation.
Update: I can not reproduce your use-case! 58597E68 becomes "58597E68" with str_getcsv() and with fgetcsv() It is not autoconverted to float! See https://3v4l.org/oXkBu for details! I suspect there is something wrong with the data you provide us or your validation.
I have a huge issue, I cant find any way to sort array entries. My code:
<?php
error_reporting(0);
$lines=array();
$fp=fopen('file.txt, 'r');
$i=0;
while (!feof($fp))
{
$line=fgets($fp);
$line=trim($line);
$lines[]=$line;
$oneline = explode("|", $line);
if($i>30){
$fz=fopen('users.txt', 'r');
while (!feof($fz))
{
$linez=fgets($fz);
$linez=trim($linez);
$lineza[]=$linez;
$onematch = explode(",", $linez);
if (strpos($oneline[1], $onematch[1])){
echo $onematch[0],$oneline[4],'<br>';
}
else{
}
rewind($onematch);
}
}
$i++;
}
fclose($fp);
?>
The thing is, I want to sort items that are being echo'ed by $oneline[4]. I tried several other posts from stackoverflow - But was not been able to find a solution.
The anser to your question is that in order to sort $oneline[4], which seems to contain a string value, you need to apply the following steps:
split the string into an array ($oneline[4] = explode(',',
$oneline[4]))
sort the resulting array (sort($oneline[4]))
combine the array into a string ($oneline[4] = implode(',',
$oneline[4]))
As I got the impression variable naming is low on the list of priorities I'm re-using the $oneline[4] variable. Mostly to clarify which part of the code I am referring to.
That being said, there are other improvements you should be making, if you want to be on speaking terms with your future self (in case you need to work on this code in a couple of months)
Choose a single coding style and stick to it, the original code looked like it was copy/pasted from at least 4 different sources (mostly inconsistent quote-marks and curly braces)
Try to limit repeating costly operations, such as opening files whenever you can (to be fair, the agents.data could contain 31 lines and the users.txt would be opened only once resulting in me looking like a fool)
I have updated your code sample to try to show what I mean by the points above.
<?php
error_reporting(0);
$lines = array();
$users = false;
$fp = fopen('http://20.19.202.221/exports/agents.data', 'r');
while ($fp && !feof($fp)) {
$line = trim(fgets($fp));
$lines[] = $line;
$oneline = explode('|', $line);
// if we have $users (starts as false, is turned into an array
// inside this if-block) or if we have collected 30 or more
// lines (this condition is only checked while $users = false)
if ($users || count($lines) > 30) {
// your code sample implies the users.txt to be small enough
// to process several times consider using some form of
// caching like this
if (!$users) {
// always initialize what you intend to use
$users = [];
$fz = fopen('users.txt', 'r');
while ($fz && !feof($fz)) {
$users[] = explode(',', trim(fgets($fz)));
}
// always close whatever you open.
fclose($fz);
}
// walk through $users, which contains the exploded contents
// of each line in users.txt
foreach ($users as $onematch) {
if (strpos($oneline[1], $onematch[1])) {
// now, the actual question: how to sort $oneline[4]
// as the requested example was not available at the
// time of writing, I assume
// it to be a string like: 'b,d,c,a'
// first, explode it into an array
$oneline[4] = explode(',', $oneline[4]);
// now sort it using the sort function of your liking
sort($oneline[4]);
// and implode the sorted array back into a string
$oneline[4] = implode(',', $oneline[4]);
echo $onematch[0], $oneline[4], '<br>';
}
}
}
}
fclose($fp);
I hope this doesn't offend you too much, just trying to help and not just providing the solution to the question at hand.
From a csv file I need to extract the header and the values. Both are later accessed in frontend.
$header = array();
$contacts = array();
if ($request->isMethod('POST')) {
if (($handle = fopen($_FILES['file']['tmp_name'], "r")) !== FALSE) {
$header = fgetcsv($handle, 1000, ",");
while (($values = fgetcsv($handle, 1000, ",")) !== FALSE) {
// array_combine
// Creates an array by using one array for keys
// and another for its values
$contacts[] = array_combine($header, $values);
}
fclose($handle);
}
}
It works with csv files that look like this
Name,Firstname,Organisation,
Bar,Foo,SO,
I just exported my gmail contacts and tried to read them using the above code but I get following error
Warning: array_combine() [function.array-combine]: Both
parameters should have an equal number of elements
The gmail csv looks like this
Name,Firstname,Organisation
Bar,Foo,SO
Is the last missing , the reason for the error? What is wrong and how to fix it?
I found this on SO
function array_combine2($arr1, $arr2) {
$count = min(count($arr1), count($arr2));
return array_combine(array_slice($arr1, 0, $count),
array_slice($arr2, 0, $count));
}
This works but it skips the Name field and not all fields are combined. Is this because the gmail csv is not realy valid? Any suggestions?
I managed this by expanding the array size or slicing it depending on the size of the header.
if (count($header) > count($values)) {
$contacts = array_pad($values, count($header), null);
} else if (count($header) < count($values)) {
$contacts = array_slice($values, 0, count($header));
} else {
$contacts = $values;
}
Although this isn't the answer to the question you asked, it might be the answer to the source of the problem. I recently had this problem and realized I was making a silly error because I didn't understand the fgetcsv() function's parameters:
That 1000 up there denotes the maximum line length of a single line in the csv you're taking content from. Longer than that, and the function returns null! I don't know why the version given in the examples is so stingy, but it's not required; setting it to 0 allows fgetcsv() to read lines of any length. (The documentation warns this is slower. For most use cases of fgetcsv() I can hardly imagine it's slow enough to notice.)
I have a huge CSV file (10M records) in the following format.
147804,AC,34,15AUG09,09:00,15AUG09,21:00,YYZ,YVR,PLS
147816,AC,34,26AUG09,09:00,01SEP09,21:00,YYZ,YVR,PLS
I need to import them into a mysql database. How can I change all the months to numeric months and preferably into yyyy/mm/dd format.
Thanks
This is difficult to accomplish with regex and would be error prone. PHP has CSV support built-in and it’s a lot safer.
<?php
if (($if = fopen("src_file.csv", "r")) !== FALSE) {
if (($of = fopen("dst_file.csv", "w")) !== FALSE) {
while (($cols = fgetcsv($if)) !== FALSE) {
$cols[3] = date('Y/m/d',strtotime($cols[3]));
$cols[5] = date('Y/m/d',strtotime($cols[5]));
fputcsv($of, $cols);
}
fclose($of);
}
fclose($if);
}
?>
I don’t know if it would be more efficient to just store $cols in the database or create a new file and import it. I don’t have any benchmarks.
Amazingly, this seems to do the trick :
echo date('Y/m/d', strtotime('15AUG09'));
returns : 2009/08/15.
If you can manage to parse your CSV, you'll get your date in the format you want.
I think my method is lame, but I cannot think of a better way to do this.
I use Ultraedit text editor to hold all the stuff I cull out of Stackoverflow for PHP and MySQL in a text file. This is my strict format for each new entry:
#################################################
TITLE: THIS IS MY TITLE (ALL IN CAPS, FOLLOWD BY A DOTTED LINE)
-------------------------------------------------
...probably a question first (if necessary), then another shorter dotted line
-------------------
...answer(s)...
#################################################
So, here is an actual entry:
#################################################
TITLE: READING FIRST 5 FIELDS OF CSV FILE INTO PHP
-------------------------------------------------
(...with fgetcsv...)
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
// iterate over each column here
for ($c=0; $c < $num; $c++) {
// handle column data here
echo $data[$c] . "<br />\n";
// exit the loop after 3rd column parsed
if ($c == 2) break;
}
++$row;
}
fclose($handle);
-----------------
(...without fgetcsv...)
$lines = file('data.csv');
$linecount = count($lines);
for ($i = 1; $i < $linecount; $i++){
$fields = explode(',', $lines[$i]);
$sno = $fields[0];
$name = $fields[1];
$ph = $fields[2];
$add = $fields[3];
}http://stackoverflow.com/users/login?returnurl=%2fquestions%2fask
#################################################
I can get a list of titles by searching for "TITLE: *", etc. My text file now contains about 15,000 lines. Is there a better way to do this? I have asked StackOverflow before about snippet software, but after a thorough search, there is really nothing out there that fits my needs.
In a way, I'm surprised that there is not a PHP/MySQL application for doing this (collecting snippets). I can't do it because I don't have the knowledge or talent. The snippet collector in my IDE will not suffice.
Thanks!
why not build yourself a little application with a small sql backend (say SQLCE or SQLITE)?
You could build it so that you have the following tables:
Title
Code Snippet
Original Question Url
and then you can relate in the TAGS of the question via another Table to allow better searching/cross referencing.
I have used InfoSelect software for years for this purpose, and have many megabytes of searchable notes and code snippets. It isn't exclusively for code snippets. It's the software equivalent of keeping notes on index cards and being able to arrange them in hierarchies, or do a search on them.
Another similar tool is OneNote which is part of Microsoft Office.
If you remove the condition that your tool should be specifically for tracking code snippets, you may be able to broaden your choices.