Is it possible to remove extra rows when uploading in .csv file in PHP or removing specific words?
Please take a look at my screen shot. I want to exclude those encircled when I process uploading in MySQL.
Full Size
Here's my code found over the net.
<?php
$message = null;
$allowed_extensions = array('csv');
$upload_path = '';
if (!empty($_FILES['file'])) {
if ($_FILES['file']['error'] == 0) {
// check extension
$file = explode(".", $_FILES['file']['name']);
$extension = array_pop($file);
if (in_array($extension, $allowed_extensions)) {
if (move_uploaded_file($_FILES['file']['tmp_name'], $upload_path.'/'.$_FILES['file']['name'])) {
if (($handle = fopen($upload_path.'/'.$_FILES['file']['name'], "r")) !== false) {
$keys = array();
$out = array();
$insert = array();
$line = 1;
while (($row = fgetcsv($handle, 0, ',', '"')) !== FALSE) {
foreach($row as $key => $value) {
if ($line === 1) {
$keys[$key] = $value;
} else {
$out[$line][$key] = $value;
}
}
$line++;
}
fclose($handle);
if (!empty($keys) && !empty($out)) {
$db = new PDO('mysql:host=localhost;dbname=attendance', 'root', 'root');
$db->exec("SET CHARACTER SET utf8");
foreach($out as $key => $value) {
$sql = "INSERT INTO `report` (`";
$sql .= implode("`, `", $keys);
$sql .= "`) VALUES (";
$sql .= implode(", ", array_fill(0, count($keys), "?"));
$sql .= ")";
$statement = $db->prepare($sql);
$statement->execute($value);
}
$message = '<span class="green">File has been uploaded successfully</span>';
}
}
}
} else {
$message = '<span class="red">Only .csv file format is allowed</span>';
}
} else {
$message = '<span class="red">There was a problem with your file</span>';
}
}
?>
Well, without more info - it looks like the rows you want to exclude have less rows than the others (I'm assuming you don't want the blank rows either).
You can use the function
$arr= str_getcsv();
to read a line of CSV data into an array the do
if(sizeof($arr)!=$validCount)
on the resulting array you get to exclude lines not of a certain length ($validCount).
Obviously we could get much more in depth for exclusion, but I'm guessing this will get your started.
Note - this requires PHP5.3 or greater.
HTH
R
Related
I have a php code that I connect to an oracle database and with a request to retrieve the information I want, here is my code :
$query = "SELECT ACTIVE_SIZES FROM ADA_ACTIVE_SIZE2_VIEW WHERE ADA_STYLE = 'SCPCL4'";
$result = odbc_exec($connect, $query);
while($final = odbc_fetch_array($result)) {
print_r($final); //Array ( [ACTIVE_SIZES] => XS-S-M-L-XL-2XL )
}
Now I'm reading a csv file and I would like to adapt this code to add the results of my queries in a column at the end of the file. I already add two columns at the end of this one, but my query doesn't return anything in the csv file, how can I do please?
<?php
//Modifications on csv file
$delimiter = ";";
$csv_data = array();
if (($handle = fopen($nomcsv, 'r')) !== FALSE) {
while (($data = fgetcsv($handle, 10000, $delimiter)) !== FALSE) {
//Add two columns at the end
$data['Pictures Names'] = (!empty($data[4]) ? ($data[7] ?: '') . "_" . $data[4] . '.jpg' : '');
$data['Color-Description'] = (!empty($data[3]) ? (ltrim($data[4], '0') ?: '') . "-" . $data[3] : '');
//Query
$query = "SELECT ACTIVE_SIZES FROM ADA_ACTIVE_SIZE2_VIEW WHERE ADA_STYLE = '".$data[4]."'";
$result = odbc_exec($connect, $query);
while($final = odbc_fetch_array($result)) {
$data['Sizes'] = $final;
var_dump($final); //array(1) { ["ACTIVE_SIZES"]=> string(8) "XS-S-M-L" }array(1) { ["ACTIVE_SIZES"]=> string(8) "XS-S-M-L" }...
}
$csv_data[] = $data;
var_dump($csv_data); //["Pictures Names"]=> string(15) "SCJEG4_1041.jpg" ["Color-Description"]=> string(12) "1041-MUSTARD" ["Sizes"]=> array(1) { ["ACTIVE_SIZES"]=> string(15) "XS-S-M-L-XL-2XL" } } }
}
fclose($handle);
}
if (($handle = fopen($nomcsv, 'w')) !== FALSE) {
foreach ($csv_data as $data) {
fputcsv($handle, $data, $delimiter);
}
fclose($handle);
}
?>
At the end I have this in my csv file:
Try this, only thing is you need to define your ActiveSizes index and push it to the sizes index
$delimiter = ";";
$csv_data = array();
if (($handle = fopen($nomcsv, 'r')) !== FALSE) {
while (($data = fgetcsv($handle, 10000, $delimiter)) !== FALSE) {
//Add two columns at the end
$data['Pictures Names'] = (!empty($data[4]) ? ($data[7] ?: '') . "_" . $data[4] . '.jpg' : '');
$data['Color-Description'] = (!empty($data[3]) ? (ltrim($data[4], '0') ?: '') . "-" . $data[3] : '');
//Query
$query = "SELECT ACTIVE_SIZES FROM ADA_ACTIVE_SIZE2_VIEW WHERE ADA_STYLE = '".$data[4]."'";
$result = odbc_exec($connect, $query);
while($row = odbc_fetch_array($result)) {
$data['Sizes'][] = $row['ACTIVE_SIZES'];
}
$csv_data[] = $data;
}
fclose($handle);
}
//var_dump($csv_data);
if (($handle = fopen($nomcsv, 'w')) !== FALSE) {
foreach ($csv_data as $data) {
fputcsv($handle, $data, $delimiter);
}
fclose($handle);
}
I have a PHP code that is supposed to save values fro a CSV file into a MySql database table. Everything works fine except that only the first row of the CSV is added. Here's the code:
<?php
public function saveProductsFromCsv($productId, $val) {
$productId = (int) $productId;
$data = array();
$fieldNames = $this->_config->getCsvColumnNames();
array_shift($fieldNames);
$numberOfFields = count($fieldNames);
$lines = explode("\n", $val);
foreach ($lines as $line) {
$line = trim($line);
if (empty($line))
continue;
$values = explode(',', $line);
if (count($values) != $numberOfFields){
throw new Exception();
return;
}
$make = trim($values[0]);
$model = trim($values[1]);
$yearFrom = (int) $values[2];
$yearTo = (int) $values[3];
$engine = trim($values[4]);
if ($yearFrom > 0){
if ($yearFrom < 1950){
$yearFrom = 1950;
} elseif ($yearFrom > 2030){
$yearFrom = 2030;
}
}
if ($yearTo > 0){
if ($yearTo < 1950){
$yearTo = 1950;
} elseif ($yearTo > 2030){
$yearTo = 2030;
}
}
$data[] = array($productId, $make, $model, $yearFrom, $yearTo, $engine);
}
if (count($data) > 0){
$this->saveValues($data);
}
}
public function saveValues($data)
{
$valuesStr = '';
foreach ($data as $values){
$cell = '';
foreach ($values as $value)
$cell .= ",'" . esc_sql(trim($value)). "'";
$valuesStr .= ($valuesStr != '' ? ',' : '') . "(NULL{$cell})";
}
$this->_wpdb->query("INSERT IGNORE INTO {$this->_mainTable} VALUES {$valuesStr}");
}
?>
Below is the CSV:
product_sku,make,model,year_from,year_to,engine
63118,Toyota,FJ Cruiser,2000,2008,V6 4.7L 2UZ-FE 20R/22R 1st Gen FJ Cruiser
28216,Toyota,GX470,1992,1997,V8 4.7L 2UZ-FE GX470
62687,Toyota,Land Cruiser,1998,2007,V8 4.7L 2UZ-FE 100-Series
28485,Toyota,Land Cruiser,2007,2018,V8 5.7L 3UR-FE 200-Series
Incidentally, if I use this:
product_sku,make,model,year_from,year_to,engine
63118,Toyota,FJ Cruiser,2000,2008,engine1
28216,Toyota,GX470,1992,1997,engine2
62687,Toyota,Land Cruiser,1998,2007,engine3
28485,Toyota,Land Cruiser,2007,2018,engine4
everything gets inserted fine. I think the problem is with the last column because if I use the same values in the last column in the second or third columns, everything works fine.
Never mind, I've solved it. The problem was with the encoding of the CSV file. I changed it to UTF-8 and everything is working fine.
I am trying to import a csv file into the data base without defining any of the rows as it will be automatic when the page loads -
$file = '../csv/file.csv';
$table = 'table_name';
// get structure from csv and insert db
ini_set('auto_detect_line_endings',TRUE);
$handle = fopen($file,'r');
// first row, structure
if ( ($data = fgetcsv($handle) ) === FALSE ) {
echo "Cannot read from csv $file";die();
}
$fields = array();
$field_count = 0;
for($i=0;$i<count($data); $i++) {
$f = strtolower(trim($data[$i]));
if ($f) {
// normalize the field name, strip to 20 chars if too long
$f = substr(preg_replace ('/[^0-9a-z]/', '_', $f), 0, 255);
$field_count++;
$fields[] = $f.' VARCHAR(255)';
}
}
$sql = "CREATE TABLE $table (" . implode(', ', $fields) . ')';
$conn->query($sql);
while ( ($data = fgetcsv($handle) ) !== FALSE ) {
$fields = array();
for($i=0;$i<$field_count; $i++) {
$fields[] = '\''.addslashes($data[$i]).'\'';
}
$sql = "Insert into $table values(" . implode(', ', $fields) . ')';
$conn->query($sql);
}
fclose($handle);
ini_set('auto_detect_line_endings',FALSE);
It loads the data in the page if I echo it and it creates the table correctly just doesn't load the data into the table and I cant find out why..
Thanks!
UPDATED
Here is the first row I am getting when I echo $sql -
Insert into 1001_inventory values('New', '581613', '88888888888888888', '2016', 'Toyota')
So I found that the varchar(255) was to small for some of the fields so I expanded it and the problem is solved, Thanks for all the help!
I am trying to read text from files in a directory and have it be displayed as description text below an image. I have been able to use the STRIPOS function to separate out each part of the text, except am having trouble with the last section. The last section is titled "Description:" and actually runs into multiple lines. I don't know how to display more than just the line that reads "Description:". I want to print from "Description:" to the end of the file. I will post my code and the text file in this message.
$dirname = 'data';
$dirhandle = opendir($dirname);
$housestextarray = array();
if ($dirhandle)
{
while (false !==($file = readdir($dirhandle)))
{
if ($file !='.' && $file !='..')
{
array_push($housestextarray, $file);
}
}
closedir($dirhandle);
}
sort($housestextarray);
foreach ($housestextarray AS $housedescription)
{
$housetext = '';
$description = '';
$pos = stripos($housedescription, 'house_');
if ($pos === false)
{
//nothing
} else {
$lines_in_file = count(file($housedescription));
$fp=fopen($housedescription,"r");
for ($cntr = 1; $cntr <= $lines_in_file; $cntr++)
{
$cityline=fgets($fp);
$priceline=fgets($fp);
$bedroomsline=fgets($fp);
$bathsline=fgets($fp);
$footageline=fgets($fp);
$realtorline=fgets($fp);
$grabberline=fgets($fp);
$descriptionline=fgets($fp);
//print $cityline;
//print $descriptionline;
//$housetext .= $line;
$citypos = stripos($cityline, 'City:');
if ($citypos === false) //found the city line first time
{
//nothing
}
else
{
$city= $cityline."<br />\n";
//print $city;
}
$pricepos = stripos($priceline, 'Price:');
if ($pricepos === false) //found the city line first time
{
//nothing
}
else
{
$price = $priceline."<br />\n";
//print $price;
}
$bedroomspos = stripos($bedroomsline, 'Bedrooms:');
if ($bedroomspos === false) //found the city line first time
{
//nothing
}
else
{
$bedrooms = $bedroomsline."<br />\n";
//print $bedrooms;
}
$bathspos = stripos($bathsline, 'Baths:');
if ($bathspos === false) //found the city line first time
{
//nothing
}
else
{
$baths = $bathsline."<br />\n";
//print $baths;
}
$footagepos = stripos($footageline, 'Footage:');
if ($footagepos === false) //found the city line first time
{
//nothing
}
else
{
$footage = $footageline."<br />\n";
//print $footage;
}
$realtorpos = stripos($realtorline, 'Realtor:');
if ($realtorpos === false) //found the realtor line first time
{
//nothing
}
else
{
$realtor = $realtorline."<br />\n";
//print $realtor;
}
$grabberpos = stripos($grabberline, 'Grabber:');
if ($grabberpos === false) //found the grabber line first time
{
//nothing
}
else
{
$grabber_formatted = str_replace('Grabber:','', $grabberline);
$grabber = "<h3>".$grabber_formatted."</h3><br />\n";
//print $grabber;
}
$descriptionpos = stripos($descriptionline, 'Description: ');
if ($descriptionpos === false) //found the description line first time
{
//nothing
}
else
{
$description .= $descriptionline."<br />";
//print $description;
}
}
$output = $grabber."<br/>".$city.$bedrooms.$baths;
$output .= $price.$footage.$realtor."<br />";
$output .= "<br />".$description."<br />";
print $output;
}
And here is the text file contents example (one of six files):
City: OceanCove
Price: $950,000
Bedrooms: 5
Baths: 3
Footage: 3000 sq. ft.
Realtor: Shirley Urkiddeng
Grabber: Fantastic Home with a Fantastic View!
Description:
You will never get tired of watching the sunset
from your living room sofa or the sunrise
from your back porch with a view overlooking
the gorgeous coral canyon. Once in a lifetime
opportunity!
UPDATED CODE WITH Branden's help:
function houseDescriptions()
{
//$DOCUMENT_ROOT = $_SERVER['DOCUMENT_ROOT'];
//$dirname = $DOCUMENT_ROOT.'data';
$dirname = 'data';
$dirhandle = opendir($dirname);
$housestextarray = array();
if ($dirhandle)
{
while (false !==($file = readdir($dirhandle)))
{
if ($file !='.' && $file !='..')
{
array_push($housestextarray, $file);
}
}
closedir($dirhandle);
}
sort($housestextarray);
foreach ($housestextarray AS $housedescription)
{
$housetext = '';
$description = '';
$data ="";
$pos = stripos($housedescription, 'house_');
if ($pos === false)
{
//nothing
} else {
$file_handle = fopen($housedescription, "r");
$data = "";
while (!feof($file_handle)) {
$filestrings .= fgets($file_handle);
}
fclose($file_handle);
//You'll need to double check the Regex if it doesn't work.
$data = preg_split('#\b(City:|Bedrooms:|Baths:|Footage:|Realtor:|Grabber:|Description:)\b#', $filestrings);
$city = $data[0];
$bedrooms = $data[1];
$baths = $data[2];
$footage = $data[3];
$realtor = $data[4];
$grabber = $data[5];
$description = $data[6];
$output = $grabber."<br />".$city.$bedrooms.$baths;
$output .= $price.$footage.$realtor."<br />";
$output .= "<br />".$description."<br />";
print $output;
}
}
//return $output;
}
Updated with proper regex
fgets can have issues when trying to get a string out of text file, especially a long string, there may be a line break within it that you may not see in your text editing program. A better way to do this is to get all of the information from the text file and store it in a string, then handle all searches with preg_split(). This code below (assuming my regex is correct, you may need to recheck it) will get all of the variables from the text file for you.
//Append the file location to the file name
$housedescription = $dirname."/".$housedescription;
//Get all contents of the file and save them into $filestrings
$file_handle = fopen($housedescription, "r");
$data = "";
while (!feof($file_handle)) {
$filestrings .= fgets($file_handle);
}
fclose($file_handle);
//Remove any pesky \n newlines that were added by the text file
$filestring = preg_replace("/[\n\r]/","",$filestrings);
//Case Sensitive Regex Search, split the string into an array based on the keywords
$data = preg_split('/(City:|Price:|Bedrooms:|Baths:|Footage:|Realtor:|Grabber:|Description:)/', $filestring, -1, PREG_SPLIT_NO_EMPTY);
//Save all the keywords to vars
$city = "City: ".$data[0];
$bedrooms = "Bedrooms: ".$data[1];
$baths = "Baths: ".$data[2];
$footage = "Footage: ".$data[3];
$realtor = "Realtor: ".$data[4];
$grabber = "Grabber: ".$data[5];
$description = "Description: ".$data[6];
Notice the addition of $filestring = preg_replace("/[\n\r]/","",$filestrings); this will remove any extra new lines that were in the text file so there are no 's where you don't want them.
Of course the absolute ideal would be to store all of your data in a mysql database instead of .txt files, as it is more secure and much faster for data access. But if you prefer .txt try not to open the same file more than once.
Some notes to take from this: How to use Regular Expressions, fopen, fgets, preg_replace, preg_split
I am using the following function to import csv files to mysql:
function csv_2_mysql($source_file, $target_table, $max_line_length=10000) {
if (($handle = fopen("$source_file", "r")) !== FALSE) {
$columns = fgetcsv($handle, $max_line_length, ",");
foreach ($columns as &$column) {
$column = preg_replace('/[^a-z0-9]/i', '', $column);
}
$insert_query_prefix = "INSERT INTO $target_table (".join(",",$columns).")\nVALUES";
while (($data = fgetcsv($handle, $max_line_length, ",")) !== FALSE) {
while (count($data)<count($columns))
array_push($data, NULL);
$query = "$insert_query_prefix (".join(",",quote_all_array($data)).");";
mysql_query($query);
}
fclose($handle);
}
}
function quote_all_array($values) {
foreach ($values as $key=>$value)
if (is_array($value))
$values[$key] = quote_all_array($value);
else
$values[$key] = quote_all($value);
return $values;
}
function quote_all($value) {
if (is_null($value))
return "NULL";
$value = "'" . mysql_real_escape_string($value) . "'";
return $value;
}
The problem is, that sometimes the headers are not on the first row due to cutting and merging of csv files at the source so for example it may end up looking like this:
value1,value2,value3,value4
value1,value2,value3,value4
value1,value2,value3,value4
header1,header2,header3,header4
value1,value2,value3,value4
value1,value2,value3,value4
value1,value2,value3,value4
value1,value2,value3,value4
header1,header2,header3,header4
value1,value2,value3,value4
value1,value2,value3,value4
value1,value2,value3,value4
value1 is unique so I know that there is never a duplicate row except for headers. How can I adjust the function so that the duplicate header rows are removed if they exist and to ensure that the remaining header row is used for $columns? I would just set the column values manually except each csv may have a different number of columns (except header1 and value1 which are always present as it is a unique timestamp).
UPDATE:
Well, I figured it out but it feels wrong using both fopen and file_get_contents. Will I run into problems with this on large csv's?
function csv_2_mysql($source_file, $target_table, $uid, $nid, $max_line_length=10000) {
if (($handle = fopen("$source_file", "r")) !== FALSE) {
$handle2 = file_get_contents($source_file) or exit;
$handle_row = explode("\n", $handle2);
foreach ($handle_row as $key => $val) {
$row_array = explode(',', $val);
foreach ($row_array as $key => $val) {
$row_array[$key] = trim(str_replace('"', '', $val));
}
if(!in_array('header1', $row_array)) {
unset ($row_array);
}
else {
$columns = $row_array;
}
}
foreach ($columns as &$column) {
$column = preg_replace('/[^a-z0-9]/i', '', $column);
}
$insert_query_prefix = "INSERT INTO $target_table (".join(",",$columns).")\nVALUES";
while (($data = fgetcsv($handle, $max_line_length, ",")) !== FALSE) {
while (count($data)<count($columns))
array_push($data, NULL);
$query = "$insert_query_prefix (".join(",",quote_all_array($data)).");";
mysql_query($query);
}
fclose($handle);
}
}
function quote_all_array($values) {
foreach ($values as $key=>$value)
if (is_array($value))
$values[$key] = quote_all_array($value);
else
$values[$key] = quote_all($value);
return $values;
}
function quote_all($value) {
if (is_null($value))
return "NULL";
$value = "'" . mysql_real_escape_string($value) . "'";
return $value;
}
I guess you could have an array to push the value of the first column (since you say it is unique) and check it for duplicates. If it is a duplicate ignore the line and continue.