PHP CSV MySql Import - php
I have the following issue that I hope ya'll can shed some light on. I am trying to import a csv that although it looks normal (, for seperators) yet when I attempt to import there is always a line break after the 40th ish column.
This is my code for the import:
<?php
// include mysql database configuration file
require_once "layouts/config.php";
if (isset($_POST['submit']))
{
// Open uploaded CSV file with read-only mode
$csvFile = fopen(__DIR__. '/uploads/upload.csv', 'r');
//fgetcsv($csvFile);
$values = array();
while (($getData = fgetcsv($csvFile, 300, ",")) !== FALSE)
{
$values[]= "('".$getData[0]."' , '".$getData[1]."' ,'".$getData[2]."' ,'".$getData[3]."' ,'".$getData[4]."' ,
'".$getData[5]."' , '".$getData[6]."' ,'".$getData[7]."' ,'".$getData[8]."' ,'".$getData[9]."' ,
'".$getData[10]."', '".$getData[11]."','".$getData[12]."','".$getData[13]."','".$getData[14]."',
'".$getData[15]."', '".$getData[16]."','".$getData[17]."','".$getData[18]."','".$getData[19]."',
'".$getData[20]."', '".$getData[21]."','".$getData[22]."','".$getData[23]."','".$getData[24]."',
'".$getData[25]."', '".$getData[26]."','".$getData[27]."','".$getData[28]."','".$getData[29]."',
'".$getData[30]."', '".$getData[31]."','".$getData[32]."','".$getData[33]."','".$getData[34]."',
'".$getData[35]."', '".$getData[36]."','".$getData[37]."','".$getData[38]."','".$getData[39]."',
'".$getData[40]."', '".$getData[41]."','".$getData[42]."','".$getData[43]."','".$getData[44]."',
'".$getData[45]."', '".$getData[46]."','".$getData[47]."','".$getData[48]."','".$getData[49]."',
'".$getData[50]."', '".$getData[51]."','".$getData[52]."','".$getData[53]."','".$getData[54]."',
'".$getData[55]."', '".$getData[56]."','".$getData[57]."')";
}
$sql = "INSERT INTO machine (PresetNumber, Dates, Starts, Stops, ProductCode, ProductName, Proper, TargetWeight1, Totals, Mean, StandardD, Maxs, Mins,
ProductRange, Under, Over, OverScale, ReCheckCnt, TotalNegErr, Preset, TargetWeight2, BagLength, BagWidth, StartIPS, StopIPS, TotalTime, PlannedTime, ITPSDownTime,
ITPSOff, LowProductTime, DownStreamDownTime, OperatingTime,FullProductTime,GoodBagMakingTime, NetOperatingTime, FacultyBagMakingTime, PlannedShutdown, Speed,
ITPSOEE, SystemAvalability, WeigherEfficiency, BagMakerEfficiency, WeightLossRatio, ProductWaste, InputProduct, OutputProduct, AverageWeight,
StandardDeviation, CombinationCount, FilledBags, GoodBags, GoodProductBags, FilmWaste, InputFilm, OutputFilm, FilmWasteUpstream, FilmWasteMiddle, FilmWasteDownStream)
VALUES " . implode(',',$values);
//fclose($csvFile);
if ($link->query($sql) === TRUE)
{
echo "File(s) uploaded successfully!";
}
else {
echo "Error: " . $sql . "<br>" . $link->error;
}
}
''''
My CSV File is
5,2022/05/17,16:22,11:21,AJ,,112965,39.0 g,4485.5 kg,39.707 g,0.495 g,42.0 g,39.0 g,3.0 g,1035,2218,451,58,0 g,5,39.0g,1.92inch,1.27inch,2022.5.19. 09:52,2022.5.19. 11:01,68min34sec,68min34sec,7min25sec,4min30sec,2min55sec,0min0sec,61min10sec,0min0sec,0min0sec,0min0sec,61min9sec,0,0bpm,0.0%,89.1%,0%,0.0%,0.0%,0.2%,226.3kg,225.8kg,0.000g,0.000g,5811,0,0,0,0.0%,0.00feet,0.00feet,0.0,0.0,0.0
Yet for some reason when I import this CSV the line always breaks after the 40th column. Now If I import a CSV with some dummy data (sequential count from 1 to 58) this file will import fine. Im pretty new to this so I am not sure what I am missing.
Thanks Much
Related
php Undefined offset csv file upload
I'm trying to import and fetch data from CSV file and store it in my database my database table schema is like this users ( ID , Fname , Lname, user_type , major , Bday , Email,password , image , status ) csv ( 111 , Leo , pichon, , business, , leo#yzu.cn ) and this is my csv file schema , columns only matching the required filed of my database schema its somthing like this : '' the empty filed either have default value or null '' after importing i get the following notice for almost each row of my data file : Undefined offset: 1 in php csv here's my php function : if ($_GET['action'] = 'UsersCvs') { if (isset($_FILES['users_cvs']['name'])) { $file = $_FILES['users_cvs']['tmp_name']; if ($_FILES["users_cvs"]["size"] > 0) { $handle = fopen($file, "r"); $c = 0; while (($files = fgetcsv($handle, 2000, ",")) !== false) { // $id = mysqli_real_escape_string($link , $files[0]); // $fname = $files[1]; // $lname = $files[2]; // $user_type = $files[3]; // $major = $files[4]; // $email = $files[6]; // $gender = mysqli_real_escape_string($link, $filesop[6]); //$sql = "INSERT INTO users(ID , Fname,Lname , user_type , major , Email) values ('" . $id . "','" . $fname . "' ,'" . $lname . "' ,'" .$user_type. "','" . $major . "','" . $email . "')"; // mysqli_query($link, $sql); $num = count($files); echo "<p> $num fields in row $c: <br /></p>\n"; $c++; for ($i = 0; $i < $num; $i++) { echo $files[$i] . "<br />\n"; } } fclose($file); // if ($sql) { // echo 1 ; // } else { // echo mysqli_error($link); ; // } } else { } }else{ echo 'no file found'; } } what seems strange to my and after inspection my data gets encoded to unreadable string somthing like this <p> 3 fields in line 2: <br /></p> �Ѷѩ\h�1������� ��drKK(G~�S⼥���Nw|x��� �s�k�Ŗ�X���J�fb�fWш�eʬ�y�`hc�g� 1 fields in line 3: ���y��Q�b~���^�)�Ϸ�������LD?��r�:��DN��(����C��*T�?��C��!�&�e���!_�8��+[�|��PK�N�#}�":[docProps/core.xml��QK�0���C�{��cm*{r 8Q|��lҐD��{�v����c�9���^R������:��� Ah���%zٮ���gZ���P�8��no 2 fields in line 4: I've really tried my best and did online research but i couldn't find any matching reference i would highly appreciate help thanks in advance
The file you are trying to process is not a CSV file, but an Excel workbook (a ".xlsx" file). There are two clues here: First, the data you're seeing clearly isn't human-readable text. A CSV file is just a text file formatted to a particular convention, whereas this looks very much like binary data intended only for machine use. The very short snippet you posted happened to include the string "docProps/core.xml". If you search for that online, you'll get lots of references to the "Office Open XML" format used by modern versions of Microsoft Office. The files are structured as a zip file containing multiple XML files, and that's the name of one of the internal files in the zip. The first thing you should do is add some validation to your code. Input from the user always needs validating, and in this case a check for the expected number of fields, or a mandatory header row, would have given you an error straight away. The second thing you should do is either a) tell the user to export the spreadsheet as a CSV file, or b) re-write the code using a library that can read Excel files, such as PhpSpreadsheet.
CSV UPLOAD TO MYSQL WITH PHP
Hi I currently have this php file that will import csv data into mysql database each time its triggered however I want it to overwrite the current data in the database not just keep adding to it everytime it is triggered . Here is the file minus my details : <?php //database connection details $connect = mysql_connect('mydetails','mydetails','mydetails'); if (!$connect) { die('Could not connect to MySQL: ' . mysql_error()); } //your database name $cid =mysql_select_db('mydetails',$connect); // path where your CSV file is located define('CSV_PATH','myfile location'); // Name of your CSV file $csv_file = CSV_PATH . "my file"; if (($handle = fopen($csv_file, "r")) !== FALSE) { fgetcsv($handle); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { $num = count($data); for ($c=0; $c < $num; $c++) { $col[$c] = $data[$c]; } $col1 = $col[0]; $col2 = $col[1]; $col3 = $col[2]; $col4 = $col[3]; // SQL Query to insert data into DataBase $query = "INSERT INTO orders(OrderNumber,OrderCustomer,OrderReqdQty,OrderStatus) VALUES('".$col1."','".$col2."','".$col3."','".$col4."')"; $s = mysql_query($query, $connect ); } fclose($handle); } echo "File data successfully imported to database!!"; mysql_close($connect); ?>
It seems that you've got unique OrderNumber. Add proper key to table and use ON DUPLICATE UPDATE syntax. More on this topic here: http://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html --- EDIT --- First you need to add correct unique key in your table: ALTER TABLE orders add UNIQUE(OrderNumber); This query should add unique key and from now on there are no duplicated OrderNumber possible. If there are any in database this query will throw proper error message in your face. If you simply try add another entry with the same OrderNumber you will be also gracefully welcomed by error message. Now you modify your script with something like this: INSERT INTO orders(OrderNumber, OrderCustomer, OrderReqdQty, OrderStatus) VALUES('".$col1."','".$col2."','".$col3."','".$col4."') ON DUPLICATE KEY UPDATE OrderCustomer='".$col2."', OrderReqdQty='".$col3."', OrderStatus='".$col4."'"; Then everytime OrderNumber is found within table instead of throwing an error, there will be UPDATE performed on this very row.
Fastest way to import csv file into MYSQL
This method uploads csv file to mysql .But upon thousands of data in csv file it takes lots of time to upload the data which is annoying. $deleterecords = "TRUNCATE TABLE discount"; //empty the table of its current records mysql_query($deleterecords); //readfile($name); //Import uploaded file to Database $handle = fopen($name, "r"); $i=0; while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { if($i>0){ $import="INSERT into discount(id,title,expired_date,amount,block)values('".$data[0]."','".$data[1]."','".$data[2]."','".$data[3]."','".$data[4]."')"; //imports data serially to the allocated columns. mysql_query($import) or die(mysql_error());//query } $i=1; } fclose($handle); //closing the handle // print "Import done "; ?> Can anyone suggest faster method for uploading data ?
Instead of writing a script to pull in information from a CSV file, you can link MYSQL directly to it and upload the information using the following SQL syntax. To import an Excel file into MySQL, first export it as a CSV file. Remove the CSV headers from the generated CSV file along with empty data that Excel may have put at the end of the CSV file. You can then import it into a MySQL table by running: load data local infile 'uniq.csv' into table tblUniq fields terminated by ',' enclosed by '"' lines terminated by '\n' (uniqName, uniqCity, uniqComments) as read on: Import CSV file directly into MySQL
Use the LOAD DATA INFILE statement. https://dev.mysql.com/doc/refman/5.1/en/load-data.html Load the data in a temporary table and use that for inserting to your main table with one statement.
You can insert data this way. This is default way to insert rows in a table. $deleterecords = "TRUNCATE TABLE discount"; //empty the table of its current records mysql_query($deleterecords); //readfile($name); //Import uploaded file to Database $handle = fopen($name, "r"); $i=0; $ins = "INSERT into discount(id,title,expired_date,amount,block) values "; while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { if($i>0){ $import .= $ins."('".$data[0]."','".$data[1]."','".$data[2]."','".$data[3]."','".$data[4]."'),"; //imports data serially to the allocated columns. } $import = rtrim($import,','); mysql_query($import) or die(mysql_error());//query $i=1; } fclose($handle); //closing the handle // print "Import done "; ?>
Instead of having multiple insert, build a big query and execute a single insert. <?php $deleterecords = "TRUNCATE TABLE discount"; //empty the table of its current records mysql_query($deleterecords); //readfile($name); //Import uploaded file to Database $handle = fopen($name, "r"); $i=0; $what_to_insert = array(); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { if($i>0){ array_push($what_to_insert, "('".$data[0]."','".$data[1]."','".$data[2]."','".$data[3]."','".$data[4]."')"); } $i=1; } fclose($handle); if (count($what_to_insert)>0){ $import="INSERT into discount(id,title,expired_date,amount,block) values " . implode(",", $what_to_insert); mysql_query($import) or die(mysql_error());//query } ?>
If phpMyAdmin is available you can use the CSV import feature.
Loading CSV file into mysql database using php
I ma struggling to load information from a csv file using PHP and want to save the contents of csv (mobile phone numbers) into mysql database table.The file contents look like this: CSV file contents ( individual record per single line with no commas) 44762126064 447508751 4474669756 44771466603 444584871 445574805 447455471039 44777451345 447460345819 44793342963 44734838672 44752845528 4474537291 44779645078 I am try to upload csv file using form and submit.The code read csv file and tries to write the content into mysql table.The problem is that code is returning all the csv records in one array element like this: Array( [0] => 44762126064 447508751 4474669756 44771466603 444584871 445574805 447455471039 44777451345 447460345819 44793342963 44734838672 44752845528 4474537291 44779645078 ); and inserting the whole array as one record rather one mobile number per row in mysql table. The code : if (isset($_POST['submit'])) { if (is_uploaded_file($_FILES['filename']['tmp_name'])) { echo "<h1>" . "File ". $_FILES['filename']['name'] ." uploaded successfully." . "</h1>"; echo "<h2>Displaying contents:</h2>"; # readfile($_FILES['filename']['tmp_name']); } //Import uploaded file to Database $handle = fopen($_FILES['filename']['tmp_name'], "r"); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { $import="INSERT into mobilecsv(phoneMobile,status) values('$data[0]',0)"; mysql_query($import) or die(mysql_error()); } fclose($handle); print "Import done"; } I have tried explode(), array_slice() etc but none of them helped me to split this array contents to save them as individual records/phone numbers in mysql database. How can i split individual contents of first array element(as in my case) to save them as separate individual records as appearing in my CSV? I am a new programmer and would appreciate your help in this regard.Thanks
The problem is fgetcsv isn't detecting a comma which is at the heart of a COMMA separated value (CSV) file. Add this line at the top of your file after the php opening tag: ini_set("auto_detect_line_endings", true); change your import code to: //Import uploaded file to Database $handle = fopen($_FILES['filename']['tmp_name'], "r"); while(($data = fgetcsv($handle)) !== FALSE){ $phoneMobile = $line[0]; $import="INSERT into mobilecsv(phoneMobile,status) values('$phoneMobile',0)"; mysql_query($import) or die(mysql_error()); } fclose($handle); print "Import done"; }
Importing .csv file into mysql in php
I have a .csv file with me but i am unable to import it into the database. I have parsed my .csv file using the below query. Can you please help me how to insert data into MySql. My code is:- $fp = fopen('test.csv','r') or die("can't open file"); print "<table>\n"; while($csv_line = fgetcsv($fp,1024)) { print '<tr>'; for ($i = 0, $j = count($csv_line); $i < $j; $i++) { print '<td>'.$csv_line[$i].'</td>'; $data[] = $csv_line[$i]; } print "</tr>\n"; } print '</table>\n'; fclose($fp) or die("can't close file");
In MySQL we can import data from CSV file to a table very easily: LOAD DATA LOCAL INFILE 'import.csv' INTO TABLE from_csv FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (FIELD1, FIELD2, FIELD3); http://dev.mysql.com/doc/refman/5.0/en/load-data.html Example: http://www.techtrunch.com/linux/import-csv-file-mysql-table
The code seems to take the csv file and export it out to the browser as a tabular data table. You mentioned importing into a mysql table, but there are no mysql information listed. Create a mysql table and try to map your fields you are importing to database fields in a table. The easiest way for me is to use phpmyadmin to generate the load data sql although the load data local infile mentioned in an answer will work as well if you understand enough to change it. When I learned how to do it I used this tutorial. http://vegdave.wordpress.com/2007/05/19/import-a-csv-file-to-mysql-via-phpmyadmin/
you can use this code. I hope this will be helpfull //after uploading csv file from a upload file field $handle = fopen($_FILES['csvfile']['tmp_name'], "r"); $header = fgetcsv($handle); while(! feof($handle)){ while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { $import="INSERT into csvtest($header[0], $header[1], $header[2], $header[3], $header[4], $header[5],$header[6]) values ('$data[0]', '$data[1]', '$data[2]', '$data[3]', '$data[4]', '$data[5]', '$data[6]')"; mysql_query($import) or die(mysql_error()); } } fclose($handle);