upload and import csv file into multiple table in mysql [closed] - php

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I would like to create and upload page in php and import the uploaded csv file data into multiple tables. tried searching here but looks like can't find any which is importing from a csv to multiple table. any help here is greatly appreciated. thank you.

As the another variant proposed above, you can read your CSV line-by-line and explode each line into fields. Each field will corresponds one variable.
$handle = fopen("/my/file.csv", "r"); // opening CSV file for reading
if ($handle) { // if file successfully opened
while (($CSVrecord = fgets($handle, 4096)) !== false) { // iterating through each line of our CSV
list($field1, $field2, $field3, $field4) = explode(',', $CSVrecord); // exploding CSV record (line) to the variables (fields)
// and here you can easily compose SQL queries and map you data to the tables you need using simple variables
}
fclose($handle); // closing file handler
}

If you have access to PHPmyadmin, you can upload the CSV into there. Then copy if over to each desired table

In response to your comment that some data is going to one table and other data is going to another table, here is a simple example.
Table1 has 3 fields: name, age and sex. Table2 has 2 fields: haircolour, shoesize. So your CSV could be laid out like:
john smith,32,m,blonde,11
jane doe,29,f,red,4
anders anderson,56,m,grey,9
For the next step you will be using the function fgetcsv. This will break each line of the csv into an array that you can then use to build your SQL statements:
if (($handle = fopen($mycsvfile, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
// this loops through each line of your csv, putting the values into array elements
$sql1 = "INSERT INTO table1 (`name`, `age`, `sex`) values ('".$data[0]."', '".$data[1]."', '".$data[2]."')";
$sql2 = "INSERT INTO table2 (`haircolour`, `shoesize`) values ('".$data[3]."', '".$data[4]."')";
}
fclose($handle);
}
Please note that this does not take any SQL security such as validation into account, but that is basically how it will work.

the problem seems to me to differentiate what field is for which table.
when you are sending a header like
table.field, table.field, table.field
and then split the header, you'll get all tables and fields.
could that be a way to go?
all the best
ps: because of your comment ...
A csv file has/can have a first line with fieldnames in it. when there is a need too copy csv data into more than one tables, then you can use a workaround to find out which field is for which table.
user.username, user.lastname, blog.comment, blog.title
"sam" , "Manson" , "this is a comment", "and I am a title"
Now, when reading the csv data you can work over the first line, split the title at the dot to find out wich tables are used and also the fields.
With this method you are able to copy csv data to more than one table.
But it means, you have to code it first :(
To split the fieldnames
// only the first line for the fieldnames
$topfields = preg_split('/,|;|\t/')
foreach( $topfields as $t => $f ) {
// t = tablename, f = field
}

if (($handle = fopen($mycsvfile, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
// this loops through each line of your csv, putting the values into array elements
$sql1 = "INSERT INTO table1 (`name`, `age`, `sex`) values ('".$data[0]."', '".$data[1]."', '".$data[2]."')";
$sql2 = "INSERT INTO table2 (`haircolour`, `shoesize`) values ('".$data[3]."', '".$data[4]."')";
}
fclose($handle);
}
in above code you use two insert queries how you gonna run these queries ?

Related

Read a CSV file and upload data to correct table in PHP

I have a SQL database with 5 tables and I also have 5 CSV files, one for each of those tables.
I am struggling to create a PHP script that can be used to read each files and then upload the data into the correct table.
How can I go about this?
URL - http://php.net/manual/en/function.fgetcsv.php
<?php
$filename = "test.csv";
//Open the csv file in the read mode
if (($handle = fopen($filename, "r")) !== FALSE) {
//loop into the each row and do the stuff
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
//$data will have all the related columns
//Make sure you will use INSERT query here
}
fclose($handle);
}
If you want to write to specific tables there are many way to do so
Create 5 files where in each and every file while loop you will have that respective table name.
Eg : Think that you want to insert for customers.csv file
Then in while loop you may use the following way
while(/*Code*/){
//INSERT INTO customers (col1, col2) VALUES (val1, val2);
}
In case if your lazy than in one file only you can make a switch operation as follows
$filename = 'customers.csv';
switch( $filename ) {
case 'customers.csv':
while(){
//Insert
}
break;
case 'products.csv':
while(){
//Insert
}
break;
}

csv file with delimiter other than comma in php

I insert data into table as bulk upload,
$handle = fopen($_FILES['file_clg']['tmp_name'], "r");
fgetcsv($handle);
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$collegename = trim(str_replace(array("’","'"),"'",$data[0]));
$description = trim(str_replace(array("’","'"),"'",$data[1]));
$sql1 = $db->selectquery("insert into $tbl(name,details)values('" .$collegename."','" .$description."')");
}
fclose($handle);
Only two fields is mentioned here: morethan 25 columns in my bulkupload csv
The problem is that the csv delimiter is the comma (',') but in some cases 'details' field contents include commas, as in this case record not inserted properly..
how to solve this case???
And a problem in insertion section,
College name : Alva’s Institute of Engineering & Technology (AIET)
and its saved in table as below format :
Alva�s Institute of Engineering & Technology (AIET)
I try below code:
$collegename = htmlentities(iconv("cp1252", "utf-8", trim(str_replace(array("’","'"),"'",$data[0]))), ENT_IGNORE, "UTF-8");
but its not working, how can i solve the issue in single quotes
And i placed : header("Content-Type: text/html; charset=ISO-8859-1");
into the header section..
I'd need to see some samples to say anything with confidence, but there are specs on quoting values with commas to preserve the value count.
https://stackoverflow.com/a/769675/2943403
Alternatively, you could create a new fputcsv() code block that will generate a semi-colon (or other non-conflicting character) delimited csv file. I am not providing the actual snippet for this. There are many available resources on SO, including:
php fputcsv use semicolon separator in CSV
Export to CSV via PHP
PHP How to convert array into csv using fputcsv function
Then your while loop could use ; (or whatever) as a delimiter.
As for safely placing your values (which may have single quotes) into your query, use prepared statements
// I realize your query will be much larger than the sample...
// declare $collegename and $decription values
$stmt=$db->prepare("INSERT INTO `$tbl` (`name`,`details`) VALUES (?,?)");
$stmt->bind_param("ss", $collegename,$description);
if($stmt->execute()){
echo "success";
}else{
echo "Error: ",$db->error;
}

PHP - insert into mutiple tables, each table with identifcal fields

I'm going to explain with my best efforts what my goal is here. Everything I've searched for online hasn't been relevant enough for me to gain an idea.
First off, this is a PHP assignment where we have to load CSV files into a MySQL database.
Now, each table (total of 4) have the exact same field values. What I am trying to accomplish is using a for each loop that populates each table with the information from the CSV file. I know I can do this by having a while loop for each table and CSV file but I'm trying to go above the requirements and learn more about PHP. Here is my code for what I'm trying to accomplish:
$files = glob('*.txt'); // .txt required extension
foreach($files as $file) {
if (($handle = fopen($file, "r")) !== FALSE) {
while (($data = fgetcsv($handle,4048, ",")) !== FALSE) {
echo $data[0]; // making sure data is correct
$import = "INSERT INTO".basename($file)."(id,itemName,price) VALUES('$data[0]','$data[1]','$data[2]')";
multi_query($import) or die (mysql_error());
}
fclose($handle);
}
else {
echo "Could not open file: " . $file;
}
}
Each CSV file contains the id, itemName and price. Hopefully this is understandable enough. Thank you
The way you are importing data into MySQL is OK for small volume of data. However, if you are importing huge volumes(thousands of rows), the best way would be to import it directy into MySQL is by using infile. Fo example:
LOAD DATA LOCAL INFILE '/path/to/your_file.csv'
INTO TABLE your_table_name
FIELDS TERMINATED BY ','
ENCLOSED BY '"' LINES
TERMINATED BY '\n' (id, itemName, price)
That's a smarter way to import your CSV data :)

get details on file then insert to database

I have a .txt file with details of all countries in this format:
Country,City,AccentCity,Region,Population,Latitude,Longitude
ad,aixas,Aixàs,06,,42.4833333,1.4666667
ad,aixirivali,Aixirivali,06,,42.4666667,1.5
ad,aixirivall,Aixirivall,06,,42.4666667,1.5
ad,aixirvall,Aixirvall,06,,42.4666667,1.5
ad,aixovall,Aixovall,06,,42.4666667,1.4833333
ad,andorra,Andorra,07,,42.5,1.5166667
ad,andorra la vella,Andorra la Vella,07,20430,42.5,1.5166667
ad,andorra-vieille,Andorra-Vieille,07,,42.5,1.5166667
ad,andorre,Andorre,07,,42.5,1.5166667
ad,andorre-la-vieille,Andorre-la-Vieille,07,,42.5,1.5166667
ad,andorre-vieille,Andorre-Vieille,07,,42.5,1.5166667
ad,ansalonga,Ansalonga,04,,42.5666667,1.5166667
I've to insert these data into 3 tables like cities, states and country with out duplication.
Any possible way to read the datas from the .txt file and insert it to the database?
How can I get the state and city database?
Read the file as a CSV and then insert:
if (($handle = fopen("cities.txt", "r")) !== FALSE) {
while (($data = fgetcsv($handle)) !== FALSE) {
// Craft your SQL insert statement such as:
$sql = "INSERT INTO cities (country, city, accent_city, etc.) VALUES ('{$data[0]}','{$data[1]}','{$data[2]}', etc.)";
// Use the appropriate backend functions depending on your DB, mysql, postgres, etc.
}
}
If you database is mysql, exists an utility to bulk insert, documentation
If not, probably your database has one too, but if you want to do this using PHP, #davidethell's example is good for do the task

PHP MYSQL import CSV and then compare and remove redundant entries

I am stuck with a peculiar issue here. I have a script that basically imports a CSV file into a database using fgetcsv() in php. There is no problem in doing this at all and I am able to update old entries as well using MySQL syntax ON DUPLICATE KEY UPDATE (I am in no way a MySQL expert, hence me asking here).
Here is that part of the code:
$handle = fopen($file,"r");
fgetcsv($handle, 1000, ",");//skip first row since they are headers
while(($fileop = fgetcsv($handle, 1000, ",")) !== false) //read line by line into $fileop
{
//read array values into vars
$item1 = $fileop[0];
$item2 = $fileop[1];
$key = $fileop[2];
// and a couple more
// now INSERT / UPDATE data in MySQL table
$sql = mysql_query("INSERT INTO table (item1,item2,key)
VALUES ('$item1','$item2','$key')
ON DUPLICATE KEY UPDATE item1='$item1',item2='$item2'");
}
This all works fine. What I am stuck with is the fact that some entries may have been removed from the actual CSV (as in the key may no longer be existant). What I would like to do is remove the entries from the MySQL table that are no longer present in the CSV.
Meaning if $key is gone from CSV also remove that row in the database table. I suppose I would do it before I run the Insert / Update query on the MySQL table?
I would appreciate any help guys.
Just keep an account of your keys.
Save every $key in an array in your while, and in the end run a query that says
DELETE FROM tabel WHERE key NOT IN (listofcommaseparatedkeysgoeshere)
$arrayThatYouNeedToTest = array();
$handle = fopen($file,"r");
fgetcsv($handle, 1000, ",");//skip first row since they are headers
while(($fileop = fgetcsv($handle, 1000, ",")) !== false) //read line by line into $fileop
{
//read array values into vars
$item1 = $fileop[0];
$item2 = $fileop[1];
$key = $fileop[2];
// and a couple more
// now INSERT / UPDATE data in MySQL table
$sql = mysql_query("INSERT INTO table (item1,item2,key)
VALUES ('$item1','$item2','$key')
ON DUPLICATE KEY UPDATE item1='$item1',item2='$item2'");
$arrayThatYouNeedToTest[] = $key;
}
$stringThatYouNeedToInspect = implode(",",$arrayThatYouNeedToTest);
$queryYouREALLYneedToCheckFirst = "DELETE FROM tabel WHERE key NOT IN (".$stringThatYouNeedToInspect.")";
//$result = mysql_query($queryYouREALLYneedToCheckFirst);
I do something very similar to this with an affiliate website - having just under 500,000 products.
In your database, simply add another column named "update_flag" or something similar. Set the default to be 0. As you add items from the CSV file, set the update_flag to be "1". In your 'on duplicate statement', set the filed to be "2". I also went and added 2 other fields: "date_added" and "date_updated".
After your import is complete, you can count the old items (to be deleted), newly added items, and those that have been updated. You can then simple delete from table where update_flag = 0
I hope this helps.

Categories