Upload CSV file into SQL database - php

I am curently trying to upload a CSV file into a sql table, but I don't want to upload the first line which contains the name of every column value that will be inserted. I used a variable c to count the number of rows that will be inserted. For the first line c=0 so I used a condition to upload the values from the row only if c!==0, but it stil uploades the values from the first line.
My code looks like this:
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
$c = 0;
if ($_FILES["file"]["type"]=='application/vnd.ms-excel')
{
while(($filesop = fgetcsv($handle, 3000, ",")) !== false)
{
$tid = trim($filesop[0]);
$beneficiar = ucwords(strtolower(trim($filesop[1])));
$locatie = ucwords(strtolower(trim($filesop[2])));
$localitate = ucwords(strtolower(trim($filesop[3])));
$judet = ucwords(strtolower(trim($filesop[4])));
$adresa = ucwords(strtolower(trim($filesop[5])));
$model = trim($filesop[6]);
$qry = mysql_query("SELECT id FROM beneficiari WHERE `nume` = '".$beneficiar."'");
while ($row = mysql_fetch_assoc($qry)){
$id_client=$row['id'];
}
$qry_id_model=mysqli_query("SELECT id FROM modele WHERE `model` = '".$model."'");
while ($row = mysql_fetch_assoc($qry_id_model)){
$id_model=$row['id'];
}
$adresa1 = $adresa.",".$localitate;
if ($c!==0){
$sql = mysql_query(
"INSERT INTO pos_equipments_other
(id_client, model, tid, beneficiar, adresa, agentie, judet)
VALUES
('$id_client','$id_model','$tid','$beneficiar','$adresa1',
'$locatie','$judet')"
);
}
$c = $c + 1;
}
}

Related

Import a single CSV file into multiple tables in MySQL

I would like to import a single CSV with 7 columns into 2 tables in MySQL.
Columns 1, 2 and 3 go into a single row in table1. Columns 4 and 5 go as a row in table2. Columns 6 and 7 go as a row again in same table2.
How can this be done using PHP or mysql directly?
First fetch all values from csv and store it in an array. Then maintain your array as per your requirement and then start looping and inserting.
<?php
$efected = 0;
$file_temp = $_FILES["file"]["tmp_name"];
$handle = fopen($file_temp, "r"); // opening CSV file for reading
if ($handle) { // if file successfully opened
$i=0;
while (($CSVrecord = fgets($handle, 4096)) !== false) { // iterating through each line of our CSV
if($i>0)
{
list($name, $gender, $website, $category, $email, $subcat) = explode(',', $CSVrecord); // exploding CSV record (line) to the variables (fields)
//print_r($subcat); sub_cat_ids
if($email!='')
{
$chk_sql = "select * from employees where employee_email='".$email."'";
$chk_qry = mysqli_query($conn, $chk_sql);
$chk_num_row = mysqli_num_rows($chk_qry);
$cat_array = array(trim($category));
$subcat_array = explode( ';', $subcat );
if(count($subcat_array)>0)
{
$subcat_array_new = array();
foreach($subcat_array as $key => $val)
{
$subcat_array_new[] = trim($val);
}
}
$cat_sub_cat_merge = array_merge($cat_array, $subcat_array_new);
$cat_subcat_comma = implode( "','", $cat_sub_cat_merge );
foreach($cat_array as $cat_key => $cat_val)
{
$chk_cat_sql = "select * from employee_cats where cats_name = '".$cat_val."'";
$chk_cat_qry = mysqli_query($conn, $chk_cat_sql);
$chk_cat_num_row = mysqli_num_rows($chk_cat_qry);
//$all_cat_array = array();
if($chk_cat_num_row == 0)
{
$new_cat_ins = "insert into employee_cats set cats_name = '".$cat_val."', parent_cat_id = '0' ";
$new_cat_ins_qry = mysqli_query($conn, $new_cat_ins);
}
}
foreach($subcat_array_new as $subcat_key => $subcat_val)
{
$chk_subcat_sql = "select * from employee_cats where cats_name = '".$subcat_val."'";
$chk_subcat_qry = mysqli_query($conn, $chk_subcat_sql);
$chk_subcat_num_row = mysqli_num_rows($chk_subcat_qry);
//$all_cat_array = array();
if($chk_subcat_num_row == 0 && trim($subcat_val)!='')
{
//$category
$get_catid_sql = "select * from employee_cats where cats_name = '".trim($category)."'";
$chk_catid_qry = mysqli_query($conn, $get_catid_sql);
$fetch_cat_info = mysqli_fetch_array($chk_catid_qry);
$fetch_cat_id = $fetch_cat_info['cats_id'];
$new_subcat_ins = "insert into employee_cats set cats_name = '".$subcat_val."', parent_cat_id = '".$fetch_cat_id."' ";
$new_subcat_ins_qry = mysqli_query($conn, $new_subcat_ins);
}
}
$get_cat_sql = "select * from employee_cats where cats_name in ('".$cat_subcat_comma."')";
$get_cat_qry = mysqli_query($conn, $get_cat_sql);
$get_cat_num_row = mysqli_num_rows($get_cat_qry);
$sub_category_ids = array();
if($get_cat_num_row>0)
{
while($fetch_cat_id = mysqli_fetch_array($get_cat_qry))
{
if($fetch_cat_id['parent_cat_id']==0)
{
$category_id = $fetch_cat_id['cats_id'];
}
else
{
$sub_category_ids[] = $fetch_cat_id['cats_id'];
}
}
$sub_cat_id_vals_comma = implode(",", $sub_category_ids);
}
else
{
$category_id = 0;
$sub_cat_id_vals_comma = "";
}
if($chk_num_row>0)
{
// and here you can easily compose SQL queries and map you data to the tables you need using simple variables
$update_sql = "update employees set
employee_name='".$name."',
employee_gender='".$gender."',
employees_website='".$website."',
employees_cat_id='".$category_id."',
sub_cat_ids='".$sub_cat_id_vals_comma."'
where employee_email='".$email."'";
$mysqli_qry = mysqli_query($conn, $update_sql);
}
else
{
// and here you can easily compose SQL queries and map you data to the tables you need using simple variables
$insert_sql = "insert into employees set
employee_name='".$name."',
employee_gender='".$gender."',
employees_website='".$website."',
employees_cat_id='".$category_id."',
sub_cat_ids='".$sub_cat_id_vals_comma."',
employee_email='".$email."'";
$mysqli_qry = mysqli_query($conn, $insert_sql);
}
$efected = 1;
}
}
$i++;
}
fclose($handle); // closing file handler
}
if($efected==1)
{
header('location:'.$site_url.'?import=success');
}
else
{
header('location:'.$site_url.'?import=failed');
}
?>

Cannot upload large csv files

I wrote a php script to extract data from a csv script and insert it into an mysql database. It does the job but does not extract data and insert into database for large csv files.(I tried extracting and inserting from a csv file with 40,000 rows. Did not work but worked when i tested it out with a far smaller (60 rows) size csv file). I get no error message. It just doesn't work. During debugging i realised even echoing a string after selecting a large csv file and clicking my upload button does not work. I've searched and tried all the possible configurations in php.ini as well as LimitRequestBody for apache config as well as setting configs directly in my code to no success. Some help would be greatly appreciated. My code is below.
<?php
include ("connection.php");
if(isset($_POST["submit"]))
{
ini_set('auto_detect_line_endings',TRUE);
$file_name = $_FILES['file']['tmp_name'];
$sql10 = "insert into upload (filename) values ('$file_name')";
if(mysqli_query($conn, $sql10)){
$last_id = mysqli_insert_id($conn);
echo "New record created successfully. Last inserted ID is: " .$last_id;
}
//mysqli_close($conn);
$file = $_FILES['file']['tmp_name'];
//ini_set("auto_detect_line_endings", true);
$handle = fopen($file, "r");
$c = 0;
while(($filesop = fgetcsv($handle, ",")) !== false)
{
if ($c > 0) {
$event_id = $filesop[0];
$conf_session_id = $filesop[1];
$service_provider_id = $filesop[2];
$service_provider_id2 = $filesop[3];
$billed_flag = $filesop[4];
$offering_id = $filesop[5];
$time_start = $filesop[6];
$time_ended = $filesop[7];
$duration = $filesop[8];
$orig_calling_code = $filesop[9];
$orig_area_code = $filesop[10];
$termination_reason = $filesop[11];
$rtp_encoding = $filesop[12];
$rtp_clock_rate = $filesop[13];
$sip_from = $filesop[14];
$sip_to = $filesop[15];
$sip_call_id = $filesop[16];
$orig_nbr = $filesop[17];
$dest_nbr = $filesop[18];
$popd_account_number = $filesop[19];
$intl_dest_flag = $filesop[20];
$sip_status = $filesop[21];
$gw_ip_ingress = $filesop[22];
$gw_port_egress = $filesop[23];
$phone_number = $filesop[24];
$orig_flag = $filesop[25];
$subscriber_sip_caller_id = $filesop[26];
$orig_route_type = $filesop[27];
$term_route_type = $filesop[28];
$call_type = $filesop[29];
$mny_BasePrice = $filesop[30];
$call_cost = $filesop[31];
$uploadID = $last_id;
$sql11 = "insert into cdr (event_id, conf_session_id, service_provider_id, service_provider_id2, billed_flag, offering_id, time_start, time_ended, duration, orig_calling_code, orig_area_code, termination_reason, rtp_encoding, rtp_clock_rate, sip_from, sip_to, sip_call_id, orig_nbr, dest_nbr, popd_account_number, intl_dest_flag, sip_status, gw_ip_ingress, gw_port_egress, phone_number, orig_flag, subscriber_sip_caller_id, orig_route_type, term_route_type, call_type, mny_BasePrice, call_cost, uploadID, date_uploaded) values ('$event_id', '$conf_session_id', '$service_provider_id', '$service_provider_id2', '$billed_flag', '$offering_id', '$time_start', '$time_ended', '$duration', '$orig_calling_code', '$orig_area_code', '$termination_reason', '$rtp_encoding', '$rtp_clock_rate', '$sip_from', '$sip_to', '$sip_call_id', '$orig_nbr', '$dest_nbr', '$popd_account_number', '$intl_dest_flag', '$sip_status', '$gw_ip_ingress', '$gw_port_egress', '$phone_number', '$orig_flag', '$subscriber_sip_caller_id', '$orig_route_type', '$term_route_type', '$call_type', '$mny_BasePrice', '$call_cost', '$uploadID', CURRENT_DATE)";
if(mysqli_query($conn, $sql11)){
// $last_id = mysqli_insert_id($conn);
// echo "New record created successfully. Last inserted ID is: " .$last_id;
$updatequery = "UPDATE cdr JOIN client ON cdr.orig_nbr = client.phonenumber SET cdr.clientname = client.clientname";
mysqli_query($conn, $updatequery);
} else{
echo "error: ".mysqli_error($conn);
}
}
$c++;
}
fclose($handle) ;
}
?>

How to check for duplicate values for excel sheet values before storing in database

I'm getting data from excel sheet and storing it in database. It is storing the data successfully. I had created two tables in database, one is to import values from excel sheet & store temporarily and other is to get values from temporarily stored table,also checking for duplicate records.
I can insert data in second table but please suggest me how to check duplicate records before storing in original table.
This is the code:
<?php
if(isset($_POST["submit"]))
{
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
$c = 0;
$row = 1;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$row++;
if($row != 1)
{
$custid = $filesop[0];
$zone = $filesop[1];
$city = $filesop[2];
$category = $filesop[3];
$focus = $filesop[4];
$customer_type = $filesop[5];
$lead_source = $filesop[6];
$exhibition = $filesop[7];
$organization_name = $filesop[8];
$assign_to = $filesop[9];
$description = $filesop[10];
$division = $filesop[11];
$product = $filesop[12];
$grade = $filesop[13];
$potential = $filesop[14];
$firstname = $filesop[15];
$lastname = $filesop[16];
$designation = $filesop[17];
$mobile = $filesop[18];
$primary_phone = $filesop[19];
$primary_email = $filesop[20];
$address_type = $filesop[21];
$address1 = $filesop[22];
$address2 = $filesop[23];
$state = $filesop[24];
$country = $filesop[25];
$hgf = "INSERT INTO temp_const set custid='$name', zone='$zone', city='$city', category='$category', focus='$focus', customer_type='$customer_type', lead_source='$lead_source', exhibition='$exhibition', organization_name='$organization_name', assign_to='$assign_to', description='$description', division='$division', product='$product', grade='$grade', potential='$potential', firstname='$firstname', lastname='$lastname', designation='$designation', mobile='$mobile', primary_phone='$primary_phone', primary_email='$primary_email', address_type='$address_type', address1='$address1', address2='$address2', state='$state', country='$country' ";
$sql = mysql_query($hgf);
$c = $c + 1;
}
}
if($sql){
echo "You database has imported successfully. You have inserted ". $c ."recoreds";
}else{
echo " Sorry! There is some problem.";
}
}
?>
You can create a unique index on the fields where you want to check on duplication, lets say it is fieldA, fieldB and fieldC
ALTER TABLE destTable add unique key idx_abc (fieldA,fieldB,fieldC);
after this you can copy your tmp table into the destTable with the option IGNORE
INSERT IGNORE INTO destTable SELECT * FROM temp_const;
or use only the field you want.

How to remove duplicate row records when importing CSV files to mysql using php?

I need to remove duplicate records when importing my current CSV files into the database. My files has been successfully updated, but when I tried to upload the same files again, it straight away inserted into it again. is there any way of I can remove duplicate records if i am importing the files?
<?
if(isset($_POST["submit"])){
$file = $_FILES['file']['tmp_name'];
//echo 'upload file name: '.$file.' ';
$handle = fopen($file, "r");
$c = 0;
$count =0;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$count ++;
$ID = $filesop[0];
$Name = $filesop[1];
$Contact = $filesop[2];
$Email =$filesop[3];
if($count>1){
$sql = "INSERT INTO clients(id,name,contact,email,)VALUES($ID,'$Name',$Contact,'$Email',')";
$resultsql = mysqli_query($link, $sql);
//echo $resultsql; //how am i going to remove duplicates when if there is a duplicate record ?
1) Before inserting, check existing data.
<?php
if(isset($_POST["submit"])){
$file = $_FILES['file']['tmp_name'];
//echo 'upload file name: '.$file.' ';
$handle = fopen($file, "r");
$c = 0;
$count =0;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$count ++;
$ID = $filesop[0];
$Name = $filesop[1];
$Contact = $filesop[2];
$Email =$filesop[3];
$checkExistingData = "SELECT * FROM clients WHERE name='$Name' AND contact='$Contact' AND email='$Email'";
$resultcheckExistingData = mysqli_query($link, $checkExistingData);
$countExistingData = mysqli_num_rows($resultcheckExistingData);
if($countExistingData == 0)
{
if($count>1) {
$sql = "INSERT INTO clients(id,name,contact,email,)VALUES($ID,'$Name',$Contact,'$Email',')";
$resultsql = mysqli_query($link, $sql);
//echo $resultsql; //how am i going to remove duplicates when if there is a duplicate record ?
.
.
}
.
.
}?>
2) If data got inserted and you want to delete duplicate rows from table. You can try this too.
DELETE c1 FROM clients c1, clients c2 WHERE c1.name = c2.name AND c1.contact = c2.contact AND c1.email = c2.email AND c1.id > c2.id;
Add unique key to one or more of your columns, column with unique key will store unique values only, duplicate values will not be inserted.

Import CSV file with a column having date or text inside

I have a CSV file with a column having a date in format d/m/Y or the word "Illimité" inside meaning unlimited in French.
I would like each time it reads "Illimité" it puts NULL value inside MySQL table.
Here is my current PHP code :
if (isset($_POST['submit'])) {
$query = "TRUNCATE TABLE `formation` ";
$result = mysql_query($query);
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
while (($fileop = fgetcsv($handle, 1000, ";")) !== false) {
$nom = $fileop[0];
$prenom = $fileop[1];
$formation = $fileop[16];
$validite = $fileop[26];
if (is_numeric($validite)) {
$validite = date_format(date_create_from_format('d/m/Y', $validite), 'Y-m-d');
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation','$validite','importCSV')");
} else {
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation',NULL,'importCSV')");
}
}
Sadly this isn't working. MySql shows no errors and it puts NULL all the time.
Any help would me much appreciate.
Try this:
function myFunc($CSVValue){
if(validateDate($CSVValue)){
//your mysql logic where you put in the date
}else{
//your mysql logic where you put in the null
}
}
function validateDate($date)
{
$d = DateTime::createFromFormat('d/m/Y', $date);
return $d && $d->format('d/m/Y') == $date;
}
function was copied from this answer or php.net
-- update --
I don't know how your code looks other than what you have provided.
If you were to put this in your code it could look something like this:
if (isset($_POST['submit'])) {
$query = "TRUNCATE TABLE `formation` ";
$result = mysql_query($query);
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
while (($fileop = fgetcsv($handle, 1000, ";")) !== false) {
$nom = $fileop[0];
$prenom = $fileop[1];
$formation = $fileop[16];
$validite = $fileop[26];
$d = DateTime::createFromFormat('d/m/Y', $validite);
if ($d && $d->format('d/m/Y') == $validite) {
$validite = date_format(date_create_from_format('d/m/Y', $validite), 'Y-m-d');
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation','$validite','importCSV')");
} else {
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation',NULL,'importCSV')");
}
}

Categories