I am trying to insert data from csv to MYSQL database like below and its working fine
$file = fopen('../assets/uploads/'.$file_name, "r");
while (($column = fgetcsv($file, 10000, ",")) !== FALSE) {
$fname = "";
if (isset($column[0])) {
$fname = mysqli_real_escape_string($mysqli, $column[0]);
}
$lname = "";
if (isset($column[1])) {
$lname = mysqli_real_escape_string($mysqli, $column[1]);
}
$email = "";
if (isset($column[2])) {
$email = mysqli_real_escape_string($mysqli, $column[2]);
}
$sqlInsert = "INSERT into $lead_data_table (lfname,llname,lemail,lead_id,lead_user_id) VALUES (?,?,?,?,?)";
$stmt = $mysqli->prepare($sqlInsert);
$stmt->bind_param('sssii', $fname, $lname,$email,$lead_insert_id,$lead_user_id);
$stmt->execute();
$stmt->close();
$insertId = mysqli_insert_id($mysqli);
However For some reason I want insert data from CSV like 0 to 100 or 100 to Remain All. I am not getting idea how I can do it? Let me know if anyone here can help me for do the same.
Thanks!
Count the csv lines and skip. Something like this will process starting with the hundredth line.
$file = fopen('../assets/uploads/'.$file_name, "r");
$lineCount = 0;
while (($column = fgetcsv($file, 10000, ",")) !== FALSE) {
$lineCount ++;
if ($lineCount >= 100) {
/* process the line */
}
}
And by the way, you're doing your prepared statements correctly. So you don't need to use mysqli_real_escape_string() to mung your data before inserting it.
Related
How import this file to mysql with spaces between rows?
Through phpmyadmin it is possible, but I need do it via website.
<?php
// Check if file was uploaded & there were no errors
if ($_FILES && $_FILES['csv-file']['error'] == 0) {
$extension = pathinfo($_FILES['csv-file']['name'],PATHINFO_EXTENSION);
// Check if extension is csv then proceed to import
if($extension == 'csv'){
// Open file for reading
$file = fopen($_FILES['csv-file']['tmp_name'], 'r');
// Loop through all rows of file and insert them to database table
while (!feof($file)) {
// Get current row as recordset
$row = fgetcsv($file);
if (!empty($row)) {
$data = [];
$data['numbers'] = htmlentities($row[0]);
$data['tids'] = htmlentities($row[1]);
$data['date'] = htmlentities($row[2]);
$data['time'] = htmlentities($row[3]);
$data['zero'] = htmlentities($row[4]);
$data['terminal_sn'] = htmlentities($row[5]);
$data['space'] = htmlentities($row[6]);
$records[] = $data;
mysqli_query($dbcon,"INSERT INTO employees (".implode(",",array_keys($data)).") VALUES ('".implode("','",array_values($data))."')");
}
}
}else{?>
And this is my .csv file with data:
10222157120501 T0040922 07/09/2020 18:13:56 0 315-525-348 1
10223157120502 T0040923 07/09/2020 18:15:24 0 318-027-497 1
10224157120503 T0040924 07/09/2020 18:15:36 0 316-176-614 1
10225157120504 T0040925 07/09/2020 18:16:25 0 317-377-077 1
// prepare the insert query once outside the loop
$sql = 'INSERT INTO employees (`numbers`, `tids`, `date`, `time`,
`zero`, `terminal_sn`, `space`)
VALUES(?,?,?,?,?,?,?)';
$stmt = $dbcon->prepare($sql);
if (($handle = fopen($_FILES['csv-file']['name'], "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, " ")) !== FALSE) {
// space delimiter ^
// bind the columns to the query parameters
$stmt->bind_param('sssssss', $row[0], $row[1], $row[2], $row[3],
$row[4],$row[5], row[6]);
// execute the query with parameters replaced with data
$stmt->execute();
}
fclose($handle);
}
problem is in my excel 369 rows are there. when I echo/print that data it showing correct but when I am inserting same data in DB table in inserted only 18 - 30 records.
if (isset($_POST['Submit'])) {
$file = $_FILES['csv_file']['tmp_name'];
$handle = fopen($file, "r");
if ($file == NULL) {
error(_('Please select a file to import'));
redirect(page_link_to('excel_data_upload'));
}else {
$conn = connect();
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$num3 = $filesop[3];
$num8 = $filesop[8];
$num9 = $filesop[9];
$num20 = $filesop[20];
if($num3!='ExpiryDate' &&$num8!='VCNO' &&$num20!='TotalB2CAmount' && $num9 !='STBNO'){
$insertAgent = mysqli_query($conn, "INSERT INTO `upload_billing_data`
(`vc_number`,`stb_number`,`operator_id`,`expiry_date`,`monthly_bill_amount`)
VALUES ('$num8','$num9',140,'$num3','$num20')");
if($insertAgent)
{
echo 'succss';
}else{
echo 'error';
}
}
}
close($conn);
}
}
I am fetching from the excel data. I want to insert all records
Change the code as below and you might get to save all data using one query to the database:
$query_insert = array();
while(($filesop = fgetcsv($handle, 1000, ",")) !== false) {
$num3 = filterString($filesop[3]);
$num8 = filterString($filesop[8]);
$num9 = filterString($filesop[9]);
$num20 = filterString($filesop[20]);
if ($num3!='ExpiryDate' &&$num8!='VCNO' &&$num20!='TotalB2CAmount' && $num9 !='STBNO') {
$query_insert[] = "('{$num8}', '{$num9}', 140, '{$num3}', '{$num20}')";
}
}
// If no row matched your if, then there will be no row to add to the database
if (count($query_insert)>0) {
$conn = connect();
$query_insert_string = implode(', ', $query_insert);
$query = "INSERT INTO `upload_billing_data` (`vc_number`, `stb_number`, `operator_id`, `expiry_date`, `monthly_bill_amount`) VALUES {$query_insert_string};";
$insertAgent = mysqli_query($query);
// The rest of you code
...
close($conn);
}
// This function makes sure that you string doesn't contain characters that might damage the query
function filterString($string) {
$string = str_replace(array("\'", '"'), array('', ''), $string);
$string = filter_var($string, FILTER_SANITIZE_STRING);
return $string;
}
Please check this modified code
if (isset($_POST['Submit'])) {
$file = $_FILES['csv_file']['tmp_name'];
$handle = fopen($file, "r");
if ($file == NULL) {
error(_('Please select a file to import'));
redirect(page_link_to('excel_data_upload'));
}else {
$conn = connect();
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$num3 = $filesop[3];
$num8 = $filesop[8];
$num9 = $filesop[9];
$num20 = $filesop[20];
if($num3!='ExpiryDate' &&$num8!='VCNO' &&$num20!='TotalB2CAmount' && $num9 !='STBNO'){
$insertAgent = mysqli_query($conn, "INSERT INTO `upload_billing_data`
(`vc_number`,`stb_number`,`operator_id`,`expiry_date`,`monthly_bill_amount`)
VALUES ('".mysqli_real_escape_string($num8)."','".mysqli_real_escape_string($num9)."',140,'".mysqli_real_escape_string($num3)."','".mysqli_real_escape_string($num20)."')");
if($insertAgent)
{
echo 'succss';
}else{
echo 'error';
}
}
}
close($conn);
}
}
BY using mysqli_real_escape_string() you will be able to avoid sqlinjection issues and you will be able to handle issue of quotes which might be causing an issue.
in your else block where you are echo "error". you can use mysqli_error($conn); to get exact what error is occurring while performing an insert
I am trying to read CSV file using fgetcsv() php function but It doesn't fetch detail as it supposed to be. I found out that the CSV file is cluttered and has multiple irrelevant commas in starting and in-between. How do I make this CSV cleaner?
I used str_replace() php function to remove triplets of commas but the commas in starting are still giving me a problem. I tried ltrim() also but that didn't work too.
<?php
$file = "grid.csv";
$s = file_get_contents($file);
$s = str_replace(",,,", "", $s);
//$s = ltrim($s,",");
$f = "grid1.csv";
$handle = fopen($f, "w");
fwrite($handle, $s);
?>
I expect the output of this code to be a clean csv file. But I get multiple commas in starting now also in the new file.
This is the Main Code where I was trying to read the file using fgetcsv().
if(isset($_POST["submit"])){
echo "in submit</br>";
if($_FILES['csv_info']['name']){
echo "some file</br>";
$filename = explode(".", $_FILES['csv_info']['name']);
if(end($filename) == 'csv'){
echo "file is csv</br>".$_FILES['csv_info']['tmp_name'];
$handle = fopen($_FILES['csv_info']['tmp_name'],"r");
$sid = 0;
//$query = "select exists(select 1 from tblMarks)";
//$choice = mysqli_query($conn, $query);
while($data = fgetcsv($handle)){
if($sid == 0){
$sid = $sid + 1;
continue;
}
//echo $data;
$name = mysqli_real_escape_string($conn, $data[0]);
$physics = mysqli_real_escape_string($conn, $data[1]);
$maths = mysqli_real_escape_string($conn, $data[2]);
$chemistry = mysqli_real_escape_string($conn, $data[3]);
$bio = mysqli_real_escape_string($conn, $data[4]);
$sst = mysqli_real_escape_string($conn, $data[5]);
echo "</br>inserting sid".$sid." name=".$name." physics=".$physics." maths=".$maths." chemistry=".$chemistry." bio=".$bio." sst=".$sst."</br>";
//$query = "insert into tblMarks (sid, name, physics, maths, chemistry, bio, sst) values ('$sid', '$name', '$physics', '$maths','$chemistry','$bio','$sst') on duplicate key update name = '$name', physics = '$physics',maths = '$maths', chemistry = '$chemistry', bio = '$bio', sst ='$sst'";
//mysqli_query($conn, $query);
$sid = $sid + 1;
}
fclose($handle);
}
else{
$message = '<label class="text-danger">Please Select CSV File Only</lable>';
}
}
else{
$message = '<label class="text-danger">Please Select File</label>';
}
}
The output was this:
OUTPUT
here is the correct method to read CSV file row by row. There are many rows in the CSV files which have blank values, to remove theme array_filter has been used.
$temp = array();
if (($h = fopen("grid.csv", "r")) !== FALSE)
{
// Convert each line into the local $data variable
while (($data = fgetcsv($h, 1000, ",")) !== FALSE)
{
$data = array_filter($data);
if(count($data) > 0){
$temp[] = $data;
}
}
fclose($h);
}
//Write csv file
$fp = fopen('grid1.csv', 'w');
foreach ($temp as $fields) {
fputcsv($fp, $fields);
}
fclose($fp);
I am curently trying to upload a CSV file into a sql table, but I don't want to upload the first line which contains the name of every column value that will be inserted. I used a variable c to count the number of rows that will be inserted. For the first line c=0 so I used a condition to upload the values from the row only if c!==0, but it stil uploades the values from the first line.
My code looks like this:
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
$c = 0;
if ($_FILES["file"]["type"]=='application/vnd.ms-excel')
{
while(($filesop = fgetcsv($handle, 3000, ",")) !== false)
{
$tid = trim($filesop[0]);
$beneficiar = ucwords(strtolower(trim($filesop[1])));
$locatie = ucwords(strtolower(trim($filesop[2])));
$localitate = ucwords(strtolower(trim($filesop[3])));
$judet = ucwords(strtolower(trim($filesop[4])));
$adresa = ucwords(strtolower(trim($filesop[5])));
$model = trim($filesop[6]);
$qry = mysql_query("SELECT id FROM beneficiari WHERE `nume` = '".$beneficiar."'");
while ($row = mysql_fetch_assoc($qry)){
$id_client=$row['id'];
}
$qry_id_model=mysqli_query("SELECT id FROM modele WHERE `model` = '".$model."'");
while ($row = mysql_fetch_assoc($qry_id_model)){
$id_model=$row['id'];
}
$adresa1 = $adresa.",".$localitate;
if ($c!==0){
$sql = mysql_query(
"INSERT INTO pos_equipments_other
(id_client, model, tid, beneficiar, adresa, agentie, judet)
VALUES
('$id_client','$id_model','$tid','$beneficiar','$adresa1',
'$locatie','$judet')"
);
}
$c = $c + 1;
}
}
I need to remove duplicate records when importing my current CSV files into the database. My files has been successfully updated, but when I tried to upload the same files again, it straight away inserted into it again. is there any way of I can remove duplicate records if i am importing the files?
<?
if(isset($_POST["submit"])){
$file = $_FILES['file']['tmp_name'];
//echo 'upload file name: '.$file.' ';
$handle = fopen($file, "r");
$c = 0;
$count =0;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$count ++;
$ID = $filesop[0];
$Name = $filesop[1];
$Contact = $filesop[2];
$Email =$filesop[3];
if($count>1){
$sql = "INSERT INTO clients(id,name,contact,email,)VALUES($ID,'$Name',$Contact,'$Email',')";
$resultsql = mysqli_query($link, $sql);
//echo $resultsql; //how am i going to remove duplicates when if there is a duplicate record ?
1) Before inserting, check existing data.
<?php
if(isset($_POST["submit"])){
$file = $_FILES['file']['tmp_name'];
//echo 'upload file name: '.$file.' ';
$handle = fopen($file, "r");
$c = 0;
$count =0;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$count ++;
$ID = $filesop[0];
$Name = $filesop[1];
$Contact = $filesop[2];
$Email =$filesop[3];
$checkExistingData = "SELECT * FROM clients WHERE name='$Name' AND contact='$Contact' AND email='$Email'";
$resultcheckExistingData = mysqli_query($link, $checkExistingData);
$countExistingData = mysqli_num_rows($resultcheckExistingData);
if($countExistingData == 0)
{
if($count>1) {
$sql = "INSERT INTO clients(id,name,contact,email,)VALUES($ID,'$Name',$Contact,'$Email',')";
$resultsql = mysqli_query($link, $sql);
//echo $resultsql; //how am i going to remove duplicates when if there is a duplicate record ?
.
.
}
.
.
}?>
2) If data got inserted and you want to delete duplicate rows from table. You can try this too.
DELETE c1 FROM clients c1, clients c2 WHERE c1.name = c2.name AND c1.contact = c2.contact AND c1.email = c2.email AND c1.id > c2.id;
Add unique key to one or more of your columns, column with unique key will store unique values only, duplicate values will not be inserted.