problem is in my excel 369 rows are there. when I echo/print that data it showing correct but when I am inserting same data in DB table in inserted only 18 - 30 records.
if (isset($_POST['Submit'])) {
$file = $_FILES['csv_file']['tmp_name'];
$handle = fopen($file, "r");
if ($file == NULL) {
error(_('Please select a file to import'));
redirect(page_link_to('excel_data_upload'));
}else {
$conn = connect();
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$num3 = $filesop[3];
$num8 = $filesop[8];
$num9 = $filesop[9];
$num20 = $filesop[20];
if($num3!='ExpiryDate' &&$num8!='VCNO' &&$num20!='TotalB2CAmount' && $num9 !='STBNO'){
$insertAgent = mysqli_query($conn, "INSERT INTO `upload_billing_data`
(`vc_number`,`stb_number`,`operator_id`,`expiry_date`,`monthly_bill_amount`)
VALUES ('$num8','$num9',140,'$num3','$num20')");
if($insertAgent)
{
echo 'succss';
}else{
echo 'error';
}
}
}
close($conn);
}
}
I am fetching from the excel data. I want to insert all records
Change the code as below and you might get to save all data using one query to the database:
$query_insert = array();
while(($filesop = fgetcsv($handle, 1000, ",")) !== false) {
$num3 = filterString($filesop[3]);
$num8 = filterString($filesop[8]);
$num9 = filterString($filesop[9]);
$num20 = filterString($filesop[20]);
if ($num3!='ExpiryDate' &&$num8!='VCNO' &&$num20!='TotalB2CAmount' && $num9 !='STBNO') {
$query_insert[] = "('{$num8}', '{$num9}', 140, '{$num3}', '{$num20}')";
}
}
// If no row matched your if, then there will be no row to add to the database
if (count($query_insert)>0) {
$conn = connect();
$query_insert_string = implode(', ', $query_insert);
$query = "INSERT INTO `upload_billing_data` (`vc_number`, `stb_number`, `operator_id`, `expiry_date`, `monthly_bill_amount`) VALUES {$query_insert_string};";
$insertAgent = mysqli_query($query);
// The rest of you code
...
close($conn);
}
// This function makes sure that you string doesn't contain characters that might damage the query
function filterString($string) {
$string = str_replace(array("\'", '"'), array('', ''), $string);
$string = filter_var($string, FILTER_SANITIZE_STRING);
return $string;
}
Please check this modified code
if (isset($_POST['Submit'])) {
$file = $_FILES['csv_file']['tmp_name'];
$handle = fopen($file, "r");
if ($file == NULL) {
error(_('Please select a file to import'));
redirect(page_link_to('excel_data_upload'));
}else {
$conn = connect();
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$num3 = $filesop[3];
$num8 = $filesop[8];
$num9 = $filesop[9];
$num20 = $filesop[20];
if($num3!='ExpiryDate' &&$num8!='VCNO' &&$num20!='TotalB2CAmount' && $num9 !='STBNO'){
$insertAgent = mysqli_query($conn, "INSERT INTO `upload_billing_data`
(`vc_number`,`stb_number`,`operator_id`,`expiry_date`,`monthly_bill_amount`)
VALUES ('".mysqli_real_escape_string($num8)."','".mysqli_real_escape_string($num9)."',140,'".mysqli_real_escape_string($num3)."','".mysqli_real_escape_string($num20)."')");
if($insertAgent)
{
echo 'succss';
}else{
echo 'error';
}
}
}
close($conn);
}
}
BY using mysqli_real_escape_string() you will be able to avoid sqlinjection issues and you will be able to handle issue of quotes which might be causing an issue.
in your else block where you are echo "error". you can use mysqli_error($conn); to get exact what error is occurring while performing an insert
Related
I am trying to insert data from csv to MYSQL database like below and its working fine
$file = fopen('../assets/uploads/'.$file_name, "r");
while (($column = fgetcsv($file, 10000, ",")) !== FALSE) {
$fname = "";
if (isset($column[0])) {
$fname = mysqli_real_escape_string($mysqli, $column[0]);
}
$lname = "";
if (isset($column[1])) {
$lname = mysqli_real_escape_string($mysqli, $column[1]);
}
$email = "";
if (isset($column[2])) {
$email = mysqli_real_escape_string($mysqli, $column[2]);
}
$sqlInsert = "INSERT into $lead_data_table (lfname,llname,lemail,lead_id,lead_user_id) VALUES (?,?,?,?,?)";
$stmt = $mysqli->prepare($sqlInsert);
$stmt->bind_param('sssii', $fname, $lname,$email,$lead_insert_id,$lead_user_id);
$stmt->execute();
$stmt->close();
$insertId = mysqli_insert_id($mysqli);
However For some reason I want insert data from CSV like 0 to 100 or 100 to Remain All. I am not getting idea how I can do it? Let me know if anyone here can help me for do the same.
Thanks!
Count the csv lines and skip. Something like this will process starting with the hundredth line.
$file = fopen('../assets/uploads/'.$file_name, "r");
$lineCount = 0;
while (($column = fgetcsv($file, 10000, ",")) !== FALSE) {
$lineCount ++;
if ($lineCount >= 100) {
/* process the line */
}
}
And by the way, you're doing your prepared statements correctly. So you don't need to use mysqli_real_escape_string() to mung your data before inserting it.
I searched everywhere in the internet but I couldn't find a perfect solution for my question, regarding on how to validate the CSV Contents before it uploads and save it to database, I am using msqli for the database and PHP for the language script. For example I have 3 columns in localhost database:
These are my following headers I made both in locahost database and in CSV File (1.)Date (2.)Email (3.)Phone Number.
Before uploading and saving to the localhost database it should meet the restrictions of the contents are the ff:
For the Date: It should mm/dd/yy ----> 1/31/2018 or 12/31/2018
For the Email: It should name#domain.com ----> saffron#gmail.com
For the Number: It should 12 digits number only and the format is ----> 0906-021-0156
If those restrictions meets perfectly the CSV File will be uploaded and save to the database, if not it will throw an error or pop-up message.
I really don't know how to start the execution of program. I am really new to PHP so please help me with this.
This is the code I worked, and I am stack here...
<?php
$dbHost = 'localhost';
$dbUsername = '';
$dbPassword = 'root';
$dbName = 'dbUpload';
$db = new mysqli($dbHost, $dbUsername, $dbPassword, $dbName);
if ($db->connect_error) {
die("Unable to connect database: " . $db->connect_error);
}
if(isset($_POST['submit'])){
$row = 1;
$mycsvfile = array(); //define the main array.
if ($_FILES['csv']['size'] > 0) {
//get the csv file
$file = $_FILES['csv']['tmp_name'];
$handle = fopen($file, "r");
$i = 0;
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
$row++;
$mycsvfile[] = $data; //add the row to the main array.
$headerDate = $mycsvfile[0][0];
$headerEmail = $mycsvfile[0][1];
$headerPhone = $mycsvfile[0][2];
if ($headerDate !== 'Date' || $headerEmail !== 'Email' || $headerPhone !== 'Phone Number') {
$qstring = '?status=invalid_header';
fclose($handle);
}
else {
if ($i > 0) {
$import = "INSERT into upload (techDate, techEmail, techPhone)values('$data[0]','$data[1]','$data[2]')";
$db->query($import);
$qstring = '?status=succ';
}
}
$i++;
}
fclose($handle);
}
else{
$qstring = '?status=err';
}
}
header("Location: uploadvalid.php".$qstring);
?>
You can use the function preg_match().
I took the regex (for date and email) from StackOverflow. Links are below this code. I made the regex for your phone format.
<?php
$dbHost = 'localhost';
$dbUsername = '';
$dbPassword = 'root';
$dbName = 'dbUpload';
$db = new mysqli($dbHost, $dbUsername, $dbPassword, $dbName);
if ($db->connect_error) {
die("Unable to connect database: " . $db->connect_error);
}
if(isset($_POST['submit'])){
$row = 1;
$mycsvfile = array(); //define the main array.
if ($_FILES['csv']['size'] > 0) {
//get the csv file
$file = $_FILES['csv']['tmp_name'];
$handle = fopen($file, "r");
$i = 0;
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
$row++;
$mycsvfile[] = $data; //add the row to the main array.
$headerDate = $mycsvfile[0][0];
$headerEmail = $mycsvfile[0][1];
$headerPhone = $mycsvfile[0][2];
if ($headerDate !== 'Date' || $headerEmail !== 'Email' || $headerPhone !== 'Phone Number') {
$qstring = '?status=invalid_header';
fclose($handle);
}
else {
if ($i > 0) {
$patternDate = '^(?:(?:31(\/|-|\.)(?:0?[13578]|1[02]))\1|(?:(?:29|30)(\/|-|\.)(?:0?[1,3-9]|1[0-2])\2))(?:(?:1[6-9]|[2-9]\d)?\d{2})$|^(?:29(\/|-|\.)0?2\3(?:(?:(?:1[6-9]|[2-9]\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\d|2[0-8])(\/|-|\.)(?:(?:0?[1-9])|(?:1[0-2]))\4(?:(?:1[6-9]|[2-9]\d)?\d{2})$';
$patternEmail = '(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")#(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])';
$patternPhone = '0906\d021\d0156';
// Check if the row has the correct format
if (preg_match($data[0], $patternDate)
&& preg_match($data[1], $patternEmail)
&& preg_match($data[2], $patternPhone)) {
// Format is OK, let's insert
$import = "INSERT into upload (techDate, techEmail, techPhone)values('$data[0]','$data[1]','$data[2]')";
$db->query($import);
$qstring = '?status=succ';
} else {
// The row doesn't have the right format
echo "The row $row doesn't have the right format";
}
}
}
$i++;
}
fclose($handle);
}
else{
$qstring = '?status=err';
}
}
header("Location: uploadvalid.php".$qstring);
?>
link to the StackOverflow date regex
link to the StackOverflow email regex
In the following code i try to import from CSV to Database in WordPress.
The data is inserted but I get this error
WordPress database error: []
INSERT INTO wp_trav_accommodation_vacancies (date_from, date_to, accommodation_id, room_type_id, rooms, price_per_room, price_per_person) VALUES ('2017-08-30', '2017-09-10', '1', '1', '3', '600.00', '0.00')
This is the code I run:
global $wpdb;
if(isset($_POST["submit"]))
{
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
$c = 0;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false) {
$date_from = $filesop[0];
$date_to = $filesop[1];
$accommodation_id = $filesop[2];
$room_type_id = $filesop[3];
$rooms = $filesop[4];
$price_per_room = $filesop[5];
$price_per_person = $filesop[6];
$sql = "INSERT INTO wp_trav_accommodation_vacancies (date_from, date_to, accommodation_id, room_type_id, rooms, price_per_room, price_per_person) VALUES ('$date_from', '$date_to', '$accommodation_id', '$room_type_id', '$rooms', '$price_per_room', '$price_per_person')";
if ($wpdb->query($sql) === TRUE) {
echo "You database has imported successfully";
} else {
echo $wpdb->print_error();
my_print_error();
}
}
}
Can anyone tell what is wrong?
I have a CSV file with a column having a date in format d/m/Y or the word "Illimité" inside meaning unlimited in French.
I would like each time it reads "Illimité" it puts NULL value inside MySQL table.
Here is my current PHP code :
if (isset($_POST['submit'])) {
$query = "TRUNCATE TABLE `formation` ";
$result = mysql_query($query);
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
while (($fileop = fgetcsv($handle, 1000, ";")) !== false) {
$nom = $fileop[0];
$prenom = $fileop[1];
$formation = $fileop[16];
$validite = $fileop[26];
if (is_numeric($validite)) {
$validite = date_format(date_create_from_format('d/m/Y', $validite), 'Y-m-d');
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation','$validite','importCSV')");
} else {
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation',NULL,'importCSV')");
}
}
Sadly this isn't working. MySql shows no errors and it puts NULL all the time.
Any help would me much appreciate.
Try this:
function myFunc($CSVValue){
if(validateDate($CSVValue)){
//your mysql logic where you put in the date
}else{
//your mysql logic where you put in the null
}
}
function validateDate($date)
{
$d = DateTime::createFromFormat('d/m/Y', $date);
return $d && $d->format('d/m/Y') == $date;
}
function was copied from this answer or php.net
-- update --
I don't know how your code looks other than what you have provided.
If you were to put this in your code it could look something like this:
if (isset($_POST['submit'])) {
$query = "TRUNCATE TABLE `formation` ";
$result = mysql_query($query);
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
while (($fileop = fgetcsv($handle, 1000, ";")) !== false) {
$nom = $fileop[0];
$prenom = $fileop[1];
$formation = $fileop[16];
$validite = $fileop[26];
$d = DateTime::createFromFormat('d/m/Y', $validite);
if ($d && $d->format('d/m/Y') == $validite) {
$validite = date_format(date_create_from_format('d/m/Y', $validite), 'Y-m-d');
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation','$validite','importCSV')");
} else {
$sql = mysql_query("INSERT INTO formation (nom,prenom,formation,validite,usermaj) VALUES ('$nom','$prenom','$formation',NULL,'importCSV')");
}
}
I have created a data-structure in Mysql with table name(CSV filename) and field names(CSV column names).
Right now I am importing the data from csv to Mysql table successfully Where as I am hard-coding csv file name and field name in script. How to dynamical fetch bec I have manny csv files to import into mysql.
<?php
include "db.php";
$filename = "C:\REQ\Status.csv";
if (($handle = fopen($filename, 'r')) !== FALSE)
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
print_r($data);
$import="INSERT into status(status) values('$data[1]')";
mysql_query($import) or die(mysql_error());
}
fclose($handle);
?>
I have implement this code and it is tested code. I think it is very use full
You have follow some rule:-
1.your csv file according to database table name (ex: db table name is users then csv should be users.csv)
2.Your csv file's first row should be db table fields name (ex: Id, name etc) after the start your data entry
3.you can download data source class from :- http://code.google.com/p/php-csv-parser/
because i have require below the code: require_once 'CSV/DataSource.php';
<?php
ini_set('memory_limit','512M');
$dbhost = "localhost";
$dbname = "excel_import";
$dbuser = "root";
$dbpass = "";
$conn=mysql_connect ($dbhost, $dbuser, $dbpass) or die ("I cannot connect to the database because: " . mysql_error());
mysql_select_db($dbname) or die("Unable to select database because: " . mysql_error());
require_once 'CSV/DataSource.php';
$filename = "users.csv";
$ext = explode(".",$filename);
$path = "uploads/".$filename;
$dbtable = $ext[0];
import_csv($dbtable, $path);
function import_csv($dbtable, $csv_file_name_with_path)
{
$csv = new File_CSV_DataSource;
$csv->load($csv_file_name_with_path);
$csvData = $csv->connect();
$res='';
foreach($csvData as $key)
{
$myKey ='';
$myVal='';
foreach($key as $k=>$v)
{
$myKey .=$k.',';
$myVal .="'".$v."',";
}
$myKey = substr($myKey, 0, -1);
$myVal = substr($myVal, 0, -1);
$query="insert into ".$dbtable." ($myKey)values($myVal)";
$res= mysql_query($query);
}
if($res ==1)
{
echo "record successfully Import.";
}else{
echo "record not successfully Import.";
}
}
Something like this:
function integrate($filename)
{
if (($handle = fopen($filename, 'r')) !== FALSE)
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
print_r($data);
$import="INSERT into status(status) values('$data[1]')";
mysql_query($import) or die(mysql_error());
}
fclose($handle);
}
$files = scandir('folder/');
foreach($files as $file) {
//pre-checking (if already integrated, if CSV etc..)
integrate($file);
}
If you run it as a PHP script, you could get the filename as an argument with $argv[1] You would need to do some checking that the file exists, though.
If the file name and table name match, then wherever you're entering or retrieving the file name from, you can use that same value in the query:
$filename = "Status";
$table_name = strtolower( $filename );
$filename = "C:\REQ\{$filename}.csv";
if (($handle = fopen($filename, 'r')) !== FALSE)
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
print_r($data);
$import="INSERT into {$table_name}(status) values('$data[1]')";
For the column names, I'd recommend putting them in the first row of your CSV files and parsing them from there, e.g.:
if (($handle = fopen($filename, 'r')) !== FALSE)
$columns = fgets($handle, 1000);
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
print_r($data);
$import="INSERT into {$table_name}({$columns}) values('{$data[1]}')";
I'd also recommend considering building up a single SQL query string for a multiple-row INSERT. E.g.
$values = array();
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
print_r($data);
$values[] = "('{$data[1]}')";
...
}
$values = join( ",\n\n", $values );
$insert = "INSERT into {$table_name}({$columns}) values {$values}";