incomplete insertion from csv to mysql database php - php

Question

I am facing an error while inserting data records from CSV file to mysql database. I have a script which works fine. It inserts data from CSV file to mysql database. It has around 40,000 records. But after running the script, it inserts only around 1000 records and stops without error. What may be the problem? Please help.
Below is my script for inserting data.
include('dbconnection.php');
if (($handle = fopen("hotels.csv", "r")) !== FALSE) {
$flag = true;
$id=1;
while (($data = fgetcsv($handle, 100000000, ",")) !== FALSE) {
if($flag){
$flag = false;
continue;
}
$hotelid = mysql_real_escape_string($data[0]); //var_dump($hotelid); exit();
$hotelname = mysql_real_escape_string($data[1]);
$address1 = mysql_real_escape_string($data[2]);
$address2 = mysql_real_escape_string($data[3]);
$country = mysql_real_escape_string($data[4]);
$city = mysql_real_escape_string($data[5]);
$postcode = mysql_real_escape_string($data[6]);
$telephone = mysql_real_escape_string($data[7]);
$hotelfax = mysql_real_escape_string($data[8]);
$hotelemail = mysql_real_escape_string($data[9]);
$longitude = mysql_real_escape_string($data[10]);
$latitude = mysql_real_escape_string($data[11]);
$website = mysql_real_escape_string($data[12]);
$location = mysql_real_escape_string($data[13]);
$description = mysql_real_escape_string($data[14]);
$starrating = mysql_real_escape_string($data[15]);
$statecode = mysql_real_escape_string($data[16]);
$select_query = "SELECT * FROM `hotelDB`.`hotels` WHERE `hotelid` = $hotelid";
if(mysql_query($select_query) != '$hotelid'){
$sql = "INSERT IGNORE INTO `hotelDB`.`hotels` (`id`,`hotelid`, `hotelname`, `address1`,`address2`,`country`,`city`,`postcode`,`telephone`,`hotelfax`,`hotelemail`,`longitude`,`latitude`,`website`,`location`,`description`,`starrating`,`statecode`) VALUES ('$id','$hotelid', '$hotelname', '$address1','$address2', '$country', '$city', '$postcode', '$telephone', '$hotelfax', '$hotelemail', '$longitude', '$latitude', '$website', '$location', '$description', '$starrating', '$statecode')";
echo $sql;
$retval = mysql_query($sql,$conn);
if(! $retval )
{
die('Could not enter data: ' . mysql_error());
}
echo "<p style='color: green;'>Entered data having id = " .$id. " successfully</p><br>";
$id++;
}
}
echo "<br><p style='color: orange;'>Congratulation all data successfully inserted</p>";
fclose($handle);
}
mysql_close($conn);`
The insertion stops after inserting some values. This is example screenshot where the code stops executes.

Answer 1

You can try one query with multiple-row INSERT. In below example I am inserting 500 row with one insert query. You can change it to 1000.
<?php
include('dbconnection.php');
if (($handle = fopen("grnconnect-dump-hotels.csv", "r")) !== FALSE) {
//var_dump( $handle);
$flag = true;
$id=1;
$counter=0;
$sql = "INSERT IGNORE INTO `hotelDB`.`hotels` (`id`,`hotelid`, `hotelname`, `address1`,`address2`,`country`,`city`,`postcode`,`telephone`,`hotelfax`,`hotelemail`,`longitude`,`latitude`,`website`,`location`,`description`,`starrating`,`statecode`) VALUES";
$saveString="";
while (($data = fgetcsv($handle, 100000000, ",")) !== FALSE) {
//var_dump($data); exit();
if($flag){
$flag = false;
continue;
}
$hotelid = mysql_real_escape_string($data[0]); //var_dump($hotelid); exit();
$hotelname = mysql_real_escape_string($data[1]);
$address1 = mysql_real_escape_string($data[2]);
$address2 = mysql_real_escape_string($data[3]);
$country = mysql_real_escape_string($data[4]);
$city = mysql_real_escape_string($data[5]);
$postcode = mysql_real_escape_string($data[6]);
$telephone = mysql_real_escape_string($data[7]);
$hotelfax = mysql_real_escape_string($data[8]);
$hotelemail = mysql_real_escape_string($data[9]);
$longitude = mysql_real_escape_string($data[10]);
$latitude = mysql_real_escape_string($data[11]);
$website = mysql_real_escape_string($data[12]);
$location = mysql_real_escape_string($data[13]);
$description = mysql_real_escape_string($data[14]);
$starrating = mysql_real_escape_string($data[15]);
$statecode = mysql_real_escape_string($data[16]);
if(trim($counter)=="500") {
$sql = $sql.''.rtrim($saveString,",");
echo $sql;echo "<br /";
mysql_query($sql);
$counter=0;
$saveString="";
}
$saveString .="('".$id."','".$hotelid."', '".$hotelname."', '".$address1."','".$address2."', '".$country."', '".$city."', '".$postcode."', '".$telephone."', '".$hotelfax."', '".$hotelemail."', '".$longitude."', '".$latitude."', '".$website."', '".$location."', '".$description."', '".$starrating."', '".$statecode."'),";
$counter++;
}
if(trim($saveString)!=='') {
$sql = $sql.''.rtrim($saveString,",");
echo $sql;echo "<br /";
mysql_query($sql);
}
echo "<br><p style='color: orange;'>Congratulation all data successfully inserted</p>";
fclose($handle);
}
mysql_close($conn);
?>

Answer 2

Is there a particular reason to insert via script ?
Else I'd suggest you just insert manually or let the DB do it, sinces it's more efficient anyway. MySQL provides functions
for this: http://www.mysqltutorial.org/import-csv-file-mysql-table/