I tried to upload text file to insert into mysql. It works fine with a small number of rows but when rows more than 1000 it's hanging.
My probem is:
code is very slow and hanging with large text file
process bar not working
how it works with text and cvs, xls, xlsx files
The code:
//Import uploaded file to Database
$handle = fopen($_FILES['filename']['tmp_name'], "r");
$t = "0";
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
$mobile=$data[0];
$mobile = clean_value(mobile_filter($mobile));
if (preg_match('/^[0-9]{12,12}$/i',$mobile)) {
$check_mob = mysql_query("SELECT * FROM book WHERE mobile = '$mobile' AND owner = '$cp_username' AND groupname = '$groupname'");
if(mysql_num_rows($check_mob) == 0) {
$import="INSERT into book(nickname, mobile, groupname, owner) values('$nickname','$mobile','$groupname','$cp_username')";
mysql_query($import) or die(mysql_error());
$t = $t+1;
echo "<div class=\"progress\">
<div class=\"progress-bar\" role=\"progressbar\" aria-valuenow=\"$t\" aria-valuemin=\"0\" aria-valuemax=\"100\" style=\"width:$t%\">
$t%
</div>
</div>";
}
}
}
You actually can do this with using only PHP too.
This is a example code.
$sql="INSERT INTO .....";
$count++;
if($count==500){
// wait for 5 seconds
usleep(5000000);
$count=0;
}
You will need to put this code in the loop that you save line by line into database. When the system saves first 500 record, it will sleep for 5 seconds and continue.
Related
I want to import the excel sheet data into mysql table with php but i am getting these errors someone please get me out from this.. please look on php code only ignore html stuff.
<?php
include ("connection.php");
if(isset($_POST["submit"]))
{
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
$c = 0;
while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$name = $filesop[1];
$email = $filesop[2];
$sql = mysql_query("INSERT INTO co (name, email) VALUES ('$name','$email')");
$c = $c + 1;
}
fcose($file);
if($sql){
echo "You database has imported successfully. You have inserted ". $c ." recoreds";
}else{
echo "Sorry! There is some problem.";
}
}
?>
this is excel sheet
in database it is showing different format
There are libraries excellibrary/php-excel-reader/excel_reader2.php and excellibrary/SpreadsheetReader.php
you can use these libraries to read your content and convert it into array.
Once you convert your spreadsheet data into an array, you can easily insert into database using iterations.
here is my code to import csv data to my database
if (isset($_POST["submit"])) {
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
$c = 0;
**while(($filesop = fgetcsv($handle, 1000, ",")) !== false)
{
$name = $filesop[0];
$email = $filesop[1];
$sql = mysql_query("INSERT INTO selleruser (emaili) VALUES ('$name')");
$c = $c + 1;
}**
if ($sql) {
echo "You database has imported successfully. You have inserted ". $c ." recoreds";
} else {
echo "Sorry! There is some problem.";
}
}?>
</div>
I have a csv file where there is a column which contains emails
the import is done suvcessfull but the issue is that it just imports the value in another format like in other language or encripted or something un readable
Try this query to import csv from phpmyadmin or any other mysql client
Before firing query, please make sure that table matching the fields is already created into db.
LOAD DATA LOCAL INFILE 'local machine path to csv file' INTO TABLE `selleruser`
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(emaili);
I have created a website which requires me to upload a csv file into the mysql database (Wamp server). Since I have never done this before a detailed answer with steps will be really helpful. I need the user to upload file using html input file option and then a php code to upload this file to mysql database. Iam using this code
<?php
$con=mysql_connect("localhost","","");
mysql_select_db("sg",$con);
define('CSV_PATH','C:/Users/mkutbudd/Desktop/');
$csv_file = CSV_PATH . "dum.csv";
if (($getfile = fopen($csv_file, "r")) !== FALSE) {
$data = fgetcsv($getfile, 1000, ",");
while (($data = fgetcsv($getfile, 1000, ",")) !== FALSE) {
$num = count($data);
for ($c=0; $c < $num; $c++) {
$result = $data;
$str = implode(",", $result);
$slice = explode(",", $str);
$col1 = $slice[0];
$col2 = $slice[1];
$col3 = $slice[2];
$query = "INSERT INTO dummy(dum1,dum2,dum3)
VALUES('".$col1."','".$col2."','".$col3."')";
$s=mysql_query($query, $con );
}
}
}
echo "File data successfully imported to database!!";
mysql_close($con);
?>
Your first step is to allow user to upload a file, and then you need to handle the errors if there are, validate the file, and if everything is correct, then move the uploaded file to its place: http://php.net/manual/en/features.file-upload.php
Then you need to parse your file, with a php function for it: http://hu1.php.net/manual/en/function.fgetcsv.php
After this, you need to create a connection to the database: http://hu1.php.net/manual/en/book.mysqli.php
And then you need to insert your data into your table with a loop, like foreach on your array, what you created from fgetcsv: http://hu1.php.net/manual/en/control-structures.foreach.php
There are an alternative way, if you have right to run system or execute functions and you have right to run mysql command, then you can use the mysql load data command: http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Done.
i know how to upload a csv file into mysql database but only through cmd. i want to know how to upload csv file into mysql database using php form and will disregard some information on the excel and will only start importing starting from a certain line. ? kindly help me.
(PHP 4, PHP 5)
fgetcsv
See php manual http://php.net/manual/en/function.fgetcsv.php
<?php
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
?>
Try this it's working well, you can add as many values as possible depending on the number of columns you have in the CSV file. Then in the HTML code put the uploading syntax in the tag.
**
$fname = $_FILES['csv_file']['name'];
$chk_ext = explode(".",$fname);
$filename = $_FILES['csv_file']['tmp_ name'];
$handle = fopen($filename, "r");
if(!$handle){
die ('Cannot open file for reading');
}
while (($data = fgetcsv($handle, 10000, ",")) !== FALSE)
{
$query = "INSERT INTO tablename (col1_csv, col2_csv)
values ('$data[0]', '$data[1]');
mysql_query($query) or die(mysql_error ());
}
fclose($handle);
?>
**
You can use the MySQL LOAD DATA INFILE statement to bulk-insert thousands of records at once. PHP can handle the file upload. The PHP code would be something similar to:
$query = sprintf("
LOAD DATA LOCAL INFILE '%s'
INTO TABLE `table1`
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\\r\\n'
IGNORE 1 LINES
",
mysql_real_escape_string($FILES["file1"]["tmp_name"])
);
The LOCAL keyword should allow you to workaround some security restrictions. Change the FIELDS TERMINATED BY and LINES TERMINATED BY parameter to match the separators used by excel while exporting. IGNORE 1 LINES tells MySQL to skip the header row(s).
Note: Excel does not seem to use an escape character; but it will (i) enclose the fields that contain , and " with " (ii) use "" to escape a single " inside data. I believe MySQL will understand this encoding and import the data correctly.
You could use the "LOAD DATA INFILE " statement with the " IGNORE ... LINES " option which you can use from the command line as well as from PHP.
try this:
$filename=$_FILES["upload_file"]["name"];
$extension = end(explode(".",$filename));
if ($extension=='csv') {
$tmp_file=$_FILES["upload_file"]["tmp_name"];
$handle = #fopen($tmp_file, "r");
//specify your own database connection parameter
$db = new PDO('mysql:host=localhost;dbname=demo','user','password');
$stmt = $db->prepare("INSERT INTO writers (writer_name, writer_email) VALUES (?, ?)");
if ($handle) {
while (($buffer = fgets($handle, 4096)) !== false) {
$array=explode(",",$buffer);
$count=1;
foreach ($array as $value) {
$stmt->bindParam($count, $value);
$count++;
}
$stmt->execute();
}
if (!feof($handle)) {
echo "Error: unexpected fgets() fail\n";
}
fclose($handle);
}
$db = null;
echo "<p>Success</p>";
}
else {
$error="<p style='color:red;'>Invalid file type</p>";
}
Refer to http://pradipchitrakar.com.np/programming/upload-csv-mysql-php/
I am using the following script to import data into my mysql database from CSV files. The CSV is setup like this :
Name Postcode
fred hd435hg
bob dh345fj
Above is what it looks like in excel, in raw csv format viewed in notepad it looks like this :
name,postcode
frank,ng435tj
The problem I am having is for some reason the postcode column isnt getting imported at all, also the header row is getting imported as a record too, is it possible to make it skip the first row ?. I have been through the code and cant see why the postcode is not being pulled in, it is very odd.
<?php
//database connect info here
//check for file upload
if(isset($_FILES['csv_file']) && is_uploaded_file($_FILES['csv_file']['tmp_name'])){
//upload directory
$upload_dir = "./csv";
//create file name
$file_path = $upload_dir . $_FILES['csv_file']['name'];
//move uploaded file to upload dir
if (!move_uploaded_file($_FILES['csv_file']['tmp_name'], $file_path)) {
//error moving upload file
echo "Error moving file upload";
}
//open the csv file for reading
$handle = fopen($file_path, 'r');
while (($data = fgetcsv($handle, 1000, ',')) !== FALSE) {
//Access field data in $data array ex.
$name = $data[0];
$postcode = $data[1];
//Use data to insert into db
$sql = sprintf("INSERT INTO test (name, postcode) VALUES ('%s',%d)",
mysql_real_escape_string($name),
$postcode
);
mysql_query($sql) or (mysql_query("ROLLBACK") and die(mysql_error() . " - $sql"));
}
//delete csv file
unlink($file_path);
}
?>
Your CSV file seems to be a TSV file actually. It doesn't use commas, but tabulators for separating the fields.
Therefore you need to change the fgetcsv call. Instead of ',' use the tab:
while (($data = fgetcsv($handle, 1000, "\t") ...
And to also skip the header row, add another faux fgetcsv before the while block:
fgetcsv($handle);
while (...) {
That will skip the first line. (A simple fgets would also do.)
Oh, just noticed: The postcode might also get dropped because you concat it into the string as decimal with the sprintf placeholder %d for $postcode. Should that field actually contain lettery, like in your example, then that wouldn't work. -- Though I presume that's just a wrong example printout.
Try this one.
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file,"r");
while(($fileop = fgetcsv($handle,1000,",")) !==false)
{
$username = $fileop[0];
$name = $fileop[1];
$address = $fileop[2];
$sql = mysql_query("INSERT INTO ...... ");
}