Writing a CSV File to SQL - php

Hope someone can help me with what I think will be something minor (I'm still learning...). I'm trying to write the entire contents of a CSV File server based to an SQL database here is the code I presently have. The line // out writes perfectly and generates a new record. The $ar0 values generate no entries into the table named order - even though the csv file is about 100 lines long I just get
Error: INSERT INTO order (Picker,Order_Number,Timestamp,System)values ('','','','')
$file = "Pal.ORD.csv";
$tbl = "order";
$f_pointer=fopen("$file","r"); // file pointer
while(! feof($f_pointer)){
$ar=fgetcsv($f_pointer);
//$sql="INSERT INTO `order` (Picker,Order_Number,Timestamp,System)values ('Me','9999','23-01-2015','ORD')";
$sql="INSERT INTO `order` (Picker,Order_Number,Timestamp,System)values ('$ar[0]','$ar[1]','$ar[2]','$ar[3]')";
echo $sql;
echo "<br>";
}
if ($connect->query($sql) === TRUE) {
echo "New records created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}

What I think may be going on is that your file probably has an empty line/carriage return as the last line in the file and is using that to insert the data as blank entries.
I can't be 100% sure about this since you have not provided a sample of your CSV file, however that is what my tests revealed.
Based on the following CSV test model: (Sidenote: blank lines will be ignored)
a1,a2,a3,a4
b1,b2,b3,b4
c1,c2,c3,c4
Use the following and replace with your own credentials.
This will create a new entry/row for each line found in a given file based on the model I have provide above.
<?php
$DB_HOST = 'xxx';
$DB_USER = 'xxx';
$DB_PASS = 'xxx';
$DB_NAME = 'xxx';
$db = new mysqli($DB_HOST, $DB_USER, $DB_PASS, $DB_NAME);
if($db->connect_errno > 0) {
die('Connection failed [' . $db->connect_error . ']');
}
$file = "Pal.ORD.csv";
$delimiter = ',';
if (($handle = fopen("$file", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, $delimiter)) !== FALSE) {
foreach($data as $i => $content) {
$data[$i] = $db->real_escape_string($content);
}
// echo $data[$i].""; // test only not required
$db->query("INSERT INTO `order`
(Picker, Order_Number, Timestamp, System)
VALUES ('" . implode("','", $data) . "');");
}
fclose($handle);
}
if($db){
echo "Success";
}
else {
echo "Error: " . $db->error;
}

At a quick glance it seems like this:
$f_pointer=fopen("$file","r"); // file pointer
Should be this:
$f_pointer=fopen($file,"r"); // file pointer
You might not be reading anything from the file. You can try outputting the file contents to see if that part is working, since you've confirmed that you can insert into the DB.

Related

How to save long text from textarea-input line per line

How to save long text from textarea-input line per line
i have a form with a text area, i wanna save a long text line per line in mysql
i have no idea
$handle = fopen("inputfile.txt", "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
// process the line read.
}
fclose($handle);
}
You need to learn how mysql interacts with the DB. You will likely need to use VARCHAR datatype depending on how big you text area is. So if the field from the from is up to 250 characters then the text area column's datatype would be VARCHAR(250).
You would do a POST request to a file with something like this:
$post = $_POST;
//set other fields here, I recommend sanitizing your inputs.
...
$textarea = $_POST['text_area'];
$servername = "HOST";
$username = "username";
$password = "password";
// Create connection
$conn = new mysqli($servername, $username, $password);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$sql = "INSERT INTO MyGuests (...other columns you have, textarea)
VALUES (..., $textarea)";
if ($conn->query($sql) === TRUE) {
echo "New record created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
Two links I highly suggested looking at:
How to filter inputs via php(used before sql execution)
PHP MySQL Insert Data
It would have been easier if I had a look at your form data. I'll assume my own form data to try and answer your question.form.php
<form action="processing.php" method="POST">
<textarea required name="records" class="form-control" rows="8" cols="4" placeholder="Enter records separated by new line"></textarea>
<button type="submit" name="addRecords" class="btn btn-warning">Add Records</button>
</form>
Then processing.php
if (isset($_POST['addRecords'])) {
$record = $_POST['records'];
//explode records based on new line \n
$records = explode("\n", $record);
foreach ($records as $new) {
$data = $new;
//Here you'll write your sql code to insert records in the database
}
}

TXT to SQL that automatically removes that line of txt once complete

I have a wordpress plugin that exports form entries to a txt file. So I need to write a php script to add them to a sql database as I want the submissions added to a database on a different domain (otherwise I’d just get the plugin to do it for me). I’m fine about how I get it to connect to the database, it’s just how I code it to interpret the data as the column names are always next to the field as shown.
{"Entry_ID":"235","Name":"matt","Email":"matt#gmail.com","Date":"03/10/2017"}{"Entry_ID":"236","Name":"matt","Email":"matt#btinternet.com","Date":"10/10/2017"}
Is there a way to get it to ignore the column name and only interpret the data within the “” after the : ?
Once these have been added to the sql database I would then need to get the lines removed from the txt
So far I have this but it isn't working...
$file= fopen('http://mpcreations.staging.wpengine.com/wp-content/themes/red-seal-resources/test.txt', 'r');
while (($data = fgetcsv($file)) !== FALSE) {
$object = json_encode($data[0]);
$servername = "";
$username = "";
$password = "";
$dbname = "";
// Create connection
$conn = mysqli_connect($servername, $username, $password, $dbname);
// Check connection
if (!$conn) {
die("Connection failed: " . mysqli_connect_error());
}
$query = "INSERT INTO 'wp_forms' LINES TERMINATED BY '\n';
if (mysqli_multi_query($conn, $query)) {
echo "New records created successfully";
} else {
echo "Error: " . $query . "<br>" . mysqli_error($conn);
}
mysqli_close($conn);
}
Any help would be greatly appreciated.
Thank you
Each line in the txt file has JSON data? Process the txt file, parse the data and INSERT it into the database table.
$file= fopen('file.txt', 'r');
while (($data = fgetcsv($file)) !== FALSE) {
$object = json_encode($data[0]);
// Prepare INSERT query here...
}

how to upload a csv file from any direcory to database

I am adding data from a file to my database. Currently the location of the files are limited to only those inside directory D:/. I want to be able to support adding files from multiple directories.
<?php
$servername = "localhost";
$username = "root";
$password = "root";
$dbname = "stdprt";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$filename = "d:/" . $_POST['fname'];
$handle = fopen($filename, "r");
while (($data = fgetcsv($handle)) !== FALSE) {
$num = count($data);
$row;
$sql = "INSERT into marks(regno,semister,subcode,subname,internals,externals,credits)values('$data[0]','$data[1]','$data[2]','$data[3]','$data[4]','$data[5]','$data[6]')";
//echo "INSERT into marks(regno,semister,subcode,subname,internals,externals,credits)values('$data[0]','$data[1]','$data[2]','$data[3]','$data[4]','$data[5]','$data[6]')";
if ($conn->query($sql) === TRUE) {
// echo "New record created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
echo "<br>";
}
?>
<h2>Uploaded Successfully....</h2>
back
If you are wanting to choose a file in another drive you can modify this line in your code and change the directory at the start.
$filename="d:/".$_POST['fname'];
So for example if you wanted to change the directory to drive F it would be like so:
$filename="f:/".$_POST['fname'];
If you wanted to enable the ability to specify a custom directory in your request then you could pass it through the same way you are passing fname. Say for example you passed your custom directory along in a key named cust_dir you could add it as the directory like so.
if($_POST['cust_dir']{
$filename=$_POST['cust_dir'].$_POST['fname'];
} else {
$filename="d:/".$_POST['fname'];
}
The code above would use a custom directory path that you passed in the $_POST variable if you passed one. If you do not pass cust_dir then it will default to directory d:/.

Content appended twice in the CSV file when CSV is generated using PHP button

This is my code to generate csv file.When I click php button to generate Csv file,which is filled withthe contents based on the category column from the database.But my problem here is when the contents are getting populated twice in the csv file as shown below.Please help to out where i have to modify the code so that i can get only one time populated content as shown below as expected.Thanks in advance.
createcsv.php
<?php
$servername = "localhost";
$username = "user";
$password = "";
$dbname = "stats";
define("DB_SERVER", "localhost");
define("DB_NAME", "stats");
define("DB_USER", "user");
define("DB_PASSWORD", '');
$dbconn = #mysql_connect(DB_SERVER, DB_USER, DB_PASSWORD);
$conn = #mysql_select_db(DB_NAME,$dbconn);
// Create connection
//$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
echo "DB connection failed";
}
// Query DB to fetch hit count for each category and in turn create corresponding .csv file
function createCSVFile($type) {
$msql = "SELECT TRIM(TRAILING '.000000' from UNIX_TIMESTAMP(hitdate)*1000) as unixdate,count from h_stats where category='".$type."' order by unixdate asc";
$query = mysql_query($msql);
$type = str_replace(' ', '', $type);
$tmp_file = "data/tmp_".$type.".csv";
$fp = fopen("$tmp_file", "w");
// Write the query contents to temp file
while($row = mysql_fetch_array($query))
{
fputcsv($fp, $row);
}
fclose($fp);
// Modify the contents of the file as per the high chart input data format
$fp = fopen("$tmp_file", 'r+');
rewind($fp);
$file = "data/".$type.".csv";
$final = fopen("$file", 'w');
while($line = fgets($fp)){
trim($line);
$line = '['.$line.'],';
fputs($final,$line);
}
// Append var $type and remove the trailing ,
$final = file_get_contents($file);
$content = 'var '.$type .'= [' . rtrim($final, ","). ']';
file_put_contents("$file",$content);
}
// Query DB to fetch success/failure count for Hits and in turn create corresponding .csv file
function createHitOutcomeCSVFile($type,$category) {
$sql = "SELECT TRIM(TRAILING '.000000' from UNIX_TIMESTAMP(hitdate)*1000) as unixdate,".$type." from h_stats where category='".$category."' order by unixdate asc";
$query = mysql_query($sql);
$tmp_file = "data/tmp_".$type."_".$category.".csv";
$fp = fopen("$tmp_file", "w");
// Write the query contents to temp file
while($row = mysql_fetch_array($query)){
fputcsv($fp, $row);
}
fclose($fp);
// Modify the contents of the file as per the high chart input data format
$fp = fopen("$tmp_file", 'r+');
rewind($fp);
$category = str_replace(' ', '', $category);
$file = "data/".$type."_".$category.".csv";
$final = fopen("$file", 'w');
while($line = fgets($fp)){
trim($line);
$line = '['.$line.'],';
fputs($final,$line);
}
// Append var $type and remove the trailing ,
$final = file_get_contents($file);
$content = 'var '.$type.'_'.$category.'= [' . rtrim($final, ","). ']';
file_put_contents("$file",$content);
}
// Invoke function to create the Hits.csv file
createCSVFile('Hits');
// Invoke function to get Three Hits csv file
createHitOutcomeCSVFile('TCount','Hits');
// Invoke function to get O2 Hits csv file
createHitOutcomeCSVFile('BCount','Login');
echo "Generated successfully";
?>
not expected csv file with twice populated data:
var Login_Hits= [[1427826600000,1427826600000,8763,8763
]]
Expected csv file as per highcharts format:
var Login_Hits= [[1427826600000,8763
]]
Try to debug it...
it will be easier than seeing typo or so...
it looks like the tmp file is already corrupted...
try to display the $row variable and the $query...
the problem may come from here...
In while loop I have used mysql_fetch_assoc instead of mysql_fetch_array at both the functions
while($row = mysql_fetch_assoc($query))
{
fputcsv($fp, $row);
}
The content is not repeating twice in the Csv file.This works try it!

import csv file from external FTP Site

I have the below script to import data from a csv file on my server, the script works correctly.
I need to change the location of the file from my server to an FTP server which requires authentication. The csv file name on the FTP server will cahnge according to the time stamp it was generated. After importing into the MySQL database, the file on the FTP server needs to be deleted.
I then need to schedule this job with cron jobs to run every 5 minutes.
Any assistance will be appreciated.
Thanks.
<?php
/********************************/
/* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/
/* Edit the entries below to reflect the appropriate values
/********************************/
$databasehost = "localhost";
$databasename = "dbname";
$databasetable = "dbtable";
$databaseusername ="username";
$databasepassword = "password";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "filenamewithtimestamp.csv";
/********************************/
/* Would you like to add an empty field at the beginning of these records?
/* This is useful if you have a table with the first field being an auto_increment integer
/* and the csv file does not have such as empty field before the records.
/* Set 1 for yes and 0 for no. ATTENTION: don't set to 1 if you are not sure.
/* This can dump data in the wrong fields if this extra field does not exist in the table
/********************************/
$addauto = 0;
/********************************/
/* Would you like to save the mysql queries in a file? If yes set $save to 1.
/* Permission on the file should be set to 777. Either upload a sample file through ftp and
/* change the permissions, or execute at the prompt: touch output.sql && chmod 777 output.sql
/********************************/
$save = 0;
$outputfile = "output.sql";
/********************************/
if(!file_exists($csvfile)) {
echo "File not found. Make sure you specified the correct path.\n";
exit;
}
$file = fopen($csvfile,"r");
if(!$file) {
echo "Error opening data file.\n";
exit;
}
$size = filesize($csvfile);
if(!$size) {
echo "File is empty.\n";
exit;
}
$csvcontent = fread($file,$size);
fclose($file);
$con = #mysql_connect($databasehost,$databaseusername,$databasepassword) or die(mysql_error());
#mysql_select_db($databasename) or die(mysql_error());
$lines = 0;
$queries = "";
$linearray = array();
foreach(split($lineseparator,$csvcontent) as $line) {
$lines++;
$line = trim($line," \t");
$line = str_replace("\r","",$line);
/************************************
This line escapes the special character. remove it if entries are already escaped in the csv file
************************************/
$line = str_replace("'","\'",$line);
/*************************************/
$linearray = explode($fieldseparator,$line);
$linemysql = implode("','",$linearray);
if($addauto)
$query = "insert into $databasetable values('','$linemysql');";
else
$query = "insert into $databasetable values('$linemysql');";
$queries .= $query . "\n";
#mysql_query($query);
}
#mysql_close($con);
if($save) {
if(!is_writable($outputfile)) {
echo "File is not writable, check permissions.\n";
}
else {
$file2 = fopen($outputfile,"w");
if(!$file2) {
echo "Error writing to the output file.\n";
}
else {
fwrite($file2,$queries);
fclose($file2);
}
}
}
echo "Found a total of $lines records in this csv file.\n All records imported";
?>
I thought I would just post the code that solved my issue; it downloads a csv file from an external FTP site onto my local server. It then imports the csv file into my local mysql database table. Once imported the file on the ftp server is deleted. I schedule this job in cpanel with cronjob.
I cannot take credit for the script, I have just modified a script found on another site. I'm sure there are better ways of doing this with some error checking. However, this is the best I could do with my limited skills.
I hope this assists future visitors with the same issue. Thanks to all those that assisted me.
Here is the code I used:
<?php
$source = "DespGoods.csv";
$target = fopen("DespGoods.csv", "w");
$conn = ftp_connect("ftp.server.com") or die("Could not connect");
ftp_login($conn,"ftpusername","ftppassword");
ftp_fget($conn,$target,$source,FTP_ASCII);
echo "file downloaded.\n";
/********************************/
/* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/
/* Edit the entries below to reflect the appropriate values
/********************************/
$dbhost = "localhost";
$dbname = "dbname";
$dbtable = "despgoods";
$dbusername ="dbusername";
$dbpassword = "dbpassword";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "DespGoods.csv";
/********************************/
/* Would you like to add an ampty field at the beginning of these records?
/* This is useful if you have a table with the first field being an auto_increment
/* integer and the csv file does not have such as empty field before the records.
/* Set 1 for yes and 0 for no. ATTENTION: don't set to 1 if you are not sure.
/* This can dump data in the wrong fields if this extra field does not
/* exist in the table.
/********************************/
$addauto = 0;
/********************************/
/* Would you like to save the mysql queries in a file? If yes set $save to 1.
/* Permission on the file should be set to 777. Either upload a sample file
/* through ftp and change the permissions, or execute at the prompt:
/* touch output.sql && chmod 777 output.sql
/********************************/
$save = 0;
$outputfile = "output.sql";
/********************************/
if(!file_exists($csvfile)) {
echo "File not found. Make sure you specified the correct path.\n";
exit;
}
$file = fopen($csvfile,"r");
if(!$file) {
echo "Error opening data file.\n";
exit;
}
$size = filesize($csvfile);
if(!$size) {
echo "File is empty.\n";
exit;
}
$csvcontent = fread($file,$size);
fclose($file);
$con = #mysql_connect($dbhost,$dbusername,$dbpassword) or die(mysql_error());
#mysql_select_db($dbname) or die(mysql_error());
$lines = 0;
$queries = "";
$linearray = array();
foreach(split($lineseparator,$csvcontent) as $line) {
$lines++;
$line = trim($line," \t");
$line = str_replace("\r","",$line);
/************************************
/* This line escapes the special character.
/* Remove it if entries are already escaped in the csv file
/************************************/
$line = str_replace("'","\'",$line);
/*************************************/
$linearray = explode($fieldseparator,$line);
$linemysql = implode("','",$linearray);
if($addauto) {
$query = "insert into $dbtable values('','$linemysql');";
}
else {
$query = "insert into $dbtable values('$linemysql');";
}
$queries .= $query . "\n";
#mysql_query($query);
}
#mysql_close($con);
if($save) {
if(!is_writable($outputfile)) {
echo "File is not writable, check permissions.\n";
}
else {
$file2 = fopen($outputfile,"w");
if(!$file2) {
echo "Error writing to the output file.\n";
}
else {
fwrite($file2,$queries);
fclose($file2);
}
}
}
echo "Found a total of $lines records in this csv file.\n";
$source = "DespGoods.csv";
$target = fopen("DespGoods.csv", "w");
$conn = ftp_connect("ftp.server") or die("Could not connect");
ftp_login($conn,"ftpusername","ftppassword");
ftp_delete($conn,$source);
ftp_close($conn);
echo "file deleted";
mysql_close($con);
?>
Is this what you're looking for?
http://www.php.net/manual/en/ftp.examples-basic.php

Categories