I'm trying to insert data into MySQL table using the load data infile query. I have an HTML form that accepts a text file for upload, and then tries to process.
File
00|Oakleaf Forest|Norfolk|VA|United States|PC
01|Igo|Lowell|MA|United States|PC
02|Mint|Lawrence|MA|United States|PC
03|Hosa|Boston|MA|United States|PC
HTML
<p>Please upload file below:</p>
<form action="process.php" method="post" enctype="multipart/form-data">
<input type="file" name="file" accept="text/plain">
<input type="submit">
</form>
PHP
/* processFile
* Process the submitted file
*
* #param Array - file for processing
*/
function processFile($file) {
// Get global variable
global $DB;
// file contents
$file_contents = $file["tmp_name"];
$SQL_statement = "LOAD DATA INFILE '$file_contents' INTO TABLE organization FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n' IGNORE 2 LINES
(ein, legal_name, city_name, state_name, country_name, description);";
// Run SQL query
$DB->execute($SQL_statement);
}
When I upload the file, nothing is inserted into the table.
I've tried debugging with echo($SQL_statement);, which returns the following query:
LOAD DATA INFILE '/Applications/XAMPP/xamppfiles/temp/phpOtL2q1' INTO TABLE organization FIELDS TERMINATED BY '|' LINES TERMINATED BY ' ' IGNORE 2 LINES (ein, legal_name, city_name, state_name, country_name, description);
What am I missing here?
LOAD DATA INFILE requires that the file be readable by the database process. I doubt that the temporary upload file is world-readable, it's probably only readable by the webserver userid.
One thing you can try is changing the permissions on the temp file first:
chmod($file["tmp_name"], 0444);
However, this still might not work if the directory containing the temp files is not world-readable. Another solution is to use LOAD DATA LOCAL INFILE. This reads the file in PHP, not the database server, so any file that the PHP process can read can be loaded. It's not as efficient, since all the file data has to be sent over the database connection, but it avoids permission problems.
So try the first method first, if it doesn't work try the second.
Related
I tried to change from insert to to load data infile for uploading a large CSV file.
I explored some examples on how to use load data infile. But it's not working and giving this Error:
SQLSTATE[HY000]: General error: 29 File '/direct1/#tmp/phpFZLLYA' not found (Errcode: 13 "Permission denied").
This is my code:
$target_dir = '/direct1/#tmp/';
$target_file = $target_dir . basename($_FILES["file"]["tmp_name"]);
$stmt = $dbcon->prepare("LOAD DATA INFILE '$target_file' INTO TABLE Rain FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n' IGNORE 1 LINES(id, date, vol, day)");
$stmt->execute();
When I try to change to load data local infile based on related discussion, error code does not appear but the data is not inserted to the database. I am using INNODB storage engine.
I solved the error by using move_uploaded_file function and the file can be access.
I would like to store data downloaded from a website into my mysql database.
I use my function "CallAPI("GET", $url, $data = false)" to access the database using a url such as "http://www.xflow1.com/xGlobalHist.csv/"...
So my call $results = CallAPI($method, $url, $data = false); returns a comma delimted array that is saved in the variable "$results". I can echo $results in a web page and it show me the data, all comma delimited. All good `til here.
To upload the csv to my mysql database I want to use the "LOAD DATA INFILE" function as so:
$upload = <<<eof
LOAD DATA INFILE $results
INTO TABLE X_Adjusted_All
FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(Cusip, Date, Price)
eof;
The snag arises as "LOAD DATA INFILE $results" does not work as LOAD DATA INFILE only wants a filename, so I would like to store $results as a csv file in the memory to avoid creating and deleting files all the time. I thought this may work:
$fp = fopen('php://memory', 'w');
fputcsv($fp, $results);
Alas no. Does anyone have any idea how to take the downloaded csv file and save it to the phps memory as a csv file for use in the LOAD DATA INFILE function?
MySQL cannot read directly from PHP's memory. You need to create an external file using tempnam(). That file can then be read by MySQL. You could also create a temporary file and find out the filename as mentioned in this question.
The example of tempnam() shows how to write to that file:
<?php
$tmpfname = tempnam("/tmp", "FOO");
$handle = fopen($tmpfname, "w");
fwrite($handle, $results);
fclose($handle);
// do something
unlink($tmpfname);
This will store the contents of $results to a file called $tmpfname which you can store in mysql using LOAD DATA INFILE $tmpfname.
As mentioned in the comments, you could create a ram drive to store that file, if performance is an issue.
Originally $filepath is like this
$filepath=$_SERVER['DOCUMENT_ROOT']."/".SCRIPTFOLDER."somefile.txt";
$sqlquerynew = "LOAD DATA INFILE '".$filepath."' INTO TABLE mytable FIELDS TERMINATED BY ','";
mysql_query($sqlquery);
That works. It sits on the server.
But what if I want to put somefile.txt outside the same server? If I put that in dropbox, and reuse the code like the following, it doesn't do anything.
$sqlquerycreate = / ** Here insert DB scheme code **/
$filepath="http://example.com/somefile.txt";
$sqlquerynew = "LOAD DATA INFILE '".$filepath."' INTO TABLE mytable FIELDS TERMINATED BY ','";
mysql_query($sqlquery);
What should I do instead? The text file contains the data for the table.
Thanks.
$data_file = file_get_contents('http://dl.dropbox.com/u/14655573/john/nyccrash.txt');
file_put_contents('C:\xampp\htdocs\import\nyccrash.txt', $data_file);
$filepath = "C:\xampp\htdocs\import\nyccrash.txt";
echo $filepath;
$sqlquery = "LOAD DATA INFILE '".$filepath."' INTO TABLE nyccrash FIELDS TERMINATED BY ',' (crash_year, accident_type, collision_type,weather_condition,light_condition,x_coordinate,y_coordinate)";
Okay. I see this file now. But it didn't get import to the DB. This is XAMPP. But I will, eventually, move to Ubuntu. What's the problem with this code?
Try this:
$file = file_get_contents("http://example.com/somefile.txt");
$filepath=$_SERVER['DOCUMENT_ROOT']."/".SCRIPTFOLDER."somefile.txt";
file_put_contents($filepath, $file);
$sqlquerycreate = / ** Here insert DB scheme code **/
$sqlquerynew = "LOAD DATA INFILE '".$filepath."' INTO TABLE mytable FIELDS TERMINATED BY ','";
mysql_query($sqlquery);
Hope it helps..
The LOAD DATA INFILE reads a file from the client or from the server, not from the internet.
You could use copy() to first download the file to a temporary directory, and then point LOAD DATA INFILE to that directory.
Create first a temporary file on your server:
$tmpFile = tempnam('my-tmp-dir', 'TMP');
$ok = copy('http://example.com/somefile.txt', $tmpFile);
if ( !$ok ) exit('ERROR!');
// Then use $filepath in your SQL query
$sqlquerynew = "LOAD DATA INFILE '".$filepath."' INTO TABLE mytable FIELDS TERMINATED BY ','";
mysql_query($sqlquery);
unlink($tmpFile);
You can first import data in local file with
//read data from remote file
$data_file = file_get_contents($http_url);
// put data in a local file
$temp_file = tempnam(sys_get_temp_dir(),'TMP');
file_put_contents($temp_file,$data_file);
//now use $temp_file into the SQL query
....
//Delete temp file
unlink($temp_file);
Please note that PHP has functions to get the system's TEMP path, and to generate a temporary filename.
In the /tmp you should not have permissions issues.
I have csv file exported from access. (dates formatted to match mysql)
I need to import the data into a mysql DB through code.
when I import the file through PhpMyAdmin it worked beautifully.
$fname = $_FILES['sel_file']['name'];
$filename = $_FILES['sel_file']['tmp_name'];
$sql="LOAD DATA INFILE '../filepath/data.txt' INTO TABLE table1 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"';";
mysql_query($sql)or die(mysql_error());
I tried using the file path or using the variable $filename and both gave me the same error
I get the following error:
Access denied for user 'uname'#'%' (using password: YES)
I set the permissions to be 777, The databse was created allowing direct access. (I am able to opload the file and read from it using an "INSERT" statement, but not load data.)
A. Is the LOAD DATA statement wrong?
B. What other things do I have to take care of in order to use the LOAD DATA command.
Thank you all!
Try LOAD DATA LOCAL INFILE
It may be a authentication problem. Please check the permissions for Import/export data in mysql.
If I use the following in a mysql_query command:
SELECT *
FROM mytable
INTO OUTFILE '/tmp/mytable.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n';
Where is the tmp file relative to, to the MySQL database somehow or to the PHP file?
If it does not exist will it be created?
If I would like it to appear 1 folder up from the PHP file which does it, how would I do that?
According to The Documentation On Select, it's stored on the server and not on the client:
The SELECT ... INTO OUTFILE 'file_name' form of SELECT writes the selected rows to a file. The file is created on the server host, so you must have the FILE privilege to use this syntax. file_name cannot be an existing file, which among other things prevents files such as /etc/passwd and database tables from being destroyed. As of MySQL 5.0.19, the character_set_filesystem system variable controls the interpretation of the file name.
And, more to the point:
The SELECT ... INTO OUTFILE statement is intended primarily to let you very quickly dump a table to a text file on the server machine. If you want to create the resulting file on some other host than the server host, you normally cannot use SELECT ... INTO OUTFILE since there is no way to write a path to the file relative to the server host's file system.
So, don't use it in production to generate CSV files. Instead, build the CSV in PHP using fputcsv:
$result = $mysqli->query($sql);
if (!$result) {
//SQL Error
}
$f = fopen('mycsv.csv', 'w');
if (!$f) {
// Could not open file!
}
while ($row = $result->fetch_assoc()) {
fputcsv($f, $row);
}
fclose($f);
Where is the tmp file relative to?
A: The file will have the result of the select * from mytable
If it does not exist will it be created?
A: yes
If I would like it to appear 1 folder up from the php file which does it, how would I do that?
A: if you want one folder up from the fileYouAreRunning.php then make path like that: "../mytable.csv"
Your current query has an absolute path. So the outfile will not be relative to anything, but saved to /tmp/mytable.csv.
I'd say, the safest bet would be to keep useing absolute paths, so check in your php what your absolute path to the parent is, and then add this to your query using a variable.