I am running a windows machine for local development and attempting to load large csv files into mysql. My code looks like this:
$sql_query = 'LOAD DATA LOCAL
INFILE "' . $tempLoc . '"
INTO TABLE users
FIELDS
TERMINATED BY " "
ENCLOSED BY "\'"
LINES
TERMINATED BY "\\n"
(
id,
name,
value
)';
$statement = $this->adapter->query($sql_query);
$resultSet = $statement->execute();
$this->adapter->getDriver()->getConnection()->commit();
When I load the file I got this error:-
"PDOStatement::execute() [pdostatement.execute]: LOAD DATA LOCAL INFILE forbidden in..."
I have searched for an answer, but I have not been able to resolve the issue.
But when i connect the db with local db in my local m/c(ie, localhost with root) it runs well
my application is in local machine
DB is connected to another server
Make sure the server allows LOAD DATA LOCAL INFILE
[server]local-infile=1
Make sure that your MySQL user has 'FILE' privilege
Set the PDO attribute in your PHP script,
$pdo = new PDO($dsn, $username, $password, array(PDO::MYSQL_ATTR_LOCAL_INFILE=>1))
When you use LOAD DATA LOCAL, you're telling the MySQL server process to open a local file, on its own file system, and read it. Most MySQL operations use a TCP/IP connection between your client and the MySQL server to send and receive all data. But this operation is different.
If your client code is running on one machine and the server is running on another, this won't work unless they are sharing a file system. Hint: if you're using one of those $5 per month hosting services, this probably won't work.
Related
I am looking to backup a MySQL database. When the following command is entered into the command line it works:
SELECT * INTO OUTFILE 'backup.txt' FROM table_name;
The file backup.txt is created. All is well.
When the above command is submitted via the MySQL query in a php file:
mysql_query("SELECT * INTO OUTFILE 'backup.txt' FROM table_name");
it does not work. The file backup.txt is not created.
Next, the privileges were looked at and the following command was entered using the command line:
GRANT FILE ON *.* TO 'root'#'localhost';
This command was accepted but the MySQL query still does not work.
The complete php file is shown below:
// connect to db host
//mysql_connect(DATABASE_HOST, DATABASE_USERNAME, DATABASE_PASSWORD)
$connection = mysql_connect("localhost", "mysql -u root", "")
or die('Could not connect: ' . mysql_error($connection));
// select db
$db = mysql_select_db("test", $connection);
// change privileges
mysql_query("GRANT FILE ON *.* TO 'root'#'localhost'");
// create query
$query = "SELECT * INTO OUTFILE 'backup.txt' FROM table_name";
//perform query
mysql_query($query);
// close mysql connection
mysql_close($connection);
Can anybody explain what is happening here and how I can get the MySQL query to work and create the outfile backup.txt.
WampServer on a Windows machine is being used.
At the moment I run mysql command line (MCL), I am an o/s user that has rights (let's call them creds for credentials) at some level to all o/s directories. Let's say the creds don't change for the duration of this code attempt, can't imagine they would. Point is, I am that user with those creds.
Without a path, full or relative just as you did, MCL writes to the data directory for the schema in the outfile call. So for instance on my system at the moment that would be
C:\Users\All Users\MySQL\MySQL Server 5.6\data\so_gibberish
where so_gibberish is the schema/db name that I would have gotten into with the use so_gibberish command or supplied as the db to use upon running mysql command line with a switch. Or it would be some linux path equivalent.
Via the MCL, I would have gotten into a db (use), as opposed to an MCL connect with no use. Meaning that I would not have issued the outfile command prior to getting into a db sandboxed, most likely (I am making the point later as it relates to Error 1046). And your command would have dumped the .txt file there (the above path a ways up) on my system. And it did in my test.
Now on to PHP.
If you are running a PHP script that connects but does not use the mysql_select_db function, then
Error 1046: No database selected
would have returned only if you were running code like the below to check for an error:
<?php
//error_reporting(E_ALL);
//ini_set('display_errors', 1);
...
... (load credential variables used below)
...
$link = mysql_connect($dbhost, $dbuser, $dbpass) or die("Unable to Connect to '$dbhost'");
//mysql_select_db($dbname) or die("Could not open the db '$dbname'");
echo "I made it here<br/>";
$test_query = "SELECT * INTO OUTFILE 'file456.txt' FROM mytable";
$result = mysql_query($test_query);
echo mysql_errno($link) . ": " . mysql_error($link) . "<br>";
mysql_close($link);
Not that people tend to check for errors. I would wager you are not in this call. They just plow foward thinking all is well.
Note though that error_reporting has no impact on this. What does impact it is whether or not you are checking for errors after mysql_query. You can rem out or clear rems and test this theory as I did. So you could be getting the 1046 Error and not know it.
Now there is the case for what o/s users and therefore the creds that the php to mysql process is masquerading as. This is driven by the original setup. Now why is this important? Because that user/those creds can very well be different than those used when at the o/s prompt you did the very first part of this using the MCL.
It is possible that the file is created thru php, but as you are not pathing with a full path to the file, just the filename, it might be sitting somewhere on your system without you knowing it, if you were to check for errors and none arrive. To test this theory, include the above error check after mysql_user without a full path, and do a directory scan to find it.
So in my test, in MCL I write out file123.txt, and in php I write out file456.txt (or whatever). And barring any error messages, I scan the file system to see where they showed up.
You would not be the first person to think OUTFILE failed, only later, maybe months later, find residue files in some directory and have a eureka moment: Oh yeah, I remember those files, what are they doing here?
It is possible that the call simply fails because of a creds issue from PHP, having to do with user or group world or some other setup chmod issue.
Via PHP, fully pathing to the outfile such as /full/path/here/out123.txt can have a good solution if the o/s user masquerading has the creds. But, in hosted environments, you can't simply say make it /tmp/out123.txt, as you will fail with permissions there. So there is no broad brush stroke "plug in this answer" that is going to solve it without a somewhat decent tinkering session by you.
So in summary for PHP, I would look into the following:
The file is being written out, you just don't know where.
The file fails but you don't know it because of no error checking after
mysql_query (such as a general mysql error, no db selected,
whatever).
Creds issue due to the o/s user masquerading from PHP into
mysql and is creds at the o/s file system level required for that
file i/o.
As for the Error 1221 error, as mentioned in comments and link to that part of it, you cannot GRANT FILE on a single db. That answer type was provided here
Alright, so after a week of trying all the different ideas answers I have found with no success, I am going to ask. This is for the LOAD DATA LOCAL INFILE, for MySQL. There have been many posts regarding this, but I am still unable to get it work from the web browser, but am able to run it from the mysql command prompt on the server, with the same user and pass connecting to the same database. What I have tried so far
MySQL version 5.5
Ubuntu 12.04
PHP version 5.3
In my.cnf
local-infile=1
in mysqld,mysql,mysql-safe
loose-local-infile=1 client
Restarted MySQL Server. At this point I was then able to run the query from the command prompt, and previously had not.
I have given the directory in which the files are being pulled from 777 access.
I have confirmed the php.ini has the local file parameter enabled.
I have updated apparmor.
Actual Query:
LOAD DATA LOCAL INFILE '/var/www/ui/uploads/r_import.csv' INTO TABLE r_data FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 ROWS (first_name,last_name,apt,user_id)
Above query works from the mysql command with no special arguments in the connection to the server.
If anyone has anymore ideas on this, I would be happy to try anything....
Thanks in advance.
<?php
include 'includes/header.php';
if($_FILES['file']['type'] != "application/vnd.ms-excel"){
die("This is not a CSV file.");
}
elseif(is_uploaded_file($_FILES['file']['tmp_name'])){
$filename = $_FILES['file']['tmp_name'];
$name = $_FILES['file']['name'];
copy( $filename, 'uploads/'.$name ) or die( "Could not copy file!");
$file_to_import = '/var/www/ui/uploads/'.$name;
$query = 'LOAD DATA LOCAL INFILE \''.$file_to_import.'\' INTO TABLE r_data FIELDS TERMINATED BY \',\' ENCLOSED BY \'"\' LINES TERMINATED BY \'\n\' IGNORE 1 ROWS (first_name,last_name,apt,user_id)';
echo $query;
$result = mysqli_query($link,$query) or die(mysqli_error($link));
}
else{
die("You shouldn't be here");
}
?>
$link = mysqli_init();
mysqli_options($link, MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_real_connect($link, 'localhost', $username, $password, $database);
The connection string to the database is what worked for me. I found it in the last comment in the link given by developerwjk
Your problem is the use of the LOCAL keyword. When you use LOCAL the server expects the MySQL client to read the file and send it. This applies when the client software is running on a remote machine, or when you're running the MySQL client on the server itself. (This is why you can run your query from the server command line).
If you're running PHP there is no client software involved. PHP makes calls directly to the server, so the LOCAL keyword is invalid in this context.
To use LOAD DATA INFILE from PHP you must make sure that the file is placed in a location in the server filesystem that the MySQL server has read access to, and that the full path to that file is passed as part of your query. Don't use the LOCAL keyword.
If you're trying to load a file from a remote client you'll need to upload the file to the file system first, then execute your query.
Note this sentence from the MySQL manual: If LOCAL is specified, the file is read by the client program on the client host and sent to the server.
Reference: http://dev.mysql.com/doc/refman/5.5/en/load-data.html
We're moving our database from being on the webserver to a separate server (from an Amazon EC2 webserver to an RDS instance.)
We have a LOAD DATA INFILE that worked before that is going to need the LOCAL keyword added now that the database will be on a different machine to the webserver.
Testing on my dev server, it turns out that it doesn't work:
I can still LOAD DATA INFILE from php as I have been
I can LOAD DATA LOCAL INFILE from mysql commandline (with --local_infile=1)
I can't LOAD DATA LOCAL INFILE from php.
Between those 2 things that do work, it rules out:
problems with the sql or php code
problems with the upload file, including syntax and file permissions
mysql server settings problems
The error I get is:
ERROR 1148 (42000): The used command is not allowed with this MySQL version
(I get that error from the mysql commandline if I don't use --local_infile=1)
A few other bits of relevant info:
Ubuntu 12.04, mysql 5.5.24, php 5.3.10
I'm using php's mysql_connect (instead of mysqli, because we're planning on using facebook's hiphop compiler which doesn't support mysqli.)
Because of that, the connect command needs an extra flag set:
mysql_connect($dbHost, $dbUser, $dbPass, false, 128);
I've used phpinfo() to confirm that mysql.allow_local_infile = On
I've tried it on Amazon RDS (in case it was a problem in my dev server) and it doesn't work there either. (With the local_infile param turned on.)
The only thing I've read about that I haven't tried is to compile mysql server on my dev server with the flag turned on to allow local infile... but even if I get that working on my dev server it's not going to help me with Amazon RDS. (Besides which, LOAD DATA LOCAL INFILE does work from the mysql commandline.)
It seems like it's specifically a problem with php's mysql_connect()
Anybody using LOAD DATA LOCAL INFILE (maybe from Amazon RDS) that knows the trick to getting this to work?
I've given up on this, as I think it's a bug in php - in particular the mysql_connect code, which is now deprecated. It could probably be solved by compiling php yourself with changes to the source using steps similar to those mentioned in the bug report that #eggyal mentioned: https://bugs.php.net/bug.php?id=54158
Instead, I'm going to work around it by doing a system() call and using the mysql command line:
$sql = "LOAD DATA LOCAL INFILE '$csvPathAndFile' INTO TABLE $tableName FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\\\"' ESCAPED BY '\\\\\\\\' LINES TERMINATED BY '\\\\r\\\\n';";
system("mysql -u $dbUser -h $dbHost --password=$dbPass --local_infile=1 -e \"$sql\" $dbName");
That's working for me.
Here's a check list to rule out this nasty bug:
1- Grant the user FILE privileges in MySQL, phpMyAdmin generaly does not cover this privilege:
GRANT FILE ON *.* TO 'db_user'#'localhost';
2- Edit my.cnf in /etc/mysql/ or your mysql path:
[mysql]
local-infile=1
[mysqld]
local-infile=1
3- In php.ini at /etc/php5/cli/ or similar:
mysql.allow_local_infile = On
Optionally you can run ini_set in your script:
ini_set('mysql.allow_local_infile', 1);
4- The database handler library must use the correct options.
PDO:
new PDO('mysql:host='.$db_host.'.;dbname='.$db_name, $db_user, $db_pass,
array(PDO::MYSQL_ATTR_LOCAL_INFILE => 1));
mysqli:
$conn = mysqli_init();
mysqli_options($conn, MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_real_connect($conn,server,user,code,database);
5- Make sure that the INFILE command uses the absolute path to the file and that it exists:
$sql = "LOAD DATA INFILE '".realpath(is_file($file))."'";
6- Check that the target file and parent directory are readable by PHP and by MySQL.
$ sudo chmod 777 file.csv
7- If you are working locally you can remove the LOCAL from your SQL:
LOAD DATA INFILE
Instead of:
LOAD DATA LOCAL INFILE
Note: Remember to restart the MySQL and PHP services if you edit their configuration files.
Hope this helps someone.
As referred in this post, adding 3rd and 4th parameter to mysql_connect are required to get LOAD LOCAL DATA INFILE working. It helped me. Any other suggestions (apparmor, local-infile=1 in my.cnf widely discussed in internet) did not help. Following PHP code worked for me!
mysql_connect(HOST,USER,PASS,false,128);
True, this is in manual, too.
use the following line that client activates with infile true
mysql --local-infile=1 -u root -p
If you're doing this in 2020, a tip for you is to check your phpinfo.php or php --ini for the location of the configuratin file. For me I was using virtualmin and changing the php ini file but my site had it's own specific ini file. Once I located it's location and changed it everything went back to normal.
I am trying to load an .xml file from my ftp server into my mysql DB.
When I do this in the DB as an sql command it works perfectly fine.
But when I try to do it with PHP it does not work. I´ve tried many different ways.
SQL Command.
LOAD XML LOCAL INFILE 'min_url' INTO TABLE table_name ROWS IDENTIFIED BY '<product>'
My PHP looks like this.
$sql = " LOAD XML LOCAL INFILE 'min_url' INTO TABLE table_name ROWS IDENTIFIED BY '<product>'";
$result = mysql_query($sql);
So I am trying to do the exact same thing just with PHP. I even "created the php code for the command in my DB" and it still does not work. Everything untill the mysql_query is fine.
Versions
Serverversion: 5.5.24-0ubuntu0.12.04.1-log
Apache/2.2.12 (Ubuntu)
MySQL klientversion: 5.1.37
PHP-tillägg: mysqli
I am trying to use LOAD DATA INFILE to insert some records into a table. Unfortunately, it's not working.
Here are some details
If I use this instruction:
LOAD DATA INFILE 'file.txt'
INTO TABLE table_ex
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(field1, field2, field3, field4);
It works using the MySQL client program and a PHP application. In this way it will look for the file in the Data Directory of my MySQL installation.
Now if I try to execute the instructions using the LOCAL option, it only works if I use the mysql client, but not from PHP:
LOAD DATA LOCAL INFILE 'path/to/file/file.txt'
INTO TABLE table_ex
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(field1, field2, field3, field4);
Again.. it works with MySQL client but not from the PHP application... I get this error:
LOAD DATA LOCAL INFILE forbidden in /path/to/my/application
I read that the problem is related to the compilation of PHP and using mysqlnd. I am using PHP 5.3.8 and MySQL 5.5.15, but I haven't found a solution.
Additional information: until now the only help I've found was an open PHP bug:
Check docs http://php.net/manual/en/ref.pdo-mysql.php.
Basically you need:
PDO::MYSQL_ATTR_LOCAL_INFILE => true
Set at instantiation.
Example:
$conn = new \PDO("mysql:host=$server;dbname=$database;", "$user", "$password", array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
));
had this problem today and solved it by setting the following in php.ini
mysqli.allow_local_infile = On
I didn't get the exact error you get, but you need no ensure the following:
Enable by adding to your my.cnf:
[mysql]
local-infile=1
[mysqld]
local-infile=1
Tell the connection in PHP that it may use LOCAL INFILE
Using mysql:
mysql_connect(server,user,code,false,128); // 128 enables LOCAL INFILE
mysql_select_db(database);
Using mysqli:
$conn = mysqli_init();
mysqli_options($conn, MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_real_connect($conn,server,user,code,database);
Give MySQL user FILE permission
When using LOCAL this shouldn't be necessary, though. LOCAL says that the file is located on the client server (where you have PHP is installed), otherwise it looks at server location (where MySQL is installed).
GRANT FILE ON *.* TO 'mysql_user'#'localhost'
Easier work around is to use exec()
exec("mysql -u myuser -pMyPass -e \"USE mydb;TRUNCATE mytable;LOAD DATA INFILE '" . $file . "' IGNORE INTO TABLE mytable;\"; ");
2019+ relevant answer with a bit more background:
In PHP >7.2.16 and >7.3.3 the default ini configuration of mysqli.allow_local_infile, which controls this, changed from '1' to '0' (so it is now disabled by default).
This directive is only configurable via PHP_INI_SYSTEM so ini_set() will not work.
The only option is to add the following directive to your php.ini file, not forgetting to reload apache.
[MySQLi]
mysqli.allow_local_infile = On
According to the MySQL manual MySQL must be compiled with --enable-local-infile. From a comment at that link:
You MUST have compiled PHP using the full path to MySQL, otherwise it
will use it's internal handlers, which don't work with the "new" LOAD
DATA.
--with-mysql=/usr/local/mysql (assuming your MySQL is located here)
You MUST start the MySQL daemon with the option '--local-infile=1'
The solution whish worked for me is below. Adding mysqli_options was required on second server I've setup same script.
$mysqli = new
mysqli("$db_server_name","$db_user_name","$db_password","$database_name");
// force LOCAL_INFILE
mysqli_options($mysqli, MYSQLI_OPT_LOCAL_INFILE, true);
LOAD DATA LOCAL INFILE executes regardless of the warnings. it works on mysql client since it allows the execution of queries, ignoring warnings. Though it later prints out the warnings. It refuses in PHP though because a warning will halt the script.
Easiest solution, that may work on some servers is to remove LOCAL like:
Original:LOAD DATA LOCAL INFILE
New/ It should be: LOAD DATA INFILE
Strange, but I have found this solution to work on my local machine, with xampp but it did not work on a live server with CentOS, so I'd to revert the code back and add 'LOCAL'.
I had exactly the same problem on a EC2 Ubuntu 12.04 LTS instance when accessing a MySQL on RDS: LOAD DATA LOCAL INFILE... works fine on a mysql console but not from PHP. Accidentaly i found out that it worked fine on another almost identical machine that used MariaDB (a binary compatible drop in replacement for MySQL).
So i replaced the MySQL clients with the ones from MariaDB and it worked.
If you use an Ubuntu server, you can try to install php5-mysqlnd :
sudo apt-get install php5-mysqlnd
To resolve the same problem in PHP Symfony application, this flag needs to be enabled in the yml config file. Here is an example:
# Doctrine Configuration
doctrine:
dbal:
driver: pdo_mysql
options:
!php/const PDO::MYSQL_ATTR_LOCAL_INFILE: true
# Skip the rest
Also note how to reference PHP constant here in yml file, and this format is used for Symfony 3.4. For older version, check out Symfony doc.
uncomment 'mysqli.allow_local_infile = On' in php.ini.