PHP LOAD DATA INFILE Error Reading File - php

Solved
So, it was a type of permission error. Earlier in this script, I used flock() on the file to make sure the file wasn't being written to by another script. Removing flock() allows the query to run. Now I just need to determine a way to not load a file if it is still being written to...
I'm having trouble getting LOAD DATA INFILE to work in my php script. Here's the relevant portions of the script:
... //set $host, $user, etc.
$dsn = "mysql:host=$host;dbname=$database";
$pdo = new PDO($dsn, $user, $password, array(PDO::MYSQL_ATTR_LOCAL_INFILE => 1));
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
... //set $sqlFile and $table
$sql = "LOAD DATA INFILE '$sqlFile' REPLACE INTO TABLE `$table`";
echo $sql . "\n";
$rows = $pdo->exec($sql);
Running the script then produces:
LOAD DATA INFILE 'D:/pathToTemp/emdr/emdrorders/emdrorders_160314-1947UTC.txt'
REPLACE INTO TABLE `emdrorders`
[PDOException]
SQLSTATE[HY000]: General error: 2 Error reading file 'D:\pathToTemp\emdr\emdrorders\emdrorders_160314-1947UTC.txt'
(Errcode: 13 - Permission denied)
However, if I run the same query through the mysql cli it works.
mysql> LOAD DATA INFILE 'D:/pathToTemp/emdr/emdrorders/emdrorders_160314-1947UTC.txt'
REPLACE INTO TABLE `emdrorders`;
Query OK, 5487 rows affected (0.10 sec)
Records: 5355 Deleted: 132 Skipped: 0 Warnings: 0
I've also tried using LOCAL, but instead of throwing an exception $pdo->exec() returns 0 and the data is not loaded into the database.
My Mysql is 5.6.12 and PHP is 5.4.16 on a Windows machine and planning to put it on linux server. (I'm also doing this within the Laravel framework but I don't think that would cause this problem.)
Since the query works in the mysql cli but not through php, I can only assume the problem is in the php settings or the pdo. What do I need to change?

What is it that you plan on doing with $rows?
You pass queries through PDO::QUERY or PDO::PREPARE.
I assume from the use of PDOStatement::execute that you wish to use PDO::PREPARE.
$query = $pdo->prepare($sql); $query->execute();
To obtain responses from the query:
$query->fetch() or $query->fetchAll()
See link for documentation on preparing a query in PDO:
http://www.php.net/manual/en/pdo.prepare.php

Related

Syntax error when executing batch of statements via PDO but fine via SSMS

I'm writing code to create a MSSQL database from scratch using PHP PDO. I have the database created, and now need to execute a bunch of statements loaded from a .SQL file to create the tables, data, etc. The first few lines are
EXEC sp_dbcmptlevel 'myDbName', 120
GO
IF (1 = FULLTEXTSERVICEPROPERTY('IsFullTextInstalled'))
BEGIN
EXEC [myDbName].[dbo].[sp_fulltext_database] #action = 'enable'
END
GO
ALTER DATABASE [myDbName] SET ANSI_NULL_DEFAULT OFF
GO
EXEC sp_dbcmptlevel 'myDbName', 120
...
If I run the .SQL file directly in SSMS, it runs perfectly. If I load it and execute it via PHP PDO, I get
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[42000]: [Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Incorrect syntax near 'GO'.' in ...
Looking at the profiler, my query is being prefaced/wrapped with
declare #p1 int
set #p1=NULL
exec sp_prepexec #p1 output,NULL,N'EXEC sp_dbcmptlevel ''myDbName', 120
...
and it appears that with the prepared execution wrapper, the server doesn't like it. I'm executing the script that I load from the SQL file with
$db = new PDO('sqlsrv:Server='.$serverName.';Database='.$databaseName, $username, $password);
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$db->setAttribute(PDO::SQLSRV_ATTR_ENCODING, PDO::SQLSRV_ENCODING_UTF8);
$db->prepare($mySqlScript)->execute();
How can I fix the syntax error to execute the entire set of commands PHP loads from the SQL file?
You had to split your script by the "GO" clause into multiple statements, then run them one by one.
Something like:
$script = file_get_contents('script.sql');
$statements = explode( 'GO', $script );
foreach( $statements as $statement ) {
// here execute the $statement
$sth = $dbh->prepare($statement);
$sth->execute();
}
This is what SSMS do internally, splits the statements by the "GO's" before execute them.

PHP PDO SQL - update inner join query freezes during execution

I need to run a daily PHP script that downloads a data file and executes a bunch of SQL Queries in sequence to import and optimize the data.
I'm having a problem executing one of my queries within PHP which appears to freeze the mysqld process on my server. Oddly, running the same query does not case a similar problem when run from the Sequel Pro Database client program (Mac).
The query is running an update on a large table with over a million rows. Here is the stored procedure I'm using:
DELIMITER ;;
CREATE PROCEDURE spSetTripIdInStopTimes()
BEGIN
-- SET META_TRIP_ID IN IMPORT_STOP_TIMES
UPDATE import_stop_times
INNER JOIN ref_trips ON
(
import_stop_times.trip_id = ref_trips.trip_id
AND import_stop_times.meta_agency_id =ref_trips.meta_agency_id
)
SET import_stop_times.meta_trip_id = ref_trips.meta_trip_id;
END;;
DELIMITER ;
When the procedure is called with
CALL spSetTripIdInStopTimes;
inside Sequel Pro, the result is 1241483 rows affected and the time taken is around 90 seconds.
With PHP PDO I run the same command with
$result = $database->runExecQuery("CALL spSetTripIdInStopTimes");
However, it gets stuck on this query for over 24 hrs and still has not completed. When I cancel the PHP script I can see that the mysqld process is still taking %99.5 CPU. At this point I restart SQL with 'sudo service mysql restart'.
I also tried using PHP's mysqli, but the freezing also occurs with this method.
$mysqli->query("CALL spSetTripIdInStopTimes")
Would anyone be able to reason why this is happening or suggest another method?
Thank you in advance.
Note: I also tried using the older mysql on PHP, but the version I'm using (5.5.9-1ubuntu4.14) tells me the command is deprecated and stops the script.
[UPDATE]
I've also tried running the stored procedure directly on the command line:
mysql --user=[username] --password=[password] --execute="call spSetTripIdInStopTimes()" [tablename]
which worked.
So I tried running the same command with PHP's exec() function:
exec("mysql --user=[username] --password=[password] --execute=\"call spSetTripIdInStopTimes()\" [table name]");
Unfortunately, it stills hangs. I'm starting to wonder if this is due to the limitation or overhead of PHP.
[UPDATE 2]
Here is the array of PHP PDO connection options I use:
array(
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
// Allow file reading, need following settings to import data from txt files
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => true,
PDO::ATTR_EMULATE_PREPARES => true)
[UPDATE 3]
I'm using a custom database object, so I'll show the function for $database->runExecQuery() for clarification:
function runExecQuery($queryString)
{
$db = $this-> getConnection(); // Connect to database
try{
return array(
"success"=> true,
"data"=>$db->exec($queryString)
);
}
catch (PDOException $ex)
{
return array(
"success"=> false,
"errMessage"=> "ERROR[{$ex->getCode()}]".($this-> Debug ? "{$ex}" : "")
);
}
}
The variable $db is the connection variable that is initialized as follows:
// Create address string
$db_address =
"mysql:host={$settings['db_hostname']};".
"dbname={$settings['db_name']};".
"charset={$settings['db_charset']}";
// Create connection to database
$db = new PDO($db_address, $settings['db_user'], $settings['db_pw'], $options);
where $options is the array from [Update 2].
[Update 4]
Mihai's suggestion of changing PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false had some promising results as query appeared to finish. However, after more testing I found that the PHP script will sometimes still hang on the Query about 50% of the time it is run. This is true even with the same set of data in the SQL tables.

mysql_connect with --local-infile parameter

Can not load data from uploaded (local) file since upgrade of mysql (current version: Server version: 5.5.44-0+deb8u1 (Debian)), files implied are:
dbconnection.php
<?php
$server = "localhost";
$user = "TheUser";
$pass = "ThePass";
$db_name = "DbName";
$link = mysql_connect($server, $user, $pass);
mysql_select_db($db_name);
mysql_set_charset('utf8', $link);
?>
send2db.php
<?php
include 'dbconnection.php';
mysql_select_db("DbName") or die(mysql_error());
$query = "LOAD DATA LOCAL INFILE '$file' INTO TABLE `T1` FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '\"' ";
mysql_query($query) or die(mysql_error());
?>
The error says:
ERROR 1148 (42000): The used command is not allowed with this MySQL version
Inside mysql:
SHOW GLOBAL VARIABLES LIKE 'local_infile';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile | ON |
+---------------+-------+
1 row in set (0.00 sec)
But if I access mysql this way, files can be loaded:
mysql --local-infile -p
So my question is, can I set this option in the dbconnection.php file, I've tried many ways already with no success, I've been reading posts about my.cnf configuration and some other stuffs but nothing works for me, any suggestion?
Thanks
UPDATE:
I've been away changing the code of the entire web to mysqli, ufff!!, well following the suggestions from the answers bellow I did the next code but no success, I still get the message: "The used command is not allowed with this MySQL version". Implied files are next:
acessdb.php
<?php
$link = new mysqli($server, $user, $pass, $dbname);
?>
send2db.php
<?php include 'acessdb.php';
$link->options(MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_query($link, "LOAD DATA LOCAL INFILE 'upfiles/file.csv' INTO TABLE `T1` FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '\"'") or die(mysqli_error($link));
$link->options(MYSQLI_OPT_LOCAL_INFILE, false);
?>
Any suggestions?
Set the option in my.cnf (or mysql configuration file on your system):
local-infile=1
Restart MySQL service and this should fix the problem for you.
UPDATE
Another option to try with PHP
$conn = mysqli_init();
$conn->options(MYSQLI_OPT_LOCAL_INFILE, true);
Try that and see if that works. Options link mysqli options
Ok, finally I found the way, here is the working code:
file: connectdb.php
$link = mysqli_init();
mysqli_options($link, MYSQLI_OPT_LOCAL_INFILE, true);
mysqli_real_connect($link, $server, $user, $pass, $dbname);
file: send2db.php
mysqli_query($link, "LOAD DATA LOCAL INFILE 'upfiles/file.csv' INTO TABLE `T1` FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '\"'") or die(mysqli_error($link));
I hope it helps.
LOCAL INFILE is a mechanism by which the database server can request more or less any file from the database client (= the PHP server) If you can't fully trust your server and network this can be dangerous. Therefore LOCAL INFILE has to be allowed both on the server (as you did) as on the database client.
You are using the old outdated and not maintained mysql extension of PHP. That extension doesn't support setting the flag. You should switch to mysqli which has the MYSQLI_OPT_LOCAL_INFILE option to enable this mode.
<?php
$m = new mysqli(...);
$m->option(MYSQLI_OPT_LOCAL_INFILE, true);
$m->query("LOAD DATA LOCAL INFILE ....");
$m->option(MYSQLI_OPT_LOCAL_INFILE, false);
?>
Look through the comments above and create your checklist of sorts as follows:
Can I run "Load Data Local Infile" via mysql command line? If no, make sure this is enable by checking your global variable as discussed above.
Can I run a basic mysql script via my php code? If yes, then replace with the "Load Data Local Infile" script show about (same as what was used in step 1 via mysql).
Check database to determine if the load worked. If yes, "great job". If no, you need to turn on this Load feature via PHP. Look at the instructions above where you add $xxx->options(MYSQL_OPT_LOCAL_INFILE, true); this should be followed by a command to turn this off.
I struggled through this same issue for 5+hours. These are the summary steps that would have helped me troubleshoot and solve this issue faster. I would try these steps before messing with config files, which could lead to unintended consequences. Thanks for everyone's contributions.
As mysql_connect() is deprecated since PHP 5.5.0 (and will be deleted in the future), it's not impossible the issue comes from your web server (Apache ?) and not your SQL version.
Another thing that can cause this issue is the content of your file which eventually contains depreacted command (that's what the error message seems to tell you).

error when update table with odbc in php

I have database with DBF format, and use the ODBC to access the database.
When i do update on the table, it returns error:
Warning: odbc_execute() [function.odbc-execute]: SQL error: [Microsoft][ODBC dBase Driver] Index not found., SQL state S0012 in SQLExecute in C:\xampp\htdocs\payroll\index.php on line 16
According to the error msg, it seems like the index not found. what index is that? how to fix?
Below is the PHP script that i use to do the query.
$odbc = odbc_connect ('payroll', '', '') or die('Error connecting to server. Server says: '.odbc_errormsg());
$upd_q = "UPDATE paytran SET empno = 22 WHERE empno = 888";
$update = odbc_prepare($odbc, $upd_q);
$result = odbc_execute($update);
odbc_close($odbc);
Same method was used to insert new record. Insert query works successfully, but not for DELETE and UPDATE query.
According to the last post in this thread,
If you have a .dbf file nowadays, more often than not, it is from a
FoxPro application. And, since you have a .cdx file, that tells you
it is a FoxPro compound index file and dBASE can't use it and
certainly can't take advantage of it to get any speed advantage that
you would like in your queries.
Do you have a .cdx file? If so, you may want to check which version you have and then find the FoxPro driver or OLE DB provider, and use that instead of ODBC. There's information on how to do it here and here; the driver and DB provider are here.

Executing SQL directly; no cursor in PHP when calling a stored procedure

I'm getting the following error when trying to execute a stored procedure from PHP using sqlsrv driver:
Executing SQL directly; no cursor
Add a cursor option to your connection:
$connection = odbc_connect('host','username','password', 1)
or die('Connection failed!');
After password insert cursor_option 1 <-- this option was missing.
I got the same error when I used code that basically did this:
$options = array( "Scrollable"=>SQLSRV_CURSOR_STATIC);
$sql = "exec dbs_webgetnextcount 'expense'";
sqlsrv_query($db,$sql,null,$options);
I fixed it by not setting the cursor.
$sql = "exec dbs_webgetnextcount 'expense'";
sqlsrv_query($db,$sql);
PS. I was doing the CURSOR_STATIC option on my select statements so that sqlsrv_num_rows() would give a correct number.
I had the same problem as Jon but I figured since it is just a warning generated by the sql data access component and the query still completes successfully you can disable it in sqlsrv.exe using sqlsrv_configure("WarningsReturnAsErrors", 0);
Could be a permissions issue:
http://griffo.info/tellme/2009/05/executing-sql-directly-no-cursor/

Categories