I have this test code:
session_start();
session_write_close();
set_time_limit(0);
for ($i = 1; $i < 1000; $i++)
{
if (rand(1,5) == 1)
{
$file_db = new PDO('sqlite:x.db');
$file_db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$file_db->exec('DROP TABLE IF EXISTS messages');
$file_db = null;
}
$file_db = new PDO('sqlite:x.db');
$file_db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$file_db->exec('CREATE TABLE IF NOT EXISTS messages (id INTEGER PRIMARY KEY AUTOINCREMENT,title TEXT,message TEXT,time INTEGER)');
$file_db->query('INSERT INTO messages VALUES (null, "'.rand(1,9999).'", "'.rand(1,99).'", date("now"))');
$file_db = null;
}
$file_db = new PDO('sqlite:x.db');
$file_db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$i = 0;
foreach ($file_db->query('SELECT * FROM messages') as $m)
{
$i++;
echo $i.' -> '.$m['id'].'<br>';
}
with this code Im testing if the PDO 3 works with multiple concurrencies. If this code runs in one thread, all is ok. But as soon as I start two instance of this, I get this message:
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 17 database schema has changed' in code.php:* Stack trace: #0 code.php(*): PDO->query('INSERT INTO mes...') #1 {main} thrown in code.php on line *
the error line is where the INSERT INTO is. Why does this happen and how to dodge it? (Im creating a cache-system where there is do a possibility that tables gets created and dropped. I know there is TRUNCATE, I just want to know why this happens)
[...] as soon as I start two instance of this, I get this message: [...] SQLSTATE[HY000]: General error: 17 database schema has changed [...] Why does this happen and how to dodge it?
Sqlite is a single file database. You open it in parallel and you then change the database structure. Inserting data while you just dropped the table to insert into just does not work because the table is gone. This explains why it happens.
As you hopefully now have understood that inserting into a non-existing table is not possible, the solution might be clearly visible: Only insert into an existing table.
To ensure that you create a table after you've dropped it, it might be helpful to do such operations in transactions: https://www.sqlite.org/lang_transaction.html
Try$file_db = new PDO(realpath('sqlite:x.db'));
for
$file_db = new PDO('sqlite:x.db');
Related
I am doing a pretty simple script to check if a given url exists in a database. I have checked and EVERY column, table, and the database itself is set for 'latin1_general_ci' so I don't see how it is even possible this is showing :
PHP Fatal error: Uncaught exception 'PDOException' with message
'SQLSTATE[HY000]: General error: 1267 Illegal mix of collations
(latin1_general_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for
operation '='' in /home/user/public_html/checkurl.php:17
//get url passed
$url = $_GET['u'];
//dating
$stmt = $db->prepare("
SELECT *
FROM testing
WHERE url = :url
");
$stmt->execute(array(':url' => $url));
$result = $stmt->fetch(PDO::FETCH_ASSOC);
if($result)
{
$db = null;
exit("1");
}
I thought maybe this might have to do with cached queries so I reset the cache as well using RESET QUERY CACHE, but it still occurs.
I'm currently looping to create a MBTiles map, and add information to my database each time.
Here's how I configured my connection and execute actions during the loop:
if ($pdo_mbtiles == null) {
echo "Opening new database connection".PHP_EOL;
$pdo_mbtiles = new PDO('sqlite:'.$filename,
'',
'',
array(
PDO::ATTR_PERSISTENT => true
)
);
$pdo_mbtiles->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$pdo_mbtiles->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
$q = $pdo_mbtiles->prepare("INSERT INTO tiles (zoom_level, tile_column, tile_row,tile_data) VALUES (:zoom_level, :tile_column, :tile_rowTMS, :tile_data)");
$q->bindParam(':zoom_level', $zoom_level);
$q->bindParam(':tile_column', $tile_column);
$q->bindParam(':tile_rowTMS', $tile_rowTMS);
$q->bindParam(':tile_data', $tile_data, PDO::PARAM_LOB);
$q->execute();
After 1018 times looping (this number doesn't change no matter how many times I try), I get this error message:
SQLSTATE[HY000]: General error: 14 unable to open database file
I checked the solution written here:
How to prevent SQLITE SQLSTATE[HY000] [14]?
but the echoed message only appears at the first time of the loop, so I assume the PDO connection isn't closed.
I didn't find other documentation relevant to this error code.
What could possibly go wrong here?
I tried to move the prepare and bind commands in a condition as follows. The exception isn't raised, but only the first tile is saved (or every tile is saved on top of the first one, not sure):
if ($pdo_mbtiles == null) {
echo "Opening new database connection".PHP_EOL;
$pdo_mbtiles = new PDO('sqlite:'.$filename,
'',
'',
array(
PDO::ATTR_PERSISTENT => true
)
);
$pdo_mbtiles->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$pdo_mbtiles->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
if ($q == null) {
$q = $pdo_mbtiles->prepare("INSERT INTO tiles (zoom_level, tile_column, tile_row,tile_data) VALUES (:zoom_level, :tile_column, :tile_rowTMS, :tile_data)");
$q->bindParam(':zoom_level', $zoom_level);
$q->bindParam(':tile_column', $tile_column);
$q->bindParam(':tile_rowTMS', $tile_rowTMS);
$q->bindParam(':tile_data', $tile_data, PDO::PARAM_LOB);
}
$q->execute();
Here are the files during the generation:
And here after the exception is raised:
Also, when the exception is raised, I do a var_dump of my $pdo_mbtiles, and here's the result (exactly the same as when I do it with a success):
object(PDO)#116 (0) {
}
Edit: Still trying to solve this problem, I simplified the code to create the MBTiles file. No success yet, but here's a sample if anyone want's to reproduce the issue. You can download it from https://www.dropbox.com/s/33vqamc9tn4c3ux/sample_PHP_MBTiles_generation_bug.zip?dl=0
I suggest you to reuse the connection if it is open.
Create a property: private $pdo;
And check if it's null before creating a new object:
function opendatabase(){
try{
if($this->pdo==null){
$this->pdo =new PDO("sqlite:database/database.db","","",array(
PDO::ATTR_PERSISTENT => true
));
}
return $this->pdo;
}catch(PDOException $e){
logerror($e->getMessage(), "opendatabase");
print "Error in openhrsedb ".$e->getMessage();
}
}
The error message was misleading. After many hours of debugging, I found that it was completely unrelated to my database connection.
It's juste that I used fopen() to get tiles data, and didn't fclose() after registration, thus reaching the limit of 1024.
1024 is because I used six require or require_once statements, so 1018 tile requests + 6 require = 1024 opened connections.
I have a database table with an unsigned int that represents a wallet balance. this column is called wallet. It is in a table called users.
The following query fails via mysql cli:
UPDATE users set wallet = `wallet` - 550000000 WHERE username = 'user'
With error message:
ERROR 1690 (22003): BIGINT UNSIGNED value is out of range in
'(database.users.wallet - 550000000)'
The issue is, when executed via PDO with ERRMODE_EXCEPTION, it brings wallet balance to 0 and continues execution without raising the mentioned error in an exception
SELECT ##sql_mode returns nothing in both the code and mysql cli.
Here is the function that creates my database handle and returns it to the query object
//This function connects to a database
public function connect($dbname,DBConfig $config = NULL)
{
//If the connection is already set return it
if(isset($this->dbs[$dbname]))
return $this->dbs[$dbname];
//If the config object is not already set, and still null, throw exception
if(!isset($this->_config[$dbname]))
{
if($config === NULL)
throw new PDOdatabasesException('Database configuration object is not set',1);
$this->_config[$dbname] = $config;
}
$config = $this->_config[$dbname];
//Create a PDO object
$this->dbs[$dbname] = new PDO(
$config::type .
':host=' . $config::$host .
';port=' . $config::$port .
';dbname=' . $dbname,
$config::$user,
$config::$password
);
//Tell the handle to throw exceptions
$this->dbs[$dbname]->setAttribute( PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION );
$this->dbs[$dbname]->setAttribute( PDO::ATTR_EMULATE_PREPARES, false );
//Return a reference to the newly created PDO object
return $this->dbs[$dbname];
} //End connect()
Trying to save the need to paste a bunch of unnecessary code.. my query object takes a database handle from above, and prepares and executes this statement:
'UPDATE users SET wallet = wallet - ? WHERE id = ?'
binding the amount (approximately 10 million) and the user id, then executing.
Wallet balance can be at 0 and the query will still execute successfully. Why does this happen when the cli cannot? It does not make sense!!
I believe i need to once again reiterate This query SHOULD fail if it drops wallet below 0!
It succeeds through pdo, but not through mysql cli, my question is WHY??
The result of wallet - 550000000 should be less than 0 and your column is UNSIGNED. Try to change your column type from BIGINT UNSIGNED to BIGINT
This is apparently an unsolved bug in PDO::bindValue() and PDO::bindParam()
EDIT:
I am apparently stupid and did not realize that data_type was REQUIRED to be defined when binding an integer to a query.
I have simplified the code to be the following:
$db = new PDO('mysql:dbname=test;host=127.0.0.1','omitted','omitted');
$db->setAttribute(PDO::ATTR_ERRMODE,PDO::ERRMODE_EXCEPTION);
try{
$statement = $db->prepare("UPDATE test set number = number - ? WHERE id = 1");
$statement->bindValue(1,1000000000);
$statement->execute();
}catch(PDOException $e)
{
die($e->getMessage());
}
changing the statement to (and not using bindValue or bindParam):
$statement = $db->prepare("UPDATE test set number = number - 100000000 WHERE id = 1");
then the code throws the expected exception
CHANGING AGAIN TO:
$statement>bindValue(1,1000000000,PDO::PARAM_INT);
throws as expected
I have a .txt file with a list of 60,000 English words. I wanted to insert those to my database, so I simply just did as show here.
$file = new SplFileObject('list.txt');
foreach ($file as $line => $word) {
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test');
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
Now, after I run this script, I get the following error:
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000] [1040] Too many connections' in /var/www/skillz/test/curl/index.php:17 Stack trace: #0 /var/www/test/index.php(17): PDO->__construct('mysql:host=loca...', 'root', 'test') #1 {main} thrown in /var/www/test/index.php on line 4
That line 4 is where the new PDO('mysql:') is located. So, I tried to search this error, and found this answer that seemed a solution. And I edited mysql accordingly, as
$ vi /etc/my.cnf
max_connections=250
But I still get the same error, I have MySql 5.5.38 running PHP-FPM, NGINX in CentOS 6.5
Don't open a new Connection for every word. You only need one connection open for the lifetime of your inserts. I'm not sure about the true lifetime of a PDO object, I know they're cleaned up when they're not used, but garbage collection might not do that for a couple of minutes, and for 60,000 words, you're going to hit your limit of connections to the database faster than it can clean them up.
$file = new SplFileObject('list.txt');
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test');
foreach ($file as $line => $word) {
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
You should declare the SQL connection outside the foreach.
Because it will make 60.000 connections.
$file = new SplFileObject('list.txt');
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test');
foreach ($file as $line => $word) {
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
You only need to declare the SQL connection once and you can use it anytime as you want.
If you put it in foreach it will make a SQL connection every word, that's why you got that message.
Solution 1
Use batch insert statement :
INSERT INTO words (col1, col2) VALUES ('val1', 'val2'), ('val3', 'val4'), ...('val3n', 'val4n');
This fails if you also want to check if some rows failed or not. So, below is another solution.
Solution 2
Create a persistent database connection. This will use the same connection in all iterations of the loop.
$file = new SplFileObject('list.txt');
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test', array(
PDO::ATTR_PERSISTENT => true)); //Persistent Database Connection
foreach ($file as $line => $word) {
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
$p = null; //Destroy Connection
I have a table in my db. My table has several fields including an auto-incremented id field set as primary key, and one other field called 'reference' that I did set as unique. To populate that table, I have a php script that insert records in that table using pdo. Every time an insert has been made successfully (meaning the 'reference' did not exist in the table), I increment a variable called $newOnes. If the value 'reference' is already in the table, an exception with the code 23000 is triggered. In that case, I increment another variable called $doublons. Unfortunately my script is triggering a fatal error with exception 23000 when the while loop is "handling" the last record of the table. And I do not get it. Thank you in advance for your help. Cheers. Marc.
My php code:
try {
$connexion = connexion('localhost', 'user', 'user', 'mydb');
$connexion->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$qry_bat = $connexion->query('SELECT...');
$ins_db = $connexion->prepare('INSERT...');
}
catch (PDOException $e) {
echo $e->getMessage();
}
while($row = $qry_bat->fetch(PDO::FETCH_ASSOC)) {
try {
$ins_db->execute(array(...));
$newOnes++;
}
catch (PDOException $e) {
if ($e->getCode() != 23000) {
echo '<span class="msg-alert">'.$e->getMessage().'</span>';
} else {
$doublons++;
}
}
}
The fatal error I am getting (note that line 22 refers to the while(...) line):
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[23000]:
Integrity constraint violation: 1062 Duplicate entry 'theFieldContentOfTheLastRecordOfTheTable' for key 'theFieldNameReference' in
/myFilePath/file.php:22 Stack trace: #0 /myFilePath/file.php(22): PDOStatement->fetch(2)
#1 {main} thrown in /myFilePath/file.php on line 22
EDIT //////////
original table (things to mentionne):
auto incremented id
table to insert in (things to mentionne):
auto incremented on id field
UNIQUE INDEX on reference
(Upgrading to an answer)
Looks like this bug, which is still open after almost five years; try instead:
while (true) {
try {
$row = $qry_bat->fetch(PDO::FETCH_ASSOC);
if (!$row) break;
$ins_db->execute(array(...));
$newOnes++;
}
catch (PDOException $e) {
if ($e->getCode() != 23000) {
echo '<span class="msg-alert">'.$e->getMessage().'</span>';
} else {
$doublons++;
}
}
}
The stack trace shows the exception taking place in the fetch which is outside of the try/catch blocks.
The SELECT statement is triggering:
Integrity constraint violation: 1062 Duplicate entry 'theFieldContentOfTheLastRecordOfTheTable'
There's a unique key named theFieldContentOfTheLastRecordOfTheTable that it's conflicting with.
Can you post the schema for us to see how this could affect integrity?
Look for dirty / duplicate data ALREADY in that column. Some apps allow a "loose" constraint to be applied AFTER data is already captured. This may have happen in your case.