I have a Symfony 4 project and I want to store mysql queries in as a string in a mysql database. However, before storing the strings I want to make sure they are valid mysql syntax. Is there a way of doing this?
Thanks!
I didn't test it but it should work.
Use the database API you already use in your project to prepare the SQL statements you want to validate then discard them; do not execute the prepared statements.
For example, using PDO, use PDO::prepare() to ask the server to prepare the statement. It returns a PDOStatement object on success (i.e. when the query is correct). Do not call execute() on the returned statement, just discard it (using unset()).
PDO::prepare() returns FALSE or throws an exception on error, depending on how the PDO's error handling is configured.
The easiest way would be to run a query in a new transaction and then roll it back. SQL can get complex to validate especially if you plan to allow MySQL-specific functions. What if a new function gets introduced in next MySQL release? Writing and maintaining a separate SQL validation library seems counterproductive.
Why not to try following:
Create a new user for running these queries in your database. This will allow you to manage security e.g. allowing only to use SELECT statement so no one will run DROP DATABASE.
Run the user provided statement using the new user created in point 1. Start a new transaction using START TRANSACTION, execute the user provided statement, and rollback with ROLLBACK. Ensure SET autocommit=0 is set as per 13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Syntax.
If the user provided statement executes without errors it's valid. You don't have to read all the returned rows in your PHP code.
Make sure to check on performance because some statements will be expensive to execute. This functionality can DOS your application.
I'd probably create procedure or function in the database. That's what they are for. Storing SQL in a table just to query it and then execute only results in a redundant round trip between the database and the application.
Related
This is odd. I'm running a query with just a single INSERT, preceded by a SET statement. The query looks something like this:
SET #discount:=(SELECT discount * :applyDiscount FROM fra_cus WHERE customerID=:customerID AND franchiseID=:franchiseID);
INSERT INTO discounts_applied (unitID, franchiseID, customerID, amount)
VALUES(:unitID, :franchiseID, :customerID, #discount * :price);
It appears that if I prepare these as two separate PDO queries, lastInsertID() works fine... but if I prepare them and execute them in the same statement, lastInsertID() returns nothing.
It's not the end of the world, but it's annoying. Anyone know why this would be the case? For the record, there's a reason I need to define #discount as a variable (pertains to triggers on one of the tables). Also this is all taking place within a larger transaction.
First of all, I would strongly recommend to run every query in a distinct API call. This is how an Application Programming Interface is intended to work.
It won't only prevent situations like this but also will make your code a multitude times more readable and maintainable.
And it will make your code much safer too. You can run multiple statements in a single call only at the expense of the native prepared statements. However virtual this vulnerability is, why taking chances at all?
Why not to make a regular SELECT query instead of SET, get the resulting value into a PHP variable and then use it among other variables, just through a placeholder? I don't see any reason why there should be such a complex way to deal with simple data.
In case I failed to convince you, the reason is simple. You are running two queries, and the first one doesn't trigger any insert ids. And obviously, you need this query's metadata (errors, affected rows, whatever), not the other one's first. So you get it. And to get the second'query's metadata you have to ask a database for it. The process is explained in my article: Treating PHP delusions - The only proper PDO tutorial: Running multiple queries with PDO. Basically PDOStatement::nextRowset() is what you need.
PDO provides functions to initiate, commit, and roll back transactions:
$dbh->beginTransaction();
$sth = $dbh->prepare('
...
');
$sth->execute(); // in real code some values will be bound
$dbh->commit();
Is there any reason to use the PDO functions over simply using the transaction statements in MySQL? I.e:
$sth = $dbh->prepare('
START TRANSACTION;
...
COMMIT;
');
$sth->execute(); // in real code some values will be bound
UPDATE: Just a note to anyone else looking into this, after some testing I actually found the second case above (using START TRANSACTION and COMMIT in prepare()) will result in an exception being thrown. So in order to use transactions with a prepared statement, you must use the PDO functions shown in the first case.
From a portability standpoint, you're better off using the interface that PDO provides in case you ever want to work with a different DBMS, or bring on another team member who's used to another DBMS.
For example, SQLite uses a slightly different syntax; if you were to move to an SQLite database from MySQL, you would have to change every string in your PHP code that contains the statement START TRANSACTION; because it would no longer be valid syntax for your database. SQL Server 2014 is another example that doesn't use this syntax.
Of course, you can also use BEGIN; in MySQL to start a transaction, and that would work fine in SQLite and SQL Server. You could just use that instead.
You'll often be able to find a syntax that you like and that is reasonably portable but why spend the time and energy to even think about it if you don't have to? Take advantage of the fact that there are a dozen PDO drivers available to make your life easier. If you care at all about consistency, favor the API over implementation-specific SQL syntax.
The difference between the PDO and mysql transaction is nothing. EXCEPT
You can for example start your transaction, make some querys run some php code do more querys based on your code and such, and you could rollback at the end of that code simply execute $PDO->rollback(); this is way easier than creating 2 - 3 more querys instead of using $pdo->beginTransaction();
Also using $pdo->rollback(); is a few lines shorter and in my opinion it's also clearer than creating another query and executing it.
So, let's say I'm using two drivers at the same time (in the specific mysql and sqlite3)
I have a set of changes that must be commit()ted on both connections only if both dbms didn't fail, or rollBack()ed if one or the another did fail:
<?php
interface DBList
{
function addPDO(PDO $connection);
// calls ->rollBack() on all the pdo instances
function rollBack();
// calls ->commit() on all the pdo instances
function commit();
// calls ->beginTransaction() on all the pdo instances
function beginTransaction();
}
Question is: will it actually work? Does it make sense?
"Why not use just mysql?" you would say! I'm not a masochist! I need mysql for the classic fruition via my application, but I also need to keep a copy of a table that is always synchronized and that is also downloadable and portable!
Thank you a lot in advance!
I suspect you put the cart before the horses! If
two databases are in sync
a transaction commits successfully on one DB
No OS-level error occures
then the transaction will also commit successully on the second DB.
So what you would want to do is:
- Start the transaction on MySQL
- Record all data-changing SQL (see later)
- Commit the transaction on MySQL
- If the commit works, run the recorded SQL against SQlite
- if not, roll back MySQL
Caveat: The assumption above is only valid, if the sequence of transactions is identical on both DBs. So you would want to record the SQL into a MySQL table, which is subject to the same transaction logic as the rest. This does the serialization work for you.
You mistake PDO with a database server. PDO is just an interface, pretty much like the database console. It doesn't perform any data operations of its own. It cannot insert or select data. It cannot perform data locks or transactions. All it can do is to send your command to database server and bring back results if any. It's just an interface. It doesn't have transactions on it's own.
So, instead of such fictional trans-driver transactions you can use regular ones.
Start two, one for each driver, and then rollback them accordingly. By the way, with PDO one don't have to rollback manually. Just set PDO in exception mode, write your queries and add commit at the end. In case one of queries failed, all started transactions will be rolled back automatically due to script termination.
I have some kind of (basic) logging with user actions in a postgreSQL database.
In order to gain performance, I execute all log inserts asynchronously to let the script continue and not wait until log entry is created.
I use prepared statements everywhere to prevent SQL injections and load them in an as-needed basis.
The problem comes when there are pending results to be fetched from a previous async query when I prepare a statement. (PostgreSQL says there are pending results to be fetched prior to prepare a new statement)
So as a workarround, I gather all pending results (if any) and ignore them to make PHP and PostgreSQL happy before preparing any statement.
But with that workarround (as I see it), I miss the performance I could gain by executing asyncronously as I have to gather the results anyway.
Is there any way to asynchronously execute a prepared statement and deliberatelly tell postgres to ignore results?
Inside my PostgreSQL class, I am calling prepared statements with
pg_send_execute($this->resource, $name, $params);
and prepairing them with
//Just in case there are pending results (workarround)
while (pg_get_result($this->resource)!==FALSE);
$stmt = pg_prepare($this->resource, $stmtname, $query);
Any help will be apreciated.
UPDATE: All asynchronous queries I am using are only INSERT ones, so it should be safe (theoretically) to ignore their results.
The only thing that is asynchronous is your communication with PostgreSQL server - your database has to process everything sequentially.
My proposal:
If you have to use PostgreSQL for logging, use a separate database connection for logging purposes and get a connection pool sitting between your script and database - auth in PostgreSQL is costly and takes some time, this will cut it down. Acquiring a second connection will take some time, but if you use this method it will be faster than one without connection pool.
Depending on your reliability requirements you should use autocommit (to never lose a log entry when PHP crashes). You may want to use an UNLOGGED table (available since PostgreSQL 9.1) if you don't care about reliability on databse end (faster inserts as your data skips WAL) or if you don't use replication or don't need to have logs replicated.
As a speed optimization, your log table should have no indexes because they would have to be updated on each insert. If you need them, create a second table and move data in a batch (every X minutes or every hour).
I'm sorry, this is a very general question but I will try to narrow it down.
I'm new to this whole transaction thing in MySQL/PHP but it seems pretty simple. I'm just using mysql not mysqli or PDO. I have a script that seems to be rolling back some queries but not others. This is uncharted territory for me so I have no idea what is going on.
I start the transaction with mysql_query('START TRANSACTION;'), which I understand disables autocommit at the same time. Then I have a lot of complex code and whenever I do a query it is something like this mysql_query($sql) or $error = "Oh noes!". Then periodically I have a function called error_check() which checks if $error is not empty and if it isn't I do mysql_query('ROLLBACK;') and die($error). Later on in the code I have mysql_query('COMMIT;'). But if I do two queries and then purposely throw an error, I mean just set $error = something, it looks like the first query rolls back but the second one doesn't.
What could be going wrong? Are there some gotchas with transactions I don't know about? I don't have a good understanding of how these transactions start and stop especially when you mix PHP into it...
EDIT:
My example was overly simplified I actually have at least two transactions doing INSERT, UPDATE or DELETE on separate tables. But before I execute each of those statements I backup the rows in corresponding "history" tables to allow undoing. It looks like the manipulation of the main tables gets rolled back but entries in the history tables remain.
EDIT2:
Doh! As I finished typing the previous edit it dawned on me...there must be something wrong with those particular tables...for some reason they were all set as MyISAM.
Note to self: Make sure all the tables use transaction-supporting engines. Dummy.
I'd recommend using the mysqli or PDO functions rather than mysql, as they offer some worthwhile improvements—especially the use of prepared statements.
Without seeing your code, it is difficult to determine where the problem lies. Given that you say your code is complex, it is likely that the problem lies with your code rather than MySQL transactions.
Have you tried creating some standalone test scripts? Perhaps you could isolate the SQL statements from your application, and create a simple script which simply runs them in series. If that works, you have narrowed down the source of the problem. You can echo the SQL statements from your application to get the running order.
You could also try testing the same sequence of SQL statements from the MySQL client, or through PHPMyAdmin.
Are your history tables in the same database?
Mysql transactions only work using the mysqli API (not the classic methods). I have been using transactions. All I do is deactivate autocommit and run my SQL statements.
$mysqli->autocommit(FALSE);
SELECT, INSERT, DELETE all are supported. as long as Im using the same mysqli handle to call these statements, they are within the transaction wrapper. nobody outside (not using the same mysqli handle) will see any data that you write/delete using INSERT/DELETE as long as the transaction is still open. So its critical you make sure every SQL statement is fired with that handle. Once the transaction is committed, data is made available to other db connections.
$mysqli->commit();