the issue
So I bumped into something curious this morning when I was updating my database. I executed a collation change in my database, changing it from latin1 to uft8. However, my queries failed suddenly on my table. After some debugging, (rebuilding the table even with its original setup, but to no such avail) and receiving 500 internal errors, i realized it had to do with the prepared statement, so i tore it out, and replaced it with a regular mysqli_query, and it surprisingly worked. So now I am wondering, was my prepared statement wrong the whole time, or did it fail because of a change in the database.
the setup
This is the current table set up. I changed it back to latin (and its innoDB) yet it didnt gave me the results back i wanted when i changed everything back to the original settings (which is how it is now)
the code
the original code was this and it worked fine until the change
require_once '../db/dbControl.php';
$id = mysqli_real_escape_string($con,$_GET["id"]);
$sql = "SELECT *
FROM project
WHERE project.ProjectId = ? ";
$stmt1 = mysqli_prepare($con, $sql);
mysqli_stmt_bind_param($stmt1,'i',$id);
mysqli_stmt_execute($stmt1);
mysqli_stmt_bind_result($stmt1,$ProjectId,$ProjectTitel,$ProjectOmschrijving, $ProjectOmschrijving,$ProjectDatum,$ProjectClient,$ProjectUrl);
while (mysqli_stmt_fetch($stmt1)){
the code itself of the page
}
So right now I am just using a regular mysqli_query in order to make it work
require_once '../db/dbControl.php';
id = mysqli_real_escape_string($con,$_GET["id"]);
$sql = "SELECT *
FROM project
WHERE project.ProjectId = '". $id ."'";
$result = mysqli_query($con,$sql);
while($rows=mysqli_fetch_array($result)){
$ProjectId = $rows['ProjectId'];
$ProjectTitel = $rows['ProjectTitel'];
$ProjectExpertise = $rows['ProjectExpertise'];
$ProjectOmschrijving = $rows['ProjectOmschrijving'];
$ProjectDatum = $rows['ProjectDatum'];
$ProjectClient = $rows['ProjectClient'];
$ProjectUrl = $rows['ProjectUrl'];
the code itself of the page
}
I am a little bit confused (maybe i overlooked something here because to focussed on a little bit of code) but it only happens on the project table. I checked it against code that involves readouts, and they work all fine with prepped statements.
Hope anyone can spot what I couldn't
I wont be able to tell you what happened but here are two thought's.
The prepared statement execution consists of two stages: prepare and
execute. At the prepare stage a statement template is sent to the
database server. The server performs a syntax check and initializes
server internal resources for later use.
A prepared statement can be executed repeatedly. Upon every execution
the current value of the bound variable is evaluated and sent to the
server. The statement is not parsed again. The statement template is
not transferred to the server again.
Maybe this is what happened, it could be that the prepared statements never reseted after you changed to utf8.
Every prepared statement occupies server resources. Statements should be closed explicitly immediately after use. If not done explicitly, the statement will be closed when the statement handle is freed by PHP.
Using a prepared statement is not always the most efficient way of
executing a statement. A prepared statement executed only once causes
more client-server round-trips than a non-prepared statement. This is
why the SELECT is not run as a prepared statement above.
Maybe the server memory (is/was) full?
Tekst from:
http://php.net/manual/en/mysqli.quickstart.prepared-statements.php
Related
I'm trying to insert into my database and have been frustratingly not been able to get my statement(s) to work. I'm using PHP's MySQL Improved (mysqli) procedural interface. It might be worth noting that I'm using the c9.io IDE (pre-AWS) and everything including the server that my application is running on is through c9.
What I've noticed is that the statements have been working randomly. Initially, I was making very subtle changes to my INSERT statements until it worked, but after the working trial, it would fail again. So, eventually I started hitting the refresh button (same inputs, no modifications to my code) repeatedly until I hit a success.
In terms of code:
$sql = "INSERT INTO `users` (`email`,`password`) VALUES ('example#mail.com','1234')";
$result = mysqli_query($connection,$sql);
gives
$result = false
very consistently, but every random nth trial
$result = true
(same inputs, no change to my code).
I do not believe it is an error with my SQL syntax considering the random successes, nor do I believe it is an error with my connection. All of my SELECT statements have been working fine.
Thus, I have a hunch that for some reason it may be an issue with c9? If you have ever had a similar issue with any of MySQL, SQL, PHP, or c9, please help!
You Should try this
<?php
if (!mysqli_query($connection,"INSERT INTO Persons (FirstName) VALUES ('Glenn')"))
{
echo("Error description: " . mysqli_error($connection));
}
?>
Use myqli_error() which will give you a error message which should help clarify the issue with your code
I have checked everywhere thoroughly, and have gone through everything possible to find an answer to this. Besides saying "the code doesn't work" which obviously is not enough, I have yet to find anything that will even come close to this. I'm probably going to get downvotes, but let's see how this goes.
I am learning how to do prepared statements for a search query from the user end, and I have to do it for multiple queries. I have to bind parameters to these multiple queries, and then execute them and use them and receive multiple rows. This is most of my code, and what I currently have is not reporting any errors whatsoever. It just returns a blank white page.
I am doing this from a simple test.php file, and those are the results I'm getting.
Now for the code.
$prep1 = $test->prepare("SELECT * FROM sb__bans WHERE sb__bans.authid=? ORDER BY sb__bans.bid DESC");
$prep2 = $test->prepare("SELECT * FROM sb__bans AS bans INNER JOIN sb__admins AS admins ON bans.aid = admins.aid WHERE bans.authid=? ORDER BY bans.bid DESC");
$prep3 = $test->prepare("SELECT * FROM sb__bans AS bans INNER JOIN sb__servers AS servers ON bans.sid = servers.sid WHERE bans.authid=? ORDER BY bans.bid DESC");
$search = "steam";
$prep1->bind_param("s", $search);
$prep2->bind_param("s", $search);
$prep3->bind_param("s", $search);
$prep1->execute();
$prep2->execute();
$prep3->execute();
while($row = $prep1->fetch() && $admin = $prep2->fetch() && $sv = $prep3->fetch()) {
echo $row['test'];
echo $admin['test'];
echo $sv['test'];
}
The database is initialized above this as $test = new mysqli("localhost", "test", "test", "test");
$search = "steam" steam would be replaced with the the post variable of course, but for testing reasons I've removed that for now and am testing with just a simple variable.
What seems to be the problem here?
Thanks in advance.
Regarding the general question you asked.
There is not a single problem with having multiple queries prepared. While speaking of getting results from a prepared query, there is indeed a problem caused by the result buffering. In order to be able to execute another query, you have to call store_result()/get_result() right after execute.
Regarding the particular problem you have.
To get errors you have to ask PHP for them.
There is absolutely no point in making three queries, you have to make just one. If you have a trouble making one, ask another question marking it with mysql tag and bringing your 3 queries along.
Even for multiple queries it's just wrong idea to do multiple fetches in a single loop. Fetch your query results one by one.
Your mysqli syntax even for a single query is incomplete. You need to re-read your tutorial and practice on a single query first.
Two points:
Based on personal experience, you can only have one prepared statement in existence at a time. I suspect this is because the db requires each PS to have a session-unique name, and the PHP layer is passing some common default name rather than generating a unique name for each PS. By comparison, the PostgreSQL driver allows an optional name for each PS, but still allows only one unnamed PS to exist. Essentially this means that you must prepare, bind, execute and fetch one PS completely before you can prepare the next PS.
You're misusing mysqli_stmt::fetch(). fetch() returns only true or false, and is used to update variables which have previously been bound with mysqli_stmt::bind_result(). To retrieve values into a $row array, you must first call mysqli_stmt::get_result() to return a mysqli_result, and then call mysqli_result::fetch_array().
I am obviously missing something. I am using this sequence to prepare a statement and execute it.
$result = pg_prepare($link,"getacctname",'SELECT accounts.acctname FROM accounts WHERE accounts.acctname = $1') or die(pg_last_error());
$result = pg_execute($link,"getacctname",array($this->valuetofind)) or die(pg_last_error());
I always get the error "prepared statement already exists". I have read the other posts and even tried adding before the prepare and same result.
pg_query($link, "DEALLOCATE getacctname");
I even tried changing the name of the query and rebooting my machine to make sure there was nothing stuck that I was not seeing. same result.
Using pg_query($link, "DEALLOCATE ALL"); before the prepare fixed the problem.
However the troubling issue is that the only place prepare statement was used for that name query name was at that location. After turning off my machine and coming back to puzzle I tried the process that ran the code and it said that the statement already existed even though that was the first time the code was run.
I'm trying to loop data from a api and then post these values to a MySQL db.
something like this:
$values = json_decode(file_get_contents("my-json-file"));
$SQL = new mysqli(SQL_HOST, SQL_USER, SQL_PASS, DB_NAME);
$SQL->autocommit(FALSE);
foreach($values As $item)
{
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2.";)";
$SQL->query($query);
if(!$SQL->commit())
{
echo "ERROR ON INSERT: [" . $query . "]<hr/>";
}
}
$SQL->close();
Since the loop is too fast, the SQL can't catch up. (Yea!)
I would then need something like this:
foreach($values As $item)
{
/**** STOP/PAUSE LOOP ****/
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2.";");
$SQL->query($query);
if($SQL->commit())
{
/**** START THE LOOP AGAIN ****/
}
else
{
echo "ERROR ON INSERT: [" . $query . "]<hr/>";
}
}
Or how should I do this the right way?
EDIT: It inserts random posts every time.
EDIT 2: This is just example code. It does escape and all that, and yes the semi colon is wrong here but since so many commented on it i will not change it. This was not the problem in the real case.
I tried to run it on another server and there it worked. The problem was fixed by restarting MAMP.
Firstly, your idea that the loop runs too fast for MySQL to keep up is completely totally wrong. The $SQL->query() call will wait for the MySQL to return a response before proceeding, so the loop won't run any faster than MySQL is responding.
Now onto the actual problem.... your query:
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2.";)";
There's a semi-colon in there at the end, after value2 which is invalid. I guess you intended to type a quote mark there? The semi-colon will be causing all your queries to fail and throw errors.
This may be the cause of your problem but you haven't got any error checking in there, so you won't know. Add some error checking to your code after calling the query; even if the query is right, it's still possible to get errors, and your code should check for them. See the examples on this manual page: http://www.php.net/manual/en/mysqli-stmt.error.php
Finally, since you're using the mysqli API, it's worth mentioning that your code would be a lot better and probably more secure if you used prepared statements. See the examples in PHP manual here: http://www.php.net/manual/en/mysqli-stmt.bind-param.php
[EDIT]
Another possible reason your query is failing is that you're not escaping the input values. If any of the input values contains a quote character (or any other character that is illegal in SQL) then the query will fail. In addition, this problem makes your code vulnerable to a SQL injection hacking attack.
You need to escape your input using $SQL->real_escape_string() OR by changing your query to use prepared statements (as recommended above).
Your query is inside the loop, which means that the loop will wait until your query finished executing before it continue, php code is processed in order...
Has #phpalix said, PHP goes in order, and waits for the previous action to finish.
I think you SQL is wrong. Try replacing your INSERT with this:
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2."');";
And don't forget to run at least mysql_real_escape_string for each variable, for security measures.
As many of the answers and comments say, it does not continue until the SQL is done. The problem was in my local apache/mysql server. It was fixed by restarting it. Yes, stupid post.
Nowadays, "Prepared statements" seem to be the only way anyone recommends sending queries to a database. I even see recommendations to use prepared statements for stored procs. However, do to the extra query prepared statements require - and the short time they last - I'm persuaded that they are only useful for a line of INSERT/UPDATE queries.
I'm hoping someone can correct me on this, but it just seems like a repeat of the whole "Tables are evil" CSS thing. Tables are only evil if used for layouts - not tabular data. Using DIV's for tabular data is a style violation of WC3.
Like wise, plain SQL (or that generated from AR's) seems to be much more useful for 80% of the queries used, which on most sites are a single SELECT not to be repeated again that page load (I'm speaking about scripting languages like PHP here). Why would I make my over-taxed DB prepare a statement that it is only to run once before being removed?
MySQL:
A prepared statement is specific to
the session in which it was created.
If you terminate a session without
deallocating a previously prepared
statement, the server deallocates it
automatically.
So at the end of your script PHP will auto-close the connection and you will lose the prepared statement only to have your script re-created it on the next load.
Am I missing something or is this just a way to decrease performance?
:UPDATE:
It dawned on me that I am assuming new connections for each script. I would assume that if a persistent connection is used then these problems would disappear. Is this correct?
:UPDATE2:
It seems that even if persistent connections are the solution - they are not a very good option for most of the web - especially if you use transactions. So I'm back to square one having nothing more than the benchmarks below to go on...
:UPDATE3:
Most people simply repeat the phrase "prepared statements protect against SQL injection" which doesn't full explain the problem. The provided "escape" method for each DB library also protects against SQL injection. But it is more than that:
When sending a query the normal way,
the client (script) converts the data
into strings that are then passed to
the DB server. The DB server then uses
CPU power to convert them back into
the proper binary datatype. The
database engine then parses the
statement and looks for syntax errors.
When using prepared statements... the
data are sent in a native binary form,
which saves the conversion-CPU-usage,
and makes the data transfer more
efficient. Obviously, this will also
reduce bandwidth usage if the client
is not co-located with the DB server.
...The variable types are predefined,
and hence MySQL take into account
these characters, and they do not need
to be escaped.
http://www.webdesignforums.net/showthread.php?t=18762
Thanks to OIS for finally setting me strait on this issue.
unlike the CSS tables debate, there are clear security implications with prepared statements.
if you use prepared statements as the ONLY way to put user-supplied data in to a query, then they are absolutely bullet-proof when it comes to SQL injection.
When you execute a sql statement on the database, the sql parser needs to analyse it beforehand, which is the exact same process as the preparation.
So, comparing executing sql statements directly to preparing and executing has no disadvantages, but some advantages:
First of all, as longneck already stated, passing user input into a prepared statement escapes the input automatically. It is as if the database has prepared filters for the values and lets in only those values that fit.
Secondly, if use prepared statements thoroughly, and you come in the situation where you need to execute it multiple times, you don't need to rewrite the code to prepare and execute, but you just execute it.
Thirdly: The code becomes more readable, if done properly:
$sql = 'SELECT u.id, u.user, u.email, sum(r.points)
FROM users u
LEFT JOIN reputation r on (u.id=r.user_id)
LEFT JOIN badge b on (u.id=b.user_id and badge=:badge)
WHERE group=:group';
$params = array(
':group' => $group,
':badge' => $_GET['badge']
);
$stmt = $pdo->prepare($sql);
$result = $stmt->execute($params);
Instead of
$sql = 'SELECT u.id, u.user, u.email, sum(r.points)
FROM users u
LEFT JOIN reputation r on (u.id=r.user_id)
LEFT JOIN badge b on (u.id=b.user_id and badge="'.mysql_real_escape_string($_GET['badge']).'")
WHERE group="'.mysql_real_escape_string($group).'"';
$result = mysql_query($sql);
Imagine you had to change the sql statement, which code would be your favourite? ;-)
Prepared Statements come in handy in several situations:
Great separation of query data from untrusted user data.
Performance increase when the same query is executed multiple times
Performance increase when binary data is being transmitted as the prepared statement can use the binary protocol, whereas a traditional query will end up doing encoding and such.
There is a performance hit under normal circumstances (not repeated, no binary data) as you now have to do two back and forths. The first to "prepare" the query, and the second to transmit the token along with the data to be inserted. Most people are willing to make this sacrifice for the security benefit.
With regards to persistent connections:
MySQL has one of the fastest connection build up times on the market. It's essentially free for most set ups, so you're not going to see too much of a change using persistent connections or not.
The answer has to do with security and abstraction. Everyone else has already mentioned security, but the real upside is that your input is completely abstracted from the query itself. This allows for a true database agnosticism when using an abstraction layer, whereas inlining the input is usually a database-dependent process. If you care anything for portability, prepared statements are the way to go.
In the real world, I rarely ever write DML queries. All of my INSERTS / UPDATES are automatically built by the abstraction layer and are executed by simply passing an input array. For all intents and purposes, there really is no "performance hit" for preparing queries and then executing them (save for connection latency in the initial PREPARE). But when using a UDS (Unix Domain Socket) connection, you're not going to notice (or even be able to benchmark) a difference. It's usually on the order of a few microseconds.
Given the security and abstraction upsides, I'd hardly call it wasteful.
The performance benefit doesn't come from less parsing - it comes from only having to calculate access paths once rather than repeatedly. This helps a lot when you're issuing thousands of queries.
Given mysql's very simple optimizer/planner this may be less of an issue than with a more mature database with much more sophisticated optimizers.
However, this performance benefit can actually turn into a detriment if you've got a sophisticated optimizer that is aware of data skews. In that case you can often be better off with getting a different access path for the same query using different literal values rather than reusing a preexisting path.
When using sql queries like SELECT x,y,z FROM foo WHERE c='mary had a little lamb' the server has to parse the sql statement including the data + you have to sanitize the "mary had..." part (a call to mysql_real_escape() or similar for each parameter).
Using prepared statements the server has to parse the statement, too, but without the the data and sends back only an identifier for the statement (a tiny tiny data packet). Then you send the actual data without first sanitizing it. I don't see the overhead here, though I freely admit I've never tested it. Have you? ;-)
edit: And using prepared statements can eliminate the need to convert each and every parameter (in/out) to strings. Probably even more so if your version of php uses mysqlnd (instead of the "old" libmysql client library). Haven't tested the performance aspect of that either.
I don't seem to be finding any good benefits to use persistent connections - or prepared statements for that mater. Look at these numbers - for 6000 select statements (which will never happen in a page request!) you can barely tell the difference. Most of my pages use less than 10 queries.
UPDATED I just revised my test to
include 4k SELECT and 4k INSERT
statements! Run it yourself and let me
know if there are any design errors.
Perhaps the difference would be greater if my MySQL server wasn't running on the same machine as Apache.
Persistent: TRUE
Prepare: TRUE
2.3399310112 seconds
Persistent: FALSE
Prepare: TRUE
2.3265211582184 seconds
Persistent: TRUE
Prepare: FALSE
2.3666892051697 seconds
Persistent: FALSE
Prepare: FALSE
2.3496441841125 seconds
Here is my test code:
$hostname = 'localhost';
$username = 'root';
$password = '';
$dbname = 'db_name';
$persistent = FALSE;
$prepare = FALSE;
try
{
// Force PDO to use exceptions for all errors
$attrs = array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION);
if($persistent)
{
// Make the connection persistent
$attrs[PDO::ATTR_PERSISTENT] = TRUE;
}
$db = new PDO("mysql:host=$hostname;dbname=$dbname", $username, $password, $attrs);
// What type of connection?
print 'Persistent: '.($db->getAttribute(PDO::ATTR_PERSISTENT) ? 'TRUE' : 'FALSE').'<br />';
print 'Prepare: '.($prepare ? 'TRUE' : 'FALSE').'<br />';
//Clean table from last run
$db->exec('TRUNCATE TABLE `pdo_insert`');
}
catch(PDOException $e)
{
echo $e->getMessage();
}
$start = microtime(TRUE);
$name = 'Jack';
$body = 'This is the text "body"';
if( $prepare ) {
// Select
$select = $db->prepare('SELECT * FROM pdo_insert WHERE id = :id');
$select->bindParam(':id', $x);
// Insert
$insert = $db->prepare('INSERT INTO pdo_insert (`name`, `body`, `author_id`)
VALUES (:name, :body, :author_id)');
$insert->bindParam(':name', $name);
$insert->bindParam(':body', $body);
$insert->bindParam(':author_id', $x);
$run = 0;
for($x=0;$x<4000;++$x)
{
if( $insert->execute() && $select->execute() )
{
$run++;
}
}
}
else
{
$run = 0;
for($x=0;$x<4000;++$x) {
// Insert
if( $db->query('INSERT INTO pdo_insert (`name`, `body`, `author_id`)
VALUES ('.$db->quote($name).', '. $db->quote($body).', '. $db->quote($x).')')
AND
// Select
$db->query('SELECT * FROM pdo_insert WHERE id = '. $db->quote($x)) )
{
$run++;
}
}
}
print (microtime(true) - $start).' seconds and '.($run * 2).' queries';
Cassy is right. If you don't prepare/compile it, the dbms would have to in any case before able to run it.
Also, the advantage is you could check the prepare result and if prepare fail your algo can branch off to treat an exception without wasting db resources to run the failing query.