I've created a basic class called Content for fetching content from a MySQL database, with a constructor like so:
$this->handle = new PDO('mysql:host=localhost;dbname=test', $user, $pass);
and some methods utilizing this handle, chiefly one that fetches data and puts it into an array:
function select($query) {
$sth = $this->handle->prepare($query);
$sth->execute();
$result = $sth->fetchAll();
$sth->closeCursor();
return $result;
}
Nothing fancy, really. And it works terrific. Except, after a dozen or so various queries, fetchAll all of a sudden returns an empty result. When I run the same query directly on the database, I do get a result. Also, when I run the query before running the other ones, I get a result as well. Even more oddly, the fetchAll at this point in my script used to work fine, but when the queries that preceded it got more numerous, it started returning empty results.
Based on these findings, I assumed the scripting and database where just fine, and it most likely had something to do with PDO. Eventually, I 'fixed' it by creating a second object from my Content class in order to fetch the data I was missing:
$contentTwo = new Content();
Of course, this should not be necessary! I'm looking for a real solution, and also an understanding as to what's going on. I've turned on all sorts of error reporting (PHP, PDO, MySQL) but there are no errors. Could it be some internal PDO mixup? Is there a maximum amount of prepared statements? Any ideas/solutions would be appreciated.
Could it be some internal PDO mixup?
No.
Is there a maximum amount of prepared statements?
No.
Any ideas/solutions would be appreciated.
Yes.
The problem you faced with is not something unusual.
Every developer has to deal with such a case once or twice a day. This is quite normal for our profession, when something goes wrong. A program is so complex a facility that sometimes it goes wrong. It is a duty of a programmer to find out and to fix the error.
So, you have to investigate the cause.
For such a purpose create a comprehensive logging system, and log everything you can hold of in case of an empty result returned. State of PDO object, input data, all involved variables, etc.
This way you most likely will be able to spot the problem
Related
There is a long living large project in PHP, in the development of which at different times were involved a number of different developers. Potentially, there may have been places where duplicate database queries were made during bug fixing. Some developer to solve a local problem could write and execute a query returning data that were previously obtained. Now on the project there is a problem of DB performance and in light of it a have a question:
Are there any tools (besides General Log) that allow you to see what database queries were made as part of a single PHP script execution?
The question is related to the fact that there are a lot of API endpoints in the project and it will take a long time to check them all just by reading the code (which is sometimes very ornate).
Most PHP frameworks use a single connection interface - some kind of wrapper - that makes it possible to log all queries through that interface.
If that does not exist another approach would be to enable logging at the MySQL-level. But for some, probably legit, reason you don't want to do that either. If you want to prevent downtime you can actually enable query logging without restarting the MySQL server.
If none of the above solutions is possible you have to get your hands dirty :-)
One way could be to add your own logging in the PHP code.
Writing new lines to a file is not a performance issue if you write to the end of the file.
I'm not sure how you query the database, but if you..
A) ..use the procedural mysqli functions
If you use the procedural mysqli functions like mysqli_query() to call the database you could create your own global custom function which writes to the log file and then call the real function.
An example:
Add the new global function:
function _mysqli_query($link, $query, $resultmode = MYSQLI_STORE_RESULT )
{
// write new line to end of log-file
file_put_contents('queries.log', $query . "\n", FILE_APPEND);
// call real mysqli_query
return mysqli_query($link, $query, $resultmode);
}
The do a project-wide search and replace for mysqli_query( with _mysqli_query(.
Then a line like this:
$result = mysqli_query($conn, 'select * from users');
Would look like this after the search and replace:
$result = _mysqli_query($conn, 'select * from users');
B) ..use the mysqli class (object oriented style)
If you use the object oriented style by instantiation the mysqli class and then calling the query methods e.g. $mysqli->query("select * from users") an approach could be to create your own database class which extends the mysqli class, and inside the specific methods (e.g. the query function) you add the logging like in the above example.
In the highly recommendable book "High Performance MySQL" from O'Reilly they go through code examples how to do this and add extra debug information (timing etc.). Some of the pages are accessible at google books.
In general
I would consider using this as an opportunity to refactor some of the codebase to the better.
If you have written automated tests that is a good way to ensure that the refactoring did not break anything.
If you do not practice automated testing you have to test the whole application manually.
You said that you are afraid that your performance issues comes from duplicate database queries, which might very well be the case.
In most cases I find it to be missing indexes though or a need to rewrite single queries.
The performance schema in MySQL can help you debug these things if you find it realistic that it could be other causes to the performance issues.
This is something I have been trying to figure out for a bit, it is the most simplest of queries that does not seem to want to work for me (only in php mysqli, works in console sql)
First I am using a prepared statement, merely looking for a match on a specialized id (from another service) to update the relation to use my primary key for easier searching on my end.
The query is as follows:
$query = "SELECT id
FROM {$this->config->dbprefix}{$table}
WHERE sf_id = ?
LIMIT 1";
I use this as one line, I split it up for better readability here
I then check that the prepare statement is valid (I do this in multiple places and it works everywhere else.
if(!($ret = $this->dbo->prepare($query))){
//handle error, this part is never called
}else{
//everything is fine, code in here is below
}
Up to here everything seems fine. Checking table and prefix manually shows they are working and referencing the proper table.
$ret->bind_param('s',$id);
$ret->execute();
$ret->bind_result($retId);
$ret->fetch();
$count = $ret->num_rows;
The problem is here, the query always returns 0 for the num_rows. Checking the query manually and trying it in console returns 1 result as it should. So far with it being such a simple query I just cannot wrap my head around why it would work elsewhere, but not here. I am sure this is the proper way to build it (I have many other queries structured similar).
Is there any kind of confusion I may be experiencing? Something easy to miss that could cause a query like this to not return results?
EDIT:
I have attempted further error handling, and trying an if test on execute does not trigger an error, though I will try more.
To expand I have a raw output of the $query variable and the id variable. By combining them and manually attempting the query in console I get the proper result. My thoughts are on somehow the prepare statement is escaping, causing the string variable $id to not properly match. Though that is just speculation.
You need to call store_result() before trying to access num_rows. Without it, the statement handle does not know how many rows are in the result set.
$ret->bind_param('s',$id);
$ret->execute();
$ret->bind_result($retId);
$ret->store_result();
$count = $ret->num_rows;
// perhaps add error handling based on number of rows here
$ret->fetch();
I'm a bit obsessed now. I'm writing a PHP-MYSQL web application, using PDO, that have to execute a lot of queries. Actually, every time i execute a query, i also check if that query gone bad or good. But recently i thought that there's no reason for it, and that's it is a wast of line to keep checking for an error.
Why should a query go wrong when your database connection is established and you are sure that your database is fine and has all the needed table and columns?
You're absolutely right and you're following the correct way.
In correct circumstances there should be no invalid queries at all. Each query should be valid with any possible input value.
But something still can happen:
You can lose the connection during the query
Table can be broken
...
So I offer you to change PDO mode to throw exception on errors and write one global handler which will catch this kind of errors and output some kind of sorry-page (+ add a line to a log file with some details)
I'm sorry, this is a very general question but I will try to narrow it down.
I'm new to this whole transaction thing in MySQL/PHP but it seems pretty simple. I'm just using mysql not mysqli or PDO. I have a script that seems to be rolling back some queries but not others. This is uncharted territory for me so I have no idea what is going on.
I start the transaction with mysql_query('START TRANSACTION;'), which I understand disables autocommit at the same time. Then I have a lot of complex code and whenever I do a query it is something like this mysql_query($sql) or $error = "Oh noes!". Then periodically I have a function called error_check() which checks if $error is not empty and if it isn't I do mysql_query('ROLLBACK;') and die($error). Later on in the code I have mysql_query('COMMIT;'). But if I do two queries and then purposely throw an error, I mean just set $error = something, it looks like the first query rolls back but the second one doesn't.
What could be going wrong? Are there some gotchas with transactions I don't know about? I don't have a good understanding of how these transactions start and stop especially when you mix PHP into it...
EDIT:
My example was overly simplified I actually have at least two transactions doing INSERT, UPDATE or DELETE on separate tables. But before I execute each of those statements I backup the rows in corresponding "history" tables to allow undoing. It looks like the manipulation of the main tables gets rolled back but entries in the history tables remain.
EDIT2:
Doh! As I finished typing the previous edit it dawned on me...there must be something wrong with those particular tables...for some reason they were all set as MyISAM.
Note to self: Make sure all the tables use transaction-supporting engines. Dummy.
I'd recommend using the mysqli or PDO functions rather than mysql, as they offer some worthwhile improvements—especially the use of prepared statements.
Without seeing your code, it is difficult to determine where the problem lies. Given that you say your code is complex, it is likely that the problem lies with your code rather than MySQL transactions.
Have you tried creating some standalone test scripts? Perhaps you could isolate the SQL statements from your application, and create a simple script which simply runs them in series. If that works, you have narrowed down the source of the problem. You can echo the SQL statements from your application to get the running order.
You could also try testing the same sequence of SQL statements from the MySQL client, or through PHPMyAdmin.
Are your history tables in the same database?
Mysql transactions only work using the mysqli API (not the classic methods). I have been using transactions. All I do is deactivate autocommit and run my SQL statements.
$mysqli->autocommit(FALSE);
SELECT, INSERT, DELETE all are supported. as long as Im using the same mysqli handle to call these statements, they are within the transaction wrapper. nobody outside (not using the same mysqli handle) will see any data that you write/delete using INSERT/DELETE as long as the transaction is still open. So its critical you make sure every SQL statement is fired with that handle. Once the transaction is committed, data is made available to other db connections.
$mysqli->commit();
This sounds like a really simple question, but I am new to PHP. If I have a statement like this:
$r =& $db->query("insert into table (col1, col2) values (10, 20)");
Do I have to still execute it, or does it get executed when I reference it? I have another case where I have a select query, which seems logically to run only when I call fetchrow, but the code I am copying from does not call execute or fetch. I would have expected it to, so I cannot tell if it is just that I don't get it, or that the missing execute statement is the problem. It also does not insert the record, but it does not throw an error I can find.
Also, I am a little confused by the =& notation. I looked it up on google, and found a few mentions of it, but I am still not clear on it.
Thanks.
It will be executed when you call query()
The =& notation is obsolete... it used to make the function return a reference to the resource object. But current versions of PHP (>5.0, I think) always pass (and return) objects by reference, so it doesn't really change anything anymore.
The query gets executed when you call the query function. when you talk about code that needs to be fixed, what is broken, and what does the code that "need[s] to be fixed" (according to who?) look like?
& is used in several contexts and it means by reference. You should start reading from here:
http://es.php.net/manual/en/language.operators.assignment.php
http://es.php.net/manual/en/language.references.return.php
In your code snippet it's most likely unnecessary (although you give little clue about what $db is) because the result set is probably an object and objects no longer need to be assigned by reference since that's the default behaviour. If you are learning PHP, be careful with outdated tutorials.
$db->query is a method contained by a class, it's not functional in php out of context. It is possible that this example of yours comes from a larger application that uses a database abstraction layer like ADODB or any of its kind.
If this is the case, then you could refer to the documentation specific to that db abstraction layer, because the query could be contained in a transaction for example and it would not be executed as soon as you call query();
To be sure the query is executed immediately try testing with a simple mysql function:
mysql_query("SHOW TABLES");