I'm currently developing an application in PHP which uses PDO. I'm writing an import which reads in a CSV file, checks the database for a record, and updates, deletes, etc....
Something which I've noticed is the memory being used by this script seems very high and it seems like it could be to do with the way I'm executing the query. see below for example query which is executed for each line in the CSV:
$qry = "SELECT * FROM company WHERE id = 1";
$sth = $this->prepare($qry);
$sth->execute();
$sth->setFetchMode(PDO::FETCH_INTO, new Company());
$sth->fetch();
for the above memory_get_peak_usage() = 6291456
When using the below:
$qry = "SELECT * FROM company WHERE id = 1";
$sth = $this->prepare($qry);
$sth->execute();
$sth->setFetchMode(PDO::FETCH_CLASS, "Company");
$sth->fetch();
for the above memory_get_peak_usage() = 524288
As you can see the difference is fairly big.
I guess I've 3 questions..
Is there a memory leak when using PDO::FETCH_OBJ in PHP 5.3.5?
Is there any difference between using FETCH_CLASS as opposed to FETCH_OBJ?
Has anyone else experienced the same issue?
Company Class is simple:
class Company {
function __construct(){}
/**classvars**/
public $_tablename = 'company';
public $transient;
public $id;
public $name;
/**endclassvars**/
}
Looking at the PHP changelog, there does appear to be a relevant fix in 5.3.4 where a memory leak was fixed in PDO FETCH_INTO.
From what you've said, I suspect that yes, this is the bug you're seeing. The solution, of course, is to upgrade -- there really is no point in sticking with an old patch release.
Even if this isn't the bug you're seeing, there have been a very large number of PDO fixes in the versions between 5.3.3 and now; I'm sure there's a good chance that at least some of them are relevant to you.
Note: the original answer was given before the OP changed PDO::FETCH_OBJ to PDO::FETCH_INTO
After that update I've tried to reproduce the behaviour using PHP 5.3.10-1ubuntu3.4. There where no significant difference in memory cosumption between both fetch modes. I've tested using a large MySQL table and a large SQLite database.
As #SDC mentioned the bug is known and was fixed after 5.3.5. (At least in 5.3.10 as I've seen).
Conclusion: You have to upgrade your PHP version.
Although the behaviour is interesting and should be investigated you are using PDO::setFetchMode() in a wrong way. When $mode - the first param - is PDO::FETCH_OBJ no second param is expected. If you use a second param the call to setFetchMode() will fail (returnin false) and the default fetch mode FETCH_BOTH will be used.
You can see this error when enabling PDO::ERRMODE_EXCEPTION :
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$stmt = $db->query('....');
// the following line will trigger an exception
$stmt->setFetchMode(PDO::FETCH_OBJ, new Company());
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: fetch mode doesn't allow any extra arguments'
When you are expecting that result rows should objects of a specific class, then PDO::FETCH_CLASS is the working attempt. PDO::FETCH_OBJ will return objects from type StdClass
FETCH_INTO: Means fetching into an existing Object (that was created with new e.g.)
FETCH_CLASS: Means fetching into an new Object (the constructor is called every row)
Be careful, if the constructor in your Company class has dependencies, they are called for every row. Therefore the constructor should not contain functions or classes that do an DB connect e.g. only simple initializing...
How does your Company class look like?
Related
I'm working with drupal 7, on PHP 7, on Xampp on Windows, and suddenly I start getting the following error:
Call to undefined method DatabaseStatementBase::setFetchMode()
Where DatabaseStatementBase extends PDOStatement directly. When reducing the code to the following minimum:
<?php
$dbh = new PDO('mysql:host=localhost;dbname=test', 'test', 'test');
$pdostatement = $dbh->prepare('SELECT * FROM items WHERE id=?');
$pdostatement->setFetchMode(PDO::FETCH_CLASS);
$success = $pdostatement->execute([1]);
// do stuff...
It still throws an error on the line regarding setFetchMode. When I comment that line out, no error is thrown, but I get an associative array instead of an object — not what drupal expects. Especially since setFetchMode should exist (see http://php.net/manual/en/pdostatement.setfetchmode.php)
Finally, when I then try to find the methods of the $pdostatement using reflection, I get garbage for some of the names — or, more accurately, the name seems about 1.5MB long and contains a lot of unreadable characters and some of the method names, as if an entire DLL was loaded in there or something. Here's an example of what var_dump (php7 & xdebug) make of it:
object(ReflectionMethod)[17]
public 'name' => string '����&������p�aZ������������ ���bindParam�������{�nZ���������������setAttribute����f�kZ����������j����FETCH_ORI_FIRST�a�pZ���������q��
���CURSOR_SCROLL���l�}Z���������������fetchColumn������zZ��������������wph�����&��������Z���������������debugDumpParams���Z��������.�����children����������Z������������wphX����&��������Z��������(��
���nextrowset��������Z������������
���__toString������ ��Z������������wph(����&������4��Z��������'... (length=1752201104)
public 'class' => string 'PDOStatement' (length=12)
How can I fix this?
The solution was: try turning it off and on again — in this case the Apache server. Apparently something got corrupted in memory, and restarting the server fixed it.
I assumed this was already done by me shutting down my computer yesterday, after the problem appeared, and booting it again this morning. Now I know: windows 8 and up use a hybrid shutdown/hibernate instead of a real shutdown, so your apache service doesn't really get restarted if you think you restarted your PC.
Somehow this makes me feel so dumb...
I'm writing a library which requires a database connection, so it requires users to pass a PDO object as a parameter of the constructor.
But as many of you know, PDO has 3 different error reporting mechanisms: Silent mode, warning mode and exception mode (http://www.php.net/manual/en/pdo.error-handling.php).
The problem is that each mode requires a different type of error handling code. Accepting that I can't force the user to use my preferred mode, i think that my options are:
Setting my preferred mode on the inputted PDO object, and worrying about nothing
Cloning the inputted PDO object and then setting my preferred mode
Writing error handling code for all 3 modes and then detecting and employing the appropriate one with the help of PDO::getAttribute() method
Setting my preferred mode before each method call and then revoking it after each one
So 1 can break user's code, 2 looks like unnecessary duplication, 3 is terribly awkward, and 4 is not-so-terribly-awkward-but-still-awkward and still susceptible to breaking user's code.
So i'm asking to library writers out there, how do you handle this?
Ok, not sure if this is the right site for this sort of question, but first, let me give you the short run-down of your options, and tell you what IMO is the better option. Then I'll explain all of it in future edits.
That's not the way forward. The user is passing an instance, which you then change, behind the users' back. Don't change what isn't yours
Don't clone a DB connection. Just don't
A library shouldn't deal with errors that are the result of the user's code. That's the user's problem/fault, they should then also deal with it. No way a lib can anticipate on every possible abuse
No, really... This is just a silent way of doing what you're doing in point 1: changing an object you never really owned.
What, then, would I do? Simple: Provide an API for DB connections that could, in its heart have a PDO instance, but then at least the user has a clear API, and knows what the result of possible errors are (for example PDO + setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION) => your API will always throw exceptions
Now, why is your first option not a viable approach? (again: this is all my opinion)
Suppose I were to use your code, and have something like this:
//code
$this->db = new PDO($dsn, $usr, $pass, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_SILENT));
$this->dependency = new Your\Lib\Stuff($this->db);
//code
$this->db->query('bad query');
Now if you set PDO to throw exceptions, I'm not catching any. My code wasn't writtin in a way to deal with PDOException instances, so it'll cause the entire app to grind to a halt. What's more, who's to say that I'm not going to add this line, while debugging, when that happens:
$this->dependency = new Your\Lib\Stuff($this->db);
$this->db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_SILENT);//override your setting
This isn't a safe path to go down, so don't.
Next: cloning PDO just won't work. If you have a silly person who wrote his own class to extend from PDO, this is possible:
class BadIdea extends PDO
{
public function __clone()
{//disable clone
return false;
}
}
In this case $db = new BadIdea() will still pass as an instance of PDO (test function foo(PDO $arg){echo 'argument is instance of PDO';} and then call foo(new BadIdea), it will work). Now you can't clone, and your lib fails.
A lib or framework should be written generically enough so you can reuse it. If you're writing the code, while thinking of a particular use-case in mind, you're probably going find yourself editing the code every time you wish to use it again.
Unless you're going to be creating your own lib (in which case the user wouldn't have to pass a self-made DB connection anyway), error handling shouldn't be the task of your lib.
Your code should just throw exceptions when something happens that is unexpected. You can't be expected to write code that deals with all of these situations:
$yourInstance->pass('Invalid query');
$yourInstance->select('INSERT ...');//valid query, wrong method
$your instance->query(array('invalid', 'argument'));
Or worse:
try
{
$yourInstance->beginTransaction();
$yourInstance->query($q1);
$q2 = $anotherObj->composeComplexQuery();//might be the cause of Exceptions, too
$yourInstance->query($q2);
$yourInstance->commit();
} catch(){}
When you deal with errors inside of your lib, then how will your user be able to deal with situations like these? a transaction is a concious decision made by the user, any exception that might occur during a transaction, be it an error in the queries or an exception thrown by third-party methods can result in the transaction having to be rolled back.
Your lib's scope isn't wide enough to pick up on the exceptions that are thrown in the user's code.
I'm doing my first own database class at the moment and currently I'm doing the prepare function.
What this function does is to take in an SQL-query and then an array containing the variables for the statement. I'm having problems with binding the parameters to the statement.
This is how the function looks like now
public function prepare($query, $var) {
$types = '';
foreach($var as $a) {
$type = gettype($a);
switch($type) {
case 'integer':
$types .= 'i';
break;
case 'string':
$types .= 's';
break;
default:
exit('Invalid type: ' . $a .' ' . $type . '(' . count($a) . ')');
break;
}
}
$stmt = self::$connection->prepare($query);
$stmt->bind_param($types, $var); // How do I do here??
$stmt->execute();
$result = $stmt->get_result();
while($row = $result->fetch_assoc()) {
print_r($row);
}
}
Everything works as I want it to (I know this function could do some polishing but it does what it needs to do). I have commented the line where I'm having trouble figuring out what to do. $var is an array and if I recall things correctly the variables needs to be passed seperately seperated with a comma. This is where I'm clueless.
The very idea of your own database class is great.
Very few people here do share it, for some reason prefers raw api calls all over their code.
So, you're taking great step further.
However, here are some objections:
Don't use mysqli for your first own database class if you're going to use native prepared statements.
Use PDO instead. It will save you a ton of headaches.
Despite of the fact this function works all right for you, it makes not much sense:
switch($type) code block is useless. Mysql can understand every scalar value as a string - so you can bind every value as s with no problem.
most integers coming from the client side have string type anyway.
there are some legitimate types like float or NULL or object that can return a string. So, automation won't work here. If you want to distinguish different types, you have to implement type hinted placeholders instead.
Never use exit in your scripts. throw new Exception('put here your error message') instead.
This is not actually a prepare function as it does execute as well. So, give it more generic name
But now to your problem
It is direct consequence of using mysqli. It is a nightmare when dealing with prepared statements. Not even only with binding but with retrieving your data as well (because get_result() works not everywhere, and after creating your application locally you will find it doesn't work on the shared hosting). You can make yourself an idea looking at this bunch of code served for this very purpose - to bind dynamical number of variables.
So, just keep away from mysqli as far as as you could.
With PDO your function will be as simple as this
public function query($query, $var=array())
{
$stmt = self::$connection->prepare($query);
$stmt->execute($var);
return $stmt->fetchAll(PDO::FETCH_ASSOC);
}
// and then used
$data = $db->("SELECT 1");
print_r($data);
You can take a look at my class to get some ideas.
Feel free to ask any questions regarding database classes - it's great thing, and I am glad you're going this way.
To answer questions from the comments.
To let you know, you're not the only user of the site. There are also some innocent visitors. Unlike you, they don't need no error messages, and they get scared with some strange behavior and lack of familiar controls.
exit() with error message does many evil things
throws an error message out, revealing some system internals to a potential attacker
scaring innocent user with strange message. "What's that? Who is invalid? Is it mine fault or what? Or may be it's a virus? Better leave the site at all" - thinks them.
killing the script in the middle, so it may cause torn design (or no design at all) shown
killing the script irrecoverably. while thrown exception can be caught and gracefully handled
When connecting to PDO, no need to throw anything here as the exception already thrown by PDO. So, get rid of try ... catch and just leave it one line:
self::$connection = new PDO($dsn, $user, $pass);
then create a custom exception handler to work in 2 modes:
on a development server let it throw the message on the screen.
on a live server let it log the error while showing generic error page to the user
Use try ... catch only if you don't want to whole script die - i.e. to handle recoverable issue only.
By the way, PDO don't throw exception on connect by default. You have to set it manually:
$opt = array( PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION );
self::$connection = new PDO($dsn, $user, $pass, $opt);
Similar questions to this my have been asked a lot of times before. But since I did not find a solution in any of the questions asked before me, I take the liberty of asking it again.
My program uses a class made by me which handles all database connections for the program. Several modules I've used before used the same class without fail but when I chose to do a new module using the same class, the warning shows as-
Warning: mysql_query(): 7 is not a valid MySQL-Link resource in wherever... on line 49.
The warning happens when I execute a MySQL query through a function I made. The function is as follows-
public function runquery($_query)
{
$result = mysql_query($_query,$this -> connection); //line 49
if (! $result) die(mysql_error());
else return $result;
}
The function belongs to a class named mysql and it has not been tampered with or made changes to. So the function should technically work as expected, as every other module relying on the same class for database connectivity works just fine.
The query execution is successful however and I manage to update tables with no problems (except the warning). The block of code in the main program where the runquery() function is called from is as follows-
$phpmyadmin = new mysql();
$phpmyadmin->connect('localhost', 'root', '');
$phpmyadmin->setdb('test_db');
$result = $phpmyadmin->runquery($Query);
unset($phpmyadmin);
So the mysql's functions work just as fine as ever and the query executes just fine. But the warning shows for a reason I cannot understand. Any help?
The symptoms suggest that the database connection has been closed or dropped. Look for unwanted mysql_close() calls in your code. Additionally, you can use the following functions to troubleshoot the issue:
is_resource() and get_resource_type() to confirm that $this->connection is a valid data type.
mysql_ping() to find out if the database connection is alive.
If it's a rare issue, log stuff into a file and wait until it happens again.
There should not be spaces.
$result = mysql_query($_query,$this->connection);
I'm building a newsletter CMS and I want to loog any errors to a database with information like timestamp, error, userID and function info. What's the best way to do this?
I'm thinking I should build a class that handles the errors and inputs them into a MYsql table using PDO.
pseudo code:
class sbmtErr
{
private $errStrng;
protected function sndErr($err, $usrData, $mthd){
$sndErrPDO = new myPDO;
$this->errStrng = "INSERT INTO errTbl (error, userID, method) VALUES ('".$err."', ".usrData['usrID'].", '".$mthd."')";
$sndErrPDO->sqlQry($errStrng);
}
}
My problem here is I don't know how to isolate the method that threw the error.
Is there a better way to do this?
Thanks.
Extend the exception class. CMSErrorException , when thrown (i.e. On construct) use reflection to find out things like function, line number, etc. how you find the user's id depends on how you keep it.
Then, regardless of what you do when you catch it, you call a method (again from the construct) which logs it into the database.
Careful not to throw these custom exceptions inside of the exception code, as it might cause an infinite loop in case of a database error.