Optimization issues in multitier architecture w/ PDO queries - php

I've been working on a PHP project for about 6 month where I'm using a multitier architecture (Data Access Layer , Business Logic Layer, ...).
One of the page of my website has a really long loading time, so I investigated and I realised it was the function from a BLL file that took a longer time to process.
Here's a example of a function in my DAL (here called DAL_Releve) files :
public static function getReleveByAffectationID($id){
$conn = MgtConnexion::getConnexion();
$sql = "SELECT *"
. "FROM releve "
. "WHERE Affectation_ID = :id "
. "AND Date_Releve >= :date ;";
$req = $conn->prepare($sql);
$req->bindValue(":id", $id);
$req->bindValue(":date", $_SESSION["year"]-1 ."-01-01");
$req->execute();
MgtConnexion::closeConnexion($conn);
return $req;
}
Here's a example of a function in my BLL (here called MgtReleve) files :
public static function getReleveByAffectationID($id){
$req = DAL_Releve::getReleveByAffectationID($id);
$releveArray = array();
while($row = $req->fetch(PDO::FETCH_ASSOC)){
$releveArray[] = new releve($row);
}
return $releveArray;
}
As you can see, I return the PDOStatement object $req from the DAL and then fetch it in the BLL.
My problem is here : In my interface file, I'm looping through a array of Affectation and each of them have Releves (so each Releves have an attribute Affectation_ID), like this:
$lesAff = MgtAffectation::getAllAffectation();
//This doesn't return an array of objects but just an Array of arrays like : $array[<number>]["key"] = value;
foreach ($lesAff as $affect){
$lesReleves = MgtReleve::getReleveByAffectationID($affect["Affectation_ID"]);
foreach ($lesReleves as $rel){
//DO STUFF HERE
}
}
FYI : The page loads in 6.622 seconds with the code above and in 0.014 seconds if I remove everything inside the first foreach (except for html code which is not showed here).
Now I'm starting to question my understanding of the multitiers architecture :
If I decide not to fetch $req in my BLL file but in my interface (to loop only once in my queries results instead of twice), what's the purpose of the BLL then if only to return the PDOStatement object? Is there another way to return the data with more efficiency ?

There are many areas of optimization:
Technological optimization: instead of connecting to the database every time you are running a query, you should connect only once and then use this connection through the entire application. I wrote an article on the common errors of Database wrappers, you may find it interesting to read, and especially the part on the database connection.
Architectural optimization. It seems that you could benefit from lazy loading when instead of selecting database rows one by one you could do it in batches. Check this answer on how to run a PDO query with multiple ids at once. So it will take you to run only 68 queries instead of 1200
Database optimization. Make sure that all queries involved are running fast.

Related

PHP OOP Efficient DB Read Reduction

Six years ago I started a new PHP OOP project without having any experience so I just made it up as I went along. Anyway, I noticed that my rather powerful mySQL server sometimes gets bogged down far too easily and was wondering what the best way to limit some db activity, when I came up with this, as an example...
private $q_enghours;
public function gEngHours() {
if ( isset($this->q_enghours) ) {
} else {
$q = "SELECT q_eh FROM " . quQUOTES . " WHERE id = " . $this->id;
if ($r = $this->_dblink->query($q)) {
$row = $r->fetch_row();
$r->free();
$this->q_enghours = $row[0];
}
else {
$this->q_enghours = 0;
}
}
return $this->q_enghours;
}
This seems like it should be effective in greatly reducing the necessary reads to the db. If the object property is populated, no need to access the db. Note that there are almost two dozen classes all with the same db access routines for the "getter". I've only implemented this change in one place and was wondering if there is a "best practice" for this that I may have missed before I re-write all the classes.
I'd say that this question is rather based on wrong premises.
If you want to deal with "easily bogged down" database, then you have to dig up the particular reason, instead of just making guesses. These trifle reads you are so concerned with, in reality won't make any difference. You have to profile your whole application and find the real cause.
If you want to reduce number of reads, then make your object to map certain database record, by reading that record and populating all the properties once, at object creation. Constructors are made for it.
As a side note, you really need a good database wrapper, just to reduce the amount of code you have to write for each database call, so, this code can be written as
public function gEngHours() {
if ( !isset($this->q_enghours) ) {
$this->q_enghours = $this->db->getOne("SELECT q_eh FROM ?n WHERE id = ?", quQUOTES, $this->id);
}
return $this->q_enghours;
}
where getOne() method is doing all the job of running the query, fetching row, getting first result from it and many other thinks like proper error handling and making query safe.

Escape SQL queries in PHP PostgreSQL

I have a website with lots of PHP files (really a lot...), which use the pg_query and pg_exec functions which do not
escape the apostrophe in Postgre SQL queries.
However, for security reasons and the ability to store names with
apostrophe in my database I want to add an escaping mechanism for my database input. A possible solution is to go
through every PHP file and change the pg_query and pg_exec to use pg_query_params but it is both time consuming
and error prone. A good idea would be to somehow override the pg_query and pg_exec to wrapper functions that would
do the escaping without having to change any PHP file but in this case I guess I will have to change PHP function
definitions and recompile it which is not very ideal.
So, the question is open and any ideas that would
allow to do what I want with minimum time consumption are very welcome.
You post no code but I guess you have this:
$name = "O'Brian";
$result = pg_query($conn, "SELECT id FROM customer WHERE name='{$name}'");
... and you'd need to have this:
$name = "O'Brian";
$result = pg_query_params($conn, 'SELECT id FROM customer WHERE name=$1', array($name));
... but you think the task will consume an unreasonable amount of time.
While it's certainly complex, what alternatives do you have? You cannot override pg_query() but it'd be extremely simple to search and replace it for my_pg_query(). And now what? Your custom function will just see strings:
SELECT id FROM customer WHERE name='O'Brian'
SELECT id FROM customer WHERE name='foo' OR '1'='1'
Even if you manage to implement a bug-free SQL parser:
It won't work reliably with invalid SQL.
It won't be able to determine whether the query is the product of intentional SQL injection.
Just take it easy and fix queries one by one. It'll take time but possibly not as much as you think. Your app will be increasingly better as you progress.
This is a perfect example of when a database layer and associated API will save you loads of time. A good solution would be to make a DB class as a singleton, which you can instantiate from anywhere in your app. A simple set of wrapper functions will allow you to make all queries to the DB go through one point, so you can then alter the way they work very easily. You can also change from one DB to another, or from one DB vendor to another without touching the rest of the app.
The problem you are having with escaping is properly solved by using the PDO interface, instead of functions like pg_query(), which makes escaping unnecessary. Seeing as you'll have to alter everywhere in your app that uses the DB, you may as well refactor to use this pattern at the same time as it'll be the same amount of work.
class db_wrapper {
// Singleton stuff
private $instance;
private function __construct() {
// Connect to DB and store connection somewhere
}
public static function get_db() {
if (isset($instance)) {
return $instance;
}
return $instance = new db_wrapper();
}
// Public API
public function query($sql, array $vars) {
// Use PDO to connect to database and execute query
}
}
// Other parts of your app look like this:
function do_something() {
$db = db_wrapper::get_db();
$sql = "SELECT * FROM table1 WHERE column = :name";
$params = array('name' => 'valuename');
$result = $db->query($sql, $params);
// Use $result for something.
}

Yii sql query bindings, with params, do not work properly

I'm new to Yii and everything seems good, but the problem is, when I`m using the binding params, like (DAO stile):
$command = $this->conn->createCommand($sql);
$command->bindColumn("title", "test_title");
$result = $command->query();
or (Active Record):
$row = Movies::model()->find("m_id=:m_id", array(":m_id"=>27));
or
$row = Movies::model()->findByPk(24);
I've tried everything:
1) added a config param to mysql config. in main.php - 'enableParamLogging' => true
2) changed strings from ' to "
3) added another param just in case of mysql versions - 'emulatePrepare'=>true
Nothing works for me.
I thought that the problem is in params, but bindColumn method doesn't use it, so my presumption is that some module of Yii hasn't been include in config file or something like that.
My model looks like this (created in /models dir):
class Movies extends CActiveRecord {
public static function model($className = __CLASS__) {
parent::model($className);
}
}
Just for everybody to avoid unnecessary questions: the database is configured properly in main.php conf file, there is a table movies, there is a PKs 24 and 27 also.
All native SQL works fine, except using in DAO special methods to bind some params and if in AR using findByPk or find. I hope that this is clear, guys don't bother me with obvious simple technical possibilities, that I can (as U presume) did wrong.
PS Another helpful info - when calling
$command->bindColumn("title", "test_title");
FW's throwing an Exception - CDbCommand and its behaviors do not have a method or closure named "bindColumn". So, as mentioned above, I think that Yii don't see those special methods and this is certain. How can I repair it?
Ok, why this code don't work either? There is no Exceptions, just blank page.
$sql = "SELECT title, year_made FROM movies WHERE year_made=':ym'";
$command = $this->conn->createCommand($sql);
$command->bindParam(":ym", "2012", PDO::PARAM_STR);
$result = $command->query();
The DAO binding isn't working because there is no bindColumn. You only have bindParam, that binds a variable to a column, or bindValue, that binds a value.
I don't know what's wrong with the AR query though.
Edit: Your DAO code should look like this:
$sql = "SELECT * FROM sometable WHERE title = :title";
$command = $this->conn->createCommand($sql);
$command->bindParam(":title", $tile_var);
$result = $command->query();
Have you generated movie model by Gii generator or you wrote it by yourself?
Edit:
As I know you have to add tableName
public function tableName() {
return 'movies';
}
You may have two separate issues. At least your DAO code (if that is what you actually have in your code) is binding wrong.
Binding needs to occur before the query is done (and it's done on the command object), not on the result object. See more info here: http://www.yiiframework.com/doc/guide/1.1/en/database.dao#binding-parameters
As far as your AR queries go, those could also be problematic. You can use findByAttributes instead of this line (although your line should work):
$row = Movies::model()->find("m_id=:m_id", array(":m_id"=>27));
The below should work if you have a standard id key. If you don't (and are using m_id, have you set your model's primaryKey() function up?
$row = Movies::model()->findByPk(24);
Also, you'll be getting a model object back, not a row if that changes how you are handling the results ...

Is it better to cache to cache the object in the controller or the query in the method?

At the moment I'm using php and apc in my mvc based site, it's a custom built simple mvc so I'm trying to build it to fit my needs.
However I'm unsure where is the preferred location in which to handle caching?
I have two choices (I think) .. either do all caching in the various controllers which would mean objects were stored in the cache, or store returned data from queries in the cache inside the method:
Controller Example:
function showPage() {
$pageOb = new Page();
$key = md5('pageOb->getQuery()');
if(!$data = apc_fetch($key)){
$data = $pageOb->getQuery();
apc_add($key, $data, 600);
}
require 'template.php';
}
Method Example:
function getQuery(){
$sql = "SELECT * FROM table";
$key = md5('query'.$sql);
if(!$data = apc_fetch($key)){
$core = Connect::getInstance();
$stmt = $core->dbh->prepare($sql);
$stmt->execute();
$data = $stmt->fetchAll();
apc_add($key, $data, 600);
}
return $data;
}
It kinda depends on how you understand and implement the Model layer. This would be how I would write the cache-related code in the Service level objects:
$user = $this->domainObjectFactory->build('User');
$user->setId(42);
if ( !$this->cacheMapper->fetch('User', $user) )
{
$mapper = $this->mapperFactory->build('User');
$mapper->fetch($user);
}
If you do not understand the terms this comment (skip to "side notes") might help. It would take too long to repeat the whole thing again.
As I understand it, the cache itself is just a different form of storage. And thus it is just another part of Data Source Layer(where mappers, DAOs and similar structures come from).
You shouldn't have data-model concerns bubbling up into your Controller. This principle is encapsulated in SRP: http://en.wikipedia.org/wiki/Single_responsibility_principle
Your second solution is better, but it would be improved by further abstracting the retrieval of data from the source of data. Here is a good reference article on the subject, although the language being used is different the patterns still hold: http://www.alachisoft.com/resources/articles/domain-objects-caching-pattern.html

Code coverage with phpunit; can't get to one place

In the xdebug code coverage, it shows the line "return false;" (below "!$r") as not covered by my tests. But, the $sql is basically hard-coded. How do I get coverage on that? Do I overwrite "$table" somehow? Or kill the database server for this part of the test?
I guess this is probably telling me I'm not writing my model very well, right? Because I can't test it well. How can I write this better?
Since this one line is not covered, the whole method is not covered and the reports are off.
I'm fairly new to phpunit. Thanks.
public function retrieve_all()
{
$table = $this->tablename();
$sql = "SELECT t.* FROM `{$table}` as t";
$r = dbq ( $sql, 'read' );
if(!$r)
{
return false;
}
$ret = array ();
while ( $rs = mysql_fetch_array ( $r, MYSQL_ASSOC ) )
{
$ret[] = $rs;
}
return $ret;
}
In theory, you should be better separating the model and all the database related code.
In exemple, in Zend Framework, the quickstart guide advise you to have :
your model classes
your data mappers, whose role is to "translate" the model into the database model.
your DAOs (or table data gateway) that do the direct access to the tables
This is a really interesting model, you should have a look at it, it enables you to really separate the model from the data, and thus to perform tests only onto the model part (and don't care about any database problem/question)
But in your code, I suggest you to perform a test where you have the dbq() function to return false (maybe having the db connexion to be impossible "on purpose"), in order to have full code coverage.
I often have these kind of situations, where testing all the "error cases" takes you too much time, so I give up having 100% code coverage.
I guess the function dbq() performs a database query. Just unplug your database connection and re-run the test.
The reason you have trouble testing is that you are using a global resource: your database connection. You would normally avoid this problem by providing your connection object to the test method (or class).

Categories