I have recently begun working on a pre-existing PHP application, and am having a bit of trouble when it comes to exactly what I should to improve this application's scalability. I am from a C#.NET background, so the idea of separation of concerns is not new to me.
First off I want to tell you what exactly I am looking for. Currently this application is fairly well-sized, consisting of about 70 database tables and ~200 pages. For the most part these pages simply do CRUD operations on the database, rarely anything actually programmatically intensive. Almost all of the pages are currently using code akin to this:
$x = db->query("SELECT ID, name FROM table ORDER BY name");
if ($x->num_rows > 0) {
while ($y = $x->fetch_object()) {
foreach ($y as $key => $value)
$$key = $value;
Obviously, this is not great for an application that is as large as the one I am supposed to be working on. Now I, being from a C#.NET background, am used to building small-scale applications, and generally building a small Business Object for each object in my application (almost a 1 to 1 match with my database tables). They follow the form of this:
namespace Application
{
public class Person
{
private int _id = -1;
private string _name = "";
private bool _new = false;
public int ID
{
get{ return _id; }
}
public string Name
{
get { return _name; }
set { _name = value; }
}
public Person(int id)
{
DataSet ds = SqlConn.doQuery("SELECT * FROM PERSON WHERE ID = " + id + ";");
if (ds.Tables[0].Rows.Count > 0)
{
DataRow row = ds.Tables[0].Rows[0];
_id = id;
_name = row["Name"].ToString();
}
else
throw new ArgumentOutOfRangeException("Invalid PERSON ID passed, passed ID was " + id);
}
public Person()
{
_new = true;
}
public void save()
{
if (_new)
{
doQuery(" INSERT INTO PERSON " +
" ([Name]) " +
" VALUES " +
" ('" + Name.Replace("'", "''") + "'); ");
}
else
{
SqlConn.doNonQuery(" UPDATE PERSON " +
" SET " +
" [Name] = '" + _name.Replace("'", "''") + "' WHERE " +
" ID = " + _id);
}
}
In shorter terms, they simply have attributes that mimic the table, properties, a constructor that pulls the information, and a method called save that will commit any changes made to the object.
On to my actual question: first of all, is this a good way of doing things? It has always worked well for me. Second, and more importantly, is this also a good way to do things in PHP? I've noticed already that PHP's loose typing has caused me issues in the way I do things, also no method overloading is entirely new. If this isn't a good practice are there any good examples of how I should adapt this project to 3 tiers? Or, possibly I should not attempt to do this for some reason.
I am sorry for the length of the question, but I have been digging on the internet for a week or so to no avail, all I seem to ever find are syntax standards, not structure standards.
Thanks in advance.
There are several patterns that one can follow. I like to use the MVC pattern myself. MVC is about splitting user interface interaction into three distinct roles. There are other patterns though, but for clarity (end length of the post I will only go into MVC). It doesn't really matter which pattern you wish to follow as long as you use the SOLID principles.
I am from a C#.NET background, so the idea of separation of concerns is not new to me.
That's great. Trying to learn PHP will only be an implementation detail at this point.
Obviously, this is not great for an application that is as large as the one I am supposed to be working on.
Glad that we agree :-)
In an MVC pattern you would have:
Model: which does 'all the work'
View: which takes care of the presentation
Controller: which handles requests
When requesting a page it will be handled by the Controller. If the controller needs to get some work done (e.g. getting a user) it would ask the model to do this. When the controller has all the needed info it will render a view. The view only renders all the info.
That person object you were talking about would be a model in the MVC pattern.
I've noticed already that PHP's loose typing has caused me issues in the way I do things
PHP loose typing is pretty sweet (if you know what you are doing) and at the same time can suck. Remember that you can always do strict comparisons by using three = signs:
if (1 == true) // truthy
if (1 === true) // falsy
also no method overloading is entirely new
PHP indeed doesn't really support method overloading, but it can be mimicked. Consider the following:
function doSomething($var1, $var2 = null)
{
var_dump($var1, $var2);
}
doSomething('yay!', 'woo!'); // will dump yay! and woo!
doSomething('yay!'); // will dump yay! and null
Another way would be to do:
function doSomething()
{
$numargs = func_num_args();
$arg_list = func_get_args();
for ($i = 0; $i < $numargs; $i++) {
echo "Argument $i is: " . $arg_list[$i] . "<br />\n";
}
}
doSomething('now I can add any number of args', array('yay!', 'woo!', false));
Or, possibly I should not attempt to do this for some reason.
Of course you should attempt it. When you already have programming experience it shouldn't be too hard.
If you have any more questions you can can find me idling in the PHP chat tomorrow (I'm really need to get some sleep now :-) ).
is this a good way of doing things?
There is no ideal way; there is good and bad for determined situation.
About typing, PHP doesn't loose it; we call that dynamic typing.
About the architecture, when we develop enterprise level web applications, from medium to large scale, we tend to talk about performance on a daily basis.
A good point to start would be reading about some of the most known PHP frameworks, like:
Zend
Symfony
Kohana
From that you can start thinking about architecture.
Related
Six years ago I started a new PHP OOP project without having any experience so I just made it up as I went along. Anyway, I noticed that my rather powerful mySQL server sometimes gets bogged down far too easily and was wondering what the best way to limit some db activity, when I came up with this, as an example...
private $q_enghours;
public function gEngHours() {
if ( isset($this->q_enghours) ) {
} else {
$q = "SELECT q_eh FROM " . quQUOTES . " WHERE id = " . $this->id;
if ($r = $this->_dblink->query($q)) {
$row = $r->fetch_row();
$r->free();
$this->q_enghours = $row[0];
}
else {
$this->q_enghours = 0;
}
}
return $this->q_enghours;
}
This seems like it should be effective in greatly reducing the necessary reads to the db. If the object property is populated, no need to access the db. Note that there are almost two dozen classes all with the same db access routines for the "getter". I've only implemented this change in one place and was wondering if there is a "best practice" for this that I may have missed before I re-write all the classes.
I'd say that this question is rather based on wrong premises.
If you want to deal with "easily bogged down" database, then you have to dig up the particular reason, instead of just making guesses. These trifle reads you are so concerned with, in reality won't make any difference. You have to profile your whole application and find the real cause.
If you want to reduce number of reads, then make your object to map certain database record, by reading that record and populating all the properties once, at object creation. Constructors are made for it.
As a side note, you really need a good database wrapper, just to reduce the amount of code you have to write for each database call, so, this code can be written as
public function gEngHours() {
if ( !isset($this->q_enghours) ) {
$this->q_enghours = $this->db->getOne("SELECT q_eh FROM ?n WHERE id = ?", quQUOTES, $this->id);
}
return $this->q_enghours;
}
where getOne() method is doing all the job of running the query, fetching row, getting first result from it and many other thinks like proper error handling and making query safe.
I'm querying big chunks of data with cachephp's find. I use recursive 2. (I really need that much recursion sadly.) I want to cache the result from associations, but I don't know where to return them. For example I have a Card table and card belongs to Artist. When I query something from Card, the find method runs in the Card table, but not in the Artist table, but I get the Artist value for the Card's artist_id field and I see a query in the query log like this:
`Artist`.`id`, `Artist`.`name` FROM `swords`.`artists` AS `Artist` WHERE `Artist`.`id` = 93
My question is how can I cache this type of queries?
Thanks!
1. Where does Cake "do" this?
CakePHP does this really cool but - as you have discovered yourself - sometimes expensive operation in its different DataSource::read() Method implementations. For example in the Dbo Datasources its here. As you can see, you have no direct 'hooks' (= callbacks) at the point where Cake determines the value of the $recursive option and may decides to query your associations. BUT we have before and after callbacks.
2. Where to Cache the associated Data?
Such an operation is in my opinion best suited in the beforeFind and afterFind callback method of your Model classes OR equivalent with Model.beforeFind and Model.afterFind event listeners attached to the models event manager.
The general idea is to check your Cache in the beforeFind method. If you have some data cached, change the $recursive option to a lower value (e.g. -1, 0 or 1) and do the normal query. In the afterFind method, you merge your cached data with the newly fetched data from your database.
Note that beforeFind is only called on the Model from which you are actually fetching the data, whereas afterFind is also called on every associated Model, thus the $primary parameter.
3. An Example?
// We are in a Model.
protected $cacheKey;
public function beforeFind($query) {
if (isset($query["recursive"]) && $query["recursive"] == 2) {
$this->cacheKey = genereate_my_unique_query_cache_key($query); // Todo
if (Cache::read($this->cacheKey) !== false) {
$query["recursive"] = 0; // 1, -1, ...
return $query;
}
}
return parent::beforeFind($query);
}
public function afterFind($results, $primary = false) {
if ($primary && $this->cacheKey) {
if (($cachedData = Cache::read($this->cacheKey)) !== false) {
$results = array_merge($results, $cachedData);
// Maybe use Hash::merge() instead of array_merge
// or something completely different.
} else {
$data = ...;
// Extract your data from $results here,
// Hash::extract() is your friend!
// But use debug($results) if you have no clue :)
Cache::write($this->cacheKey, $data);
}
$this->cacheKey = null;
}
return parent::afterFind($results, $primary);
}
4. What else?
If you are having trouble with deep / high values of $recursion, have a look into Cake's Containable Behavior. This allows you to filter even the deepest recursions.
As another tip: sometimes such deep recursions can be a sign of a general bad or suboptimal design (Database Schema, general Software Architecture, Process and Functional flow of the Appliaction, and so on). Maybe there is an easier way to achieve your desired result?
The easiest way to do this is to install the CakePHP Autocache Plugin.
I've been using this (with several custom modifications) for the last 6 months, and it works extremely well. It will not only cache the recursive data as you want, but also any other model query. It can bring the number of queries per request to zero, and still be able to invalidate its cache when the data changes. It's the holy grail of caching... ad-hoc solutions aren't anywhere near as good as this plugin.
Write query result like following
Cache::write("cache_name",$result);
When you want to retrieve data from cache then write like
$results = Cache::read("cache_name");
I know this is more of PHP problem because of it's loose Typing of arrays but I see this problem all over the place in a project I took over and not sure of the best way to refactor it. Suppose you have two sets of data, both multi dimensional arrays, $results_by_entity and $target_limits and we want to check what the target is foreach result_by_entity so we can set some state
foreach ($results_by_entity AS $result_by_entity) {
foreach ($target_limits AS $target_limit) {
if ($target_limit['activity_id'] == $result_by_entity['activity_id']) {
$result_by_entity->target = $target_limit->quantity;
$result_by_entity->progress = $target_limit->score;
}
}
}
There are a couple of main problems here
1-The data is really strongly tied together, so it is really hard to refactor $results_by_entity into it's own class and $target_limits into it's own class
2-The time taken to process this grows exponentially as the data size grows
I read the Refactoring book by Martin Fowler and it was really helpful but this style of problem doesn't really show up I think mostly because his examples are in JAVA which is strongly typed. The class is super run on so really hard to debug and extend but all the data is so tied together primarily because of these types of loops so not to sure how to solve. Any recommendations would be really appreciated
What you want is to index your data pre-emptively if possible:
$results_index = array();
foreach ($results_by_entity AS $result_by_entity) {
//Index this value
$results_index[$result_by_entity['activity_id']] = $result_by_entity; //Add a & in front if it's a scalar value, but it looks like it's an object in your example
}
foreach ($target_limits AS $target_limit) {
//Find the corresponding activity id in results
if (isset($results_index[$target_limit['activity_id']])) {
$result_by_entity = $results_index[$target_limit['activity_id']];
$result_by_entity->target = $target_limit->quantity;
$result_by_entity->progress = $target_limit->score;
}
}
All of the code in my project is procedural. It isn't using any framework, or any MVC structure. Most of the PHP is form handlers with some logic. So lots of code like this:
<?php
require "headerFile.php";
$user = array(); // filled with user data
if (isset($_POST['submitButton'])) {
if (isset($_POST['money'])) {
if (is_numeric($_POST['money'])) { // I don't actually validate numbers this way
if ($_POST['money'] <= $user['money']) {
$query = mysql_query("SELECT * FROM someTable WHERE id={$user['id']}");
if($result = mysql_fetch_array($query)) {
if ($someOtherCheck == $user['someOtherData']) {
/*
* run some queries, give user some item
*
*/
} else {
echo "This isn't right.";
}
} else {
echo "You don't have a row in some table!";
}
} else {
echo "You don't have that much money!";
}
} else {
echo "Enter a valid number";
}
} else {
echo "Enter how much you want to wager.";
}
}
// some hard coded form goes here
require "footerFile.php";
?>
There are over a hundred of these forms and nested if handlers, some that are similar with more/fewer conditions.
I want to start using OOP and I've been reading up on design patterns but I can't seem to find anything that applicable to my project.
I'm adding loads of new blocks of code similar to the block above, I don't want to copy and paste and just change a couple of bits here and there, what are my options?
First off, if you are ever copying and pasting code that should be a BIG RED FLAG. This should be a warning that instead of copying and pasting, you need to write a generalized function that can be used.
Secondly, it's really time to stop using mysql_query-like functions. Even the PHP page says use of those functions is discouraged. I would start moving your SQL over to PDO which is more OO-like.
When I inherited an application that looked the code you posted, I wrote a blog post about the steps I took to start getting a handle on things. It may also be helpful to you - http://mattmccormick.ca/2011/10/13/how-to-refactor-a-php-application/
OOP is not always about application. It is about readability and structure. Honestly, how readable is the code you have posted here? There are a ton of things happening and it would take you 10 minutes to decipher them all. However, if you break it down into classes with class functions you would be able to glance at things and know what is going on.
OOP does not do a ton for php all the time, but its something you want to do for almost all other non-static load languages. With the added benefit that if you have more than one programmer on a project you can both read the code. Comments are you friend.
You're best option is to use functions with returns. Return the value and then the function does whatever it needs to do with it. I personally would do something like
$class->check
if error
$this->error_handler
and the function error handler does whatever you want it to do with the error, if its die or echo then do that, but you build the error handler, so if you ever want to change it, you chance it in one place instead of 20.
Even without bringing in OOP, you could do wonders for structuring your code to be readable. There's no need to nest your conditionals if they're not dependent on each other.
$form_is_valid = true;
if (!isset($_POST['submitButton'])) {
echo "This isn't right.";
$form_is_valid = false;
}
if (!isset($_POST['money'])) {
echo "Enter how much you want to wager.";
$form_is_valid = false;
} elseif (!is_numeric($_POST['money'])) {
echo "Enter a valid number";
$form_is_valid = false;
} elseif (!$_POST['money'] <= $user['money']) {
echo "You don't have that much money!";
$form_is_valid = false;
}
if ($form_is_valid) {
do_stuff_here();
}
Regarding Frameworks
A decent framework will help you on your way to organizing code better trough separation of concerns, but does not necessarily enforce best practices. Subjectively, I say it takes hands-on experience and making lots of mistakes before best practices are ingrained in your head.
Try to think of a framework as a delivery mechanism only. Optimally, your code is not tied to any one framework. This generally means using components to handle different aspects of your application such as routing, validation, config, events, dependencies, etc.
Regarding OOP
Something I feel you should start with would be the SOLID principle. This will help you, although not guarantee, avoid making mistakes that cost you a lot of time down the road.
Foremost, your objects should only have a single responsibility. e.g. a Product object should not be responsible for acting as a data store, persisting itself, handling an order, etc etc.
Also another biggy would be dependency injection. This is huge for unit testing your classes (which you should get in the habit of doing). In a nutshell, do not construct dependency within a class. Construct it beforehand and pass it in through either a constructor argument or a setter method.
The process of architecting an application merits having its own books, so I'm not going to attempt writing it all here. If you follow the SOLID principle though, you will be well on your way to being a better developer.
Regarding your code above
That kind of nesting is generally a smell, and I can see why you are asking the question...
Step 1 would be to separate the validation in to its own service. Thinking of this in a MVC sense, at the least, your controller would only have [pseudocode] if ($form->isValid()) { do something }. This alone eliminates the bulk of spaghetti you have.
I want to start using OOP and I've been reading up on design patterns but I can't seem to find anything that applicable to my project.
You don't have to start with patterns yet .. understand the basics and you can progress form there ..
This is a basic example
if (isset($_POST['submitButton'])) {
$db = new \mysqli("localhost", "user", "pass", "db");
$user = new User("Juddling Stack", 123456);
$user->setMoney(500.01);
$player = new PlayerRegister($db, $user);
if (! isset($_POST['money']) || ! is_numeric($_POST['money']))
$player->addError("Enter Valid Wager Money");
if ($_POST['money'] <= $user['money']) {
$player->addError("You don't have that much money!");
}
try {
$player->doSQL();
} catch ( BaseException $e ) {
foreach ( $e->getError() as $error ) {
echo $error, " <br />";
}
}
}
Classes Used
It looks like your code is combining certain components of both the model and view layers. You're running queries against a database, and in the same place, you're including hard coded forms. So, a good place for you to start would be to split these tasks up into two separate classes. For example, write a class to connect to your database and run queries against it, and another class to actually serve your content.
My advice on Design patterns is to not get too bogged down in the details. Instead, try to understand why certain patterns are so helpful, and what problems they are trying to solve. A lot of beginners get too bogged down in the how , and they end up wasting so much time learning the nuances of a framework when a simple solution would have sufficed.
Finally, as you read through your code, keep an eye out for things that could potentially be structured into a class. Also, remember that specialization is key, and by building classes that are very specialized in what they do, you are building modular components that could potentially be used in other projects,
Let's start by stating the problem you are having which is not not using OOP, but called programming-overhead or in programmer terms spaghetti-code.
If you experience a lot of overhead, meaning time wasted writing almost the exact same line of code, where only it's content is different, but the functionality is the same. Then start slicing every peace of code that's the same from it's function, but differentiate its content.
You stated there is more to be copied pasted, and even more complex, I'll just do the form validation part (something I called stage 1), which is just one simple example of how to apply logic that does all the work for you by feeding input it expects. One example might be more elegant than others.
all code below is not tested
An example for locating code of equal functionality.
// functionality on checking values is the same, but it's content is not
if (isset($_POST['submitButton'])) {
if (isset($_POST['money'])) {
if (is_numeric($_POST['money'])) {
and
// though every decision made by its content is only being produced once ..
} else
echo "You don't have that much money!";
} else
echo "Enter a valid number";
} else
echo "Enter how much you want to wager.";
Now the trick is to find a solution to logically solve this. PHP is full of build-in functions, but first grasp the idea on how to solve it. An example would be to have each key, like submitButton, money have a value that would equal to not exists if not set/present, let's say null. Second, you are comparing values with a key supplied by the $_POST, so no matter what.. your $_POST array is the decision maker.
A keen example can be seen on how the jQuery library has been build using the $.extend() method to apply default values to keys, so that a key always has a value and always a decision to make by not checking first if it exists. But so can PHP.
Let's default the values.
$_POST = array_merge(array(
'submitButton' => null,
'money' => 0,
'etc' => '...'
, $_POST);
Building a function to validate this array is a lot easier now, because you can always depend on a value being present.
You stated you have many more forms that need validation, a function that would validate certain fields would be next to figure out.
A formal representation of a valid or invalid form could be an array, e.g.
$valid_form = array(
'submitButton' => array('not_null'),
'money' => array('not_null','int'),
'etc' => '...'
);
A function to validate would be
function validateForm($values, $valid) {
// error array to be returned
$error = array();
// let's iterate over each value, remember we defaulted the $_POST array with
// all the fields it can have, so all fields should be iterated apon.
foreach($values as $key => $value) {
if(array_key_exist($key, $valid)) {
// logic content can be used by different functions, switch
// used here for simplicity
foreach($valid[$key] as $validation) {
switch($validation) {
case 'not_null':
if(is_null($value)) {
$error[] = "error logic";
continue; // skip rest
}
break;
case 'etc':
$error[] = "..";
break;
}
}
}
}
return $error ? $error : true; // true being valid
}
The error handling can be done in a lot of ways, just one simple example (depends on how extensive this project is going to be), you could bind the error content to the validation key, e.g.
$vfe = $valid_form_errors = array( // $vfe for simlicity's sake
'__no_error' => 'no error present for "%key%" validation',
'not_null' => '%key% should not be null',
'int' => '%key% expects to be an integer'
);
$valid_form = array(
'submitButton' => array('not_null'),
'money' => array('not_null','int'),
'etc' => '...'
);
A function to create a formal error message
function error_msg($key, $validation) {
global $vfe;
// error exists?
$eE = array_key_exists($validation,$vfe);
return str_replace('%key%', $eE?$key:$validation, $vfe[$eE?$validation:'__no_error']);
}
And in the simple switch, the error logic is
foreach($valid[$key] as $validation) {
switch($validation) {
case 'not_null':
if(is_null($value))
$error[] = error_msg($key, $validation);
break;
case 'etc':
$error[] = "..";
break;
}
}
So how would your code look like using a different logic?
// first stage ..
$form_valid = validateForm($_POST, $valid_form);
if ($form_valid === true) {
// second stage, same logic be applied as with validateForm, etc.
if($_POST['money'] <= $user['money']) {
$query = mysql_query("SELECT * FROM someTable WHERE id={$user['id']}");
if($result = mysql_fetch_array($query)) {
// third stage, same logic can be applied here..
if ($someOtherCheck == $user['someOtherData']) {
} else {
echo "This isn't right.";
}
} else {
echo "You don't have a row in some table!";
}
}
else {
$errors = $form_valid;
// error handling here
print_r($errors);
}
It all comes to how specific you can define values being expected. You can extend every function to more specific, like binding the errors to a form key so you can specifically target that input at a later stage. They key thing is to erase all possible duplication where a function might do all this for you by simply asking him to compare values you expect, and let him tell which value there actually are.
I know vBulletin uses bitperms, I was using them too but when I got to 10^63 they stopped working, it wouldn't handle any numbers greater than that (it being my PHP host).
I'm curious to know what myBB, PhpBB, IPB, Joomla and other scripts on the net use for permission systems, I really want to use a fast permission setup in my script. Right now I've implemented a sql variable on each user called permgroups and would have a value such as 1,4,5 and each of those numbers correspond to a permission group which has a similar variable called canseepages 1,2,3,4,1,4,1,54,6,4,5,22,6,2,3,4,1,2 which correspond to each page I have.
First I select permgroups in PHP
Then I use PHP's explode on permgroups
then I do a foreach on every perm group the user can see
within the foreach I run a sql query to grab the canseepages variable from the permissions group
I then append this to a variable so I end up with something MASSIVE like
$variable = '1,2,3,4,5,6,7,8,9,2,22,55,44,55,33,44,11,44,33,44,11,33,44,'.
'22,33,44,11,22,33,44,33,11,22,33,44,33,22,33,44,55,44,'.
'55,54,26,77,84,645,345,233,11,4,11,3,32';
That variable represents all the pages the user is allowed to view. I then explode that into an array of numbers and I use in_array() to check if the current page they're trying to view is within that array of pages they're allowed to view.
It's pretty fast now but I'm just thinking there must be a faster method of doing all this in general.
Maybe this doesn't apply for you, but typically you'd apply permissions to sections of a system, not individual pages. So, for example, you might have an 'admin' permission, that unlocks all the big adminy sections.
You could have a manager perm that unlocks the ability to add, edit, and delete users from the system. Since it is ultra rare to have a need for someone that can do one of, but not all of, those things.
An alternative is a task-specific permissions system. This site uses one, you've been around long enough to gain some of them.
I figured out a long time back that Bit masks was the best possible solution for User Permissions:
Short Example:
class UserPermissions()
{
private $Mask = 0;
//Levels
const PUBLIC_READ = 1;
const PUBLIC_WRITE = 2;
const PUBLIC_EDIT = 4
const PUBLIC_DELETE = 8;
//ETC
public function __construct($Mask)
{
$this->Mask = $Mask;
}
public function InvokePermission($Bit)
{
return ($Mask & $Bit); //True / False
}
public function AddPermission($Bit)
{
$this->Mask |= $Bit; //Add the bit to the mask
}
public function RevokePermission()
{
$this->Mask &= ~ $Bit;
}
public GetMask()
{
return $this->Mask;
}
}
Simple use like so:
$Permissions = new UserPermissions($User->PermissionsData);
if($Permissions->InvokePermission( Permissions:: PUBLIC_EDIT ))
{
//Use can edit
}
Some links:
Why should I use bitwise/bitmask in PHP?
Duplicate (From Myself)
Why not use arrays of integers as bitmasks? Then you just do something like
$ndx = $pageNo / PHP_INT_SIZE;
$bit = $pageNo % PHP_INT_SIZE;
$canAccess = $permArray[$ndx] & (1<<$bit);
$pageNo is the number of the page the user is trying to access, $permArray is the array of integers representing the permitted pages for the group. If the bit corresponding to the page is set, the user can access the page.
(Sorry if the syntax is wrong, I haven't used PHP for a long time.)