I have four forms on my website sending user input to four different tables in my MySQL database. Is it good practice to put all the queries for those four forms in a single php file?
Currently, I have a different php file for each form. Eg. form-abc.html has a abc.php to communicate with the database, form-def.html has a def.php and so on.
Since the mysqli connection in all these forms is going to be the same, I was wondering if it is possible to call the relevant function/ query from a php file containing all the queries/functions? And if it is possible, how?
On balance it is not good practice to put the code in a single file, particularly if the four queries are not obviously related in terms of their purpose, but it would also be poor design to have each file containing the code to connect to the database, and also to be hand crafting the query strings.
You should seek to factor out common code to a utility library, with your starting point being to have a single library function that returns a connection to the database. Changing connection details, the name of the database etc. then requires a single change rather than four, and so reducing the chances of a mistake. Your four files should also not need to know how to connect to the database, and removing the code achieves that.
Rather than hand crafting the queries, where there is a chance of failing to properly escape data and where you will end up writing more code than is necessary as well as locking yourself to a particular database type, you should aim to have library routines to help with this. PHP has some libraries already, or you could develop your own that are potentially better suited for the job.
Very broadly, you may end up with code such as this:
<?php
include_once "db_utils.php";
$db = DB::get_instance();
if (isset($_POST['form_submitted'])) {
// Validation etc.
$customers = $db->table('customers');
$customers->insert(array('name' => $_POST['name'], 'email' => $_POST['email']));
// ...
}
The include would include your database utilities, providing in this case a class called DB, and a table class. get_instance() would be a static method to provide database instance that encapsulates a connection. We'll assume that it knows how to obtain connection information, so not needing that to be provided to the method. The table() method would provide an instance of a database table that would know how to perform operations on a database table. The insert() method on a table instance would take an array of key/value pair data, escape the value of each item and do the insert.
Keeping the queries with the files that need them should in this case ease maintenance.
An alternative approach to your entire site would be to use a framework. In such a case, the business logic, database management and rendering code would be separated, which for non-trivial systems is generally a desirable aspiration. If one cobbles a system together with that in mind in an ad-hoc way, this could quickly lead to an unmaintainable system, with developers struggling to find the relevant code that plays a part in a request, however a framework would impose a structure through naming and layout conventions that would largely alleviate that.
For where you are now though, just aim to identify duplicate code, factor that out, look for some database libraries so that you are not hand crafting queries (if you are right now), and keep your system simple.
Related
I cant seem to find an acceptable answer to this.
There are two big things I keep seeing:
1) Don't execute queries in the controller. That is the responsibility of business or data.
2) Only select the columns that you need in a query.
My problem is that these two things kind of butt heads since what is displayed in the UI is really what determines what columns need to be queried. This in turn leads to the obvious solution of running the query in the controller, which you aren't supposed to do. Any documentation I have found googling, etc. seems to conveniently ignore this topic and pretend it isn't an issue.
Doing it in the business layer
Now if I take it the other way and query everything in the business layer then I implicitly am making all data access closely reflect the ui layer. This is more a problem with naming of query functions and classes than anything I think.
Take for example an application that has several views for displaying different info about a customer. The natural thing to do would be to name these data transfer classes the same as the view that needs them. But, the business or service layer has no knowledge of the ui layer and therefore any one of these data transfer classes could really be reused for ANY view without breaking any architecture rules. So then, what do I name all of these variations of, say "Customer", where one selects first name and last name, another might select last name and email, or first name and city, and so on. You can only name so many classes "CustomerSummary".
Entity Framework and IQueryable is great. But, what about everything else?
I understand that in entity framework I can have a data layer pass back an IQuerable whose execution is deferred and then just tell that IQueryable what fields I want. That is great. It seems to solve the problem. For .NET. The problem is, I also do PHP development. And pretty much all of the ORMs for php are designed in a way that totally defeat the purpose of using an ORM at all. And even those dont have the same ability as EF / IQueryable. So I am back to the same problem without a solution again in PHP.
Wrapping it up
So, my overall question is how do I get only the fields I need without totally stomping on all the rules of an ntier architecture? And without creating a data layer that inevitably has to be designed to reflect the layout of the UI layer?
And pretty much all of the ORMs for php are designed in a way that totally defeat the purpose of using an ORM at all.
The Doctrine PHP ORM offers lazy loading down to the property / field level. You can have everything done through proxies that will only query the database as needed. In my experience letting the ORM load the whole object once is preferable 90%+ of the time. Otherwise if you're not careful you will end up with multiple queries to the database for the same records. The extra DB chatter isn't worthwhile unless your data model is messy and your rows are very long.
Keep in mind a good ORM will also offer a built-in caching layer. Populating a whole object once and caching it is easier and more extensible then having your code keep track of which fields you need to query in various places.
So my answer is don't go nuts trying to only query the fields you need when using an ORM. If you are writing your queries by hand just in the places you need them, then only query the fields you need. But since you are talking good architectural patterns I assume you're not doing this.
Of course there are exceptions, like querying large data sets for reporting or migrations. These will require unique optimizations.
Questions
1) Don't execute queries in the controller. That is the responsibility of business or data.
How you design your application is up to you. That being said, it's always best to consider best patterns and practices. The way I design my controllers is that I pass in the data layer(IRepository) through constructor and inject that at run time.
public MyController(IRepository repo)
To query my code I simply call
repository.Where(x=> x.Prop == "whatever")
Using IQueryable creates the leaky abstraction problem. Although, it may not be a big deal but you have to be careful and mindful of how you are using your objects especially if they contain relational data. Once you query your data layer you would construct your view model in your controller action with the appropriate data required for your view.
public ActionResult MyAction(){
var data = _repository.Single(x => x.Id == 1);
var vm = new MyActionViewModel {
Name = data.Name,
Age = data.Age
};
return View();
}
If I had any queries that where complex I would create a business layer to include that logic. This would include enforcing business rules etc. In my business layer I would pass in the repository and use that.
2) Only select the columns that you need in a query.
With ORMs you usually pass back the whole object. After that you can construct your view model to include only the data you need.
My suggestion to your php problem is maybe to set up a web api for your data. It would return json data that you can then parse in whatever language you need.
Hope this helps.
The way I do it is as follows:
Have a domain object (entity, business object .. things with the same name) for Entities\Customer, that has all fields and associated logic for all of the data, that a complete instance would have. But for persistence create two separate data mappers:
Mappers\Customer for handling all of the data
Mappers\CustomerSummary for only important parts
If you only need to get customers name and phone number, you use the "summary mapper", but, when you need to examine user's profile, you have the "all data mapper". And the same separation can be really useful, when updating data too. Especially, if your "full customer" get populated from multiple tables.
// code from a method of some service layer class
$customer = new \Model\Entities\Customer;
$customer->setId($someID);
$mapper = new \Model\Mappers\CustomerSummary($this->db);
if ($needEverything) {
$mapper = new \Model\Mappers\Customer($this->db);
}
$mapper->fetch($customer);
As for, what goes where, you probably might want to read this old post.
I want to make my database schema and application code as dynamic as possible to handle "unknown" use cases and changes. Developing in PHP and MySQL. Twice now I have had to change my entire schema including table and column names and this means the developers have to go back to the application code and modify all the SQL queries and table/columns names. So to prevent this I want to if just like we do on pages where we have page content, title bar etc dynamic like a %variable%, can we do it for the schema and maybe even for the php code functions and classes somehow? It takes weeks to re-do all changes like this vs if it is dynamic it can be done in under a day.
First of all, may you have a happy new year (regardless of your coding issues :) ). Now, what I'm going to proposse to you applies to nearly any development language, but I pressume that PHP suffers this most due to the lack of OO development.
The main issue with changin schemas and functions is the definition of your architecture. To have a robust architecture, you should:
Identify the classes that compose your application
Create the interfaces to comunicate with these clases
Abstract your business(domain) classes from your data classes
Consider using (if you are not using it already) an ORM framework for mapping your database, since you are using PHP, I would suggest Propel or Doctrine, which are wonderful. Beware of depending too much on your ORM framework and converting it into your business model, you can check out this blog post I made talking on the subject.
Hope I can help,
David
What stage of development are you at? And are you talking about a redesign or just refactoring names and so on? Some thoughts:
If you're in the early stages of development, perhaps playing around to get a feel for how the design might shape up, then it's natural to rewrite your proof-of-concept application.
If you're in production, and your design really has changed (as user requirements mature - it happens in real life) then surely your old application is now out of date and has to be rewritten anyway.
If you're just renaming things, and not really redesigning, then do you really need to do it at all? Names of internal things like tables and columns and classes and variables don't need to be updated just because the user-visible terminology changes.
Basically I'm saying that a dynamic design such as you envisage strikes me as a poor foundation to build upon.
But that's not to say that you can't have a flexible design. Simplest case: normalising your database to about third normal form generally makes for flexibility, because you can add columns, create new relationships, and so on. Same goes for small classes with well-defined interfaces.
Assuming you want a strong design to build your system on, I recommend that you make a flexible design (hard work) and don't look for push-button solutions. (I wonder whether I've understood your question...?)
You could include a special PHP file at the beginning of all your scripts, and this file can contain predefined constants that you use throughout your code in place of all table and column names. Then, whenever you need to change the name of a table or column, you only need to change it in 1 place inside of this 1 PHP file.
See PHP documentation on constants here:
http://php.net/manual/en/language.constants.php
As you can see, constants exist in a global scope. That means they can be used inside of all functions and all classes without any problems.
You probably already know about the use of "include" (for loading the special PHP file at the beginning of all your scripts):http://php.net/manual/en/function.include.php
So, for example, if you wanted to write some SQL using dynamic table and column names, you can do this:
Inside constants.php:
define("USER_TABLE", "Users");
define("NAME_COLUMN", "FirstName");
Inside your scripts you do this:
include "constants.php";
...
$sql = "select " . NAME_COLUMN . " from " . USER_TABLE . " where id = ...(etc)";
The above translates to this:
$sql = "select FirstName from Users where id = ...(etc)";
If you someday later want to change the name of the table and column, you only need to change the constants.php file.
define("USER_TABLE", "DifferentUsers");
define("NAME_COLUMN", "DifferentFirstName");
That will automatically cause the changes throughout all of your scripts that use this include file.
The new translated result would now look like:
$sql = "select DifferentFirstName from DifferentUsers where id = ...(etc)";
So I am a little confused on the object oriented part of PHP. Right away I will apologize for the fact I know very little about PHP and databases.
My question is when you are making, say, a database to hold users in it, why would you want to make a class/object for that user when you can just pull info from the database?
Also, if you were to make a object/class where is the data for the objects stored? For example, if I have a class with a username and email, and I make that object, were does it get stored?
Thanks for taking your time to help a learning noob!
My question is when your making per
say a database to hold users in it,
why would you want to make a
class/object for that user when you
can just pull info from the database.
You make objects to abstract away specific functionality. What happens if you move to, say, Microsoft SQL Server (hypothetically speaking)? Rather than update your entire site, you just edit the implementation of the object.
Also if you were to make a
object/class where is the data for the
objects stored? Like a class with a
username and email, and I make that
object, were does it get stored.
The same place as any other variable.
There are a LOT of reasons why you want to use some abstraction on top of just raw database access in any reasonably large software system. If you're looking at an Object Oriented approach you should consider that one of the core ideas of the Object Oriented paradigm is that an object encapsulates both data and logic that acts on that data.
Let's take a concrete example. Say that a part of your application (the UI) needs to display user information, including a nicely formatted user name. In an OO world you could have a User object which would store a local copy of the data in the database, and expose methods like getFormattedName(), or something similar. Now the rest of your application can use that code without needing to know about the database, or even how the name is formatted. On the other hand if you were just pulling data directly from the database then the UI part of the application (which doesn't really care about databases) still has to know itself how to get information about the user from the database, and how to format the users name nicely.
To put it simply, there are logic not captured in a database table but related to the entry. The database only stores the raw data. How the data is used and how it interacts with the rest of your application should be captured in your object methods.
You're missing a fundamental of object-oriented design. Ignoring inheritence entirely, Objects combine information/data and functions/procedures/operations into a single unit called an object. This object performs operations (methods/behaviors/functions/procedures) and has attributes. A database will not have the entire set of operational/procedural information. By design, a database will only contain data, and know nothing of how the data can be used or what the data does.
Databases store data in a tabular fashion which is designed to be speedy. Objects are so much more flexible; they can be trees, they can be lists, they can be widgets, or anything else out of a million things. They can represent presentation, data, or structure. And sometimes they are even faster (when it's easier to calculate a value on the fly rather than retrieve it from a database). Databases are very powerful and important but are only appropriate for a small subset of the tasks that a web application performs. The rest are made easier by objects.
I'm reading programming best practices and it is said that when creating a function we should make it do a only single specific task.
I got model functions that retrieves data and it's related data. Example:
$this->Student->StudentAssignments();
Currently this function retrieves the student's assignments plus the question for each assignment and data about the student. I use them all. My dilemma is if I try to make separate functions that retrieves the related data (student and question datas) it's taxing since I'm producing more calls to the DB.
What would you guys suggest?
Something to keep in mind when doing this sort of refactoring...
I typically will have a Model->getSomethingAndSomethingElse functions in my models.
These functions are public and meant to be called as a substitute for doing complicated (or any) find calls from the Controller.
What I will usually do is then build up a small collection of private functions in the model.
In your case I might have something along the lines of...
Student->getStudentAssigmentsWithQuestions
that then calls some private functions i.e.
Student->getStudent which might call Student->joinStudentAssignment which in turn might call Assignment->joinAssignmentQuestion etc.
The double underscore prefixes have been removed since markdown wants to bold things because of them. If you are using php5 the underscores aren't really important anyways as long as you use the "private" or "proteced" keywords.
Basically I use the public method as a container for a group of very specific query building or association building private functions within the models. This allows me to have an api that has complex data returned, but I build the query or the result set (depending on the type of data, relationships involved or query complexity) from small pieces - that can ideally be purposed and used in more than one public function call.
I think you're doing fine. But you should reconsider renaming your function to
$this->Student->getStudentAssignmentsWithQuestions
Or whatever you think fit. I think one should try to do as few calls to the database as possible (I assume you're performing a join somewhere in there), instead of fetching each set of elements by specific methods. This can lead to the fact that you'll get more methods (and therefore have to write some more tests), but I think this is the right way to do it.
To defend the design argument:
Your method does just one single task; it fetches student's assignments with each assignment's questions.
No, if you're strictly concerned about code refactoring you should break down that blob into simpler functions that perform a single task as you said. Yes, you will hit more your database but considering how easy is to work with caching in cakephp, performance should not be an issue. And if it is, then you shouldn't worry about code refactoring at this point.
I'm developing an object-oriented PHP website right now and am trying to determine the best way to abstract database functionality from the rest of the system. Right now, I've got a DB class that manages all the connections and queries that the system uses (it's pretty much an interface to MDB2). However, when using this system, I've realized that I've got a lot of SQL query strings showing up everywhere in my code. For instance, in my User class, I've got something like this:
function checkLogin($email,$password,$remember=false){
$password = $this->__encrypt($password);
$query = "SELECT uid FROM Users WHERE email=? AND pw=?";
$result = $this->db->q($query,array($email,$password));
if(sizeof($result) == 1){
$row = $result->fetchRow(MDB2_FETCHMODE_ASSOC);
$uid = $row['uid'];
}else{
return false;
}
/* Rest of the login script */
}
What I would like to do is find out the best technique for reducing the amount of inline SQL. I understand that one way to do this would be to write functions within User for each of the queries that User makes use of (something like the following), but that could lead to quite a few functions.
function checkLogin($email,$password,$remember=false){
$password = $this->__encrypt($password);
$uid = $this->do_verify_login_query($email,$password);
/* Rest of the login script */
}
function do_verify_login_query($email,$encpw){
$query = "SELECT uid FROM Users WHERE email=? AND pw=?";
$result = $this->$db->q($query,array($email,$encpw));
if(sizeof($result) == 1){
$row = $result->fetchRow(MDB2_FETCHMODE_ASSOC);
return $row['uid'];
}else{
return false;
}
}
So...my question. What is the best technique for managing the large amount of queries that a typical database application would use? Would the way I described be the proper way of handling this situation? Or what about registering a list of queries within the DB class and associating with each a unique ID (such as USER_CHECKLOGIN) that is passed into the DB's query function? This method could also help with security, as it would limit the queries that could be run to only those that are registered in this list, but it's one more thing to remember when writing all the class functions. Thoughts?
Having the SQL pulled out into separate functions is a decent start. Some other things you can do:
Create separate classes for database access code. This will help make sure you don't have SQL functions scattered around in all of your PHP files.
Load the SQL from external files. This completely separates your SQL code and your PHP code, making both more maintainable.
Use stored procedures when you can. This removes the SQL from your PHP code altogether, and helps improve your database security by reducing the risk that external SQL will get executed.
You might want to look into implementing the ActiveRecord Pattern. Using a design pattern such as this provides some consistency in how you work with data from your tables. There can be some downsides to these sorts of approaches, mainly performance for certain types of queries but it can be worked around.
Another option can be the use of an ORM, for PHP the most powerful are:
Propel
Doctrine
Both allow you to access your database using a set of objects, providing a simple API for storing and querying data, both have their own query language, that is converted internally to the targeted DBMS native SQL, this will ease migrating applications from one RDBMS to another with simple configuration changes. I also like the fact that you can encapsulate datamodel logic to add validation for example, only by extending your model classes.
Since you say you're doing this as OO PHP, then why do you have SQL scattered through all the methods in the first place? More common models would be:
Use an ORM and let that handle the database.
Give your classes one or more 'load' methods which use a single query to pull all of an object's data into memory and a 'save' method which uses a single query to update everything in the database. All the other methods only need to do in-memory manipulation and the database interactions are confined to the load/save methods.
The first option will generally be more robust, but the second may run faster and will probably feel more familiar compared to the way you're used to doing things, if either of those are concerns.
For your login example, the way I would do it, then, would be to simply load the user by email address, call $user->check_password($entered_password), and throw an exception/return false/whatever if check_password fails. Neither check_password nor any of the login handling code need to concern themselves with the database, or even with knowing that a database is where the user gets loaded from.
Another option is to think of the queries as data and store them in the database. For instance, you can create one table that stores the query with a name and another table that stores the parameters for that query. Then create a function in PHP that takes the name of the query and an array of params and executes the query, returning any results. You could also attach other metadata to the queries to restrict access to certain users, apply post-functions to the results, etc.