I want to make my database schema and application code as dynamic as possible to handle "unknown" use cases and changes. Developing in PHP and MySQL. Twice now I have had to change my entire schema including table and column names and this means the developers have to go back to the application code and modify all the SQL queries and table/columns names. So to prevent this I want to if just like we do on pages where we have page content, title bar etc dynamic like a %variable%, can we do it for the schema and maybe even for the php code functions and classes somehow? It takes weeks to re-do all changes like this vs if it is dynamic it can be done in under a day.
First of all, may you have a happy new year (regardless of your coding issues :) ). Now, what I'm going to proposse to you applies to nearly any development language, but I pressume that PHP suffers this most due to the lack of OO development.
The main issue with changin schemas and functions is the definition of your architecture. To have a robust architecture, you should:
Identify the classes that compose your application
Create the interfaces to comunicate with these clases
Abstract your business(domain) classes from your data classes
Consider using (if you are not using it already) an ORM framework for mapping your database, since you are using PHP, I would suggest Propel or Doctrine, which are wonderful. Beware of depending too much on your ORM framework and converting it into your business model, you can check out this blog post I made talking on the subject.
Hope I can help,
David
What stage of development are you at? And are you talking about a redesign or just refactoring names and so on? Some thoughts:
If you're in the early stages of development, perhaps playing around to get a feel for how the design might shape up, then it's natural to rewrite your proof-of-concept application.
If you're in production, and your design really has changed (as user requirements mature - it happens in real life) then surely your old application is now out of date and has to be rewritten anyway.
If you're just renaming things, and not really redesigning, then do you really need to do it at all? Names of internal things like tables and columns and classes and variables don't need to be updated just because the user-visible terminology changes.
Basically I'm saying that a dynamic design such as you envisage strikes me as a poor foundation to build upon.
But that's not to say that you can't have a flexible design. Simplest case: normalising your database to about third normal form generally makes for flexibility, because you can add columns, create new relationships, and so on. Same goes for small classes with well-defined interfaces.
Assuming you want a strong design to build your system on, I recommend that you make a flexible design (hard work) and don't look for push-button solutions. (I wonder whether I've understood your question...?)
You could include a special PHP file at the beginning of all your scripts, and this file can contain predefined constants that you use throughout your code in place of all table and column names. Then, whenever you need to change the name of a table or column, you only need to change it in 1 place inside of this 1 PHP file.
See PHP documentation on constants here:
http://php.net/manual/en/language.constants.php
As you can see, constants exist in a global scope. That means they can be used inside of all functions and all classes without any problems.
You probably already know about the use of "include" (for loading the special PHP file at the beginning of all your scripts):http://php.net/manual/en/function.include.php
So, for example, if you wanted to write some SQL using dynamic table and column names, you can do this:
Inside constants.php:
define("USER_TABLE", "Users");
define("NAME_COLUMN", "FirstName");
Inside your scripts you do this:
include "constants.php";
...
$sql = "select " . NAME_COLUMN . " from " . USER_TABLE . " where id = ...(etc)";
The above translates to this:
$sql = "select FirstName from Users where id = ...(etc)";
If you someday later want to change the name of the table and column, you only need to change the constants.php file.
define("USER_TABLE", "DifferentUsers");
define("NAME_COLUMN", "DifferentFirstName");
That will automatically cause the changes throughout all of your scripts that use this include file.
The new translated result would now look like:
$sql = "select DifferentFirstName from DifferentUsers where id = ...(etc)";
Related
I have four forms on my website sending user input to four different tables in my MySQL database. Is it good practice to put all the queries for those four forms in a single php file?
Currently, I have a different php file for each form. Eg. form-abc.html has a abc.php to communicate with the database, form-def.html has a def.php and so on.
Since the mysqli connection in all these forms is going to be the same, I was wondering if it is possible to call the relevant function/ query from a php file containing all the queries/functions? And if it is possible, how?
On balance it is not good practice to put the code in a single file, particularly if the four queries are not obviously related in terms of their purpose, but it would also be poor design to have each file containing the code to connect to the database, and also to be hand crafting the query strings.
You should seek to factor out common code to a utility library, with your starting point being to have a single library function that returns a connection to the database. Changing connection details, the name of the database etc. then requires a single change rather than four, and so reducing the chances of a mistake. Your four files should also not need to know how to connect to the database, and removing the code achieves that.
Rather than hand crafting the queries, where there is a chance of failing to properly escape data and where you will end up writing more code than is necessary as well as locking yourself to a particular database type, you should aim to have library routines to help with this. PHP has some libraries already, or you could develop your own that are potentially better suited for the job.
Very broadly, you may end up with code such as this:
<?php
include_once "db_utils.php";
$db = DB::get_instance();
if (isset($_POST['form_submitted'])) {
// Validation etc.
$customers = $db->table('customers');
$customers->insert(array('name' => $_POST['name'], 'email' => $_POST['email']));
// ...
}
The include would include your database utilities, providing in this case a class called DB, and a table class. get_instance() would be a static method to provide database instance that encapsulates a connection. We'll assume that it knows how to obtain connection information, so not needing that to be provided to the method. The table() method would provide an instance of a database table that would know how to perform operations on a database table. The insert() method on a table instance would take an array of key/value pair data, escape the value of each item and do the insert.
Keeping the queries with the files that need them should in this case ease maintenance.
An alternative approach to your entire site would be to use a framework. In such a case, the business logic, database management and rendering code would be separated, which for non-trivial systems is generally a desirable aspiration. If one cobbles a system together with that in mind in an ad-hoc way, this could quickly lead to an unmaintainable system, with developers struggling to find the relevant code that plays a part in a request, however a framework would impose a structure through naming and layout conventions that would largely alleviate that.
For where you are now though, just aim to identify duplicate code, factor that out, look for some database libraries so that you are not hand crafting queries (if you are right now), and keep your system simple.
This isn't much of an issue with MySQL per-se.
The Full Story
I'm writing a very small PHP framework. It isn't like existing frameworks where they force you to use a certain methodology. It isn't either like a CMS framework. Trust me, I've seen Zend framework and I've used CMSes like Joomla and WordPress extensively, none of them come close to what I'm doing.
Introducing The Issues
I'm writing the Database abstraction part. You get class methods like ::table_exists() etc.
It is designed in a way that people can easily add different database classes and use them instead (eg; mysql, mssql, oracle, flatfile...).
They simply need to write a class which satisfies a base abstract classes'.
The Real Issue
I'm writing the functionality for ::table_create(), but have one main problem: MySQL doesn't like empty tables (ie, without a column).
I have several proposed fixes:
For each new table, create a commonly used column, such as 'id' (type=INT)
For each new table, create a temp column which doesn't use any space as much as possible (perhaps a boolean column?)
Somehow delay table creation until at least one column can be created
This approach is most certainly new, and I'd like to here some unbiased comments about it (anything on the lines of "but no one does it that way" won't do).
Well I would either go with option 1), Adding a generic ID column, which you might find you need anyway, or with option 3) Delaying the table creation. I'm assume after they call ::table_create() they will be calling table_add_col(), etc. So just delay creation until there is at least one column, OR until they actually try and use the table for the first time.
Your proposed fixes look quite good. But I would recommen them in a diffrent order. If you are able to delay the creation, tht's probably the best. My second favorite would be to have a table with only an ID, although you might be delete this column, if you want to create a many-to-many relations table with two foreign keys only.
last of your points.
its really very strange what you are doing here. creating tables on the fly? dynamically or something?
well... whatever you are trying to accomplish. you should have a look at document/object oriented databases like couchdb http://couchdb.apache.org/ ! you can create a document and dynamically add whatever fields you want. those are the closest thing to your "columns"
but as you like it...
your first attempt is ugly because it might lead to conflicts.
the second attempt is clumsy. but if you do so create a col with uniqueprefix_random so you can delete it afterwards.
but its well... i dunno what to say about that.
theird approach seems the only senseful!
lacking a fellow programmer to talk over the right approach for my problem, I decided to ask you. What is your preferred approach of mapping dictionary tables to a model in MVC paradigm, regardless of the MVC framework / environment you are using?
My problem is I have a couple of database tables that only serve as dictionaries and are related to other tables as foreign keys. A good example would be a table request having a status_id where statuses are kept in a separate status table.
Now, the latter table needs to be mapped to a model on the code-side of the application. I can either:
Define all the statuses as constants so they can be referenced in the code without poking those dreaded 'magic numbers' here and there. However, any change to the dictionary (database-side) would require a code modification.
Omit the `status` table at all and just define meaningful constant to be used across the code. Pros: one place to rule them all. Cons: all changes require diving into the code, now the database features 'magic numbers' not really being foreign keys
Try to translate statuses into the model automagically, adding a field like 'const_name' to the 'statuses' table and them creating the constants on the fly while loading the model. This one seems to have the most sense for me.
Would you mind to share your usual approach to this issue?
Best,
Bartek
If it's just going to be a set of constants that are contained in the database instead of code, you could have a static class load the status constants for everyone else to use. That way there's no duplication between db and code, and no magic numbers.
edit: since it's a static class, you could have it lazy load the constants. Don't hit the database until the first time someone asks for a status value.
I'd say if you going to change it often it's better to go with table. Otherwise static class is fine (for example no point having table to store sex, or list of states).
I want that cms\framework to create me tables like "Users" "Cameras" and so on and declare classes and simple default methods for them (like paged sql relults and so on).
I mean I say to it: I want Users to have ID, SpecialNumber and Name Flilds.
and I want to get from it class for table generation (to call it once) and class containing methods such as Search by ID, SpecialNumber and Name, Create User, Delit User and so on functions.
Is there any framework/cms like this for working with CODE not ui's and so on... so to say PHP generator or something...
The result should be as that framework free as possible.
So Is there any free, open source php CMS\framework for described case?
Cake PHP is a framework that takes the code generation approach, corresponds to your description, and is pretty mature.
I personally dont find code generation (scaffolding) that much of a deal, you end up rewriting it anyway, I would use Zend Framework and code my stuff, or write my own codegen commands if it's important for the project.
You might wish to consider SilverStripe which takes subclasses of SiteTree (if I recall correctly) and creates tables and an administrative interface for them.
What I really like about Entity framework is its drag and drop way of making up the whole model layer of your application. You select the tables, it joins them and you're done. If you update the database scheda, right click -> update and you're done again.
This seems to me miles ahead the competiting ORMs, like the mess of XML (n)Hibernate requires or the hard-to-update Django Models.
Without concentrating on the fact that maybe sometimes more control over the mapping process may be good, are there similar one-click (or one-command) solutions for other (mainly open source like python or php) programming languages or frameworks?
Thanks
SQLAlchemy database reflection gets you half way there. You'll still have to declare your classes and relations between them. Actually you could easily autogenerate the classes too, but you'll still need to name the relations somehow so you might as well declare the classes manually.
The code to setup your database would look something like this:
from sqlalchemy import create_engine, MetaData
from sqlalchemy.ext.declarative import declarative_base
metadata = MetaData(create_engine(database_url), reflect=True)
Base = declarative_base(metadata)
class Order(Base):
__table__ = metadata.tables['orders']
class OrderLine(Base):
__table__ = metadata.tables['orderlines']
order = relation(Order, backref='lines')
In production code, you'd probably want to cache the reflected database metadata somehow. Like for instance pickle it to a file:
from cPickle import dump, load
import os
if os.path.exists('metadata.cache'):
metadata = load(open('metadata.cache'))
metadata.bind = create_engine(database_url)
else:
metadata = MetaData(create_engine(database_url), reflect=True)
dump(metadata, open('metadata.cache', 'w'))
I do not like “drag and drop” create of data access code.
At first sight it seems easy, but then you make a change to the database and have to update the data access code. This is where it becomes hard, as you often have to redo what you have done before, or hand edit the code the drag/drop designer created. Often when you make a change to one field mapping with a drag/drop designer, the output file has unrelated lines changes, so you can not use your source code control system to confirm you have make the intended change (and not change anything else).
However having to create/edit xml configuring files is not nice every time you refractor your code or change your database schema you have to update the mapping file. It is also very hard to get started with mapping files and tracking down what looks like simple problem can take ages.
There are two other options:
Use a code generator like CodeSmith that comes with templates for many ORM systems. When (not if) you need to customize the output you can edit the template, but the simple case are taken care of for you. That ways you just rerun the code generator every time you change the database schema and get a repeatable result.
And/or use fluent interface (e.g Fluent NHibernate) to configure your ORM system, this avoids the need to the Xml config file and in most cases you can use naming conventions to link fields to columns etc. This will be harder to start with then a drag/drop designer but will pay of in the long term if you do match refactoring of the code or database.
Another option is to use a model that you generate both your database and code from. The “model” is your source code and is kept under version control. This is called “Model Driven Development” and can be great if you have lots of classes that have simpler patterns, as you only need to create the template for each pattern once.
I have heard iBattis is good. A few companies fall back to iBattis when their programmer teams are not capable of understanding Hibernate (time issue).
Personally, I still like Linq2Sql. Yes, the first time someone needs to delete and redrag over a table seems like too much work, but it really is not. And the time that it doesn't update your class code when you save is really a pain, but you simply control-a your tables and drag them over again. Total remakes are very quick and painless. The classes it creates are extremely simple. You can even create multiple table entities if you like with SPs for CRUD.
Linking SPs to CRUD is similar to EF: You simply setup your SP with the same parameters as your table, then drag it over your table, and poof, it matches the data types.
A lot of people go out of their way to take IQueryable away from the repository, but you can limit what you link in linq2Sql, so IQueryable is not too bad.
Come to think of it, I wonder if there is a way to restrict the relations (and foreign keys).