Related
I'm about to start coding a new website. My problem is that I'm still stuck in using old school coding methods.
I recently downloaded some open source code from Question2Answer.org and was really intrigued in how it was set out.
Does anyone know of any sources? or something that I could possibly download, a template or example to help me get started with a new site?
The site won't be anything fancy but I want to start moving into Web 2.0 and OO programming.
In short I want to do it right. Any advice would be appreciated.
It seems like this question isn't getting many answers, so I'll try my hand at it (even though I'd recommend a different language, just because PHP is such a terrible language ). PHP was the first language I ever wrote anything big in, and the one thing that I wish I had known at the time was the MVC design pattern. It has some advantages like:
Separation of logic and UI means less ad-hoc code, more functions (try to follow the "each function does one thing" rule -- It makes things much easier when you go back and look at code)
Functions are easier to verify correctness than huge function-less pages
Functions can be unit tested (do this!)
It's easier to figure out where things are (database logic is in one file, HTML in another, and "controller" logic in another)
Here's a pretty good high-level intro to MVC.
Key points:
The model talks to the database (or whatever other storage you're using).
The view displays things (HTML)
The controller does everything else
I found two good-looking PHP MVC tutorials here and here. Hopefully they're not too complicated, and feel free to ask more questions if anything doesn't make sense.
Good luck!
PS - Don't forget the point about unit tests! If you can't find a way to unit test a function, it's probably too complicated.
There is a reason why people chose PHP as a server-side scripting language. It's extremely easy to pick up and offers many different coding options. Many functions are included without having to know prior importing, and you do NOT have to write OO code if you don't want to.
This all leads to a huge downfall as well, because there are less restrictions on the structure of the code, it's much easier to write bad code.
I suggest using a framework. It saves you time, energy, and the opportunity to write bad code:
CakePHP is a rapid development framework for PHP that provides an extensible architecture for developing, maintaining, and deploying applications. Using commonly known design patterns like MVC and ORM within the convention over configuration paradigm, CakePHP reduces development costs and helps developers write less code.
http://cakephp.org/
http://www.phpframeworks.com/
IMO MVC ( and this OOP ) is nothing really fancy. It's basically a function to register a pointer to a function in an array for example like a hook. This array is looked by another function to do some stuff. A good MVC should have a callback function. IMO this OOP thing is more a bussines logic to help you to monetize your application. It's not really something difficult to understand.
I've got a big project written in PHP and Javascript. The problem is that it's become so big and unmaintainable that changing some little portion of the code will upset and probably break a whole lot of other portions.
I'm really bad at testing my own code (as a matter of fact, others point this out daily), which makes it even more difficult to maintain the project.
The project itself isn't that complicated or complex, it's more the way it's built that makes it complex: we don't have predefined rules or lists to follow when doing our testing. This often results in lots of bugs and unhappy customers.
We started discussing this at the office and came up with the idea of starting to use test driven development instead of the develop like hell and maybe test later (which almost always ends up being fix bugs all the time).
After that background, the things I need help with are the following:
How to implement a test
framework into an already existing
project? (3 years in the
making and counting)
What kind of frameworks are there
for testing? I figure I'll need one
framework for Javascript and
one for PHP.
Whats the best approach for testing
the graphical user interface?
I've never used Unit Testing before so this is really uncharted territory for me.
G'day,
Edit: I've just had a quick look through the first chapter of "The Art of Unit Testing" which is also available as a free PDF at the book's website. It'll give you a good overview of what you are trying to do with a unit test.
I'm assuming you're going to use an xUnit type framework. Some initial high-level thoughts are:
Edit: make sure that everyone is is agreement as to what constitutes a good unit test. I'd suggest using the above overview chapter as a good starting point and if needed take it from there. Imagine having people run off enthusiastically to create lots of unit tests while having a different understanding of what a "good" unit test. It'd be terrible for you to out in the future that 25% of your unit tests aren't useful, repeatable, reliable, etc., etc..
add tests to cover small chunks of code at a time. That is, don't create a single, monolithic task to add tests for the existing code base.
modify any existing processes to make sure new tests are added for any new code written. Make it a part of the review process of the code that unit tests must be provided for the new functionality.
extend any existing bugfix processes to make sure that new tests are created to show presence and prove the absence of the bug. N.B. Don't forget to rollback your candidate fix to introduce the bug again to verify that it is only that single patch that has corrected the problem and it is not being fixed by a combination of factors.
Edit: as you start to build up the number of your tests, start running them as nightly regression tests to check nothing has been broken by new functionality.
make a successful run of all existing tests and entry criterion for the review process of a candidate bugfix.
Edit: start keeping a catalogue of test types, i.e. test code fragments, to make the creation of new tests easier. No sense in reinventing the wheel all the time. The unit test(s) written to test opening a file in one part of the code base is/are going to be similar to the unit test(s) written to test code that opens a different file in a different part of the code base. Catalogue these to make them easy to find.
Edit: where you are only modifying a couple of methods for an existing class, create a test suite to hold the complete set of tests for the class. Then only add the individual tests for the methods you are modifying to this test suite. This uses xUnit termonology as I'm now assuming you'll be using an xUnit framework like PHPUnit.
use a standard convention for the naming of your test suites and tests, e.g. testSuite_classA which will then contain individual tests like test__test_function. For example, test_fopen_bad_name and test_fopen_bad_perms, etc. This helps minimise the noise when moving around the code base and looking at other people's tests. It also has then benefit of helping people when they come to name their tests in the first place by freeing up their mind to work on the more interesting stuff like the tests themselves.
Edit: i wouldn't use TDD at this stage. By definition, TDD will need all tests present before the changes are in place so you will have failing tests all over the place as you add new testSuites to cover classes that you are working on. Instead add the new testSuite and then add the individual tests as required so you don't get a lot of noise occurring in your test results for failing tests. And, as Yishai points out, adding the task of learning TDD at this point in time will really slow you down. Put learning TDD as a task to be done when you have some spare time. It's not that difficult.
as a corollary of this you'll need a tool to keep track of the those existing classes where the testSuite exists but where tests have not yet been written to cover the other member functions in the class. This way you can keep track of where your test coverage has holes. I'm talking at a high level here where you can generate a list of classes and specific member functions where no tests currently exist. A standard naming convention for the tests and testSuites will greatly help you here.
I'll add more points as I think of them.
HTH
You should get yourself a copy Working Effectively with Legacy Code. This will give you good guidance in how to introduce tests into code that is not written to be tested.
TDD is great, but you do need to start with just putting existing code under test to make sure that changes you make don't change existing required behavior while introducing changes.
However, introducing TDD now will slow you down a lot before you get back going, because retrofitting tests, even only in the area you are changing, is going to get complicated before it gets simple.
Just to add to the other excellent answers, I'd agree that going from 0% to 100% coverage in one go is unrealistic - but that you should definitely add unit tests every time you fix a bug.
You say that there are quite a lot of bugs and unhappy customers - I'd be very positive about incorporating strict TDD into the bugfixing process, which is much easier than implementing it overall. After all, if there really is a bug there that needs to be fixed, then creating a test that reproduces it serves various goals:
It's likely to be a minimal test case to demonstrate that there really is an issue
With confidence that the (currently failing) test highlights the reported problem, you'll know for sure if your changes have fixed it
It will forever stand as a regression test that will prevent this same issue recurring in future.
Introducing tests to an existing project is difficult and likely to be a long process, but doing them at the same time as fixing bugs is such an ideal time to do so (parallel to introducing tests gradually in a "normal" sense) that it would be a shame not to take that chance and make lemonade from your bug reports. :-)
From a planning perspective, I think you have three basic choices:
take a cycle to retrofit the code with unit tests
designate part of the team to retrofit the code with unit tests
introduce unit tests gradually as you work on the code
The first approach may well last a lot longer than you anticipate, and your visible productivity will take a hit. If you use it, you will need to get buy-in from all your stakeholders. However, you might use it to kickstart the process.
The problem with the second approach is that you create a distinction between coders and test writers. The coders will not feel any ownership for test maintenance. I think this approach is worth avoiding.
The third approach is the most organic, and it gets you into test-driven development from the get go. It may take some time for a useful body of unit tests to accumulate. The slow pace of test accumulation might actually be an advantage in that it gives you time to get good at writing tests.
All things considered, I think I'd opt for a modest sprint in the spirit of approach 1, followed by a commitment to approach 3.
For the general principles of unit testing I recommend the book xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros.
I've used PHPUnit with good results. PHPUnit, like other JUnit-derived projects, requires that code to be tested be organized into classes. If your project is not object-oriented, then you'll need to start refactoring non-procedural code into functions, and functions into classes.
I've not personally used a JavaScript framework, though I would image that these frameworks would also require that your code be structured into (at least) callable functions if not full-blown objects.
For testing GUI applications, you may benefit from using Selenium, though a checklist written by a programmer with good QA instincts might work just fine. I've found that using MediaWiki or your favorite Wiki engine is a good place to store checklists and related project documentation.
Implementing a framework is in most cases a complex task because you kinda start rebuilding your old code with some new solid framework parts. Those old parts must start to communicate with the framework. The old parts must receive some callbacks and returnstates, the old parts must then somehow point that out to the user and in fact you suddenly have 2 systems to test.
If you say that you application itself isn't that complex but it has become due to lack of testing it might be a better option to rebuild the application. Put some common frameworks like Zend to the test, gather your requirements, find out if the tested framework suits for the requirements and decide if it's usefull to start over.
I'm not very sure of unit testing, but NetBeans has a built-in unit testing suite.
If the code is really messy, it's possible that it will be very hard to do any unit testing. Only sufficiently loosely coupled and sufficiently well designed components can be unit tested easily. However, functional testing may be a lot easier to implement in your case. I would recommend taking a look at Selenium. With this framework, you will be able to test your GUI and the backend at the same time. However, most probably, it won't help you catch the bugs as well as you could with unit testing.
Maybe this list will help you and your mates to re-structure everything:
Use UML, to design and handle exceptions (http://en.wikipedia.org/wiki/Unified_Modeling_Language)
Use BPMS, to design your work-flow so you won't struggle (http://en.wikipedia.org/wiki/Business_process_management)
Get a list of php frameworks which also support javascript backends (e.g. Zend with jQuery)
Compare these frameworks and take the one, which matches the most to your project desgin and the coding structure used before
You should may be consider using things like ezComponents and Dtrace for debugging and testing
Do not be afraid of changes ;)
For GUI testing you may want to take a look at Selenium (as Ignas R pointed out already) OR you may wanna take a look at this tool as well: STIQ.
Best of luck!
In some cases, doing automatic testing may not be such a good idea, especially when the code base is dirty and PHP mixes it's behavior with Javascript.
It could be better to start with a simple checklist (with links, to make it faster) of the tests that should be done (manually) on the thing before each delivery.
When coding in a 3 years old minefield, better protect yourself with many error checking. The 15 minutes spent writing THE proper error message for each case will not be lost.
Use the bridging method: bridge ugly lengthy function fold() with a call to fnew() which is a wrapper around some clean classes, call both fold and fnew and compare result, log differences, throw the code into production and wait for your fishes. When doing this, always use one cycle for refactoring, anOTHER cycle for changing result (do not even fix bugs in old behavior, just bridge it).
I agree with KOHb, Selenium is a must !
Also have a look at PHPure,
Their software records the inputs and outputs from a working php website, and then automatically writes phpunit tests for functions that do not access external sources (db, files etc..).
It's not a 100% solution, but it's a great start
After reading this post I kinda felt in the same position as the guy who asked the question. I love technology and coming up with new ideas to solve real world problems just gets my neurons horny, but the other part of the equation - actually getting things done (fast) - is normally a pain in the ass to accomplish, specially when I'm doing this for myself.
Sometimes I kinda feel plain bored with code, some other times I spend more time moving the cursor in the text editor and staring at my code, trying to come up with a solution that is better than the one I already have. I heard this is a disease called perfectionism.
I've read in that same post (and also a few times here on SO too) that TDD is actually good to stop coding like a girl, however I've never given a chance at TDD - either because I'm too lazy to learn / set it up or because I don't think I need it because I can do all the tests I need inside my head.
Do you also believe that TDD actually helps to GTD?
What do I need to know about TDD?
What about alternatives to TDD?
What would be the best methodology to organize / develop a TDD web app?
What libraries should I use (if any) to make my life easier?
PS: I'm primarily (but not exclusively) working with PHP here.
Personally I think TDD is at best overkill and at worst an impediment to a the creative process of programming. Time that is spent laboriously writing unit tests for each as yet unwritten methods/classes would be better spent solving the original problem. That being said I am a big fan of unit tests and believe wholeheartedly in them. If I have a particularly complex or troublesome piece of code I'm more than happy to write 20 unit tests for a single method but generally AFTER I have solved the problem. TDD, just like every other programming paradigm, is no silver bullet. If is suits you use it if not keep looking.
But take my opinion with a grain of salt. A much more interesting one comes from Kent Beck and How deep are your unit tests?.
Do you also believe that TDD actually helps to GTD?
My biggest concern was simply not being able to test code. It was too complex. Our core libraries weren't built around an easily testable interface. So we wrote tests for whatever we could. In the end we ended up refactoring our core libraries to make life easier. Besides that, it's a change of a mindset, and i would definitely consider allocating more time on your first TDD project just to kind of flush out some of the problems you may have along the way.
What do I need to know about TDD?
TDD is not a substitute for a methodology. It is a beneficial addition or at least it's supposed to be. If done right, TDD greatly improves software design. It also acts as your internal documentation. If you want somebody to look at your API and understand how it works they can simply look at your well named an formed tests.
What about alternatives to TDD?
Like i said, i wouldn't consider this a substitute for a methodology. There is an alternative and that is not to use it :)
What would be the best methodology to organize / develop a TDD web app?
We have been fairly successful with scrum/agile, if that's what you're asking.
What libraries should I use (if any) to make my life easier?
My PHP knowledge has expired 5 years ago, and i'll let somebody else answer this.
Either way, just my 2 cents. If you feel like reading here is a good overview: http://www.agiledata.org/essays/tdd.html
I've recently started using "fat models thin controllers" http://www.amitshah.in/php/controller-vs-model.html : shovelling as much code into the model (and out of the view / controller) as possible.
I use PHPUnit (and the Zend Framework support for it) to test only a few complex models in my web apps. Writing unit tests that exhaustively check a 2 line function that executes a simple SQL query is a waste of time IMHO. Over the last couple of years I've got lazier and lazier with writing tests, and for most web apps its not worth it because the code is so simple.
But in a recent project (an ecommerce site that tracks stock levels over multiple warehouses with composite products) I started doing test-first development because I knew there was going to be a lot of subtle complexity that I couldn't keep in my head all at once. The only way to keep everything working as my thinking developed was to write some tests. With some parts it seemed more natural to write the class then the tests, other parts were test first, yet other parts didn't need tests because they were trivial. TDD is a tool, not a religion. Use it when it works, stop if it doesn't.
Where I see the benefit in TDD is in reducing complexity and increasing speed (the rate at which I can solve problems). If I write some tests that prove my code is working, then as soon as the all the tests pass, I can move on to the next problem. This puts the fun back in coding for me.
This isn't a question about what framework to use. I've learned both Rails and Django, and I write all of my webapps in PHP. My question is why bother with the frameworks? It's always taken me longer to use a framework than to reuse old MySQL code and build "models" with phpMyAdmin. I also like writing everything myself, because I know what's going on. I can still reuse functions, etc. and do things how I want, and this freedom seems to be missing from most frameworks.
I'm not saying that my way is right; in fact, I'm trying to figure out where my logic fails. The hype can't be just thin air. What am I missing?
The basic idea of a framework is to allow you to work at a higher level of abstruction and write only the code you have to write to implement your specific requirements. All the other repetitive stuff is handled for you by the framework, and probably with far fewer bugs and security holes than if you did it yourself.
It may feel like it takes longer to learn a framework than to just do it yourself using basic language features and standard APIs, but it's simply not true - not if the framework is good and the app is non-trivial, and especially not once you have learned the framework (using a different one for each new project would of course be idiotic) and factor in the time it would take to find and eliminate all the bugs and correct all the design mistakes that have long since been found, eliminated and corrected in the framework by its developer community.
Almost every developer has cowboy coder instincts that tell him "Doing things yourself is much more fun than using code others have written, and I'm sure I'm good enough to get it right the first time, so it will even be faster and better!". These instincts are almost always wrong.
Frameworks allow you to concentrate on the application itself rather than worrying about the boilerplate code that you'd otherwise have to write for every application. They allow you to structure you site in a much more logical (mostly object-oriented) way, using tried and tested design patters such as model-view-controller. The code in framework is generally more mature and of a higher standard than code you would write yourself for one-off projects as framework have a large community of developers perfecting the code perfecting the code over year. This means that framework-driven sites often perform better and are much more secure.
You also mentioned you like writing things yourself - I know where you're coming from. My solution to this was to write my own framework - I get to reuse and improve my code with every project I do and I know the entire codebase inside out.
Writing it your self may make it easier for you to understand things your self but unfortunately it can make it much harder for other developers to understand what is happening. Frameworks will often be better documented and have a larger community that can support a new developer that is working on the app that you wrote.
I think a big part of it is what you focus on. Frameworks standardize the parts that you shouldn't have to keep revisiting, which helps you focus on the application as a whole. If you reuse your own code all the time you're already using your own makeshift framework.
Your comparing a framework (Rails) to a language (PHP). A framework is going to give you pre-built components so you can spend time on what makes your project unique.
You may already have a code base that helps do this for you. Check out some of the PHP frameworks since that's where you are more comfortable. Take a look at CakePHP, CodeIgnitor and/or Zend Framework.
If you are building many small apps/sites, using a framework may make your life easier.
I think a good step for you is to create your own framework with the code you've programmed so far. ;)
Try to make your code parametrizeable, in other words: create components which you can reuse in different parts of a website (for instance: styled containers), or in different websites (form generators/validators).
You can even go further and create base-classes from which you extend new classes to build your websites. (for instance: data objects with generic select/insert/update/delete methods).
I bet this gives you the best view on why frameworks are so damn handy ;)
It'll take you longer to initially use a framework for the same reasons a PHP developer would take longer to initially use Ruby - you're not familiar with it.
Once you're familiar with them, frameworks can offer the ability to skip the mundane and focus on actually writing the important parts of the app.
You should also just use a PHP based framework like Symfony or CakePHP using them should reduce your production time considerably.
One reason to use a frame work is code separation. Take symfony for example. The model is all done with propel or doctrine libraries. Very little SQL needed. You instantiate a new object and user getters and setters, to store your data, and instead of writing SQL in your page code you create functions in the objects related to the query. When you need to access the same kind of data on different pages you are asking the model for it, keeping the business logic with the model where it should be, so there's never any difference. All the work is done in the "action controller function". You get all the data you need, and then put as little php in the display, basically just echoing the variables you got in the action controller, (with the exception of some for loops and if statements for conditionals. I have found this a more efficient way to code, and on my 2nd project saw the production time cut in half.
You don't need to learn a new language python/ruby just to use a great framework, just have to fin one that works for you.
First, PHP has frameworks too, so the question as stated misses the point.
Yes, you can write your own framework, and as Kris said, there's no shame in that. However, part of the leverage of code reuse is the collective value of the efforts of many. It's not just about reusing your own code. Frameworks encapsulate the common tasks and patterns we all share and provide well tested solutions with many iterations of improvements from the community. No individual effort is going to measure up to that, no matter who you are.
If you roll your own, it will only become world class due to the collective effort of world class people, and that will only happen if your idea merits the attention. The top frameworks out there are already proven on those criteria.
DHH is a smart guy, but the Rails we have today never could have been realized by him alone. Not even close.
If you like "writing everything yourself" as you say, then choose a framework with a core philosophy that matches yours, and start making core contributions in the areas where you can see room for improvement.
Depending on the functions of your Web Application, it can be faster to develop without a framework. For example when the Webapp is just some kind of data viewer.
But as soon as you begin to implement more advanced functions, you are much more efficient with a framework.
Try do do this from scratch:
- proper Form validation
- Handling of multiple Language and Date/Time formatting
- Authentication
See a framework as free tools and stable implemented function for you to use.
Sounds to me like you have already written your own framework in php, since you do mention code reuse.
I can imagine it being easier to use your own set of wheels instead of adapting to someone else's. No shame in that.
Frameworks are there mainly to help people who are semi-new to PHP (or the specific language it is built on) to be able to build a website to an extent that it is secure and easy enough to add on extra parts to the site without having to know a lot about the specifics like security, MySQL (or other database types). In my opinion it is a fairly good way to help break coders into a language, allowing for the fact that the framework isn't too complex of course.
EDIT The reason behind me saying they are for beginners is because myself, as a beginner has used frameworks to break myself into languages a lot better.
I want to create a 100% object oriented framework in PHP with no procedural programming at all, and where everything is an object. Much like Java except it will be done in PHP.
Any pointers at what features this thing should have, should it use any of the existing design patterns such as MVC? How creating objects for every table in the database would be possible, and how displaying of HTML templates etc would be done?
Please don't link to an existing framework because I want to do this on my own mainly as a learning excercise. You will be downvoted for linking to an existing framework as your answer and saying 'this does what you want'.
Some features I'd like to have are:
Very easy CRUD page generation
AJAX based pagination
Ajax based form validation if possible, or very easy form validation
Sortable tables
Ability to edit HTML templates using PHP
I've gone through many of problems on your list, so let me spec out how I handle it. I am also OOP addict and find object techniques extremely flexible and powerful yet elegant (if done correctly).
MVC - yes, hands down, MVC is a standard for web applications. It is well documented and understandable model. Furthermore, it does on application level what OOP does on class level, that is, it keeps things separated. Nice addition to MVC is Intercepting Filter pattern. It helps to attach filters for pre- and post-processing request and response. Common use is logging requests, benchmarking, access checking, caching, etc.
OOP representation of database tables/rows is also possible. I use DAO or ActiveRecord on daily basis. Another approach to ORM issues is Row Data Gateway and Table Data Gateway. Here's example implementation of TDG utilising ArrayAccess interface.
HTML templates also can be represented as objects. I use View objects in conjunction with Smarty template engine. I find this technique EXTREMELY flexible, quick, and easy to use. Object representing view should implement __set method so every property gets propagated into Smarty template. Additionally __toString method should be implemented to support views nesting. See example:
$s = new View();
$s->template = 'view/status-bar.tpl';
$s->username = "John Doe";
$page = new View();
$page->template = 'view/page.tpl';
$page->statusBar = $s;
echo $page;
Contents of view/status-bar.tpl:
<div id="status-bar"> Hello {$username} </div>
Contents of view/page.tpl:
<html>
<head>....</head>
<body>
<ul id="main-menu">.....</ul>
{$statusBar}
... rest of the page ...
</body>
</html>
This way you only need to echo $page and inner view (status bar) will be automatically transformed into HTML. Look at complete implementation here. By the way, using one of Intercepting Filters you can wrap the returned view with HTML footer and header, so you don't have to worry about returning complete page from your controller.
The question of whether to use Ajax or not should not be important at time of design. The framework should be flexible enough to support Ajax natively.
Form validation is definitely the thing that could be done in OO manner. Build complex validator object using Composite pattern. Composite validator should iterate through form fields and assigned simple validators and give you Yes/No answer. It also should return error messages so you can update the form (via Ajax or page reload).
Another handy element is automatic translation class for changing data in db to be suitable for user interface. For example, if you have INT(1) field in db representing boolean state and use checkbox in HTML that results in empty string or "on" in _POST or _GET array you cannot just assign one into another. Having translation service that alters the data to be suitable for View or for db is a clean way of sanitizing data. Also, complexity of translation class does not litter your controller code even during very complex transformations (like the one converting Wiki syntax into HTML).
Also i18n problems can be solved using object oriented techniques. I like using __ function (double underscore) to get localised messages. The function instead of performing a lookup and returning message gives me a Proxy object and pre-registers message for later lookup. Once Proxy object is pushed into View AND View is being converted into HTML, i18n backend does look up for all pre-registered messages. This way only one query is run that returns all requested messages.
Access controll issues can be addressed using Business Delegate pattern. I described it in my other Stackoverflow answer.
Finally, if you would like to play with existing code that is fully object oriented, take look at Tigermouse framework. There are some UML diagrams on the page that may help you understand how things work. Please feel free to take over further development of this project, as I have no more time to work on it.
Have a nice hacking!
Now at the risk of being downvoted, whilst at the same time being someone who is developing their own framework, I feel compelled to tell you to at least get some experience using existing frameworks. It doesn't have to be a vast amount of experience maybe do some beginner tutorials for each of the popular ones.
Considering the amount of time it takes to build a good framework, taking the time to look into what you like and loathe about existing solutions will pale in comparison. You don't even need to just look at php frameworks. Rails, Django etc are all popular for a reason.
Building a framework is rewarding, but you need a clear plan and understanding of the task at hand, which is where research comes in.
Some answers to your questions:
Yes, it should probably use MVC as the model view controller paradigm translates well into the world of web applications.
For creating models from records in tables in your database, look into ORM's and the Active Record pattern. Existing implementations to research that I know of include Doctrine, more can be found by searching on here.
For anything AJAX related I suggest using jQuery as a starting point as it makes AJAX very easy to get up and running.
Creating your own framework is a good way to gain an appreciation for some of the things that might be going on under the hood of other frameworks. If you're a perfectionist like me, it gives you a good excuse to agonize over every little detail (e.g. is should that object be called X or Y, should I use a static method or an instance method for this).
I wrote my own (almost completely OO framework a while ago), so here's my advice:
If you've worked with other frameworks before, consider what you liked/didn't like and make sure yours gives you exactly what you want.
I personally love the MVC pattern, I wouldn't dream of doing a project without it. If you like MVC, do it, if you don't don't bother.
If you want to do JavaScript/AJAX stuff, do use a JavaScript library. Coding all your own JavaScript from scratch teaches you a bit about the DOM and JavaScript in general, but ultimately its a waste of time, focus on making your app/framework better instead.
If you don't want to adopt another framework wholesale, take a look at whether there are other open source components you like and might want to use, such as Propel, Smarty, ADOdb, or PEAR components. Writing your own framework doesn't necessarily mean writing everything from scratch.
Use design patterns where they make sense (e.g. singletons for database access perhaps), but don't obsess over them. Ultimately do whatever you think produces the neatest code.
Lastly, I learned a lot by delving into a bit of Ruby on Rails philosophy, You may never use RoR (I didn't), but some of the concepts (especially Convention over Configuration) really resonated with me and really influenced my thinking.
Ultimately, unless your needs are special most people will be more productive if they adopt an existing framework. But reinventing the wheel does teach you more about wheels.
At the risk of sounding glib, this seems to me like any other software project, in this sense:
You need to define your requirements clearly, including motivation and priorities:
WHY do this? What are the key benefits you hope to realize? If the answer is "speed" you might do one thing, if it's "ease of coding" you might do another, if it's "learning experience" you might do a thid
what are the main problems you're trying to solve? And which are most important? Security? Easy UI generation? Scalability?
The answer to "what features it should have" really depends on answers to questions like those above.
Here are my suggestions:
Stop what you're doing.
It's already been done to death.
Click this Zend Framework or that CakePHP or maybe even this Recess Framework.
Now, my reasons:
... if you've worked with developers at all, you've worked with developers that love reinventing the wheel for no good reason. This is a very, very common failure pattern.
... they would go off and write hundreds and thousands of the crappiest languages you could possibly imagine ...
... "Oh, I'm gonna create my own framework, create my own everything," and it's all gonna be crappier than stuff you could just go out and get ...
from StackOverflow Podcast # 3.
So, save yourself some time, and work on something that solves a problem for people like a web app that lets people automatically update Twitter when their cat's litter box needs cleaning. The problem of "Object Oriented PHP Framework" is done. Whatever framework you slap together will never be as reliable or useful or feature rich as any of the freely available, fully supported frameworks available TODAY.
This doesn't mean you can't have a learning experience, but why do it in the dark, creating a framework that will grow into a useless blob of code, leaving you without anything to show for your time? Develop a web app, something for people to use and enjoy, I think you'll find the experience incredibly rewarding and EDUCATIONAL.
Like Jim OHalloran said, writing your own framework gives you a very good insight into how other frameworks do things.
That said, I've written a data-access layer before that almost completely abstracted away any SQL. Application code could request the relevant object and the abstraction layer did lots of magic to fetch the data only when it was needed, didn't needlessly re-fetch, saved only when it was changed, and supported putting some objects on different databases. It also supported replicated databases, and respected replication lag, and had an intelligent collection object. It was also highly extensible: the core was parameter driven and I could add a whole new object with about 15 lines of code - and got all the magic for free.
I've also written a CRUD layout engine which was used for a considerable percentage of a site. The core was parameter driven so it could run list and edit pages for anything, once you wrote a parameter list. It automatically did pagination, save-new-delete support etc etc, leveraging the object layer above. It wasn't object-oriented in and of itself, but it could have been made so.
In other words, a object-oriented framework in PHP is not only possible, it can be very efficient. This was all in PHP 4, BTW, and I bumped up against what was possible with PHP 4 objects a couple of times. :-)
I never got as far as a central dispatch that called objects, but I wasn't far away. I've worked with several frameworks that do that, though, and the file layout can get hairy quickly. For that reason, I would go for a dispatch system that is only as complex as it needs to be and no more. A simple action/view (which is almost MVC anyway) should get you more than far enough.
I initially started creating my own framework with similar ideals to your own. However, after a couple of months I realised I was re-creating work that had been done many times over. In the end I found an open source framework which was easily extendable and used it as a basis for my own development.
The features I implemented myself:
MVC Architecture
Authentication object
Database access class
URL rewriting config
Pagination class
Email class
Encryption
The features I looked at and thought, forget it! I'll build on top of someone elses:
Caching class
Form validation class
FTP class
Plugin-ability classes
Of course, writing a framework that outperforms the open source options is possible, but why would you bother?
It's true that some developers reinvent the wheel for no good reason. But because there are already good frameworks around doesn't mean that it's a waste of time doing one yourself. I started on one a while ago with no intention of using it for anything more than an exercise. I highly recommend doing it.
I've got the perfect link for you my friend: http://nettuts.com/tutorials/php/creating-a-php5-framework-part-1/. This is an awesome tutorial I have looked at, and its not too overwhelming. Plus look around the PHP section of that site I saw an article on CRUD. As for the AJAX look elsewhere, but you have to start somewhere, and this tutorial is awesome.
Note: this tutorial has 3 parts and I think it brings up MVC in the second instalment, but starts the first part using other methods.
The one, huge selling point I would look for in a new framework is that it would make writing testable code easy.
We typically work with Zend Framework, and it's mostly awesome, but trying to unit test/test drive ZF-based code is not far short of masochism.
If you could provide a framework that replaces the MVC parts of ZF with something that allows us to write testable code, whilst still allowing us to use the library parts of ZF, I will - quite literally - buy you a beer.
I'll buy you two beers if you ditch the AJAX. There's a huge gulf between an OO PHP framework and a JavaScript framework.
Please don't link to an existing framework
I will not, I started writing my own for learning purposes, and took a peek into some of the mainstream frameworks, and even with my limited knowledge see so many mistakes and bad ideas in them.
They're built by hardcore developers, not end users.
I'm in no way saying I could write better than the "big boys" but I (along with most of you I imagine) could point out why some things they do are bad, even if just because they're not end user/non-developer friendly...
I wonder how your framework is doing, some 6 years on?
Are you still working on it? Did you stop?
Should You Write Your Own Framework
This is probably a little late for you, but for anyone else, writing your own framework is a fantastic thing to do for learning purposes.
If, however, you are wanting to write one other than learning purposes, because you cannot work out the one you are using, or because it's too bloated, then do not!
Believe me, and don't be insulted, you would not be here contemplating it if you are a knowledgeable enough developer to do so successfully!
Last year I wanted to learn OOP/classes, and more advanced PHP.
And writing my own framework was the best thing I did (am actually still doing), as I have learned so much more than I anticipated.
Along the way I've learned (to name a few):
OOP/Classes many best practices which come with it - such as
Dependency Injection, SRP
Design patterns, which help you write code and structure your system
in such a way that it makes many things logical and easy. For an
example see Wiki - SOLID
Namespaces
PHP Error Handling and all of the functionality which that provides
A more robust (and better) understanding of MVC, and how to apply it
appropriately (as there is no clear cut way to use it, just guides
and best practices).
Autoloading (of classes for OOP)
Better code writing style and more structured layout, and better
commenting skills
Naming conventions (it's fun making your own, even if based on
common practices).
And many other basic PHP things which you invariably come across accidentally from reading something.
All of this not only vastly improved my grasp of PHP and things which come with it, to a more advanced level, but also some of the commercially/widely used methods and principles.
And this all boosted my confidence in using PHP in general, which in turns makes it easier to learn.
Why Write a Framework To Learn All of This
When you start out, you learn the basics - A (variables), then B (how to write a basic function), etc.
But it doesn't take long when you're trying to learn more advanced things, that to learn and use D and E, you also have to learn and understand F, G, H, and J, and to know those you have to know K, L, and M, and to know parts of L and M you first need to understand N and O...
It becomes a minefield as trying to learn one thing brings the need to first learn a few other things, and those other things often bring a need to understand various other things.
And you end up a mile away from where you started, your mind tingling and shooting sparks from it, and about 20 tabs open all with various advanced PHP things, none of which you are 100% comfortable with.
But over time, with practice and most certainly dedication, it will all fit into place, and you'll look back at code, even a collection of files/classes, and think "Did I write that.."?
Writing a framework helped greatly with this "minefield" because:
I had specific tasks to do, which brought about the need to learn and
implement other things, but specific things. This allowed me to focus
on less things at once, and even when something branches off to
various other things, you can reel it back in to where you started
because you are working on something specific. You can do this with
any learning, but if you do not have some goal, or specific task you
are focusing on, you can easily get distracted and lost in the ether
of things to learn.
I had something practical to work with. Often reading tutorials about
an animal class, and how cat and dog classes extend animal etc,
can be confusing. When you have a real life task in your own
framework, such as how do I manage XYZ, then you can learn how
classes work easier because you have trial and error and a solid
requirement which you understand, because you created the
requirement! Not just theory-like reading which means nothing
usually.
I could put it down when my mind was blown, although as it was my
framework (my Frankenstein's monster in the beginning :P) I wanted to
press on, because it was interesting, and a personal goal to learn
and sort the next stage, to resolve an issue I was stuck with, etc.
You can do it how you want. It might not be best practice, but as long as you are trying to learn best practice, over time you will improve, and likely easier than just reading tutorials, because you are in control of what and how you do something.
Wait, I Shouldn't Re-invent the Wheel Though
Well, firstly, you cannot reinvent the wheel, it is impossible, as you will just make a wheel.
When people say "Don't reinvent the wheel", they of course mean "there are already frameworks out there", and to be fair, they are written by skilled developers.
That's not to say the frameworks don't have problems or issues, but in general they are pretty solid, secure and well written.
But the statement is nonsensical in relation to writing your own framework!
Writing your own framework for learning purposes is really useful.
Even if you plan to use it commercially, or for your own website, you haven't just "re-invented the wheel", you've made something else.
Your framework won't be like the others, it won't have many features and functionality, which might be a major advantage to you!
As long as you understand about best security practices etc, because you can think you are writing a great system, which is super fast and without all the bloat other frameworks have, but in fact you have holes in places which someone could crawl into...
But a project for learning which you don't use on the internet is ideal - or use it, eventually, when you are advanced enough to know it's secure!
With all that said, you should write your own framework IF:
You are not needing it any time soon! It takes a lot of time as
there are so many aspects to consider, learn, and trial and error
leads to refactoring (a lot at first!)
You are willing to read, code, test, change, read, code, and read
some more. There is a lot of good advice on the internet for advanced
PHP, most of it mind blowing at first, like reading all the design
patterns. But they eventually make sense, and end up helping you
resolve problems you face, and how to do things within your
framework.
Willing to put the time in, and keep trying to improve, and head
towards best practice, especially with security. Speed issues shouldn't be an issue with a small framework, and besides, if you have a fairly decent system, you can usually refactor and make speed improvements. usually if you have significant speed issues it means you've chosen intensive operations, which can usually be addressed by doing it a different way.
.
Without previous experience, or an advanced knowledge of PHP, you will likely spend some time writing a framework, further reading and knowledge will show you that your approach is skewed, and so you might delete everything and start again.
Don't be disheartened by this.
I did exactly that, as I learned so much advanced patterns and ways of doing things along the way in the first month, I ended up where refactoring was no good, and a blank canvas with a whole new approach was the only option.
However, this was quite pleasing, as I saw a much better structure take form, and I could see not only a better framework foundation start to take place, but realised it was because I had a better understanding of advanced PHP.
Just do it! Just make sure you have a plan of what you want it to do before you even write some code.
Seriously, write down on paper how you are going to load error checking, are you going to have auto loading, or include files when needed? Are you going to have a centralised loading mechanism, which instantiates classes when you need them, or some other method?
Whatever you do, and whatever stage you are at, if you are heading into new territory, plan it first. You'll be glad of it when you hit a brick wall, can go back to your plans, and realise a slight deviation to your plans will resolve it.
Otherwise you just end up with a mess and no plan or way to re-deign it to resolve the current problem or requirement you face.
You are looking to build exactly same thing I've worked on for a few years and the result is Agile Toolkit.
Very easy CRUD page generation
$page->add('CRUD')->setModel('User');
AJAX based pagination
All pagination and many other things are implemented through a native support for AJAX and Object Reloading. Below code shows a themed button with random label. Button is reloaded if clicked showing new number.
$b=$page->add('Button')->setLabel(rand(1,50));
$b->js('click')->reload();
Ajax based form validation if possible, or very easy form validation
All form validations is AJAX based. Response from server is a JavaScript chain which instructs browser to either highlight and display error message or to redirect to a next page or perform any other javascript action.
Sortable tables
Table sorting and pagination has a very intuitive and simple implementation when you can really on object reloading.
Ability to edit HTML templates using PHP
This seems out of place and a wrong thing to do. Templates are better of in the VCS.