Many conversations on frameworks tend to discuss ease of use and speed of development. These issues are important but they tend to come from developers who are creating new low traffic projects over and over for different clients. It is important in that situation to be able to knock out a solid solution for the client in as little time as possible. I myself have operated in this area for many years and have used many MVC frameworks with much successes.
But what about when you are working on a single high traffic application for several years? Now ease of use and speed of development start to take a back seat to scalability and speed. It doesn't matter how easy it is to use or how fast you can write code if the application won't function because of speed and scale.
My question to any large scale developers out there is, are frameworks still useful in this situation and if so which have been used successfully in large scale production situations?
common frameworks: zend, symphony, codeignitor, cakephp
When it comes to large applications, it's not the framework you should be concerned about, it's the database.
First decide which database you going to use, then look for a framework that has good support for that database. If you want your app to scale to unlimited size and still be fast, you should eliminate Mysql as an option right from the start.
Look for NoSQL solution, find the one that you like. Cassandra, MongoDB, CouchDB are the top 3 choices, but there are others. All of these are fairly new, but some of the biggest sites in the Internet are using them. Do your research, choose database, then choose framework.
I don't know most of the frameworks, I looked at Zend and Symphony2, both support MongoDB, both are pretty fast. I would look at either one of these 2, no reason to look anywhere else.
But really, if you plan on writing a really large application you should really write your own code and don't use frameworks. This way you will have only the functions that you need and none of functions you don't need. Remember most frameworks offer you a lot more functionality then you really need. Unnecessary functionality usually slows down your app.
if you want a fast app, then it has to be lean, only do what you need and nothing else.
symfony is used by Yahoo on it's Bookmarks and Answers site.
You should note they ditched the ORM so the database remains as the main concern for scaling (as Dmitri stated), you can read more about it here.
Dailymotion is also using symfony.
My question to any large scale developers out there is, are frameworks still useful in this situation and if so which have been used successfully in large scale production situations?
I've seen symfony-based applications successfully scale up to many thousands of users so is a strong candidate.
The creator of php liked codeigniter(in the past) the best, because it is the fastest framework when he benchmarked it. You can listen to video about that over here.
To add on to what the others have already said:
You can start off with any framework as your starting point. As your application gets bigger (and I mean, BIG), you can start modifying the framework and customizing it to your application's specific needs. After all, frameworks are a general solution to most problems.
You should pick whatever framework you're most comfortable with, and tailor it to your needs if your application ever grows extremely large. No point worrying about a million visitors when you don't have them yet.
WordPress.com is about the 22nd most popular website in the world and it runs on (you guessed it) WordPress, which is a framework with a large amount of overhead. So it definitely is possible.
Whether you are using a framework or not, you end up working on the same kinds of problems as your site gets to massive scale. Building on top of a framework means that you'll have one less problem (no custom framework to write) while working on other scaling issues. However, you may have to extend or tweak that framework to work with whatever technology you need to scale.
And in general, what I would recommend is that you don't blaze your own path. Find an app or site that is comparable to the one you are building and figure out what technology and techniques they used to build that site (many like WordPress.com, Twitter, Google projects are well documented or have chatty developers) and use the same technologies and techniques. You'll benefit from them figuring everything out ahead of time.
Related
A little while ago I read a great article which described a number of reasons against using any of the RAD frameworks available for PHP. Basically, it argued that a good framework should get you off the ground quickly, and then should get out of your way. But none of the PHP frameworks did that. It pointed out that Django was good at doing just that (but that's obviously not a PHP framework).
For the life of me, I can't find the article now.
So I'm curious. Does anyone have any solid arguments as to why applications should not be built on top of a RAD framework? And I'm not necessarily talking about generic applications (frameworks by definition try to solve a generic problem. The quesiton is does that translate well to specific problems).
And when I say built on top of, I mean from the ground up based on the framework. I don't mean referencing the framework as a series of libraries. I mean basing the entire architecture of the application off of the framework (which then ties you into the framework).
I'm also not really talking about rapid prototyping where the code will likely be re-written anyway. I'm more looking at long-term applications that have a specific business requirement to meet, and must be supported and maintained (and modified) for a relatively long period of time.
We always hear about why we should be using a framework. There are reasons galore:
Not reinventing the wheel (although I hate this reason)
Faster development time (since architecture is skipped)
Easier to bring new developers in
Common problems are already solved
etc...
But I'm looking for the antithesis...
Any thoughts?
It is implied that frameworks being written by some unknown heroes who is smarter than you by default.
But as a matter if fact it's often not true.
It has bugs, it can be messy and unnecessary overcomplicated.
Thus, having your own helper library is often more handy than using such a monster, knowing roughly 10% of it's features.
A few ideas:
The type application you are creating does not have a commercial or free framework available (custom hardware-based apps come to mind) (this is the obvious answer)
The system has extensibility requirements that are made more difficult by using a RAD framework (i.e. framework values convention over configuration, where you really need the configuration)
The project scope is small enough that it doesn't make sense to invest the learning curve into using a RAD framework (if you already are past the learning curve, this may or may not apply)
The tools/approach/back-end/platform requirements provided by the framework will introduce unnecessary complexity (YAGNI, using a framework without a business need for its features may increase support costs as you have to support the whole thing, including unused features)
My fear has always been, what happens when a framework stops being maintained/updated (for whatever reason)? Your project is locked into whatever PHP/MySQL versions the framework supported.
I'm doing a little framework research lately.
http://matrix.include-once.org/framework/
And to some extend I have to agree. The majority of frameworks don't significantly simplify development. Particularily the "MVC" frameworks are about abstraction more than utility. There are however always features which do help for everyday tasks. ActiveRecord/ORM come to mind as reducing tedious manual work. Form helpers might do, but I haven't seen a seriously helpful one yet.
Generally I believe, the more functional library code a framework provides, the higher the productivity. But in the case of Zend Framework, it's randomly dwarfed by complexity. However, check some of the smaller frameworks out. Avoid the overly abstract. Look for scaffolding tools, not libraries.
One of the main advantage to not use a framework is that you have more flexibility in what you can code. For project with a smaller scale, it is often better to have that flexibility.
Some frameworks need shell access to deploy.
Scalability, I've read that sharding is an issue
If performance is a priority, frameworks tend to slow down execution speeds
Not all developers familiar with MVC pattern so maintainability if developers rotating in/out of project could be a challenge.
Zend Framework is a good framework but not very fast.
Can you tell whether it's worth using Zend Framework for highload projects, for example, for email marketing service that can inlude about ten or houndred thousand of users?
Is it possible to achive acceptable performance using Zend Framework?
Has anybody such an expirience?
Thank you very much.
For what I have seen, the definitive defense of Zend Framework performance and recommendations for performance optimization comes from Padraic Brady at:
PHP Framework Benchmarks: Entertaining But Ultimately Useless
In particular, note his four recommendations for performance optimization:
Don't use Zend_Application. While Zend_App is great for creating consistent complex bootstraps within a standardised structure, it doesn't come without a significant performance hit to baseline performance. A more direct bootstrap (typical of ZF until Zend_App arrived) is far faster and can also be done without configuration files.
Skip using the ViewRenderer plugin. Without the ViewRenderer, you need to manually configure Zend_View and add render() calls to Controllers. This is actually very simple to do and is fairly fast - fast was never really part of the ViewRenderer's genetics.
Use autoloading. Strip require_once calls from the framework library so unneeded files are ignored. Replace uses of Zend_Loader_Autoloader with a not-so-crazy autoloader function. In fact, pray Zend_Loader is never used - it does a lot of file ops that, to date, have never been explained to me as having any value.
Preload everything (Symfony 2 Preview does!). It buys you some performance cookies and equalises the speed baseline. Using a simple preload script is not that hard.
We've used ZF in a lot of high traffic sites, and we've had no issues so far. We did have to jump through a few low-hanging hoops, though.
Some suggestions:
use Zend_Queue to help with batch mailing
use Zend_Cache whenever possible
Use plugin loader cache
Strip require_once calls in favor of autoloading
Get rid of components you don't want. (as suggested, you would not need MVC stack for CLI / mail)
We chose Sphinx in favor of Zend_Search_Lucene (enormous performance gain)
The bottom line for us has been this: development time is much, much more expensive than hardware. The flexiblity and higher re-use of code completely trumps any minor performance losses we had to deal with. For the most part, the performance overhead was very fixed.
You ask:
Is it possible to achive acceptable performance using Zend Framework? Has anybody such an expirience?
Yes, I have experience with a site with millions of users. But you do need to use techniques to deal with the high load. Caching etc...
A CDN can help a lot. Look into developing with the cloud. Amazon might be a pain to get started with but it helps you scale if need be.
I guess what I'm saying is, the Framework may cost you a bit of performance, but helps make maintenance possible and building it faster (once you get over the learning curve). Then you you have to evaluate what needs to be done to improve performance (although it helps a lot to plan for what will be obvious problems, right from the get go).
I know of several companies that use ZF in high-performance/high-load scenarios. I don't know which ones I can state and which ones I can't, but some of them are media companies who have to handle popular TV shows. Others handle live sporting events. Others are multi-billion dollar companies who need to serve their internal organizations. So, ZF is being used by plenty of companies who run pretty high-load sites. One of our case studies is Fox Interactive (http://framework.zend.com/about/casestudies) and I know of several other customers who use it for high-performance websites.
Zend Framework MVC, out of the box, will be quite fast. My blog comes back in about 100ms without caching and there's a fair amount of stuff that happens on my front page. I could probably drop that down to 50ms with some internal caching (Full page caching could drop it down to single digit ms, but then it's not touching ZF).
Seconding Joe's answer. I've also seen ZF deployed on a few sites handling millions of requests and have yet to encounter a problem. When dealing with that amount of traffic it's a good idea to use other strategies beyond your framework, including but not limited to caching and the use of a CDN.
I've found most frameworks will call or create many class instances per request which I think is what causes people to say that framework X is slow without having any real world experience with it. Any hit you take there can be easily mitigated by using an accelerator and caching.
If you already have a team of devs you've hired, I'd suggest using what they feel most comfortable with and have the most experience with. Best case they'll be able to tune their code for that framework.
A lot of the framework, any framework really, is used for building and managing the project development but the resulting project is 'just' php, html, css etc. the same as any other php web site. So what evidence do you have, that's real timing against other framework and non-framework built sites not anecdotal evidence, that a Zend project site is slow.
Edit -- answers to below --
I don't think the structure that the framework uses will hurt performance. It may be more a question of PHP being acceptable and then how much 'overhead' is added with the site design and the optimisation of loading say JavaScript's etc. I would imagine that using the Yui guidelines of minifying JavaScript and CSS and loading them in the correct order and making sure the PHP code is efficient will help. You can also use other standard things such as DB Caching and Zend Accelerator will speed things up. One thing to be careful of would be the DB connection. The use of an ORM layer might have an impact.
However back to the original question about the framework i think it is similar to asking if using Eclipse or Textmate has an effect on the speed of the resulting site.
At my workplace we're planning a major refactor on our core product, a web application with several 'modules'. I quoted that because that's one of our main concerns: modules are not really modules, the whole thing is monolithic. The application is written in PHP with smarty templating and using Pear for accessing a MySQL database. We're not really concerned with database independence, although it would be nice if that wouldn't take months to implement.
Our main concerns are that development time/cost is increasing exponentially because of bugs popping up in unrelated places and not having a sound common architecture to rely on to get the most common functionality (each module is basically copy/paste from the previous one, then adapt).
I've got some experience with the web MVC principle, mainly in ASP.NET MVC. I like the clean separation it offers and the testability. However, when trying this on a local machine the app is simply a lot slower than it should be.
Alright, enough introduction, off to the questions:
- Should I rely on caching modules? Does this remove most of the overhead using a good architecture provide? Something like APC.
The application is mainly read. Writing is mainly single values (change a single field on a record). Any OR/M for PHP that are good at this?
Also looking for a flexible MVC framework. I know Zend, CakePHP, maybe Symfony?
The tricky part is that we don't have the luxury of being able to do a full rewrite. We'll have to incrementally improve a currently very messy codebase. This has to be done while writing new code, or fixing bugs. One thing I'd really, REALLY like to be able to do is write a regression test for a new bug before fixing it, to prevent it from popping up again later (this happens, occasionally).
The stack I'm currently considering contains:
MVC framework of choice
Logging (log4php?)
an OR/M of choice (doesn't have to be dynamic, code generation is fine too)
IoC container of choice
Smarty Templating, perhaps abstracted so we can switch it out if we need to.
Opcode cache of choice (we're using one now, forgot which one, have to ask sysadmin)
The main point that worries me is the performance implications of creating clean code in PHP. Seeing it's a parsed language opposed to something like the .NET/Java web stack, creating abstractions for otherwise in-line code (with obligatory separation in different files) might create new problems on another level.
Note: Retag if you come up with more appropriate tags, I'm not sure on the current ones.
Having a clean setup isn't a performance issue, usually. Most performance is spent with databases or other external systems you're talking to.
Except for these there are usually one or two hotspots which might be worth optimizing but for that you should start with a clean design, then use a profiler (like XDebug or ZendDebugger) to identify the bottlenecks.
A clean software design is way more important than the 0.01% performance gain by a "optimized" design. Usuallyit's even cheaper to buy and run more hardware than worry about an "optimized" codebase which is unmaintainable.
I'd stress budgeting time to build tests, with the following arguments to management:
When developers fix a bug, allow them to write a test for the bug. Bugs reoccur much more often than they probably should, and this is a cheap and effective way to stop that completely.
When developers are building new functionality, allow them to write tests under it. Since they're completely familiar with the functionality at that point, this is the least expensive time to build the "safety net" of automated testing.
Don't candy coat how long testing will take you; whether it's 1% of your time or 50%, give that to the manager straight, but stress that building automated testing as a safety net will stop users from hitting as many bugs, and will save developer time for new development instead of bugfixing.
As far as managing an MVC component with a spaghetti code component, we had a similar issue with a large project. What worked well was just taking a directory and making that the new docroot for MVC app (Zend Framework in our case) such that:
old part:
http://site.com/data.php
http://site.com/other.php
new part:
http://site.com/app/controller/action/...
Re authentication, you have a couple of choices. Probably the most logical is to redirect your login.php script to the MVC login and then pass it back to the original page that you want to go with necessary info passed as a GET parameter. This will allow legacy and new systems to exist simultaneously and transparently.
Re slowness, before I would pull out XDebug, I would try to isolate a problematic part and just output times it takes. Faster IMHO.
There isn't any good reason that well structured object oriented code should perform significantly worse than sapghetti php code in a database driven web application. You need to do some profiling to find where your bottlenecks are and optimize accordingly.
You do have a tough (but not uncommon) situation.
As far as organizing the code to minimize bugs, all I can give is a tip of the cap to DRY.
For performance issues, those are easy to find, because their very slowness shows them to you, by this technique.
I'd like to create some small websites using PHP as the programming language. I've looked at several frameworks but they all appear to be rather large and am not sure how well they will run in a shared hosting environment. I was wondering if anyone knew of frameworks that work well in shared hosting? I'm looking for something with MVC and ORM features and anything else extra would just be a bonus.
ElisLabs codeigniter(http://codeigniter.com/) should fit the bill like a glove.
This is my framework of choice when i do php projects.
Although I personally love CakePHP, and I've done many projects for clients running on shared hosts and it runs just fine, if a light footprint is really important for you, CodeIgniter would be my choice.
CakePHP. I don't see why it wouldn't work in shared hosting.
As has already been mentioned, CodeIgniter is a great PHP framework. It has a great user guide, community and a light footprint.
Notch up another vote for CodeIgniter - it's really easy and quick to learn, and is very lightweight, so good on shared hosting.
Since you are talking about small applications, it probably doesn't matter. CodeIgniter seems to be the fastest.
However for big applications (or ones that could become big (talking about load, not quantity of code)) I suggest not using a "general" frameworks at all. They may speed up development a lot, but in most cases they are real computer power killers.
According to - for example - some tests of Paul M. Jones frameworks eat up about 90% to 97% of the system ressources (CodeIgniter scored good by eating just 85%). That means that if your computer can handle 100 requests per second using a framework you'll have only 15 possible requests left before the server goes boom.
Of course the results will vary based on the application. The tests Paul performed were simple "Hello World" pages with no database and no other calculations. Obviously the "application / framework" ratio will change a lot if you add application logic.
Not trying to stop you from using a framework... Just trying to make you think before you start coding...
Kohana is a good choice too. Originally forked from CodeIgniter, it's now been rewritten as a PHP5-only, real open-source project that is optimized for speed, rapid development and flexibility.
Side note/disclosure, I was originally involved in the fork that formed into Kohana. CodeIgniter (at least about a year or two ago, when I was involved) is very not open to user contributions. I had several enhancements and bug fixes that I contributed patches for, that ended up rotting, basically, as they were ignored*. As a result, several people took up a huge interest in getting their contributions in, and while Kohana is based on all the same principles and ideas as CodeIgniter, the implementation is much better, and much easier to work with.
(*: not due to code quality, just do to simply not being interested because it didn't necessarily fit in the direct that Ellislabs was going since it was the backing for their commercial product. It was like if a change was made -- even if it made a huge improvement to the library -- that would have required them to rewrite/change anything they had built on top, it would be ignored.)
I really like CakePHP, it is simple and has pretty good documentation. I have set it up pretty easily on MediaTemple, I just had to change the webroot variable in the config file.
Qcodo is another option to review.
Take a look at Sonic. It's built to be really light and fast. The core framework is less than 3000 lines of code, and you can create an entire app with just a single 26kb file.
It has an extension framework set up to allow you to install extensions on top of it (ORM, Database, Cache, etc).
view site:
http://sonicframework.com
browse source:
http://github.com/ccampbell/sonic
I am thinking of using a PHP framework called CodeIgniter.
One of the things I am interested in is its speed. I have, however, no way to find out how fast it is, and would rather not simply take the word of their website for it. Does anybody know how I can determine its speed myself, or can someone tell me of a site that can?
Code Igniter also has some built-in benchmarking tools:
http://codeigniter.com/user_guide/general/profiling.html
Yes, the problem is you have to build your application to profile it.
At work we had a couple of projects written outside which we load-tested before putting them on our main boxes. We were quite surprised to find critical performance problems with both; one was written in CakePHP and the other was written using Drupal. I don't think this highlights a problem with any framework or CMS other than the need to do profiling and load-testing on any site which is going to get significant traffic. In both cases it was what the developer had done, rather than the characteristics of the software platform, that caused the problem. For example, there was a recursive function call the developer had created in the Cake project which instantiated the entire Cake object every recursion and this would have taken out the server had it gone live under load.
In my opinion performance should not be a deciding factor in choosing a framework; the objective differences are likely to be marginal and the way you use it is likely to cause far more performance problems than the inherent performance of the framework.
I believe that to scale any PHP application to run under load, you will need an opcode cache and you'll need to write in intelligent content caching using something like memcached or whatever built-in caching your framework supports.
If your site is database-driven I would be very surprised if your bottleneck would be the application framework. "Fast" as in faster development is what I would worry about rather than "fast" as in speedy handling of requests. Significant optimization is better done by caching strategies and optimizing your database access.
Besides database access your own code will be where most of the time for each request is spent (and even that is usually not significant compared to database access), the framework will likely not be affecting the time spent on a request, unless it is really badly written.
It way be better to look for a framework which has good caching support (which Code Igniter may have, I don't know), that will almost always save you more time than the few milliseconds you could shave off the request handling by using a slightly faster framework.
Have a look at the Zend Framework too, it has the benefit of being PHP 5, whereas Code Igniter is still PHP 4, as I understand it. That may be an issue when it comes to speed, but in favor of which framework I don't know. Zend has good caching support and a database profiler that can help you find where your bottlenecks are.
i'd recommend testing it for yourself. use xdebug's profiler to create a cachegrind compatible file and webgrind to visualize the file.
that way you end up with very reliable information.
CodeIgniter is plenty fast for most projects. Some have posted here and if you Google, you will find that it compares favorably to other frameworks with respect to speed.
I would agree with another poster that performance is usually not a big concern when it comes to framework choice. The major frameworks all have sufficient performance for most projects.
I maintain a site that gets slammed a few times a year. Last year the development team rewrote the entire site using Codeigniter and we have had much luck in terms of performance. Additionally, the time it took to perform the rewrite was minimal as this framework is quite easy to work with. CakePHP in my opinion is also a good choice if you find that you don't like Codeigniter.
For CodeIgniter and other PHP frameworks, PHP Quick Profiler is very handy for benchmarking and measuring speed especially for database queries. You must check this out:
php-quick-profiler
It's very easy to install and provides an awesome GUI for examine different benchmarking tests.