I am working on a Facebook game which is developed using Zend framework. Right now I don't have lots of traffic and already seen quite a large # of data usage / CPU time.
Actually, I'm not good at Zend. I good at coding from scratch for both PHP & JS.
so, I am curious about the performance of Zend framework. becuase I'm thinking about rebuilding the applciation using Zend as the backend to manage the data / session / logic. and use JS (native code or JQuery) for the front-end rendering UI and handle user action in the client side.
In between, use aJax to get data from Zend backend.. most likely REST.
Anyone has suggestion about this kind of structure? I want to cut down the server load by that and also easier to manage code, plus better user experience.
Appreciate if anyone has good idea. :)
(POST few days later)
so, baseline PHP should be faster and use less data transfer (if code correctly) then Zend (or any) framework, right? Code reusablility is not a big concern here. :)
One thing many php developers fall victim of is sacrificing good architecture and sound principles for what they perceive as performance.
You may decide to cut corners in your code, but remember, "premature optimization is the root of all evil". So if you need to optimize early make sure that you're actually doing something useful.
The Zend libraries are engineered with best practice in mind, not necessarily performance. The rationale is that there are many ways to speed things up later without sacrificing code maintenance and readability (caching, load balancing, hardware, queue management, etc).
That been said, I don't think what you're looking for are statistics on ZF's performance, but rather advice on how to setup your application with it. Specifically, I'd advise you to create a dedicated, very light bootstrap for ajax requests. In ajax, you would normally only need some minimal prerequisites before dropping inside your controller. For the non ajax requests, set them up normally using the recommended architecture (bootstrap, front controller+plugins, controller+helpers, model+views+helpers).
My personal rule of thumb is that if I only am going to serve about 100 requests throughout the day, then there's very little reason to optimize anything. When the application starts feeling sluggish, if it generates enough revenue, maybe I can get a dedicated server, if not, there are always solutions such as apc, memcached, beanstalkd, etc.
Here's what you're looking for: PHP framework comparison benchmarks
Zend Framework vs raw PHP code tends to be close to a 1:500 requests per second ratio.
If you have a lot of time and not a lot of money, develop in raw PHP. If you have little time and enough money, develop in whatever is fastest without regard for performance...you can always add hardware later. If you fall in between the two, balance it as you see fit.
As much as I dislike it, CakePHP is faster to develop in than ZF, so it's great for time crunches. CodeIgniter is faster than either, but doesn't have as much built-in, so it's closer to the performance end of things without going to raw PHP.
There is only one thing to do first if you think that a ZF-based app is slow. Measure it. Various tools exist that will take the profiling output of Xdebug to show you which parts are slowing down the process, then you can make some moves towards optimising those parts (such as a lighter weight initialisation, and/or some caching). One good resource is the the Zend Framework book at survivethedeepend.com, "performance optimisation for ZF Apps".
In my opinion, benchmarks of frameworks are useless. They say something about an abstract situation, but performance is very situational. You should measure your application and optimise that. Yes, there's probably a limit to how far you can make your application go if you use lots of components from Zend Framework, but then there's also a limit for how fast your application can go before you need to drop PHP and write it in C. It's possible to make Zend Framework perform just fine on a high load site, but you have to put an effort in to it. Just like you have to if you don't use it.
Related
What I would propose would probably sound strange, but I have my reasons for it.
For a long time, we've had this Spring based API, which originated as an abstract set of REST services for CRUD functionality. With time however, we started building business and presentation layers on top until we reached a state of a dead end. Do not get me wrong, Spring/Hibernate are great frameworks, and being built on top of the JVM has its definite advantages, including performance over other web technologies, such as PHP. Compared to PHP, it gives us a much deeper way of controlling aspects such as transactioning, multithreading, dealing with byte data, control over native C++ applications, using JNI, etc.
Where the stack clearly hits a hard wall, is where the requirements get changed most often, namely the business and presentation layers. Turning the applciation into a modern, user-centered social applciation, we've experienced the toughest challenge in out careers. Java EE presentation technologies are tough to work with. Also, changing the business requirements became a tremendously long cycling because of the traditional hurdles in building testing and deploying massive Java applications.
It also felt like for a large part, we are trying to reinvent the wheel. In the PHP world, so many projects exist already, which give you a complete management system, agnostic to any kind of backend system (mapping hooks to REST/SOAP endpoints) . Many of them have all these functionalities ready, allow for admin-friendly change of scenarious, and rules, have templating, etc. Plus, being PHP based means absolutely no time wasted in building and deploying. Write the change, test, asert that it works, and switch.
Our idea now is to find a way to move the business and presentation layers in such a kind of front-server PHP-based application, and leave the pure backend stuff to Spring/Hibernate. We have a few concerns though, coming from our relative inexperience with Spring.
If we implement the business methods using PHP methods, how do we keep transactional security? I mean if a business method has to make three separate HTTP requests to the JAVA, how can we guarantee that they will all be executed in the same transaction, DB-wise?
Is there a way to use proxies or promise objects between both systems? For instance, if the PHP business method needs to call a Spring search method get a collection of objects from the database, and then pass it on to another spring method, this will mean that teh whole collection will have to be sent back and forth. Perhaps, one could store it in a session object on the JAVA side, and simply sent an empty proxy back to the frontend, which the frontend can bass back to another jav method.
A lot of our Spring based functionailities rely on a structure of plugins, using Spring events. How can we make so that our frontend server also gets notified on every ackend event that happens.
I have two ideas here: a post-processing-filter that simply makes a POST request to a controller on the frontend server, using some naming convention.
or ... using some kind of a message queue, such as JMS or RabbitMQ, or why not even something like Reddis, where one can watch data for changes
Anyone who has done that before? Is this a good idea in general? Any suggestions how to resolve the aforementioned issues?
This is not what you asked, but I think it's worth saying that some of the problems you are experiencing with Java are problems that most experienced Java developers faced some day, and most have found some solution. For instance, you can use Arquillian to do a "faster" TDD in Java. And you can use JRebel to do a "code-and-refresh" approach, like in PHP. Modern IDEs also helps in refactoring, something that helps in the "requirements change" scenario.
Granted, the presentation part is still a big problem with Java. I personally don't like JSF, and (to me), most of the other presentation technologies are either non-intuitive or they are cumbersome.
I would say that this is the reason why a lot of Java developers are becoming adopters of HTML5 and Javascript (backbone.js, underscore.js, jquery, ...) for the frontend, with REST in the backend. There's no need to have PHP in the middle.
I'm afraid I cannot answer your other questions, but perhaps a good start would be to see if PHP could be run from inside a Java EE container? I know this works for Ruby and Python apps, as JRuby and Jython would take care of the bridge between the two world.
Largely what you are trying to do is what we call: SOFEA
In this case PHP is your frontend and Java is your service layer.
Concern 1
Transactions over traditional REST or WS are a PITA.
Either consider aggregating your service calls to do more work so that the whole entire transaction is in one service calls or have Java handle your frontend. Otherwise see Concern 3 with Finagle.
Concern 2
I recommend you stay stateless and do REST or RCP.
It sounds like your concerned with speed which I think you shouldn't worry about till its a problem. However if you do want to do user based caching of objects I would use Redis and its pub/sub feature (or RabbitMQ + memcache).
Concern 3
Use RabbitMQ, or Redis, or/and Finagle.
MY final Opinion
I don't think you need PHP particularly if you know Java.
So much of the web app world today is moving towards Javascript (in the client) I don't see any benefit to using something like PHP serving pages when there are better options like Node.js or just sticking with Java.
And #jpkrohling is right... templating and frotend used to suck in Java but its gotten much better particularly since most of your UI code will be in Javascript anyway.
If you need an example of more modern web stuff in Java have a look at this project: MWA and also by the same author: Handlebars in Java.
To sum it up my opinion and go back to SOFEA: backend = Java and frontend = Javascript
Haven't done this before, not sure if it's a good idea, but here are some thoughts on your questions:
First of all, in the PHP world, each request (whether it uses keep-alive or not) is run as an isolated process, so this scenario is typically unheard of; it's possible to use persistent database connections, but any non-committed transactions are automatically rolled back when the request is done. Now, this is where my knowledge about Spring is lacking, because I'm not sure whether Java can keep a database connection in the same state between requests as long as the same HTTP connection is used; if so, you can use PHP + cURL to perform three HTTP requests while reusing the network connection.
Object (un)marshaling between PHP and Java will probably not work very well; a representation of the collection would have to be made in the form of perhaps a result identifier that can be fetched via REST by PHP and passed back to Java as an opaque string.
It should be noted that PHP requests are typically short-lived, which makes it a good candidate for horizontal scaling; long running processes, such as those for Pub/Sub systems, are therefore seen less often. Personally I prefer using the alternative of sending HTTP requests so that the frontend caching can be abstracted away from the Java service.
I'll add more when I think of anything else, do let me know if my answer gives you more questions :)
There is definitely no straightforward way out. You have to re-write your backend controller layer which exposes REST calls to PHP. You need to group some fine grained services and make them coarse grained.
A word of advise - you can make things work at the end day with tweaking here and there, introducing a queue or another framework or library etc - but you would have also introduced lot of Accidental complexity..
Things can get real nasty in future during maintenance, bug fixes, upgrades etc
If speed of development is a concern, explore other alternative like Play Framework 1.2.x framework, so that you can re-use your server side code as it is.
I DON'T WANT TO COMPILE PHP TO NATIVE EXES OR ANYTHING LIKE THAT, ONLY COMPILE/CACHE (both words don't reflect what I mean) THE FRAMEWORK LOGIC ITSELF
Is there anything like a compiled PHP Framework?
What I mean is take Kohana as an example. It's extensible, overrideable, fun, etc. But it ends up loading 200 files to show one page!
What I would think is best is if you have Controller_Admin extends Controller_Admin_Template extends Controller_Template extends Kohana_Controller_Template extends Controller extends Kohana_Controller. This isn't needed... if we just copypasted method bodies to $parent->whatever() it would end up in one nice, smaller, faster file.
We already do it for JS and CSS to minimise assets downloaded; why not the whole framework? :D
Also I am looking for a compileable ORM. The Kohana ORM is very... slow... it uses magic methods, loads tables, generally is fun to work with but a pain in the... circuitry of the server. :P
If we could make ORM::factory('test')->compiled()->where('bla','=','1)->execute(); to compile into ORMC::factory('test','SELECT * FROM test WHERE bla=1')->execute(); in the production server it would be cool. This applies to a bunch of other stuff besides ORM but ORM would benefit greatly.
The overhead of dynamic frameworks doesn't seem to tip the scales by the ease of use in my opinion. With this we lose no ease and speed it up a lot. ;)
So my question is: Does something like this exist? If not, is my logic flawed?
Edit:
Because of failed answers I'll show more straight what I want to do.
We have an /application/ where there is the EXACT SAME code as without "compiling", and a /compiled_app/ where all (for example) queries that can be SIMPLIFIED are SIMPLIFIED (not object Query_Builder but SELECT blablablabla etc).
Also as much as having 50 files for one class adds a lot of umm... override vectors? :D it is an UNNEEDED 100% GUARANTEED BOTTLENECK PERFORMANCE OVERHEAD. Maybe it's not much but it's there always. And it doesn't have to.
You can check Flow3 framework and how it works. Its not really what you want but maybe you will find it interesting..
PHP is an interpreted language and doesn't compile as such.
There is APC which dynamically compiles PHP code into the bytecode that the Zend engine executes and caches it. This can gain you quite a bit of performance but it has drawbacks and limitations.
Honestly though, what you're asking for sounds an awful lot like premature optimization. The biggest factor in deciding how fast your code will run is choice of algorithm. A compiled bubble sort on a million records is still going to be slower than an uncompiled quicksort on the same number of records. PHP apps also tend to spend a lot of time communicating with external systems such as databases. You can't optimize this kind activity at all by compiling the PHP. A query that takes 10 seconds is going to take 10 seconds whether the PHP is compiled or not.
If you are having performance issues with your application then the SQL queries it's executing is usually a good place to start when it comes to optimizing. A lot of PHP code does too much querying, executes queries in loops, or queries for data that the application subsequently does nothing with. Finding and eliminating these will speed your application up no end.
However, it's important to note that you should never try to optimize your code by guessing at where the bottlenecks are. Use a tool like XDebug to generate a profile of your running code and then analyse he output to determine where the hot spots in your code are.
Most importantly, if you're not having performance problems then optimizing for its own sake is a waste of time. Premature optimization wastes developer time, it tends to make source code less readable and harder to maintain, and it has a tendency to introduce bugs. Only optimize when there is a proven need for it.
You can check Yaf. It's a framework compiled to PHP extension.
Have You ever heard about HipHop? It can compile whole PHP application into one binary file.
I have a php application that has about 50-55 code files. The file that has the maximum amount of code has about 1200 lines of code(this includes the spaces, tabs and multiple line breaks...) and rest of the code files are relatively smaller than this.
The application code in almost every file is a mix of html, sql and php(what you call spaghetti), except in a few files that are pure php include files....for example a file containing functions that are needed in many other places.
I have been considering whether it's a good idea to refactor this application to a mvc type architecture.
Now i know that a mvc application offers plenty of advantages like ease of maintenance, reuse and ease of further development etc but what about scalability and performance - specifically in this case ?
My assumption is that since this is a small application(i believe so, do you think it's small enough?), i don't envision having a hard time with maintenance or adding a few more features(at the max) which just may mean a few additions in existing files or maybe say addition of a maximum 5-10 new files.
So i am thinking i shouldn't be converting to mvc just for maintenance sake.
As far as i understand you may put each component of mvc on a separate server to spread the load so as to have a different server serving html, database and logic and do other optimization/caching further as well to make a hich is mvc application scale and perform.
I think even though in a small spaghetti application we cannot have different servers for html, database etc we can easily scale without degrading the performance by having a load balancer in front of web servers, databae server etc.(Supposing it comes to a situation where one server is not enough)
Even more so on it's own the spaghetti code should perform better than mvc, since it doesn't have any overheads like requiring includes or files or function calls from files placed under folders belonging to a different component of the mvc.
So, considering all these things do you really think it's useful to refactor a relatively small spaghetti application to mvc for scalability and performance?
Please don't tell me that refactoring will be useful in future (i know that will help and will consider if we really need to add much more code to the existing code base) but please provide me with a clear answer to
1)Do i really need to convert this application to a mvc architecture for scalability and performance ?
2)Can a spaghetti application like this scale and perform for atleast a minimum of 1 million request a day and half of which occur during some peak time?
As far as i understand you may put each component of mvc on a separate server to spread the load
I've never heard that myself - but I come from a .Net world where you'd run all your managed code on the same server anyway (it's not like in the Java world where you often have a separate "App" server and "Web" server).
The main reason you'd probably move to MVC (just as you mentioned) is for benefits in managing the code: separation of concerns, re-use, etc; not performance.
In theory you could do this with object / component based technologies like Java or .Net where the components communicate with each other - but in procedural code? i don't think so!
So, considering all these things do you really think it's useful to refactor a relatively small spaghetti application to mvc for scalability and performance?
No - assuming by scalability and performance you're refering to runtime qualities of the system, which I believe is what you meant.
If you put scalability and performance purely into the context of development (people coding - how fast and easily they can work on the system, how easily you can add developers to the project) then the answer would be yes.
2)Can a spaghetti application like this scale and perform for atleast a minimum of 1 million request a day and half of which occur during some peak time?
Nothings impossible - I love Gordons comments along those lines - but as I'm sure you'd agree, it's probably not the best footing you could be on.
No, you don't have to convert because with an infinite budget any application will scale infinitely. Just add more servers. The question is, do you have an infinite budget? If you dont have an infinite budget, find out what is cheaper: add more hardware or optimize code.
So the real answer is: maybe.
We cannot tell you what it takes for your application to reach your scalability goals. We don't know what it does, nor do you provide hard limits for the desired performance. For instance, how long may a request take until it is served? Run ab or Siege and measure your Status Quo. Run a profiler on your code and identify bottlenecks. Find out whether you are IO, CPU or RAM bound. Are you using an Opcode cache? Take all your findings and make an educated guess about cost. Then decide how to optimize.
There will be a point where the effort required to shed some microseconds is higher than simply adding better or more hardware. In practise, you will likely use a mixed strategy to find the most scalable and affordable solution for your needs.
Note that refactoring a big ball of mud into a pristine OOP application does not necessarily mean it will run faster afterwards. In fact, the more loosely coupled, the more indirection, the more layers, the slower the application will likely become. It's a tradeoff for better maintainability. Maintainbility is a cost saver too. It will cut down on your time to deliver new features.
Yes
No
50+ files is not small. Spaghetti code is unmaintainable and highly inefficient, so there is no performance advantage over a proper framework. Finally, a good framework offers well-designed, well-tested, and constantly updated plugins to achieve most common tasks, reducing the amount of code that you have to write and maintain yourself.
Back in 1995 high-traffic sites were running tangled messes of spaghetti-code. Today, you shouldn't even think about running a high-traffic site (or any kind of site!) on spaghetti code.
Would creating a custom php mvc framework from scratch be completely pointless for a small digital agency?
I have heard recently from a friend that His colleges are infact spending time on such a project within their agency. Their argument against using another such as cake or zend being that they come with so many unused features and therefore lots of unnecessary code.
I would have thought that such an endevour is completely pointless and indeed a waste of time and money. Surely it would make more sense to pick a framework that matches your needs as close as possible and then customise/extend it. On such frameworks as zend the code is surely going to be of a much better/tested level than that of 1 or 2 programmers could create in a few months.
It just doesn't make sense to me - can anyone pursuade me otherwise?
Using your own framework has one big advantage over other solutions. It means you have total i mean TOTAL! controll over it. No matter what happens to other frameworks, because one day you have open source software and another day it could be abbandoned or stopped project. And your framework will still be developed. Even if it means you will need to constantly add some features to it, with time it will pay off. Besides, PHP is like one huge framework after all, and it's evolving. Writing your own framework will keep you up to date with this technology :) Of course keeping with other frameworks or CMSes is also good idea, because maybe your clients will want to use them, and you can also learn from them and move some solutions to your software :D
No, I think you will find most here would agree with you - I certainly feel it is a complete waste of time. Especially given that they are a small agency, their limited resources could be better spent elsewhere.
Since they are concerned about a bloated framework, I suggest that instead they investigate using a lightweight PHP framework such as CodeIgniter and Kohana.
I see it pointless. But it's a curse of PHP programmers who tend to invent wheel again and again. I know what I'm talking about. I tried it too.
The best approach seems to be to learn a framework and if it doesn't suit you ask friends who knows other frameworks to learn how is the particular problem solved in other frameworks and if none of them seems good then MAYBE it's time to start work on a new framework.
The point with lots of unused code is not completely valid. Zend contains classes that can be stripped off as far as I know.
EDIT: Performance of PHP can be enhanced via eAccelerator for example. I find it easier than building a brand new framework.
MVC FW is less than 5% of average project code base, so I strongly recommend building MVC for every project and adapt it to the project specifications.
If someone has enough knowledge about MVC pattern and some experience in using and building MVC frameworks, it can be built in some days.
General arguments are speed and performance given by the custom FW.
Sometimes when people are afraid of changes and learning new things, they come up with all kinds of silly reasons why somethings is bad and should not be used.
I once worked in company that was determined to write their own web framework instead of using Django. One of the developers that was working on "admin" part of framework said i wanted him to lose his job cos Django has auto generated admin.
Unless is it very specific framework and you have very good reasons and enough fundings for project it is never a good idea to reinvent the wheel specially if you can't make it the same or better quality then what is already out there.
It depends on what kind of projects they do, and what are their needs. Never forget that in software, the 'best' always depends on what you are doing. There is no 'one golden solution' for everything. This includes mvcs.
There are various reasons to code your own mvc :
You can tailor it to your exact specific needs
You can keep it closed source, reducing exposure, increasing security due to obscurity
You can continue developing it, and then release it to the world when it is mature and fulfills a major purpose - a lot of current mvcs came to being in that way anyway.
You can continue developing it, keep it closed source code, and build massive SaaS services and applications on it, to which you will retain full rights. Can be done with apache license too though.
You can fulfill a purpose that is not fulfilled by any other mvc. Despite every mvc has its zealots who believe that they have the one and all solution, every mvc has its downsides. And believing that 'everything that can be done, has been done' because there are 15-20 mvcs out, is as stupid as saying 'all that can be discovered has been discovered'. And there were people saying the latter at the start of 20th century.
Not to mention that with an external mvc you are using, you are obliged to that mvc's project team for many things. They may already have or may introduce things that you dont like, or even hamper your specific applications.
Not to mention they may just get abandoned. Can happen to the biggest. Codeigniter was all the rage. What happened ? Ellis is dumping it basically. Development slowed down to a halt. Probably soon totally halting. Being open sourced does not guarantee a project either - there have been many open source projects which went similar ways.
What's going to happen to people who had coded major applications with Codeigniter ? Are they going to undertake the security of the Codeigniter installations they have themselves ? Therefore de facto becoming maintainers of their own mvc ? Or are they going to hire external parties to maintain the mvc for them ?
Not a good future investment.
My preference for coding my own mvc is mainly to have a mvc that fits my own style of coding. My criteria for a mvc :
Fast development
Fast modification
Low maintenance
Speed
Security
Extreme simplicity
Reliability in regard to future development
I have coded a major mvc for myself before. and i am still successfully running it on one of my hobby websites which handles ~600,000-1,200,000 unique visitors per month. At a given moment, 250+ visitors (google analytics - it doesnt include any bots) can be present on the site, hammering page after page. And it does not even blink. I like that. Despite it has various shortcomings that i dont like, it works spectacularly.
In the current mvc im doing, im using the lessons i learned from the previous one, as they apply to my specific purposes.
Not to mention it is a great way to hone and develop new skills.
But one thing is a must: plan it very well, and keep building on it, changing and improving it.
For any framework
Pros
get exact functionality you want
control over small details important to your project
familiarity with code
Cons
everything that breaks you must fix yourself
don't get to learn from the mistakes of the past
won't get as many bugs noticed due to smaller audience
Personally I recommend exploring existing frameworks before even considering writing your own. Just remember that it's okay if you don't understand it all in a day!
It's a waste of time. Unless they are building a site with the traffic of Facebook, the performance hits they will get using something like Yii vs. their own should matter very little
It's not necessarily a waste of time but I agree with you, I tried it myself too. I've been using Zend, Code Igniter and Fuel depending on what project that I'm working on.
The consequence that I had in creating my own framework is the deadline. If you're working on a project with a very tight deadline then creating your own framework would be a bad idea.
I'm going to develop a facebook game similar to chess board but not a chess game.
2 users will play against each other.
User movements are not necessary to store.
I'll use ajax/jquery for front end (NO flash)
PHP for backend and MySQL for storage.
I know it's a good practice to use framework but I'm afraid it'll become bottle neck to my app when massive concurrent users are playing. I'm expecting more than 1 million per month active users so what I need is a powerful framework to develop but should be able to handle massive dynamic concurrent requests.
I'm really appreciate your advices!
I really like the PHP "unframework" Flourish. It both might be exactly what you need (only use what you need), and you might have unique situations that normal frameworks don't fit. It also might be too small to give your project the kick it needs... but it may be worth a look see anyway.
I've only used it on small stuff, but I really enjoy it.
Some frameworks (e.g. CodeIgniter and a fork, Kohana) have lightweight designs, i.e. they do not load a lot of code by default. Some other frameworks (e.g. Zend Framework) is designed so that you can use individual classes of it instead of the whole framework.
Your framework won't be a bottleneck, unless it's incredibly horrible. There's a misconception that frameworks must be bloated and offer more than what's needed. A good framework, which most are, offers well designed components that you usually end up writing yourself.
This is one reason that I tend to recommend the Zend Framework. It doesn't provide everything. That's the benefit. You take what you want. Don't be forced into anything. If you hate the controller classes, simply replace them with something that makes sense to you.