I'm having difficulty understanding the impact of having packages included (for example with composer), and autoloaded. For example, I'm currently trying to build an API with Lumen and hitting a bunch of roadblocks, because of poor documentation and a small userbase. People have suggested I use Laravel instead and comment out the services/middleware/functionality I wont use. But at the same time, if the packages are being autoloaded, how much of an inpact does that make?
On the surface, I'd think autoloading everything in my vendor directory would lead to a bunch of stuff in memory that would never be used. However, I'd doubt that it works by loading everything (I assume it builds some sort of reference system so when the code is called, it can be executed).
Looking online, it seems that Laravel is only 2-3 times slower than Lumen, though it has far more packages. If I start disabling functionality, I can put that on the lower end, if not much closer. But from a lower level, what impact is loading all the packages in the framework having? How much should I let it impact my decision on what framework to use?
Related
We have a huge code-base (I mean huge, about 2M+ lines) in PHP. I would like to know how you guys managed to integrate composer in this kind of situation.
Specially when the code cannot be decoupled in little projects (Right now) because of the complexity (Even mixed with legacy code) and it's being hold in the same SVN repository.
Why should I be confident in the quality of the composer/packagist libraries?
What happens if packagist goes down?
What should I do if my vendor repository goes down (Github/Bitbucket/Whatever)?
What happens if some of my vendors decide to delete their library?
What if they've been hacked and set the next version tag empty?
I know that this possible problems could be over-passed in one way or another. But the fact that the life of a lot people could be depending on this makes me feel a bit crazy with this kind of decision.
What do you think? What are my best options?
For the first point - if you have legacy, 2M+ tighthly-coupled codebase, common open source projects quality shouldn't bother you ;).
For the rest - you can use staging to build your project together with dependencies and then build a full package there (by that I mean all the dependencies downloaded and bundles). Of course you will still be dependent on external packages on your development cycle, but not in deployment/production. Whenever package goes down, you have time and possibility to replace it.
Composer is a really great tool for bundling yor project together with dependencies, so it's both the answer to question "how to use external dependencies" and also to "how to be independent from them", you only need to specify the point, at which you want to bring this independency into your project.
I think that you should develop with external dependencies in mind, lowering your code base as much as possible and not put these problems on your devs shoulders, they want to use code, libraries, play with tiem... then, somewhere in your deployment process, bundle it all together (staging is a good place). Even if your dependencies will disappear and you will have to spend your development time to replace them:
It will probably still cost you less than handling all on your own.
Some background first
Our company, a small startup with only four developers, is starting the refactoring of our products into reusable modules to simplify the development process, increase productivity and, along the way, we would like to introduce unit tests where fits.
As usual on a small startup, we can't afford wasting too much development time but, as we see, this is extremely important for the success of our business on a medium and long term.
Currently, we have two end-user products. Both are Laravel (PHP) applications built on top of our own internal business layer, mainly composed of webservices, restful apis and a huge database.
This business layer provides most of the data for these products, but each of them makes completely different use of it. We plan to build other products on the near future besides maintaining and improving those two that are almost finished.
For that to happen, we intend to abstract the common logic of those (and the future) products into reusable and decoupled modules. The obvious choice seems to be Composer, even with our little knowledge about it.
Now to the real question
I would like to ask other opinions on how to develop internal packages on a test driven fashion. Should each module be a composer package with it's own unit tests and requiring it's dependencies, or should we build a single package with each module namespaced?
To clarify a bit, we would like to have, for instance, a CurlWrapper module and that would be required on our InternalWebserviceAPI module (and a few others).
I personally like the idea of having completely separate packages for each module and declaring dependencies on composer.json, which would mentally enforce decoupling and would allow us to publish some of those packages as opensource someday. It also may simplify breaking changes on those modules because we could freeze it's version on the dependents that will need to be updated.
Although, I also think this separation may add a lot of complexity and may be harder to maintain and test, since each module would need to be a project on it's own and we don't have all that man power to keep track of so many small projects.
Is really Composer the ideal solution for our problem? If so, which would recommend: single package or multiple packages?
Edit 1:
I would like to point out that most of these modules are going to be:
Libraries (ie obtaining an ID from an youtube URL or converting dates to "x seconds ago")
Wrappers (like a chainable CURL wrapper)
Facades (of our multiple webservices, those require the other two kinds)
Yes, composer is the way to go and I recommend you to use single packages.
You don't know when you need these modules. It is better to create many single packages and be able to include them all (or a single one), than creating big packages and need to put more time in breaking a package in multiple ones when you need some classes from it.
For instance, see the Symfony2 project. That is a lot of components which are all required for the full-stack Symfony2 framework, but you can also use some components in your own project (like Drupal8 is doing). Moreover, Symfony2 gets more and more packages, it seems so usefull to have small packages that people put time in breaking some big packages in pieces.
An alternative to using single packages: use separate composer.json files for each subproject.
This has the benefit of letting you keep all of your libraries in the same repository. As you refactor the code, you can also partition autoload and dependencies by sub-library.
If you get to the point that you want to spin the library off into its own versioned package, you could go the final step and check it into its own repository.
As far as I understand, every enabled module in a ZF2 application is loaded for every request (unless one uses optimization methods such as that offered by the zf2-lazy-loading-module module). I've been keeping an eye on modules that get published on modules.zendframework.org and I've come across modules which offer extremely limited functionality, such as the AkrabatFormatUkTelephone module which purpose is to format phone numbers to UK format.
Whilst I understand development should focus on creating single purpose modules that are good at doing one thing (instead of modules which do many things but not in a very good way), I'm thinking if we start using modules which offer such limited functionality as the one mentioned, we will need to combine hundreds of modules in order to build a rich application which could be disastrous for performance. Instead I would expect this sort of functionality to be put in a class (e.g. Zend\I18n?) and loaded on demand which would be more optimized. But knowing Akrabat's reputation I'm thinking I must be missing something, hence my question:
Is the loading of modules such as the one I mentioned significantly worse for performance than loading the same functionality via PHP classes (or is it similar due to the way ZF2 has been designed)? Does anybody have any figures (i.e. is it 5%, 10%, 15% slower) about module vs class loading performance?
Don't take this comment as a final answer, as hopefully someone of the ZF2 devs will shed some more insight to it, but generally only Module.php and usually module.config.php will be actively loaded. Everything else will simply be registered and be called on demand. So as long as your Module.php and module.config.php are not TOO big in filesize, the performance shouldn't be THAT big of an issue
In the case of Akrabats example, all that's happening is, the registry of a new ViewHelper. Nothing else. The same with all other view helpers inside of Zend. Performance won't really matter a lot in these cases.
Personally the Skeleton loaded with 80ms on my Webspace and with BjyAuthorize, ZfcBase, ZfcUser and my own module, the loading time ramped up to 100ms. And this is without any sort of memory caching enabled!
Loading a module is not much more than loading any class, like Sam pointed out.
As long as you don't use anything from your module and do things right, it's just beeing registered.
Now what does "do things right" mean?
Just try to put a big nonsense loop inside your module classes bootstrap() method. You will see that this slows down every request on your application, because the bootstrap method of your module is called on every request and it should be used very carefully, only for light weight tasks. The purposes you usually use the bootstrap() method for, won't even slow down your app for a millisecond, but writing a file to the disk in this method could slow down your app for many seconds in each request.
If your app becomes really heavy, you should use the classmap_autoloader and some caching wherever you can. If you did "things right", you won't have any performance problems, just because you have many modules or many classes in your app. One could say, it's just all about algorithms.
Keep going on using best practices, like the one you mentioned. Usually these aren't the bottlenecks of your application, but your own algorithms and failures are.
edit:
When you're using modules from the community, you should always check them for performance issues. Even a module that seems to be very light could be a bottleneck for your application if it has bad algorithms. But the case that you're loading an additional module is not the point of it.
Good question. I would like to contribute a little bit to the reaction of Sam.
Module performance is not solely the loading of the module (which is, as pointed out quite fast), but also the communication of the modules in-between. So this question might boil down to: how slow/fast is the ServiceLocator and Event-driven system in comparison to traditional non-modulair systems?
I recall that ZF2 was build with performance in mind. For instance, the ServiceLocator registers factories, so that objects can be instantiated on-the-fly. So this requires only a few extra in-memory objects and instantiations, I guess this does not impact the total performance for your application much. The EventManager works in much the same way and I have not seen it being overloaded with registered events, even in large applications.
What might slow down, on the other hand, is the loading of the modules configuration. I figure that using a cache might solve this problem. I'm not sure but maybe Zend Optimizer might do this already.
So, in short, applications should scale pretty wel, provided that modules behave well, and do not over-register events or misuse the ServiceLocator.
From the MVC component's perspective there are no modules at all! There's one big configuration file - a result of merge of every module's configuration. Unless your modules don't have a onBootstrap method or don't do much, module loading is as fast as invoking new Module on every one of them, which is painless and memory inexpensive.
The configuration merge procedure, which I mentioned above, happens only in DEV mode which is enabled by default.
There are number of tricks also to speed up your ZF2 application, like:
Enable merged config cache
Use EdpSuperluminal module
Return the ViewModel objects from actions, not arrays
Explicitly set the template name on the ViewModel
Use template maps instead of template path stack alone
Route order in the config matters! Its a LIFO queue (last in-first out).
Make sure you don't load Console modules in HTTP context.
Let the Composer do the autoloading, not ZF2
... and more. There's a quite good talk by Gary Hockin on the ZF2 app performance.
Authorization modules will surely slow down your app. There are number of things going down under the hood: the identity of the user needs to be fetched (from the database?), user needs to be authenticated agains your rules. Surely you can speed things up by using memcached or such, but this requires to have some knowledge about the lifecycle of the ZF2 application, about the modules you use, etc.
Also there is Zend Framework 3 going to be released soon, some things will go faster, but don't expect much. A lot of overhead is a result of your lack of knowledge about ZF2 - no offense!
How long do you normally test an update for Zend Framework before pushing it out into a productions project. We can break this question up into minor updates 1.6.0 -> 1.6.1 or maybe a major update 1.6.2 -> 1.7.0. Obviously you don't release it if it add bugs to your code.
Also, as with most other server software updates normally people have a window of time they like to wait and watch the community before even attempting an update on a development environment. How long do you even wait to start the process?
It seems like the best method would be to have a comprehensive set of tests that exercised all the functionality in your application. With a good method for testing it seems like you could push it into production pretty quickly.
Another simple thing you can do to help you make your decision would be to simply do a diff against the repository to see what changes where applied to any modules that you use. If there where no changes, then upgrading shouldn't make any difference. If something underwent a major re-write, you would probably want to investigate a lot deeper.
I'll often jump through update releases (1.7.1 -> 1.7.2) without much hesitation. When the minors roll in, it's another bag of tricks though. For example, there were a lot of changes with Zend's file upload elements, and Zend form in between 1.5, 1.6 and 1.7.
Whether or not I even move on a new release depends on what's been done. Checking the update lists provided is pretty important for deciding on whether or not to go.
As for timing, it varies. There's no set in stone process.
Finding "what breaks" is quickly accomplished with the unit tests. But, who really has a full set of unit tests for their application, right? ;)
Using unit testing will help catch some of the deltas. Zend Framework now comes with Zend_Test to make testing applications a bit easier. I updgrade between projects (so new projects that are coming up will get the latest version).
I am deciding on a framework to try out for PHP. I have narrowed it down to CakePHP and CodeIgniter. I have a couple of questions for any of you who have used or are familiar with both:
I like the fact that CakePHP keeps most of the code outside of the webroot by default. Especially since I may end up using a single framework install for multiple apps. I see CodeIgniter will do that too, but you have to configure it and move some stuff around. Is that workaround secure and reliable, or is it an afterthought hack?
Which (if not both) is easier to upgrade, and maintain over the long term? As new versions of the framework (and PHP itself) come out. I don't want to find my stuff either breaking, or becoming outdated.
Edit:
This is a very old post, but I thought I would update it with what I finally ended up doing, which was to use Kohana.
You should try both frameworks for a week or so, building something trivial (like a blog or wiki) in both, and see which you prefer using. Whatever makes the most sense to you will probably sustain you the longest through upgrades an deprecations.
CakePHP is in a bit of a volatile state right now, still unearthing bugs while pushing to release version 1.2 (which is not backward compatible). I wouldn't suggest building a critical application with it if you need something rock solid right now. If you can wait a month or two for things to settle, then it's probably a moot point.
To address your concerns:
1) Cake and CI do it the same way (iirc). They are equally secure, reliable, and hackish on this front.
2) Everything changes. If you need concrete, perpetual assurance of stability and backward compatibility, roll your own framework. There's not that much to it, and you're guaranteed that nothing changes unless you want it to.
I have deployed multiple applications on CakePHP and it's been a very, very, nice experience. You can't go wrong either way, as both are solid.
is a non-issue.
CodeIgniter has a sponsor behind it, so it's definately the one to choose for the long term. Also, it's faster.
This is a non issue. The app has a couple of lines which says where the core and your application code lies. You just need to change those lines.
You can never anticipate this one.
The state of PHP is a wildcard here. One app (CI) is built to be compatible with PHP4 the other requires PHP5. If you need to deal with the possibilty of a web host only supporting older versions of PHP then you need to go with CO.
Another issue is unit testing. If you require your framework to ship with tests, then CI is not the way to go.
Personally, I feel comfortable with CI because of the corporate backing. The company behind CI is making real profit from their efforts. Though CI is free, their paid product (ExpressionEngine) will eventually live on CI. The same could be said of the Zend Framework and even the birth of Rails (originally built for Basecamp.)
A minor correction to an above comment: both are compatible with PHP4, not just CI. Also, I don't think that having a sponsor makes CI and more or less upgradable or maintainable. Money doesn't solve those problems in the least.
I use CakePHP for a variety of applications and I've been happy with it thus far. 1.2 is a huge improvement over 1.1, and while the library source may change from RC3 to Final, I don't think any code you write will become obsolete. My only niggle is that the Manual isn't as comprehensive as it should be (in my opinion), and I end up in the API quite a bit. The trade-off there is that I now understand the code behind the scenes very well. In any case, I highly recommend it.
On the other hand, I've never played around with CI, so I can't recommend CakePHP over CI. I would take each for a spin and see which one grabs you. Whichever one you choose, study the hell out of its conventions and capabilities. When I started with Cake, I unwittingly wrote a bunch of code to do something Cake did "automagically" by having me set one variable in the controller.
CodeIgniter is very flexible as you would see once you try it. So how your application would be maintainable would fall you your hands.
I have also deployed multiple applications using the same installation. I usually create 2 applications for CMS projects (one for admin, one for the front-end).
To address both of your questions from a CodeIgniter perspective (I don't use Cake):
1) CodeIgniter doesn't keep itself outside the webroot by default, but it can do so with some very simple changes. The first part of my CI tutorial series explains how to do so, along with a walk through of the setup of a new CI instance. Once finished the only part of CI that needs to be in the webroot is a small index.php bootstrap file.
2) I've got an application which I originally developed in CI 1.4.x and I've sucessfully migrated to 1.5.x then 1.6.x. With each new release the CI dev's make available detailed upgrade instructions laying out what needs to be replaced so upgrades are fairly easy.
Jim.