I'm buidling a new appliction, eventually I'm planning on releasing it onto the Internet for free consumption.
In an effort to reduce the final download size of the package I would like to only bundle the absolute bare ZF components used by my app:
Zend
Zend_Application
Zend_Config
Zend_Config_Ini
...
I could manually do this - though I'd rather not. Is there a tool around that I can point to my application, it can scan the PHP codebase and create a package with all classes referenced.
I know ZF2 uses composer.json to take care of this - however I'm building on ZF 1.11
No, there is no tool, to do this job for you. However, I recommend not to break the ZF1 into pieces, because it's not that trivial to track down the several dependencies between the components. There is also no benefit: The autoloader takes care, that only the classes were loaded, that are required, thus you only save a small amount of disk space. Thats not worth all the effort it takes. This means you will definitely feel no difference wether or not you use ZF1 as a whole, or only partial, unless you find broken dependencies you created yourself.
Related
Some background first
Our company, a small startup with only four developers, is starting the refactoring of our products into reusable modules to simplify the development process, increase productivity and, along the way, we would like to introduce unit tests where fits.
As usual on a small startup, we can't afford wasting too much development time but, as we see, this is extremely important for the success of our business on a medium and long term.
Currently, we have two end-user products. Both are Laravel (PHP) applications built on top of our own internal business layer, mainly composed of webservices, restful apis and a huge database.
This business layer provides most of the data for these products, but each of them makes completely different use of it. We plan to build other products on the near future besides maintaining and improving those two that are almost finished.
For that to happen, we intend to abstract the common logic of those (and the future) products into reusable and decoupled modules. The obvious choice seems to be Composer, even with our little knowledge about it.
Now to the real question
I would like to ask other opinions on how to develop internal packages on a test driven fashion. Should each module be a composer package with it's own unit tests and requiring it's dependencies, or should we build a single package with each module namespaced?
To clarify a bit, we would like to have, for instance, a CurlWrapper module and that would be required on our InternalWebserviceAPI module (and a few others).
I personally like the idea of having completely separate packages for each module and declaring dependencies on composer.json, which would mentally enforce decoupling and would allow us to publish some of those packages as opensource someday. It also may simplify breaking changes on those modules because we could freeze it's version on the dependents that will need to be updated.
Although, I also think this separation may add a lot of complexity and may be harder to maintain and test, since each module would need to be a project on it's own and we don't have all that man power to keep track of so many small projects.
Is really Composer the ideal solution for our problem? If so, which would recommend: single package or multiple packages?
Edit 1:
I would like to point out that most of these modules are going to be:
Libraries (ie obtaining an ID from an youtube URL or converting dates to "x seconds ago")
Wrappers (like a chainable CURL wrapper)
Facades (of our multiple webservices, those require the other two kinds)
Yes, composer is the way to go and I recommend you to use single packages.
You don't know when you need these modules. It is better to create many single packages and be able to include them all (or a single one), than creating big packages and need to put more time in breaking a package in multiple ones when you need some classes from it.
For instance, see the Symfony2 project. That is a lot of components which are all required for the full-stack Symfony2 framework, but you can also use some components in your own project (like Drupal8 is doing). Moreover, Symfony2 gets more and more packages, it seems so usefull to have small packages that people put time in breaking some big packages in pieces.
An alternative to using single packages: use separate composer.json files for each subproject.
This has the benefit of letting you keep all of your libraries in the same repository. As you refactor the code, you can also partition autoload and dependencies by sub-library.
If you get to the point that you want to spin the library off into its own versioned package, you could go the final step and check it into its own repository.
As far as I understand, every enabled module in a ZF2 application is loaded for every request (unless one uses optimization methods such as that offered by the zf2-lazy-loading-module module). I've been keeping an eye on modules that get published on modules.zendframework.org and I've come across modules which offer extremely limited functionality, such as the AkrabatFormatUkTelephone module which purpose is to format phone numbers to UK format.
Whilst I understand development should focus on creating single purpose modules that are good at doing one thing (instead of modules which do many things but not in a very good way), I'm thinking if we start using modules which offer such limited functionality as the one mentioned, we will need to combine hundreds of modules in order to build a rich application which could be disastrous for performance. Instead I would expect this sort of functionality to be put in a class (e.g. Zend\I18n?) and loaded on demand which would be more optimized. But knowing Akrabat's reputation I'm thinking I must be missing something, hence my question:
Is the loading of modules such as the one I mentioned significantly worse for performance than loading the same functionality via PHP classes (or is it similar due to the way ZF2 has been designed)? Does anybody have any figures (i.e. is it 5%, 10%, 15% slower) about module vs class loading performance?
Don't take this comment as a final answer, as hopefully someone of the ZF2 devs will shed some more insight to it, but generally only Module.php and usually module.config.php will be actively loaded. Everything else will simply be registered and be called on demand. So as long as your Module.php and module.config.php are not TOO big in filesize, the performance shouldn't be THAT big of an issue
In the case of Akrabats example, all that's happening is, the registry of a new ViewHelper. Nothing else. The same with all other view helpers inside of Zend. Performance won't really matter a lot in these cases.
Personally the Skeleton loaded with 80ms on my Webspace and with BjyAuthorize, ZfcBase, ZfcUser and my own module, the loading time ramped up to 100ms. And this is without any sort of memory caching enabled!
Loading a module is not much more than loading any class, like Sam pointed out.
As long as you don't use anything from your module and do things right, it's just beeing registered.
Now what does "do things right" mean?
Just try to put a big nonsense loop inside your module classes bootstrap() method. You will see that this slows down every request on your application, because the bootstrap method of your module is called on every request and it should be used very carefully, only for light weight tasks. The purposes you usually use the bootstrap() method for, won't even slow down your app for a millisecond, but writing a file to the disk in this method could slow down your app for many seconds in each request.
If your app becomes really heavy, you should use the classmap_autoloader and some caching wherever you can. If you did "things right", you won't have any performance problems, just because you have many modules or many classes in your app. One could say, it's just all about algorithms.
Keep going on using best practices, like the one you mentioned. Usually these aren't the bottlenecks of your application, but your own algorithms and failures are.
edit:
When you're using modules from the community, you should always check them for performance issues. Even a module that seems to be very light could be a bottleneck for your application if it has bad algorithms. But the case that you're loading an additional module is not the point of it.
Good question. I would like to contribute a little bit to the reaction of Sam.
Module performance is not solely the loading of the module (which is, as pointed out quite fast), but also the communication of the modules in-between. So this question might boil down to: how slow/fast is the ServiceLocator and Event-driven system in comparison to traditional non-modulair systems?
I recall that ZF2 was build with performance in mind. For instance, the ServiceLocator registers factories, so that objects can be instantiated on-the-fly. So this requires only a few extra in-memory objects and instantiations, I guess this does not impact the total performance for your application much. The EventManager works in much the same way and I have not seen it being overloaded with registered events, even in large applications.
What might slow down, on the other hand, is the loading of the modules configuration. I figure that using a cache might solve this problem. I'm not sure but maybe Zend Optimizer might do this already.
So, in short, applications should scale pretty wel, provided that modules behave well, and do not over-register events or misuse the ServiceLocator.
From the MVC component's perspective there are no modules at all! There's one big configuration file - a result of merge of every module's configuration. Unless your modules don't have a onBootstrap method or don't do much, module loading is as fast as invoking new Module on every one of them, which is painless and memory inexpensive.
The configuration merge procedure, which I mentioned above, happens only in DEV mode which is enabled by default.
There are number of tricks also to speed up your ZF2 application, like:
Enable merged config cache
Use EdpSuperluminal module
Return the ViewModel objects from actions, not arrays
Explicitly set the template name on the ViewModel
Use template maps instead of template path stack alone
Route order in the config matters! Its a LIFO queue (last in-first out).
Make sure you don't load Console modules in HTTP context.
Let the Composer do the autoloading, not ZF2
... and more. There's a quite good talk by Gary Hockin on the ZF2 app performance.
Authorization modules will surely slow down your app. There are number of things going down under the hood: the identity of the user needs to be fetched (from the database?), user needs to be authenticated agains your rules. Surely you can speed things up by using memcached or such, but this requires to have some knowledge about the lifecycle of the ZF2 application, about the modules you use, etc.
Also there is Zend Framework 3 going to be released soon, some things will go faster, but don't expect much. A lot of overhead is a result of your lack of knowledge about ZF2 - no offense!
In the Symfony 2 docs it's said:
A bundle should not embed third-party libraries written in JavaScript, CSS, or any other language.
Then how should I do that? I want to install Twitter Bootstrap, DataTables, and many other things as dependencies using Composer. But the only way I can think of is creating a bundle and embedding them.
What is the correct way to do this?
You should use Bower by Twitter. It is a package manager for HTML, CSS and Javascript. It was created to address this very issue you are having.
EDIT:
As of now, there are very good package managers for JS Libraries such as Bower, Jam or Component.
Versioning system
Semantic Versioning - Composer advises to use the semantic versioning system. It uses a X.Y.Z setting, in which X is the major version, Y is the minor version and Z is the patch version. Y and Z should always be backwards compatible while X reflects changes in code which MIGHT break backwards compatibility.
Embeding
Embeding should be read as copy and pasting the code (and binary) as part of your library, rather then requiring it as a third party (vendor) package/bundle. Its like including query.js in a resource folder or copying and pasting propel code to a folder inside your bundle.
Why not embed 3rd party libs
A bundle should not embed third-party libraries written in JavaScript, CSS, or any other language.
This statement comes from a best practice point of view. Embeding (as in copy/paste) third party libraries of any kind (PHP libs especially) is usually not a good idea. For instance, imagine that BUNDLE A uses LIBRARY FOO v1.4.1, and BUNDLE B also uses LIBRARY FOO but with a different version v1.5.2. If any of the BUNDLES (A or B) embeds FOO lib, they might (most probably will) become incompatible. For instance, php classes and functions cannot be redeclared. Any of the bundles can, of course, use workarounds to mitigate this problem, such as namespacing their version of FOO or autoloading rules, but this can rise other problems as well besides surely increasing memory usage as there are 2 versions of the same thing parsed by PHP.
If a PHP package does not follow this best practice, the errors that arise are usually easy to spot (with error: cannot redeclcare function blablabla). With Javascript Libraries, however, that is not true. You can redeclare functions (as they are object properties). So if now FOO is a JS Lib instead, and BUNDLE A and B embed them into their libraries, when they are included, strange problems can arise. For instance, a function can be redeclared that lacks a crucial functionality for one of the bundles and break it.
Symfony is a PHP framework.
It deals with PHP libraries/bundles. Symfony advises to require a library as dependency instead of embedding it since it uses Composer as a Package manager, which takes care of downloading and loading the require packages. As far as I remember, when 2 bundles/packages use the same library, if they have different version requirements, the most actual is used, unless its backwards incompatible. Composer then reports a conflict that you have to manually resolve.
However... There is no way to handle javascript libraries properly. That's because Composer is a package for PHP libraries. You can workaround this in two ways I can think of: (there are probably more and best ways to handle this, I just thought of these two, read them as suggestions)
Create a PHP wrapper around the javascript library and including it (although this potentially creates the same problem if another bundle decides to do the same thing but giving the package a different name)
Creating a bundle which requires the javascript library as a third party dependency through composer. Since the javascript library won't probably have a composer.json file in it's repository (sometimes they live as a standalone minified file), this can be accomplished by creating a custom composer installer, forking the javascript repository (in gitHub for instance) adding a composer.json to it, etc... However, you will need to constantly maintain and upgrade the said library, which can be troublesome.
You will have to keep in mind that:
JS and CSS libs have to be exposed publicly, so that the client can access it (security considerations)
Symfony is a PHP framework and deals with server-side packages. JS/CSS are client side. This as to be taken in consideration so it works properly.
One of the main ideas behind symfony (as with other PHP Frameworks) is code reusability within and between projects. Pure Javascript Library are reusable in themselves. They are usually self contained. Besides, there is no real gain in "bundling" a JS library from the server side. You don't need any kind of bundle to achieve reusability.
My Approach
Since the composer system is so appealing, specially when deploying bundles/packages/libraries to other people, my approach to using third party javascript/css libraries was to create a dependency manager specific to JS/CSS that other packages/bundles could rely on to take care of their JS/CSS dependencies without worrying about this.
My sugestion
If you are planing to release your project to the public, namely as a symfony bundle, you should plan carefully how to approach this.
If your project is self contained (personal use or to a client, not widespread use) then this has much less relevancy since you (the programmer) have total control in what third party tools you use and include in your project. These are just best practice "suggestions" to avoid
future problems.
Zend framework is well known for loosely coupled components.
I would like to use XML-RPC from zend framework, is there any dependency for XML-RPC? Like if I had taken out XML-RPC folder off Zend Framework Library and try to instantiate RPC object, would it throw error?
Where can I find the proper way of separating component from the framework?
Thanks
I wrote a tool which takes ZF components and their dependencies so you can easily take just one (or several) component from ZF.
http://epic.codeutopia.net/pack/
It doesn't have the latest ZF release 1.11 (because I'm lazy), but 1.10.6 should work just fine.
You should never split single components off a framework or library independent from Zend Framework, or any other. Especially when using PHP there is also no performance reason, because with PHPs autoloading functionality it will always just include the files, which are requested.
You should literally be able to copy the XmlRpc folder from your copy of Zend Framework and use it in your own projects. The only dependency that I can see is in XmlRpc/Exception.php as it requires a file in the root directory of Zend/ (Exception.php) you could simply copy this file along with the XmlRpc folder keeping the directory structure the same and it should work....
My first question is why you would want to do that in the first place. It means that every time you upgrade Zend Framework you now need add a bunch of tooling to manage the removal of some components. One of the purposes of using a framework is so you don't have to manage a bunch of code. Removing parts of a framework is a step backwards IMHO. Disk space is cheap. Network transfer is cheap. If you are going to remove parts of a framework you should have a REALLY, REALLY good reason to do it.
I've been looking for a good PHP ORM tool to use, and I recently found a good ORM class in Kohana. It has a fairly good and simple ORM implementation. The problem is, the code is unreusable outside of the Kohana framework without a rewrite/refactor. It relies on the Kohana class loader and various framework loading strategies to even work in the first place. Further, the required classes aren't packaged into a single dependency directory, or even multiple directories.
When I do this rewrite, I intend to republish the code via sourceforge or something, and those guys can of course reuse it if they want. So, should I just package the fileset needed into one directory, and make the appropriate classes do a require_once on any dependant classes, and 2 should I stick with the original class names that are project dependent (like Kohana_exception) even though it is pretty much unrelated to the Kohana project as a whole?
A second option would be to write another classloader that is a simplified version of the Kohana Framework classloader, and only cares about the ORM related stuff...
If you think what you are doing will be a marked improvement to the Kohana project you should make your changes and submit a patch to be considered by the project's maintainers. You probably aren't the first person to appreciate a part of their framework and component-izing their framework into smaller bits may be something you could help encourage by submitting a patch (though that will require much more time on your part to engineer).
It sounds like you are extracting a piece of their framework for independent use so it doesn't sound like you're forking at all, though I can imagine removing all of those dependencies may sure seem like a fork.