I'm working on refactoring a huge PHP monolith (>1.2 million LOC, running >40 Websites in >30 countries) into some more manageable composer packages to seperate concerns. Each of these packages will be living in it's own github repository, as some of them will be shared across all or just some sites, and some of them will only be relevant to one specific site.
Everything works just fine, except for the IDE integration in PHP Storm that sucks quite a bit (PHP storm sees everything as dependency, and excludes it from auto-completion by default. it takes quite a lot of configuration effort to make features like auto completion and version control integration just work)
Is there anybody having experience woking on such a multi-package project in PHPStorm who has some a somewhat convenient way to handle this?
(Changing the structure as described is not an option. It definitely is the solution that makes most sense for us)
Cheers,
Matthias
Related
I am developing TYPO3 projects since 2006 now, and projects are getting bigger and more complex. Setting up a simple CMS site with a contact form and news listing is all routine.
Right now, we finished a bigger project: A platform for an international company with countless extensions:
Login & registration, news, listing database records, dynamic contact forms, surveys & statistics, intranet functions: document upload & download, several backend "tweaks" per TCA modifications, etc. .
The project managers got upset at us developers, because sometimes, after we finished on function X and later committed function Y to the dev server, function X was broken. This was related to typoscript settings, extension interdependencies, versioning errors or sometimes simple programming mistakes and typos.
I know how to take care of the latter, but in general:
From your experience:
How can we develop an error-proof system in TYPO3, where everything works in hand and extensions don't get in their way? In other words:
How can we secure and isolate functionalities (extensions) - and avoid those interdepency issues?
We are working in a DEV team with two developers, and we already use:
Subversion Repository
Local DEV server for development & testing
External typoscript configuration files, split into single files for each extension
Edit for Bountyhunters:
What I am looking for is a best-practice-summary that might include these topics:
General workflow habits
General coding habits
Reliability of our subversion commits (or Git)
Unit testing (PHPUnit, Selenium?)
Deployment (I haven't yet figured out how automated deployment can
help us)
Typoscript best practices
Problems we could find in large TYPO3 projects are not to much different from any development project.
General practices :
configure continuous integration platform with continuous deployment tools;
Test Driven Development with automated testing;
robust architecture (dB, URL routing, ...);
performance tests during development;
use versionning with formatted comments;
use powerful IDE as PHPStorm, Eclipse, Netbeans;
Common TYPO3 practices :
use official API;
follow TYPO3 API code guideline;
use TYPO3 hooks where You can if You need to modify the logic of Core or 3rd party extensions;
use TypoScript constants to separate data from logic in Your configuration;
Additional references :
TYPO3 Best Practice Workshop
TYPO3 Best Practices (de)
TDD & Best Practices mit TYPO3 (de)
Extensions could help to manage complex TYPO3 installation :
caretaker
additional reports
Use modern project management methodologies & tools
Scrum, Kanban, lean developement principles
Bugtrackers as Redmine, Trac
Books :
php|architect's Guide to Enterprise PHP Development
I absolutely recommmend start using PHPUnit for unit testing, but remember unit testing is really about how you create the code in the first place, not usually something you add later. But of course, better late than never.
You should consider setting up a build server like jenkins/hudson or atlassian bamboo. The latter is quite nice and integrates with zend studio which in my opinion is the better choice when developing in PHP. In general the atlassian products are widely used for software projects. (Jira + confluence + greenhopper in particular)
I would also recommend setting up phpunit on jenkins - see http://jenkins-php.org/ as a template, although I've read good feedback about Teamcity. Then, depending on the code you write you setup unit tests (for raw php code, maybe a bit with of mocks), integration tests (API and module connectivity) and system tests (selenium).
Once you have it running after each build you can be sure that at least covered functionality is working. The problem however is that you will spend more time on writing tests and their support as well as thinking about testable code. Also keep in mind that you cannot cover everything - thats not the point. You must have critical paths covered.
I'd like to ask about real life experiences with release management of PHP projects over Hudson CI server.
Our projects are separated into subprojects: frontends, framework, libraries. Everything is stored in our SVN as project of its own. Different frontends may depend on different versions of framework which itself depends on different versions of libraries.
So far we do release management by hand. We have one environment serving as test and production to avoid environment differences problems releasing the project. We tag each subproject in SVN with release number and checkout everything for testing accessible under test domain. Once we do testing and fix the bugs we tag new versions of everything in SVN as production and relink production domain to point to the new code. This manual process has its obvious problems and there is no way we are keeping it.
Under my research to move to automated solution I already installed Hudson and configured it for PHP projects (Phing, ppUnit, etc...) I am familiar with writing build scripts under Phing or Ant. I red all possible "the basics" stuff of setting up CI environment and preparing project and done that. What I did not find is an example of release management similar to what we do manually taking into account all the dependencies we have. Can you please point me out into a right direction?
I'm afraid there is no such complex thing readily available. We had to solve quite similar problem and we ended up using very similar configuration you were planning to use (or already using).
We have multiple application cores and then specific client modifications on top of that. All is stored in SVN. Additionally we use svn:externals to link Framework and other 3rd party libraries with the application.
All is done using Phing and although it took us some time, Phing helped a lot and I can really recommend it.
Client specific addons are fetched automatically using phing. Everything is configurable through build properties.
For database schema updates we are very happy with dbdeploy which is a part of Phing. Although we had to modify it slightly for our needs.
Additionally we added a support for creating pre-configured self-extracting Linux installer so the whole complicated process of creating a deployable package consists of calling one phing target and passing a correct build property file. To create those installers we use this simple technique (http://www.linuxjournal.com/node/1005818).
Again using Phing the created package is automatically uploaded to a target server, executed over SSH to do the deploy.
Then we use Hudson for automatically creating installer packages (in addition to automatically running phpunit tests and Selenium/Hmres tests) and storing them in a defined location/or as artifacts. Our support team can then grab the packages and do the production deployment themselves (our QA/Test environments are updated automatically by Hudson).
Additionally the code is automatically encoded and licensed using ZendGuard where necessary.
The brief description above is just to illustrate what can be achieved using Hudson, Phing, SVN and PHP. Full technical details would obviously be too long for this post but I would be happy to elaborate more somewhere else.
I'm deveoping a big project, I have the dev folder (connected to a specific subdomain) then the "real" folder, the live one. When I'm ready to push patches or whole new versions I'm currently copying the files individually, is there a program that can help me do this task?
Keep in mind that some files (the config one and the htacess) and folders (the dev ones) do not need to be copied in the live version.
Thank you
Yes: subversion (or any other version control system) will allow you to push changes painlessly.
A simplicistic solution would be to have one checkout where you develop and you commit to, and another checkout which is the deployment. When you are ready, you go to the deployment directory, and do a svn up, to sync it. It won't overwrite modified or excluded files.
There are build packages like Capistrano and Phing which can help with more complicated deployments. Capistrano is Ruby-based, so it is a more natual choice for RoR applications, and Phing (being PHP-based) can be a little more convenient for PHP-based projects. In my experience, Phing seems less mature than Capistrano, but is a little more flexible because it doesn't assume you are working with a Ruby project like Capistrano seems to do. That's entirely opinion of course.
Both tend to take more thought and work to configure up front, but once you've designed the deploy script, you can run a single command and have everything happen for you while you watch. Both tools can integrate with source control like SVN, and bring copies of your project out of the repository for you. You can also break your deployment out into sub-parts, like a traditional Makefile, which helps with testing and reuse. If you want the process you go through for your releases to be bulletproof and consistent, you need to use a tool that will manage all the steps involved for you so you remove the human-error component.
I've installed Eclipse 3.5.1 (PDE), which I believe I got from Zend's download site (it was a while ago on my travel laptop). I can't get auto-complete to work for any of my included libraries. I've tried both adding the libraries to the 'include path' and just linking the files to a subdirectory of the project. Neither gets me auto-complete of the library classes.
My normal development system has an older version of Eclipse which I don't update, because it currently works well, and I fear an update will change that. I find configuring eclipse more work that actual coding, and more voodoo that mod_rewrite. I doubt I'm alone.
Any secret to getting auto-complete working?
No you're definitely not alone! I have experienced this problem in many installations of Eclipse (after updates and such etc etc). Try running Build Project. I know that solution has worked in some instances for me. My last installation I was running Eclipse Galileo and once I finally got the intellisense to at least work it was extremely slow. I tried lowering the time delay for the intellisense which helped in php files (still had a 1.5 second delay at least) but made it way over-sensative in my view scripts for example.
I finally bit the bullet and switched to Zend Studio 8. (At least they've knocked the price down $100 haha).
Eclipse has a quirky auto-complete but the Zend one is excellent! :)
You can add a PHP library to the project in Project properties/PHP Include Path/Libraries. After that, autocomplete will index all php files in that directory.
Maybe it's not the best solution for you, because it doesn't care about whether the file is included or not in the given script. If I include /usr/share/pear, and now I see all PEAR libs' functions everywhere.
There are so many options when it comes to PHP development environments and you have to piece it all together yourself.
I'm wondering if someone has come up with what they think is the ideal setup that gets out of your way and lets you develop.
Right now I use vim and svn from the command-line. I write scripts to manage builds but I'm thinking about looking into Phing.
I love vim but I'm seriously thinking of trying Eclipse with the PHP plugin because I imagine it makes common SVN options a bit easier (moving files around in a project).
Something to support continuous integration on the database would be a major plus!
UPDATE: Just wanted to stress that previous line up there. I realize some frameworks will help with this, but I don't use a framework. Is there some simple module out there (included in the IDE or not) that will let me easily tie my database schemas/data to a subversion revision, letting me rollback and forward, tag, branch, etc?
Any comments on things beyond the editor? For example: Builds, managing staging/production/development environments, automated testing and building upon SVN commit, etc. Ideally we can make this post a "Go to Whoah" for setting up a professional PHP team development environment.
I recommend to use a complete featured IDE like the PDT (the eclipse PHP project), it gives you:
debugging (using Xdebug or ZendDebugger)
SVN/CVS very convinient integration
DB integration (the DTP plugin)
and much more, based on features of the PDT and eclipse plugins
if you have some money to spent, I think the Zend Studio For eclipse worth it.
It gives you better debugging, PHPUnit integration, ZendFramewrok support, Refactoring and remote system support (ftp, ssh etc.)
I'm giving Netbeans 6.5 PHP bundle a try and liking it very much. I find debugging in it is less clunky than in Eclipse PDT.
I too love vim and used to develop using the same environment as you. These days though I find Eclipse PDT, with Subclipse for SVN integration, to be invaluable. XDebug is great too - no more var_dump();exit; for debugging.
One of the best plugins for a vim fan moving to Eclipse: viPlugin. Well worth the token licence fee to have vi key bindings in Eclipse.
If you are working from the command line, using Git's SVN module eases most of the SVN pain - it handles deletes and moves automagically.
The GUI front ends (kgit or qgit) provide a very intuitive history browser.
I personally like the way that AptanaStudio has pre-packaged all the great Eclipse modules you need to have a very smooth PHP development environment