I'm looking for a script that will quickly implement package install/update functionality for my framework. It should be able to do the following things:
Must be PHP 5 compatible.
Should be able to install and update components' files and database tables.
Should allow defining of package dependencies and have a form of conflict resolving.
Should allow UPDATE SQL-statements when the developer of the package provides it. (To make database alterations without losing data).
Should be able to 'jail' an update package to a target folder.
Should be able to define and detect different package types to apply mentioned 'jail'.
Should provide an API to integrate it in custom backend.
Should allow downloading of the packages, preferably over HTTP (rather than FTP or others).
I plan on using this as a stand-alone utility to easily provide updates for modular frameworks as well as custom code for our clients.
If you have recommendations of tools that are similar or used in an open source framework to base it off that would be good as well. :)
I don't know of any magic tools to do this, but a couple that might help you do these tasks are Phrake and Composer.
Phake is a PHP version of Ruby's Rake util. You could use these scripts to set up your database, update files and run unit tests.
For package management you could use Composer to manage your dependencies (it can also download from PEAR if you depend on packages from those systems) and of course this could be incorporated into your build scripts.
Related
Sounds weird, but I thought composer was a tool that one used to install packages in PHP stacks. An efficient and robust way to make sure that php environments are setup correctly.
But I keep coming across forum posts that talk about caching and advising to do things like composer clear-cache as if it was part of the actual running application. Like it's actively doing things in the running app.
Am I missing something?
Composer is a tool for dependency management in PHP. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you(refer link.
It helps us installing/updating various requirements/components for our app such as Laravel framework, Doctrine, lodash etc..
If you have ever written anything in PHP before, you have probably found that it feels like you have to keep re-inventing the wheel anytime you want to do a common task such as User Authentication, Database Management or Request Routing.
If you were to start manually picking the bits you wanted from Laravel then it would become very difficult to manage. Each library might also have dependencies, and so you would end up in a mess, particularly if you required other people to work on your project.
This is where Composer comes in. Composer is a dependency manager for PHP. Composer will manage the dependencies you require on a project by project basis. This means that Composer will pull in all the required libraries, dependencies and manage them all in one place.
Managing your dependencies manually in any programing language is an immense pain. This is often why, in most programming languages these days you may notice that all of them have some implementation of a dependency management system or generally a package manager.
In PHP, we use NPM i.e Node Package Manager in frontend technologies like JavaScript, VueJS. For backend, Composer is the de facto dependency manager.
Laravel is itself a package of packages, hence to develop our projects smoothly among the team members, dependency management becomes a must and composer does its work under the hood, silently but efficiently.
Composer is an application-level package manager for the PHP programming language that provides a standard format for managing dependencies of PHP software and required libraries.
Composer runs through the command line and installs dependencies (e.g. libraries) for an application. It also allows users to install PHP applications that are available on "Packagist" which is its main repository containing available packages. It also provides autoload capabilities for libraries that specify autoload information to ease usage of third-party code.
Composer is used as an integral part of several popular open-source PHP projects, including Laravel.
Composer is very awesome for managing dependencies and makes life so much easier.
However, I am a bit unsure where I should place data files to be used by my composer requirements.
For example, the Maxmind GeoIP library is something I am using. Because the maxmind databases are provided independently of the PHP library and have a different cycle, there really isn't a way to make it a composer requirement.
Should I add them as packages to my composer.json? Since I have a Satis installation for serving private repos, should I create my own repo for those datafiles?
What is the accepted practice of dealing with vendor databases and datafiles when using composer?
If you want to manage the data via composer, I would suggest to configure them as an own package, which you require on project level.
There is no default workflow for non php parts. Especially databases are rarely managed via composer, as there is no standard for doing migrations.
In my opinion, migration scripts should not be part of composer, because only your application should alter your databases, and only when you want it to, not when you execute composer commands.
The GeoIP datas are a bit different, this is only a kind resource file you use. If I look at attempts regarding java-script and similar, its common to have a separate composer package for them. So like I said first, add the data in an own package, which you can maintain separate from the GeoIp Library
I'd like to ask about real life experiences with release management of PHP projects over Hudson CI server.
Our projects are separated into subprojects: frontends, framework, libraries. Everything is stored in our SVN as project of its own. Different frontends may depend on different versions of framework which itself depends on different versions of libraries.
So far we do release management by hand. We have one environment serving as test and production to avoid environment differences problems releasing the project. We tag each subproject in SVN with release number and checkout everything for testing accessible under test domain. Once we do testing and fix the bugs we tag new versions of everything in SVN as production and relink production domain to point to the new code. This manual process has its obvious problems and there is no way we are keeping it.
Under my research to move to automated solution I already installed Hudson and configured it for PHP projects (Phing, ppUnit, etc...) I am familiar with writing build scripts under Phing or Ant. I red all possible "the basics" stuff of setting up CI environment and preparing project and done that. What I did not find is an example of release management similar to what we do manually taking into account all the dependencies we have. Can you please point me out into a right direction?
I'm afraid there is no such complex thing readily available. We had to solve quite similar problem and we ended up using very similar configuration you were planning to use (or already using).
We have multiple application cores and then specific client modifications on top of that. All is stored in SVN. Additionally we use svn:externals to link Framework and other 3rd party libraries with the application.
All is done using Phing and although it took us some time, Phing helped a lot and I can really recommend it.
Client specific addons are fetched automatically using phing. Everything is configurable through build properties.
For database schema updates we are very happy with dbdeploy which is a part of Phing. Although we had to modify it slightly for our needs.
Additionally we added a support for creating pre-configured self-extracting Linux installer so the whole complicated process of creating a deployable package consists of calling one phing target and passing a correct build property file. To create those installers we use this simple technique (http://www.linuxjournal.com/node/1005818).
Again using Phing the created package is automatically uploaded to a target server, executed over SSH to do the deploy.
Then we use Hudson for automatically creating installer packages (in addition to automatically running phpunit tests and Selenium/Hmres tests) and storing them in a defined location/or as artifacts. Our support team can then grab the packages and do the production deployment themselves (our QA/Test environments are updated automatically by Hudson).
Additionally the code is automatically encoded and licensed using ZendGuard where necessary.
The brief description above is just to illustrate what can be achieved using Hudson, Phing, SVN and PHP. Full technical details would obviously be too long for this post but I would be happy to elaborate more somewhere else.
I have a folder of PHP scripts, they are mostly utility scripts. How to share those scripts among different PHP applications so that reuse and deployment are easy?
I would have to package my app into an installer, and let the user install it.
I could put the lib and hardcode the include path, but that means I haven to change the PHP code every time i deploy the web application to a new customer. This is not desirable.
Another route I consider is to copy the lib to other apps, but still, since the lib is constantly updating, that means that I need to constantly do the copying, and this will introduce a lot of problems. I want an automated way to do this.
Edit: Some of the applications are Symfony, some are not.
You could create a PEAR package.
See Easy PEAR Package Creation for more information on how to do this.
This assumes that when you say anyone, you mean outside your immediate organisation.
Updated: You do not need to upload to a website to install the PEAR package. Just extract your archive into the pear folder to use in a PHP application.
Added: Why not create a new SVN repository for your library? Lets say you create a library called FOO. Inside the repostory you could use the folder heirachy of trunk\lib\foo. Your modules could then go into trunk\lib\foo\modules and have a file called trunk\lib\foo\libfoo.php. Now libfoo.php can include once or require once all the modules as required.
PHP now supports Phar archives. There's full documentation on php.net.
There's a complete tutorial on IBM website as well.
One neat thing you can do with Phar archives is package an entire application and distribute it that way.
http://php.net/phar
http://www.ibm.com/developerworks/opensource/library/os-php-5.3new4/index.html
Ahh, libraries...
There are two conflicting purposes here:
Sanity when updating scripts (ie. not breaking 10 other apps).
Keeping things in one organized logical place for developer efficiency.
I suggest you take a close look at git and git submodules
We use git submodules extensively for this very purpose. It allows the best of both worlds because shared scripts can be upgraded at will in any project, and then that change can be moved to the other projects (deliberately) when you have time to do so and test correctly.
Of course, you need to be using git to take advantage of submodules, but if you are not using git, and you start, you'll eventually wonder how you ever lived without it.
Edit: Since the original poster is using svn, consider using SVN Externals.
UPDATED:
you just have to put the lib in some place reachable by your apps (in a place where you can reach it via http or ftp or https or something else) and include it.
If you have to update it often you can package your library in a single phar file and you can then provide your client a function to pull the library from some remote path and update a parameter in their local configuration accordingly, like:
function updateLocalLibary(){
//read the remote library in a variable
$file= file_get_content($remoteLibraryRepository.$libraryPharFile);
//give it a unique name
$newLibraryName=$libraryPharFile."_".date('Ymdhsi');
//store the library it on a local file
file_put_content($localLibraryPath.$newLibraryName,$file);
//update the configuration, letting your app point to the new library
updateLatestLibraryPathInConfig($newLibraryName);
//possibly delete the old lib
}
In your include path then you don't have necesasrily to hardcode a path, you can include a parameter based on your config, like:
include( getLatestLibraryPathFromConfig() )
(you are responsible to secure the retrieval in order to let only your clients see the library)
Your conf can be in a db, so that when you call updateLibraryPathInConfig() you can perform an atomical operation and you are sure not to have client read dirty data.
The clients can then update their library as needed. They may even schedule regular updates.
There are a lot of options:
tar + ftp/scp
PEAR (see above #Wayne)
SVN
rsync
NFS
I recommend to use a continuous integration software (Atlassian Bamboo, CruiseControl); check out your repository, build a package, and then use rsync. Automatically.
You should also look into using namespace in order to avoid conflicts with other libraries you might use. pear is probably a good idea for the delivery method, however, you can just place it in the standard path /usr/share/php/, or any other place that is set as the include path in your php settings file.
Good question, and probably one that doesn't have a definite answer. You can basically pick between two different strategies for distributing your code: Either you put commonly used code in one place and let individual applications load from the same shared place, or you use a source-control-system to synchronise between local copies. They aren't mutually exclusive, so you'll often see both patterns in use at the same time.
Using the file system to share code
You can layer the include_path to create varying scopes of inclusion. The most obvious application of this pattern is a globally maintained PEAR repository and a local application. If your it-system consists of multiple applications that share a common set of libraries, you can add a layer in between these (a framework layer). If you structure the include_path such that the local paths come before the global paths, you can use this to make local overrides of files. This is a rather crude way to extend code, since it works per-file, but it can be useful in some cases.
Use source-control
Another strategy is to make a lot of local checkouts of a single shared repository. Some benefits over the layered-include-pattern is that you can make more fine grained local changes. It can be a bit of a challenge to manage the separation between application layers (infrastructure, framework, application). svn:externals can work, but has some limitations. It's also slightly more complicated to propagate global changes to all applications. An automated deployment process can help with that.
this is a question on PHP mainly. I was wondering: How do you make sure that all necessary libaries are packaged with your application when you do a deployment to (production) servers?
A more concrete example: I have an app running on Zend Framework and each time I roll the application to a server the deployment process creates a fresh "installation" on that system. Therefore, I need to bundle Zend Framework together with my application and then copy the files to the right places together (it is done automatically). Currently, I am using a svn:externals definition to get the files out of Zend's SVN system during deployment, however, I don't want to rely on that SVN and I also don't want to put traffic on external SVNs with each deployment.
In the Java world, I am used to Maven which handles such stuff using central artifact repositries. I know that there is a Maven4PHP version, however, I am more looking for a PHP-based solution. Additionally, I don't believe that PEAR is a good way to go as it doesn't really fulfill my requirement of bundling the applicaiton (incl. libs) into a single deployable.
Is there some tool available already that I am not aware? Or do you have any great technique that I should know?
Thanks much for your help!
Michael
There's a build system called Phing which is written in PHP and based on Apache Ant.
I personally can very well live with externals.
I think the vendor branching would solve the problem from your example quite straightforward, but if you also don't like large repositories I'd recommended to keep watching on the modern toys like composer and what it solve(and maybe phark, I never heard before :) )
It isn't production ready yet but you might want to keep an eye on the Phark project. It is a port of Bundler to PHP.
While looking through the Simplify your external dependency management slides I came across a tool called pantr which can be used as a PEAR installer. pantr as PEAR installer which allows you to specify your dependencies in a project specific file.
The article Version Control != Dependency Management has some information about using the new PEAR installer called Pyrus