Not sure if this should be here or on Programmers.
Generating API documents
I would like some advice on how I should generate API documentation for an internal project. I am relatively new to Git and we are trying to implement some sound build/deploying practices.
One of the things we discussed was making sure our code base is well documented and generating documentation using something like PhpDocumentor2 or one of the many similar tools.
We have started to implement a workflow similar to the one detailed here.
Should I automate when the docs are built?
For example a pre or post commit hook in git when tagging a release. Or should when I merge develop to a release branch just manually create the docs and commit to the repository?
Is it standard practice to have docs generated for each release?
I might have misunderstood the process, should a new doc release correlate with a git release/tag?
Where do you store the generated docs?
In the same repository? a different repository? Hosted somewhere like Read The Docs or just internally?
The current project we are working on is only small, but we would like to roll out the process to other larger projects in the future if successful.
Context
The project is a Magento extension which we would like to provide API docs, unit testing, and PSR conforming code. I am lacking information on how the whole workflow integrates. PHPunit and PHPDocumentor2 are installed locally via Composer.
I have heard and looked at Travis Ci, but I'm not sure if Docs fall in to that category.
This question may seem petty and/or trivial, however, I've not much experience in integration and git workflow and I couldn't find much information around.
Generated documented generally are:
always in sync with the code source (so the question of "should a new doc release correlate with a git release/tag" becomes moot)
not stored in a version control referential (like a git repo), but rather (re-)generated at will (in any location you like).
If you look at a project with a large code source, and an extensive code documentation, you can take as an example the language Go and his repository (a mercurial repo, but you have mirror on GitHub as well)
static documentations like the specs, articles, release notes, ... are kepts within the repo, as they are not generated, but updated manually, and are tightly linked to the sources.
Code documentation is kept separately in a static web site.
Documentation for all go project is kept in a static website GoDoc, which will fetch the sources of Go projects, and generate the documentation from them.
Related
After some fiddling building a package for a project we've realised there's some issues with doing what we need to achieve as per Laravel 5 package development clarity
Maybe I should rather explain my goal and someone can suggest a direction to head in.
We've built a Laravel 5 application that now needs to be "re-used".
We had to modify Laravel and implement an Eloquent type base model as our data-source is actually C# Web Services. At the point the call would be made to a database we intercept this and make an "API" call to SOAP.
The major difference will be CSS, maybe some JS & content but all the routes/controllers/models will remain the same across all projects. Most configuration comes from endpoints.
Initially we considered creating multiple asset repositories for each site's styling and have a base repo which is the core Laravel project that gets included. This seemed to get quite complex as we couldn't simply just have a repo in a repo due to branching and multiple directory issues.
We then started experimenting with the idea of building the "core" as a Laravel package but we seem to constantly hit walls. The latest problem being including models in the package. For the models to be called we are using the root projects config/composer to access these models instead of just the service provider. It feels like the package is becoming to tightly coupled to the project config.
Are there any better ways of going about what we are trying to achieve?
Edit:
I forgot about the multiple branch solution on 1 repo but wouldn't this get ugly when it comes to feature development? Example:
master (core with releases that get pulled into _site*)
dev (master dev)
feedback-form (eg. master branch feature)
_site1 (root site with releases)
_site1-dev (_site1 dev)
_site1-reskin (eg. _site1 feature)
_site2 (root site with releases)
_site3 (root site with releases)
This leaving quite a bit of destructive merge power in the developers hands? Read access with pull requests maybe a solution to this?
So after some R&D it seems the best solution right now is to have 1 repo with multiple branches. Developers have read access and have each developer creates his own fork. Developers create pull requests and sync to parent repo through "upstream" remote and developers sync each others forks through additional remotes.
Seems a little clumsy but probably "cleanest" option.
While developing a Symfony2 project, I often come across bugs in third party bundles. Most of the time the bugs are subtle but hard to find. For example this week alone I have found three bugs where a value was tested using a simple if ( $value ) construct but required the use of ( $value !== null) or ( $value !== false ).
Without having sufficient permissions on the relevant github pages for the projects in question, the best I can do is push a pull request. It usually takes quite some time for the request to be merged. In the mean time, especially when using the master version, other pull requests are merged which in turn leads composer to update. When that happens, any local bug fixes will revert back to the original code.
Is there any method to handle this situation?
Ideally, I would like the third party bundle to update but have my modifications persist. Until the pull request is merged of course.
There is a project that allows you to apply patches after downloading packages with composer. It is created to be used with the Drupal project but I believe it should work with your own patches just as well.
https://github.com/jpstacey/composer-patcher
Otherwise, you could fork the project, make you improvements, submit a pull request and in the mean time use your own forked repository in composer. See [this answer][https://stackoverflow.com/a/14637668/3492835) for a detailed description of how to achieve that.
Edit:
The stars say it is about to be 2016 now, and a few things have changed.
jpstacey/composer-patcher is considered deprecated in favour of the netresearch/composer-patches-plugin project. This is a Composer plugin which does basically the same, but it is able to apply local patches as well.
Composer does not support this functionality out of the box. The reason is simple, one should not work with the development versions of other libraries. But fear not, you can easily work around this by forking the projects on GitHub. Of course this means a lot of overhead, but it is the best solution I can think of with which you can tackle this problem.
Note that this approach has several advantages over the patch approach:
You can directly create your pull request from your fork.
The git merge process will identify any conflicts.
A script to automate this process is easy:
#!/bin/sh
git fetch upstream
git checkout master
git merge upstream/master
You could create a Composer post update/install script which executes these command in each projects local directory if it is one of your forks. (I leave this implementation part to the reader. But one would need to create the repository locally first, since Composer only downloads the latest files without an repository data. This might add huge .git folders to a project since some projects are, well, huge.)
My team has been working on a web application based on php, msql for backend and html,css,jquery for front end. We have been working for quite sometime now without version control. The project has become quite large and complicated now and feel that it is time to use some sort of version control ( vc ).
We have been reading quite a lot about vc and have found several techniques that people have suggested. although it might be a perfect method for them. it does not effectively apply to our case.
our application is split up into various modules and we have outsourced work to a few freelancers. freelancers work independently over front end or back end depending on their skill. The work has been happening over a Private Network so far and we would like to shift to an online system.
Now the problem is that we cannot distribute the source code for the entire project to all developers. developers are only allowed to work on some common libraries and their respective modules.
Hence we cannot allow the entire project to be downloaded onto each developers Local machine. Thus we need to find a way for all developers to be working on the same branch OR on the trunk. to be able to access only certain sections of the code and on saves/commits be able to check how their changes have effected instantly.
Is there a way the above task can be achieved ? for a web application ? using a version control like subversion?
To summarize, the features we are trying to implement are as follows.
Instant effects on save/commit ( When each developer saves/commits they should be able to test the effects like normal through the browser instantly )
Limited access ( Each developer can access only a specific part of the project and not the whole project. )
Online repository / Online copy - ( we are working on a VPN connection, and would like to have the website work on an online copy. so when a commit is made, the developer can view his changes online instead of the VPN )
after a lot of searching online we were able to find the below possibilities. But not sure if it is the right way to go.
For instant effects --> Entire system is checked out in a webroot folder ( eg. wamp/www/projectName ), a bat file is called to update the current system in the webroot, everytime a commit is made, by using the post-commit hook.
Limited access --> All development for all developers happen in the trunk OR a Development branch, developers can access their respective sections, lock it if needed and commit changes when they are ready to see its effects ( note still in dev branch ).
tags are created whenever a stable release is created. these Tagged copies are never modified.
Is such a configuration achievable using subversion or should we look at other open-source version control tools ?
Trying to prevent devs from accessing the entire source tree seems a bit misguided to me. Is it company politics? Don't you trust your developers?
In any case..
The easiest way to achieve what you want would probably be to put each module into its own repository (svn, git, whatever). Then you can selectively choose who has access to which repo.
Post-commit hook, which unconditionally "do site" is in a common not-so-good idea: developer can commit semi-finished code and even don't think about testing it
Make your Working Copy public, when site became public - not-so-good idea from the POV of security and IP. For SVN 1.7 to move WC-root outside web-root is much better idea
If modules are splitted to different directories, you can create repository per module and "super-repository", which, with svn:externals, combine repositories in Project
Path-access inside tree can be easy controlled (in http-repo) by authz_svn_module
Enable creating private personal short-time branches for developers ("shelves"), it helps to avoid (with 1) huge messed up commits
Firstly, I'd recommend reading the "Continuous Delivery" book (website here). It provides many examples for how to set up this kind of thing.
Secondly, yes, SVN allows you to assign permissions on a folder level as well as repo level (but read the "do you really want to do this" section).
Thirdly, making sure developers have up to date copies is something you should instill by discipline - "before starting work, and before committing, run svn update"). Post-commit hooks have a nasty habit of breaking stuff, and you're not preventing commits which break the build.
Fourthly, I'd consider setting up a continuous integration server (also described in the Continuous Delivery book). This makes sure you have a clean, working build whenever you put together the work of your developers.
I'd like to ask about real life experiences with release management of PHP projects over Hudson CI server.
Our projects are separated into subprojects: frontends, framework, libraries. Everything is stored in our SVN as project of its own. Different frontends may depend on different versions of framework which itself depends on different versions of libraries.
So far we do release management by hand. We have one environment serving as test and production to avoid environment differences problems releasing the project. We tag each subproject in SVN with release number and checkout everything for testing accessible under test domain. Once we do testing and fix the bugs we tag new versions of everything in SVN as production and relink production domain to point to the new code. This manual process has its obvious problems and there is no way we are keeping it.
Under my research to move to automated solution I already installed Hudson and configured it for PHP projects (Phing, ppUnit, etc...) I am familiar with writing build scripts under Phing or Ant. I red all possible "the basics" stuff of setting up CI environment and preparing project and done that. What I did not find is an example of release management similar to what we do manually taking into account all the dependencies we have. Can you please point me out into a right direction?
I'm afraid there is no such complex thing readily available. We had to solve quite similar problem and we ended up using very similar configuration you were planning to use (or already using).
We have multiple application cores and then specific client modifications on top of that. All is stored in SVN. Additionally we use svn:externals to link Framework and other 3rd party libraries with the application.
All is done using Phing and although it took us some time, Phing helped a lot and I can really recommend it.
Client specific addons are fetched automatically using phing. Everything is configurable through build properties.
For database schema updates we are very happy with dbdeploy which is a part of Phing. Although we had to modify it slightly for our needs.
Additionally we added a support for creating pre-configured self-extracting Linux installer so the whole complicated process of creating a deployable package consists of calling one phing target and passing a correct build property file. To create those installers we use this simple technique (http://www.linuxjournal.com/node/1005818).
Again using Phing the created package is automatically uploaded to a target server, executed over SSH to do the deploy.
Then we use Hudson for automatically creating installer packages (in addition to automatically running phpunit tests and Selenium/Hmres tests) and storing them in a defined location/or as artifacts. Our support team can then grab the packages and do the production deployment themselves (our QA/Test environments are updated automatically by Hudson).
Additionally the code is automatically encoded and licensed using ZendGuard where necessary.
The brief description above is just to illustrate what can be achieved using Hudson, Phing, SVN and PHP. Full technical details would obviously be too long for this post but I would be happy to elaborate more somewhere else.
I have a need to create a library of Object Oriented PHP code that will see much reuse and aspires to be highly flexible and modular. Because of its independent nature I would like it to exist as its own SVN project.
I would like to be able to create a new web project, save it in SVN as its own separate project, and include within it the library project code as well. During this process, while coding the web application code and making commits, I may need to add a class to the library. I would like to be able to do so and commit those changes back to the libraries project code.
In light of all this I could manage the code in two ways
Commit the changes to the library back to a branch of its original base project code and make the branch name relevant to the web project I was using it with
Commit the changes to the library back to the original code, growing it in size regardless of any specific references that might exist.
I have two questions
How can I include this library project code into a new project yet not break the subversion functionality, i.e. allowing me to make changes to each project individually?
How I can keep the code synchronized? If I choose the first method of managing the library code I may want to grab changes from another branch and pull it in for use in another.
EDIT - I realize I can simply check out these projects individually and commit/update them individually as well, but then how can I include them together as a single project? To be more clear, how could I create a web project that includes the library code as a unified subversion project in consideration of the points I elaborated on above?
I think you can use svn:externals to achieve what you want. It will pull the library project into your website project and update it whenever you update your working copy. The only thing is you cannot commit back to the library in the same commit as you project as described in this question How do I checkin to local copy AND svn:externals subdirectories in one commit?.
Option #1 looks like the right way to go.
I think you should expect to keep separate branches of the API project for any of your sites that have site-specific modifications to the shared API. Of course, you don't need to create the branch upfront, just checkout the 'trunk' and make sure you branch before you commit any site-specific changes.
There are a couple of articles on branching/merging that I have used in the past that might help you out:
Streamed Lines: Branching Patterns for Parallel Software Development
MSDN Branching and Merging Primer
However, there are some aspects of your 'two questions' that are a bit confusing/concerning. Hopefully I'm misinterpretting what you've said, but keep the following in mind:
With your first question, I think you might be getting caught up on the physical location of the source code on your development machine and how your repositories will be structured (hint: treat the two separately).
In your second question, you mention specific references and it sounds like you might be thinking of making your API in some way dependent on the website source (hint: bad idea for an API).