I am working on a small web project, using a PHP framework and I would like to check my development into some source code management for easy deployment to different environments. I chose Laravel and Git, but I guess the exact systems are not important for this question.
I am unsure what exact folder structure is best to be submitted to the source control. Should I:
check in the whole framework, including the config files?
check in the "application" and "public" parts into separate repositories, but leave the config and system files out of source control?
If 1., how do I make sure that I am not overwriting deployment specific configs (i.e. dev, qa, prod) when I do a commit/push/pull.
If 2., how do I make sure that I keep the non-committed parts of the app in sync.
Any help/guidelines/best practices would be appreciated.
I'd check everything in and use Laravel's environments support. It sounds as if you're maintaining separate configs for separate environments, instead of using the built in features.
Alternatively, you could try creating a repository for the app, and a repository for each config, and pull both the app and the relevant configs when testing.
Related
By default, Laravel git ignores the .env file that contains environment settings and configuration.
I would like to have a local .env, and a production .env that CAN be committed to my server so that I don't need to manually create production settings or touch the .env file at all and can use push to deploy techniques.
Anyone got any good solution for this? It seems a strange design decision to make developers always manually create the production settings in a single file, and not make it easy to switch between production and local settings. I was considering using git hooks to rename .production to .env but this seems like overkill, and I want to check that this functionality isn't available out of the box and I've missed it.
Bonus points for explaining why having three files .local.env, .production.env and .env (which switches between the two) files is a bad idea, and one the original team avoided.
I'm using the latest version of Laravel (5.3).
The reasons behind this design are mainly:
Security concerns
Different environments might need different settings
Laravel makes use of the DotEnv PHP library by Vance Lucas to provide the .env functionality, so you can read more about it there as well as in the officiall Laravel docs:
Your .env file should not be committed to your application's source
control, since each developer / server using your application could
require a different environment configuration.
If you are developing with a team, you may wish to continue including
a .env.example file with your application. By putting place-holder
values in the example configuration file, other developers on your
team can clearly see which environment variables are needed to run
your application.
And from DotEnv:
The .env file is generally kept out of version control since it can
contain sensitive API keys and passwords. A separate .env.example file
is created with all the required environment variables defined except
for the sensitive ones, which are either user-supplied for their own
development environments or are communicated elsewhere to project
collaborators. The project collaborators then independently copy the
.env.example file to a local .env and ensure all the settings are
correct for their local environment, filling in the secret keys or
providing their own values when necessary. In this usage, the .env
file should be added to the project's .gitignore file so that it will
never be committed by collaborators. This usage ensures that no
sensitive passwords or API keys will ever be in the version control
history so there is less risk of a security breach, and production
values will never have to be shared with all project collaborators.
In summary, if you're certain that you will deploy to only one system and settings will always be the same, and there is no sensitive data inside the .env (or you're OK with everyone involved seeing it), then you could probably go ahead with your solution.
As a side node: Since you were asking for design decisions specifically, DotEnv was apparently inspired by The Twelve-Factor App, which advocates strict separation of config from code among other factors.
I have been approached to develop some front-end components (CSS/JS & templates) for an existing Typo3 website. The client already has a development server set up, with Typo3 & extensions installed, and a copy of their database. I have SSH & FTP access to this development server. I am looking for advice on the fastest & most practical way to begin work on the templates.
My level of experience is that I have done front-end work with Typo3 before, but always in a team with an existing version control, build & deployment workflow. In this case, I would be responsible for setting up my own workflow and tools.
I have spent some time reading through version control, build & deployment-related questions on Stack Overflow, but many of them assume:
A team with multiple developers
A long-running project with major changes to functionality, in which it makes sense to invest considerable time up-front to the build process
It is 2010.
An existing workflow (e.g existing development environments) into which an additional tool such as git will be added.
In my case, I will be the only developer working on this, and I have been hired only to make some layout updates. I have no existing development environment for Typo3, and would need to set this up. My work on this project is not intended to run for more than a couple of weeks. I would like to find a solution somewhere in between "edit files directly on the development server" (very bad practise) and "set up a fully-featured PHP application deployment service using Magellanes" (probably good practise, but a steep learning curve for me and a large investment of time).
Ideally, I want to wind up in a situation where I have:
A local development environment on my Mac with Typo3 installed where I can preview & test code changes
Version control such as git on my local system
A way to push my changes to the development site
Can anyone share with me tools or workflow suggestions for how to get to that point as quickly as possible?
My environment is similar to yours, and this is my typical setup:
Version control
The website is version controlled with git. There may or may not be a repository to regularly push to, but my local version is always included in the backups. The following things are excluded from version control:
Files in typo3temp, but not the directories (Put the .gitignore at the end of the post into a directory to keep it, but not the files in it)
The server specific configuration
deprecation logs
IDE files
...
To include the TYPO3 core, there are three variants:
Just add it to the repository
Add the core git repository as git submodule
Add symlinks to the core into the repository.
The latter may be problematic if the directory structure is different in both environments, and the first one clutters the repository, so when possible, go with the second one.
TYPO3 configuration
The TYPO3 configuration is split into three files:
Localconfiguration.php - version controlled, not manually edited (it is constantly overwritten by TYPO3).
AdditionalConfiguration.php - version controlled, manually edited.
ServerspecificConfiguration.php - not version controlled, manually edited. Contains database credentials, colorspaces for imagemagick when different on localhost and remote host, defines caching backends to use and similar stuff.
The additional configuration file includes the server specific file. Both use the \TYPO3\CMS\Core\Configuration\ConfigurationManagerConfigurationManager::setLocalConfigurationValueByPath('DB/host', 'localhost');-syntax for settings to make this work.
Deployment
To deploy the site, I have used two things:
git ftp - this is useful if there is only FTP access.
rsync - This is the better option.
To automate the deployment (and other stuff like building CSS from LESS or SASS), a task runner like grunt.js or ant is useful. Make sure you exclude the server specific configuration and typo3temp from synchronization.
Building such a setup from scratch does not take that much time, maybe 1 or 2 hours, but less then a day. This may of course differ depending on your experience.
Here is the .gitignore mentioned above:
*
!.gitignore
!*/
I am trying to work what the point of the environments folder is.
Originally I had the idea that you could point the webserver to the different dev and prod folders in the environment folder but after reading up a bit I realise this is not the case.
In Yii 1 you would solve this by just having multiple index.php's i.e.:
index.php
index-local.php
So the question is what benefit does this new environment structure actually give me over the old way?
I've found environments very useful in allowing me to keep a common code base for multiple client projects (based on Yii App Advanced) and setting up a different environment for each specific client, keeping their custom code private and separate.
To do this I store the environments folder in a separate git repo from the rest of the code and pull down the relevant folder on a client/project basis.
This lets me use a base common code for all projects and add/override any file for a specific client or project whilst still allowing separate dev/prod config settings. If the client uses other developers too, they are also catered for. In this way, only common code I choose will be shared amongst clients and custom code will be kept private.
I've also moved the composer.json file into the environments folder so I can pull in different extensions per client/project keeping those private too.
That init command can be a very powerful tool and you don't have to limit yourself to the template provided by the core developers.
If you don't need environments, then don't use them, but I assure you some people will find it very useful.
Yii2 documentation in WIP, but you should read this :
https://github.com/yiisoft/yii2/blob/master/docs/guide/apps-advanced.md#configuration-and-environments
You need to use yii init command to switch between these environments.
EDIT :
This new environment feature is more than just use different config file. You can use different folder structure, different entry script...etc
Personnaly I won't use this feature, I don't need it (I will use a different entry script as with Yii 1), but I think this is not useless.
I think you didn't get the real purpose of environments introduced in Yii2.
I'll try to explain what was the main purpose of adding environments to yii from the developers point of view on an example and hope you will really appreciate its usefulness.
Let's suppose for a moment that you are a team of developers (e.g. 5-7 person) working on mid-to-large project implemented in Yii. To effectively work on that project your team decides to use some CVS or SVN (e.g. GIT) and keep all the files of the project in repository in cloud for the whole team. That's de facto standard while working on mid-to-large projects in teams and nobody will resist that it's the only comfortable and easy way.
Ok, now let's suppose you use Yii 1.x or Yii2 with the approach of different entry scripts to differentiate between local (development) and production environments to connect to db or set some other environment specific configs. Everything is ok and working. But suppose your team members implemented something new on the project and you check out repository to work on updated version and you suddenly find out that your local config file (in this case entry script with config) is overwritten with other team member's file who pulled the changes to repository (because each of you is using your local machine db with other database name or OS, or config, or simply because your team uses one local development server db, but you are on vacation and can't use anything except your local machine).
So generally Yii2 environment adds more flexibility for using different environments each with it's own specific configurations while using also general (common) configs when working in teams on mid-to-large projects hence why the example in guide is given on advanced app project.
Surely you can overcome everything stated above with some solutions or .gitignore which is used by default to overcome the problem stated in Yii2 with environments. But:
Why bother if everything is already done?
and
It was just one little example of usefulness of Yii2 environments. More depends on the project and your imagination.
Overall Yii2 is great product. Not only it adds many new features to already great framework, but it also is more robust and flexible than Yii 1.x (despite the fact that Yii 1.x was already very robust).
As for Laravel or any other PHP framework, it really depends... Everyone will find his/her own favorite.
For those who are tired of copying files around, I created a useful script that you can run in background to keep the files in sync on your dev environment:
File sync-env-files.sh
#!/bin/bash
ENVIRONMENT_DIR="/var/www/example.com/environments/dev/"
DIR="/var/www/example.com/"
while [ true ]; do
for envFile in `find $ENVIRONMENT_DIR -type f`
do
file=${envFile/$ENVIRONMENT_DIR/$DIR}
if [ `stat -c "%Y" $file` -gt `stat -c "%Y" $envFile` ]; then
#echo "copying modified file $file to $envFile"
/bin/cp -f $file $envFile
fi
done
sleep 2
done
Then run the script in background or add to cron with flock
nohup server/sync-env-files.sh >/dev/null 2>&1 &
I would like to mention in addition to #AngelCoding, since this question still gets seen, that I use the environments folder lots now and definitely see the point of it.
The very first things I do in any open source project is create one project for the code base on GitHub and then another, private, one on Bitbucket for the configuration, in other words the environments folder.
Having this folder has made it a lot easier for me to separate my configuration into a private repository.
So the environments folder has a lot of uses and really helps to separate configuration for easier usage even if it does not seem like it initially.
I have 3 PHP projects using the CodeIgniter framework which share some exact same files such as models libraries and controllers. What's the best way I could share these files across without having to keep in sync and update the same files across?
In linux I thought of using dynamic links and extract these files to a central place but that kind of breaks our version control and would create portability issues.
Another way perhaps to use unison on these files across projects
I'm assuming that's a common problem, what are common approaches?
Separate them into a module, and use something like composer.
http://getcomposer.org/
Or just put them in a separate SCM.
One thing you can do:
Put all the shared code in libraries, helpers and models and place this in a separate folder. Then use:
$this->load->add_package_path('shared location');
Also take a look here: http://codeigniter.com/user_guide/libraries/loader.html , under application packages.
This works for most of the stuff, except controllers.
Use version control! In svn you can use externals, git has submodules or subtrees.
You don't want to use hardlinks, you'll run into weird issues like updating one project influences another project ("that I haven't touched in weeks").
The code can be in two physical places but shared under version control. There will always be only one authorative copy, namely the one in your version system. All physical copies are derivatives. It's important to see that you have control over when you update the code of a specific project, so a change at one point doesn't immediately break another project in case you made a mistake.
If you do want to catch these kinds of errors, set up a proper regression testing environment.
Sharing a development environment with another developer is also a big no. You don't want to have to wait till your colleague fixes a parse error that breaks the entire program. Each developer should have their own copy (checkout!) of a project and similarly each project should have their own copy (externaled) of shared code.
Seperate them into folders outside your project, then configure or include them in your projects.
Usually we will rewrite "autoloader" method for the project to find files in our new folders.
I have a PHP Platform that I've built.
I use a copy of that project for each project I do.
I add assets and configure each project with it's own content.
Configuring a project changes some of the core files which contain
the configuration defaults.
When I update core files in the library/platform itself and I want to disseminate those changes to all of the other projects using that platform, what would be the best approach?
I think it may be worth noting that I have each project as an SVN project in Eclipse.
UPDATED DETAILS:
The platform is based on AMFPHP 1.9.
AMFPHP has a services folder.
I have to put core services in the services folder as well as
implementation specific services and therefore cannot achieve
complete separation between the "library" and the "application".
Also, the default configuration files need to be in the "platform" folder
so they can be easily distributed with it, but when configuration is
changed, those files become "implementation specific" as well.
Is there a solution that will allow me to designate certain files to update?
Take a look at the use of svn:externals. Configure each secondary project with your core libraries as an external.
Excellent question, here's the way I solved this problem with my own PHP framework.
If you have all the project-specific and core files mixed together, and platform code interspersed with your project code, it's going to be really difficult to update the core code to all of your projects.
The best way to get around this is to make it easier for svn (or git, or any other versioning software) to update your files: if they're all neatly organized, with project and core files in their respective places. Let me give you an example to clarify.
Put all of the core files of your framework in this directory:
/App/FrameworkName
Here you can place core classes, functions, and other code that won't change for all of your different projects.
Then, any project-related content, settings or pages go in:
/App/Project
Here, you have all of the data that your projects use.
With this system, if you add a feature to your platform, all you have to do is svn the latest version of your framework to /App/FrameworkName, and your project will be using up-to-date code.
My programs are usually organised in two parts. the core framework folder, and the addon folders.my framework (in house) does all the initial work then loads the addon for additional functionality.
For example, the framework deals with all the user auth , the addon provide additional crud functions the each project needs.
That way, i can keep them in separate repository. and the framework can be checked out and used for other projects.