Does anyone have a good Drupal upgrade strategy for an install that is in production? No one talks about this in books and it's hard to find a definitive answer in forums and email lists.
Ex:
Lock down prod, don't allow
updates to data
copy prod
copy prod database to dev
turn off all modules in dev
upgrade core Drupal in dev (update db if necessary)
upgrade modules in dev (update db if
necessary)
turn on modules
test
migrate code and db to prod
turn site back on
Your strategy sounds good, but it would require a site to be in “read only” mode for quite a while. This is not always feasible. Also I am not quite sure why you would turn on and off all of the modules?
May I propose a slightly different approach
copy prod database to dev
replicate prod code in dev
upgrade core Drupal in dev
run update.php
test
For each module
. Upgrade modules in dev
. Run update.php
. Test
Put into maintenance mode
Backup database
Migrate code to production
Run update.php
Put back online test
This way there is a lot more testing but less downtime, also you will be able to work out which module breaks things if there is an error. It also dosn't rely on you uploading the DB from dev to live.
Don't think there's any need to turn off modules before running update.php anymore (except maybe between major versions). And I definitely wouldn't run update.php once per module - that doesn't make sense with the way update hooks work.
If you're at all comfortable with the command-line (and running on a Linux server) then definitely have a look at Drush. It can simplify the process and allow parts of it to be scripted.
In addition, if you're looking for a formal update process to move stuff from your dev server to production for a large site, you should also be getting up to speed on the hooks that run during install and update.
Related
At work we took back our existing store running on Magento 2 from an external development agency. I need to get the project running in local development (with docker).
I familiarized myself with a vanilla project from the official docs and managed to get it running by downloading the vanilla template with composer, granting the proper permissions on files and folder and running the magento setup:install command.
My question is how do one goes when kick starting from an existing (production running) project?
Do I need to run setup:install again? If I do, why?
What do I need to import from production to ensure any content or configuration created via the admin is also running on my local setup? Should I import the complete Database from production?
I know our setup is using more than just php and mysql, but env.php seems to be listing only db configuration and admin url. Where can I get the complete service configuration informations about what our setup uses?
Anything else I am missing to get started with an existing project for local development?
As someone who is running Magento 2 on a local environment myself, hopefully I can shed some light on this.
If you have a direct copy of the live site, you do not need to run setup:install again.
Ensure you have a copy of the entire Magento 2 site (you can technically ignore the vendor folder, as you can run composer install and it will redownload those files, but that's up to you). Also get a copy of the entire database. Magento 2 is notorious for copying the same data to multiple tables so something could break if you don't have everything.
What do you mean by "service configurations" If you are referring to Magento 2 extensions, that data is saved in the database, not the env.php file. env.php is only for server side configurations, such as the DB information, Caching, and things of that nature. On mine, I use Redis for site Cache, so that would be included in that file as well, as an example.
When you first unpack the site to your local environment, run composer update in the directory. This will ensure you have all the proper files installed. If you are going to run a local dev environment, set the mode to development with the following command: bin/magento deploy:mode:set developer. This will allow you to make changes and to view those changes by just refreshing the page, rather than flushing cache all the time.
All queries are replied correctly by Eric. I am also not sure about "service configurations" you have mentioned here. If this is about third-party extensions/services you can check config.php file for this.
I am an avid webdev hobbyist and freelance, up until now I simply edit the website live (put a maintenance message up while its being made), now all my projects up until now have also been very small.
eg I make a site, show em, take money and go, I've never had to work on a site after it's gone live.
Now my new project is pretty big and I know I will have to edit it after its gone live and maybe have a small team of devs (atm just me)
So how do people professionally handle this? I know I will need a prefix-amp app cos i run an apache server, I've also heared that people use github for versioning, but I'm not really sure because apparently its not svn?
Thanks
ps. I have a windows 7 pc, so no mac apps please
up until now i simply edit the website live
Terrible in my book ;)
so how do people professionally handle this?
First you need to setup a development server (it would be best to keep it as close as possible to the expected live environments). On this server you would install all the software you need.
You may also want to setup a staging server.
i know i will need a prefix-amp app
I hope you are not talking about those one click installers. If you would do it professionally you should install everything yourself that way you can set it up the way you need it.
ive also heared that people use github for versioning, but im not really sure because apparently its not svn?
GitHub is just a website. What you are looking for is git or svn for versioning. You could also setup a git or svn server locally instead of using services like GitHub. Basically what versioning is is that when somebody makes a change to the code he/she would need commit the changes. This way it is easy to keep track of changes in the codebase (like what was changed, when was it changed and by whom).
Local XAMP-stack (LAMP, or WAMP) for development
intranet-system for test and maybe staging
Of course the live system
Versioncontrol, I prefer git. Of course you can use SVN too, but... lets say: It's SVN.
Make changes local, test this changes local
everythings fine: Push it into the "master" vcs-repository
New version ready (or it's "sunday-night-release-time")? Push all that stuff on test/stage
Everythings fine there too: Push it into the live system
Thats very shortened of course, but it should give you an idea.
The tool where you manage your software version is not that important. Use Git, or SVN or whatever, the one you like most. But use _one_.
Equally important is that you run the "page" on two sites, a test and a live system, strictly apart. Both systems have to be very close in their layout, all changes must first be done in the test system, be verified and then done in the same manner in the live system. Do not allow changes only to be made to the live system ('cause it's just a small change'). No exceptions.
Then think about deployment: how will you transfer changed files to the target system ? You need routines for this, that run once started and don't forget a step in between.
Firstly you need some kind of versioning system: either SVN or Git. GitHub is simply an online service that provides managed Git repositories. Secondly you need a development server.
If it were just you doing development, you could host both of these on your local desktop PC, but since other developers are going to be joining, you need a remote server. If you don't want to be running a server out of your home, the best option is a VPS (virtual private server) on which you can install Git, Apache, etc. and anything else you need.
As for development software, take your pick- there are loads of options. A common choice is the NetBeans IDE and TortoiseGit combo. You use NetBeans to develop your code on, automatically uploading to your development server, then you TortoiseGit to commit and sync changes.
Only when you're ready to go live do you copy the code from the dev server to the production server.
I am in the midst of setting up the development environment (PHP/MySQL) for my start-up. We use three sets of servers:
LIVE - the servers which provide the actual application
TEST - providing a testing version before it is actually released
DEV - the development servers
The development servers run SVN with each developer checking out their local copy. At the end of each day completed fixes are checked in and then we use Hudson to automate our build process and then transfer it over to TEST. We then check the application still functions correctly using a tester and then if everything is fine move it to LIVE. I am happy with this process but I do have two questions:
How would you recommend we do local testing - as each developer adds new pages or changes functionality I want them to be able to test what they are doing. Would you just setup local Apache and a local database and have them test locally on their own machine?
How would you recommend dealing with data layer changes?
Is there anything else you would recommend doing to really make our development process as easy and efficient as possible?
Thanks in advance
+1 to each developer running her own setup, complete with Apache and database.
Keep the database schema under version control.
Possibly you could keep (maybe in a separate repository) a small but representative set of data, in a test database. Each morning you check out the latest copy of this test database, and start hacking. When you change schemas, update your test data repository accordingly.
Anyone doing development SHOULD have their own local environment. I use Mac so I run MAMP so that I can have my own LAMP environment local and independent of any other environment. This will also allow me to know that nobody else is changing / working on the same components I am and removes any possible confusion. If you are a windows user, there are also easy to install local versions of the LAMP stack such as XAMP, etc. If you are running Linux as your desktop, you will most likely already know how to install LAMP for the flavor of Linux you are running.
The database schema version is a great idea. It is what we use as well. In addition to the schema under version control, we add a schema version table to the schema and keep it updated so we can quickly tell what version is in production/qa/dev when we need to compare.
As for the data layer changes there are two things I would recommend.
Always create your migration path, forward and backward. This means that when you have the schema you want to put on production to upgrade an existing schema, you should always make it part of the release. A clear concise process for ALTERing the tables. By the same token, you need to have a working and tested ROLLBACK version as well in case something goes wrong.
What I have found helpful is using a backup of production to load on my local (or QA/DEV) so that I have the most up-to-date data / schema to play with without affecting production. If you are not performing regular backups of production, maybe now is a good time to implement a policy. Then you will kill two birds with one stone. You will have backups for any outage and a useful live schema with data you can load to test with on another machine. This will also lend itself to raising any possible issues with schema changes as the data will be matching production. So if it works locally (and on DEV/QA), it reduces the risk of something going wrong in production.
I've read a number of topics in the same sort of ballpark as this one, but in all honesty I'm still not exactly sure on the best approach (as a starting point). I am a solo developer in a small office and I have around 30 websites which are hosted on a linux VPS. I want to start using using version control (probably SVN) and also set up a staging server. At the moment, I do development either locally on my machine before using FTP to upload to the live server, or ocassionally for small changes I edit the remote files directly, which is not an ideal approach.
I'm looking for some guidance on how to improve my development environment. I imagine I should be installing SVN on the web server, which would then allow me to check out versions to my local machine (which would also require SVN i think). Also, if I want to set up a staging server, should I just set up subdomains for each of the live websites, then use these subdomains for showing clients changes to the site before making them live?
Hope this makes sense!
This is what we do at work:
We have a staging server running Apache and a Subversion server. We have a post commit hook that updates a working copy in the htdocs directory, that way, when a developer commits something it automatically gets updated on the staging server, so everyone can see the latest code.
On the client's production servers (the ones we can control) we have the Subversion client installed and the website is a working copy. When we need to update the live site we login to a shell and run svn up. If you do something like this, make sure to limit access to the .svn directories, either with .htaccess files or from the main Apache config.
We have a custom app that manages the projects, but that is only because we're lazy and don't want to setup each project by hand, the app creates the necessary directories and working copies. You could write a quick script to do this.
We never, ever, edit files via FTP on the live site. All in all we have been using this setup for almost 2 years and aside from the occasional conflict on the staging server, we never have had any problems.
You can actually install the SVN server on your local machine, which I would recommend in lieu of installing it on the web server (assuming you make backups). The easiest thing to do, since it’s only you using it, would be to use the file:// protocol, but using svnserve is a little more robust, and the preferred method if you want to take the time to do it.
#Michael, I disagree - I would say it's better to install on the linux vps, especially if you are already paying for the hosting service. I find it very helpful to be able to browse and download stuff from my svn repo wherever I am, from whatever computer I'm on.
#nicky, I started with svn (and version control) several years ago and I took baby steps which made it easier to tackle.
If I had to do it over again, I'd read the svn book to start with. The book is very well laid out and didn't take more than 1-2 days to plow thru.
While you're reading, install svn on your linux vps with an apache front end.
Once you have that up, pick one of your websites and import it into svn. This is how I structure my svn repo. For example, say my repo is hosted at http://mysvn.mydomain.com/svn/:
mywebsite1
- trunk
- tags
- branches
mywebsite2
- trunk
- tags
- branches
Don't worry about creating the perfect structure. It's pretty easy to re-organize especially when you're starting out. After you import a few projects into svn, you'll start to get a feel for which projects should have their own "trunk/tags/branches" dir structure and which can be combined.
For creating test environments, I do exactly what you describe. I use build scripts to checkout from svn and download files into dirs that are mapped to subdomains like "test.clientsite.com" (I work primarily in java and use ant and maven, but I think you can use whatever scripting language you're familiar with).
Once you get used to version control, you'll never go back, good luck!
We have a PHP project that we would like to version control. Right now there are three of us working on a development version of the project which resides in an external folder to which all of our Eclipse IDEs are linked, and thus no version control.
What is the right way and the best way to version control this?
We have an SVN set up, but we just need to find a good way to check in and out that allows us to test on the development server. Any ideas?
We were in a similar situation, and here's what we ended up doing:
Set up two branches -- the release and development branch.
For the development branch, include a post-commit hook that deploys the repository to the dev server, so you can test.
Once you're ready, you merge your changes into the release branch. I'd also suggest putting in a post-commit hook for deployment there.
You can also set up individual development servers for each of the team members, on their workstations. I find that it speeds things up a bit, although you do have some more setup time.
We had to use a single development server, because we were using a proprietary CMS and ran into licensing issues. So our post-commit hook was a simple FTP bot.
Here is what we do:
Each dev has a VM that is configured like our integration server
The integration server has space for Trunk, each user, and a few slots for branches
The production server
Hooks are in Subversion to e-mail when commits are made
At the beginning of a project, the user makes a branch and checks it out on their personal VM as well as grabs a clean copy of the database. They do their work, committing as they go.
Once they have finished everything in their own personal space they log into the integration server and check out their branch, run their tests, etc. When all that passes their branch is merged into Trunk.
Trunk is rebuilt, the full suite of tests are run, and if all is good it gets the big ol' stamp of approval, tagged in SVN, and promoted to Production at the end of the night.
If at any point a commit by someone else is made, we get an e-mail and can merge those changes into our individual branches.
Beanstalk has built-in post-commit hooks for deploying to development, staging, and production servers.
One way to use subversion for PHP development is too setup a repository for one or all three developers, and use this repository, more as a syncing tool, than true version control.
You could,
Make a repo
Add your entire PHP document structure of your project
Checkout a copy of this repo into the correct spot on your dev server
Use an svn hook, that activates on commit
This hook, will automatically update the contents of the dev sever, whenever anybody on the team checks in any code.
Hook resides in:
svn_dir/repo_name/hooks/post-commit
And could look like:
/usr/bin/svn up /path_to/webroot --username svn_user --password svn_pass
That will update your working copy on the dev server to the latest check in.
What about something distributed? You can start for example with Mercurial, try different workflows, and see which one fits you the best.
Each of you could run it locally, or on your own dev server (or even the same one with a different port...).
One possible way (there are probably better ways):
Each of you should have your own checked out version of the project.
Have a local copy of the server on your computer and test it there throughout the day. Then at the end of each day (or whenever), you merge together whatever you are ready to test, and you check it out onto the dev server and test it.
Another tool you can use for the builds is TeamCity which is free for 20 build configurations (enough for most small companies/projects.) This way you can run your tests as well as schedule builds.