Deploy thousands of files to FTP during web app deployment process - php

Recently I investigated the continuous integration development process, so I installed a TeamCity server, set up the build and tried to make it automatically deploy the build artifacts (my web app) to the web hosting server through FTP. However, I failed on the last step (deployment) because of thousands php files deploying for a very long time. So, I'm wondering if there is a way to do it more quickly, perhaps using zip-archives or something else.
So, my question is, if there is a common way to solve such problem?

Use git for deploy. It get only changes
Example using hooks: gist.github.com/noelboss/3fe13927025b89757f8fb12e9066f2fa – Łukasz Gawrylu

Related

Updating Laravel application from git on multiple servers

I am working on development of a web app (for learning) in Laravel and I`m using Bitbucket for source control. It will be deployed on couple servers (20 or so, perhaps more over time), and I would like to be able to update all of them as the app changes over time.
The problem is that I will not have SSH access to most of those servers so I wont be able to use a simple "git pull" (a server we test on does not even have git installed so shell_exec is not an option also).
My plan was to make a script that will download latest zip from Bitbucket server, unpack it overwriting the old code, and then running a Laravel script to run migrate (for eventual database changes).
Is there maybe a more sensible way of doing this?
What are you looking for is CI/CD, i.e. Continues Integration/ Continues Delivery. There are so many ways to automatically deploy or pull a code over server. You can use following methods
Automating Deployment to EC2 Instance With Git
Using Bitbucket for Automated Deployments
CI\CD workflow with BitBucket Cloud, Bamboo, AWS CodeDeploy
Bitbucket - Manage Web Hooks
Apart from this you can find so many articles on this, but if you wants to automate the process at laravel level then use Laravel Envoy

Using a Network Attached Storage (NAS) for developing

I was enjoining about getting a Network Attached Storage (NAS) so that I can work on dev sites from both my desktop and my laptop without duplicating files and always having the most current file (just in case I forget to save). My question is if I put sites on there that uses php, would I be able to run the sites off of the NAS as I would with MAMP / WAMP? Or would I still need something else to make that work?
The point of a NAS is to share files over a network. This is usually done via Windows File & Print Sharing (aka Samba aka SMB) which is supported on most platforms.
Some NAS devices might allow you to run a web server (particularly if you can install custom firmware), but it is a poor choice of platform to run anything remotely complex in terms of web server stacks.
You can certainly store your development files on a NAS, and then access them from webservers running in both your development environments.
… but that said, I'd look at using version control software (Git would be my preference), keeping your repository on the NAS and getting into the habit of saving, committing and pushing. It makes things more manageable in the long run. (You could also use a service like Bitbucket or Github and dispense with the local NAS entirely).
You could also go a step further and run a server with CI software on it that monitors your repository and automatically pulls updates from it, runs your automated tests, and then updates a local test server.
I am assuming you use windows(it's easier to do it in mac i think) with wamp what you can do is mount a network drive to w:\ let's say. Then create a virtual host that points to a folder in W:\ drive.
With mac all you need to do is mount the remote folder share to your mamp directory and all should work as you want.
Though personally I think this is a terrible idea and would much rather suggest you to use a VCS(version control system) to share code between multiple places. Lots of them are designed with this problem in mind. And it provides nice history about your code at the same time. If you want to do some research look at GIT(the most popular, currently) bitbucket has free private repositories. you can look into what a VCS can do here https://en.wikipedia.org/wiki/Version_control

Auto deployment of PHP applications

My team currently has a development web/database server and a live deployment web server and a live database server. We use SVN with the repository stored on the development server but the problem is our deployment process.
Currently when we need to deploy an update to the live application we simply use SFTP to transfer from the repository to the live web server and then amend the database on the live server to reflect the development database. This is a really slow process as we also minify all javascript and CSS files.
I have used Capistrano for Ruby and Cruise Control for java but I have never used anything for PHP. I'd rather not have to build our own if something already existed. Does anyone know of anything?
Check out my recent bounty question:
Setting up a deployment / build / CI cycle for PHP projects
I got a number of very good product recommendations. There's also a Cruise Control connector for PHP, phpUnderControl.
I did a great job with auto deployment using CruiseControl and phpUnderControl
http://topecoders.blogspot.com/2010/04/how-to-install-cruisecontrol.html
They work pretty cool with all tests and code analysis being done automatically. It also deploys projects on every successful build.

Source Control Advice

I've inherited a significantly sized internal app at the company I work for. It has two parts. One runs on a LAMP server, the other runs on a WAMP server. There is no source control to be seen.
Currently, developers will log into each server via FTP and just directly edit the PHP files that are there. Or log into phpMyAdmin and make changes to DB schema where necessary.
What is the best way to implement some source control into this situation? I'm not very knowledgeable on how to best setup CVS ans SVN. Would all developers need their own "test system"? Meaning, would I have to setup a LAMP+WAMP enviornment for each developer? Or can they would off a central dev server? What is the best way to deploy changes that have been made?
Developers all use Windows. We also use Zend Studio 5.5 for development.
Thank you in advance for any advice.
It's really not a lot of effort for a developer to have a personal LAMP/WAMP running locally or in a VM. A local dev server is worth having for testing non-live changes in a closely matched environment but it doesn't really scale if all developers are making their changes on that at once.
I'd have all developers committing to trunk in Subversion and then have a stable branch which is automatically (using a post-commit hook) deployed onto the server. Only certain people should have write access to the stable branch to ensure a junior developer doesn't accidentally screw the live server.
That's a horrible way to work on websites, especially with a team. The whole setup should be changed.
The source code should live somewhere else, and then get deployed to the website when it's ready. You could have a script that does the deployment and whatnot (copy the files over ssh, plus any other setup if needed, etc). The point is: no one should ever edit files directly on the production server.

Development environment - VCS from development to staging server to production

I've read a number of topics in the same sort of ballpark as this one, but in all honesty I'm still not exactly sure on the best approach (as a starting point). I am a solo developer in a small office and I have around 30 websites which are hosted on a linux VPS. I want to start using using version control (probably SVN) and also set up a staging server. At the moment, I do development either locally on my machine before using FTP to upload to the live server, or ocassionally for small changes I edit the remote files directly, which is not an ideal approach.
I'm looking for some guidance on how to improve my development environment. I imagine I should be installing SVN on the web server, which would then allow me to check out versions to my local machine (which would also require SVN i think). Also, if I want to set up a staging server, should I just set up subdomains for each of the live websites, then use these subdomains for showing clients changes to the site before making them live?
Hope this makes sense!
This is what we do at work:
We have a staging server running Apache and a Subversion server. We have a post commit hook that updates a working copy in the htdocs directory, that way, when a developer commits something it automatically gets updated on the staging server, so everyone can see the latest code.
On the client's production servers (the ones we can control) we have the Subversion client installed and the website is a working copy. When we need to update the live site we login to a shell and run svn up. If you do something like this, make sure to limit access to the .svn directories, either with .htaccess files or from the main Apache config.
We have a custom app that manages the projects, but that is only because we're lazy and don't want to setup each project by hand, the app creates the necessary directories and working copies. You could write a quick script to do this.
We never, ever, edit files via FTP on the live site. All in all we have been using this setup for almost 2 years and aside from the occasional conflict on the staging server, we never have had any problems.
You can actually install the SVN server on your local machine, which I would recommend in lieu of installing it on the web server (assuming you make backups). The easiest thing to do, since it’s only you using it, would be to use the file:// protocol, but using svnserve is a little more robust, and the preferred method if you want to take the time to do it.
#Michael, I disagree - I would say it's better to install on the linux vps, especially if you are already paying for the hosting service. I find it very helpful to be able to browse and download stuff from my svn repo wherever I am, from whatever computer I'm on.
#nicky, I started with svn (and version control) several years ago and I took baby steps which made it easier to tackle.
If I had to do it over again, I'd read the svn book to start with. The book is very well laid out and didn't take more than 1-2 days to plow thru.
While you're reading, install svn on your linux vps with an apache front end.
Once you have that up, pick one of your websites and import it into svn. This is how I structure my svn repo. For example, say my repo is hosted at http://mysvn.mydomain.com/svn/:
mywebsite1
- trunk
- tags
- branches
mywebsite2
- trunk
- tags
- branches
Don't worry about creating the perfect structure. It's pretty easy to re-organize especially when you're starting out. After you import a few projects into svn, you'll start to get a feel for which projects should have their own "trunk/tags/branches" dir structure and which can be combined.
For creating test environments, I do exactly what you describe. I use build scripts to checkout from svn and download files into dirs that are mapped to subdomains like "test.clientsite.com" (I work primarily in java and use ant and maven, but I think you can use whatever scripting language you're familiar with).
Once you get used to version control, you'll never go back, good luck!

Categories