Related
After many hours of reading documentation and messing around with Amazon Web Services. I am unable to figure out how to host a PHP page.
Currently I am using the S3 service for a basic website, but I know that this service does not support dynamic pages. I was able to use the Elastic Beanstalk to make the Sample Application running PHP. But i have really no idea how to use it. I read up on some other services but they don't seem to do what I want or they are just way to confusing.
So what I want to be able to do is host a website with amazon that has dynamic PHP pages. Is this possible and what services do you use?
For a PHP app, you really have two choices in AWS.
Elastic Beanstalk is a service that takes your code, and manages the runtime environment for you - once you've set it up, it's very easy to deploy, and you don't have to worry about managing servers - AWS does pretty much everything for you. You have less control over the environment, but if your server will run in EB then this is a pretty easy path.
EC2 is closer to conventional hosting. You need to decide how your servers are configured & deployed (what packages get installed, what version of linux, instance size, etc), your system architecture (do you have separate instances for cache or database, whether or not you need a load balancer, etc) and how you manage availability and scalability (multiple zones, multiple data centers, auto scaling rules, etc).
Now, those are all things that you can use - you dont have to. If you're just trying to learn about php in AWS, you can start with a single EC2 instance, deploy your code, and get it running in a few minutes without worring about any of the stuff in the previous paragraph. Just create an instance from the Amazon Linux AMI, install apache & php, open the appropriate ports in the firewall (AKA the EC2 security group), deploy your code, and you should be up & running.
Your Php must run on EC2 machines.
Amazon provides great tools to make life easy (Beanstalk, ECS for Docker ...) but at the end, you own EC2 machines.
There is no a such thing where you can put your Php code without worrying about anything else ;-(
If you are having problems hosting PHP sites on AWS then you can go with a service provider like Cloudways. They provide managed AWS servers with one click installs of PHP frameworks and CMS.
I have started making a website and was hosting on Hostgator but I am going to move it to Amazon web services before launch. There is a small problem that I previously just uploaded my files to the relevant location to Hostgator and it has all just worked. I have no experience in setting up from scratch a production worthy server setup and I need to know how. I did setup the basic lamp stack on the EC2 instance, however, I keep reading that when the EC2 instance does down it will take all the data with it and I can not have that happen. I have also read then when it dies it wont do anything and you have to start up the apache server again it is not automatic. I need it to be reliable and have the data independent so it will not crash, burn and die if the server goes. I have worked out that I will need S3 for static things such as my PDF's and images as well as using the RDS for my MYSQL database. My domain name is registered elsewhere so I believe I need to use route 53 as well.I want to use AWS for a few reasons reasons, firstly as it can scale which is really important but not sure if this is built in or it requires customization. I have been told that it is very secure the EC2 and the last reason is that I can debug my php code. The debug reason is that I have an error that only appears on the Hostgator server not my local lamp stack and I can't debug it there so I should be able to when I move to EC2.
I have done a lot of looking around online and I can't find anything comprehensive about what to setup. I have been reading (some of you may think otherwise). However, I am so overwhelmed by the amount of information there is as it is either far to complicated discussing some theory that I do not care about or to easy and does not discuss how to use anything other then a generic install of a LAMP stack on the EC2 with out using the other services.
I have seen http://bitnami.com/stack/lamp/cloud/amazon but do not think this is what I want as again the EC2 has a mysql database and I am not using the RDS
If someone can point me in the direction of a comprehensive guide to setting up a slid LAMP stack on AWS (mabey even a book has been written) that would be great as I found the amazon docs did not go into much detail and told me how to do things but not why I should do them and what purpose they had.
Thanks
I'll start with answering your q's first, and as you are a newbie I would suggest don't pressurize to learn all of AWS, you can keep migrating slowly and keep discovering the magic of cloud.
Q.
when the EC2 instance does down it will take all the data with it and
I can not have that happen. I have also read then when it dies it wont
do anything and you have to start up the apache server again it is not
automatic?
A. When an EC2 instance goes down (down could mean shutdown manual by you or Down means AWS network is down, or instances are having some other issues) only the data on "ephemeral data" or you can say data on RAM or sessions will get lost, whatever is on disk will remain on disk, And the instance will be available as soon as problem is resolved.
Apache will start itself when an instance restarts, and remains up until you manually shut it down or some other issue.
Q. I will need S3 for static things such as my PDF's and images as
well as using the RDS for my MYSQL database?
A. Its a good practice to keep static stuff on s3, but not a necessary thing to do, you can set up a ftp or manage your static content like you were used to, like keeping it on a folder of your website.
You don't necessarily need RDS to have a mysql database, I have a process running on aws with around 40 mil transactions a day, and I do it on a normal mysql at an ec2 instance.
however having RDS gets rid from the daily backup and index maintenance hustles.
Q. My domain name is registered elsewhere so I believe I need to use
route 53 as well ?
A. Again not a necessary thing, you can just go to your domain manager and change the A-name or C-name records (with static public ip of ec2) and give a static public ip to your ec2 instance or Elastic load balancer and you'll be up and running in no time.
Q. I want to use AWS for a few reasons reasons, firstly as it can
scale which is really important but not sure if this is built in or it
requires customization.
A. It can scale really well, but depends how do you want it to scale, and its highly customizable.
there are 2 kinds of scaling
vertical - you change your instance type from one type to another to get better disk / cpu or RAM or better network performance, but this will need you to stop your ec2 instance and change its type, that means there will be downtime of around 10 minutes while you do so.
horizontal - you can put your website (ec2 based) behind a load balancer (ELB - elastic load balancer) and add/remove more instances to/from it as and when you deemed suitable, or you can also have an auto scaling policy to help you do it automatically depending up on the load at your web server.
Security? - you can be very well assured its very well secure, and so much secure that I can bet my life on a secure ec2 instance, i can swear by linux thor that it works and it works like a charm.
Debugging? - I suggest you do debugging by classic means, make logs of errors and all, just treat ec2 like a normal machine and learn slowly the tricks of trade.
Now lets setup a basic solid LAMP stack for ourselves, I am assuming that you have a ready ubuntu instance, and you can ssh to it, in case you haven't been able to make one - see this.
basically.
1. create security groups - This is your firewall, makes sure which ports are open, and also makes sure which ec2 instances can talk amongst themselves.
2. Create an ec2 instance - make any ubuntu instance.
And access your instance using ssh - ssh is basically secure terminal connection to your ec2 machine which is secured by a key file (pem file) and whoever has it can access your machine's data, so keep it very very secure, and you can't afford to lose it.
3. install LAMP using - Tasksel utility
4. setup a public ip for yourself ( costs a dollar per month) - you can use this ip to redirect your www.example.com traffic using domain manager of your DNS provider - godaddy or someone alike i suppose.
I think this will be it to make you start with AWS.
Just to be safe that you have a copy of your data make an AMI of your ec2 instance with all the data on it. AMI is the image from which you can make a similar or better instance in 10 minutes flat (or even lesser).
You wil pay for - instance type you chose, public IP, traffic if its beyond a level (usually very very cheap), and disk usage (8 gb is the default disk), and AMI volume.
Have fun with AWS.
To retain data between during the down time, make sure you use EBS storage. Its default now a days. In the past, before EBS, instance storage was default and you would lose data once server is down, but with the EBS storage, data is retained during the shutdown.
You can go one of the follow two routes depending upon your needs.
1. Use AWS ElasticBeanStalk (http://aws.amazon.com/elasticbeanstalk/) if you do not need to install anything additional Its super easy and its similar to Google Apps and you can deploy your app quickly. You do not get server, but a server to deploy your app. You have to use RDS for database and S3 for storage. You can not store locally on the server where you are running.
Use EC2 server with static IP address. You can get pre-configured LAMP stacks from market place. I use bitnami cloud stacks for AWS that comes pre-configured with LAMP and many other apps. Just use their free account to create micro instance for your PHP and select a server and you are good to go. http://bitnami.com/cloud
You do not need to use Route 53 unless you need to manage DNS programatically. You can just point your server to EC2 server by adding entry in your DNS (godaddy or whoever is your domain name provider).
Bitnami service also allow scheduled backups, but if you are not storing anything locally, you do not need frequent backups.
Make sure you use Multi-AZ option in RDS which is more reliable. When you provision a Multi-AZ DB Instance, Amazon RDS automatically creates a primary DB Instance and synchronously replicates the data to a standby instance in a different Availability Zone (AZ). Also, Amazon RDS automatically patches the database software and backs up your database, storing the backups for a user-defined retention period and enabling point-in-time recovery, up to last 5 minutes.
I hope this helps.
You should be using dynamo DB (http://aws.amazon.com/dynamodb/pricing/) in with LAMP without Mysql for storage. Having a Samebox database can almost never give you reliability. So you will not loose your data what ever your Application box goes through. You can even read our application config from dynamo DB.
http://aws.amazon.com/documentation/dynamodb/
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SettingUpTestingSDKPHP.html
Do I need to use EC2 with DynamoDB?
You wont loose data when server is down. Just make sure your select EBS volume, and not Instance.
You can get ready-made server from AWS market place. I used the following for my projects, but there are many other pre-configured servers available.
https://aws.amazon.com/marketplace/pp/B007IN7GJA/ref=srh_res_product_title?ie=UTF8&sr=0-2&qid=1382655655469
This with RDS server is what you need. We use this all the time for production servers and never had any issues.
Here are two guides that look good to me:
http://shout.setfive.com/2013/04/05/amazon-aws-ec2-lamp-quickstart-guide-5-steps-in-10-minutes/
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-LAMP.html
If learning the Linux command line isn't your thing, you should consider going "up the stack" to a PaaS (Platform As A Service). They are things like Heroku, Google App Engine, and ElasticBeanStalk.
The trade-off between Infrastructure as a Service (IaaS like EC2) and a Platform as a Service (PaaS like Heroku):
- PasS is quicker to get started, less to learn. IaaS requires you to know the entire stack from the start (or hire/rent a sysadmin).
- PasS usually gets more expensive as you get bigger compared to IaaS (but it depends).
- PaaS has less control (you can't choose the language version, so you can't upgrade to get around a specific bug.)
- IaaS can literally do anything (it's just a Linux box)
- IaaS allows for more tuning (upgrade libraries to get features, switch to different instance type to trade off RAM for CPU, run HipHop for speed, add caching layers, etc)
You have a few choices:
Use only EC2. Install Apache+MySQL and your dynamic website on EC2. This will be very similar to setting it up on Hostgator except you are running a full server.
Use EC2 for "compute" (that is, the dynamic part of the site) and S3 for storage. This doesn't differ much from #1 above, except that you are using S3 for static file storage - which is great if you are expecting to host a lot of static content (multimedia, etc)
Set up your website using Amazon Elastic Beanstalk (which now supports PHP). However, if you go this route, you will need to host your database somewhere - which will likely be RDS.
I recommend going with #1. There is nothing wrong with that - yes, if EC2 goes down, it will take down your site with it, but to alleviate that, you can run two servers in two different regions (one in US East and one in US West) - I don't think two EC2 regions have ever gone down at the same time.
UPDATE: If you are concerned about backup/restore and making sure your data is safe, I recommend the following (I do this with a site in production on EC2):
Put your website code into Git/SVN source control; and pull from there
Backup your MySQL database to Amazon S3 regularly (at least once a day) using mysqldump
I think you have some misconceptions.
If EC2 as a whole goes down (which is rare) then you do NOT lose your data. The site would simply be offline until Amazon restored services.
If your particular instance goes down due to a hardware issue, then you might lose data. This is no different than if your own server went belly up. The right answer is to simply make normal backups of your database and store it in S3 or some other location. Generally you will want to create and attach a second EBS volume to your DB server which has the DB files on it as well.
If you Terminate your instance then, yes you will lose everything on that. However Amazon has the ability to make terminating instances difficult so you don't do it accidentally.
Stopping your instance is like turning the computer off. The difference being that you can remotely turn it back on when you want. You can only stop EBS backed instances - which means that your data is safe while it is offline.
I would highly suggest that if you are uncomfortable with setting up and maintaining your own server that you should investigate fully managed hosting instead. EC2 is awesome, we've been on it for 2 years. However, we have a strong tech team that understands what it takes to run and manage servers.
I am attempting to figure out a good way to push out a new commit to a group of EC2 server instances behind a ELB (load balancer). Each instance is running Nginx and PHP-FPM
I would like to perform the following workflow, but I am unsure of a good way to push out a new version to all instances behind the load balancer.
Dev is done on a local machine
Once changes are ready, I perform a "git push origin master" to push
the changes to BitBucket (where I host all my git repos)
After being pushed to bitbucket, I would like to have the new version
pushed out to all EC2 instances simultaneously.
I would like to do this without having to SSH in to each instance
(obviously).
Is there a way to configure the remote servers to accept a remote push? Is there a better way to do this?
Yes, I do this all of the time (with the same application stack, actually).
Use a base AMI from a trusted source, such as the default "Amazon Linux" ones, or roll your own.
As part of the launch configuration, use the "user data" field to bootstrap a provisioning process on boot. This can be as simple as a shell script that runs yum install nginx php-fpm -y and copies files down from a S3 bucket or do a pull from your repo. The Amazon-authored AMI's also include support for cloud-init scripts if you need a bit more flexibility. If you need even greater power, you can use a change management and orchestration tool like Puppet, Chef, or Salt (my personal favorite).
As far as updating code on existing instances: there are two schools of thought:
Make full use of the cloud and just spin up an entirely new fleet of instances that grab the new code at boot. Then you flip the load balancer to point at the new fleet. It's instantaneous and gives you a really quick way to revert to the old fleet if something goes wrong. Hours (or days) later, you then spin down the old instances.
You can use a tool like Fabric or Capistrano to do a parallel "push" deployment to all the instances at once. This is generally just re-executing the same script that the servers ran at boot. Salt and Puppet's MCollective also provide similar functionality that mesh with their basic "pull" provisioning.
Option one
Push it to one machine.
Have a git hook created on it http://git-scm.com/book/en/Customizing-Git-Git-Hooks.
Make hook run pull on other machines.
Only problem , you'll have to maintain list of machines to run update on.
Another option
Have cron job to pull from your bitbucket account. on a regular base.
The tool for this job is Capistrano.
I use an awesome gem called capistrano-ec2group in order to map capistrano roles with EC2 security groups.
This means that you only need to apply an EC2 security group (eg. app-web or app-db) to your instances in order for capistrano to know what to deploy to them.
This means you do not have to maintain a list of server IPs in your app.
The change to your workflow would be that instead of focussing on automating the deploy on pushing to bitbucket, you would push and then execute
cap deploy
If you really don't want to do to steps, make an alias :D
alias shipit=git push origin master && cap deploy
This solution builds on E_p's idea. E_p says the problem is you'd need to maintain a server list somewhere in order to tell each server to pull the new update. If it was me, I'd just use tags in ec2 to help identify a group of servers (like "Role=WebServer" for example). That way you can just use the ec2 command line interface to list the instances and run the pull command on each of them.
for i in \
`ec2din --filter "tag-value=WebServer" --region us-east-1 \
| grep "running" \
| cut -f17`\
; do ssh $i "cd /var/www/html && git pull origin"; done
Note: I've tested the code that fetches the ip addresses of all tagged instances and connects to them via ssh, but not the specific git pull command.
You need the amazon cli tools installed wherever you want this to run, as well as the ssh keys installed for the servers you're trying to update. Not sure what bitbucket's capabilities are but I'm guessing this code won't be able to run there. You'll either need to do as E_p suggests and push your updates to a separate management instance, and include this code in your post-commit hook, OR if you want to save the headache you could do as I've done and just install the CLI tools on your local machine and run it manually when you want to deploy the updates.
Credit to AdamK for his response to another question which made it easy to extract the ip address from the ec2din output and iterate over the results: How can I kill all my EC2 instances from the command line?
EC2 CLI Tools Reference: http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/Welcome.html
Your best bet might be to actually use AMI's for deployments.
Personally, I typically have a staging instance where I can pull any repo changes into. Once I have confirmed it is operating the way I want, I create an AMI from that instance.
For deployment, I use an autoscaling group behind the load balancer (doesn't need to be dynamically scaling or anything). In a simple set up where you have a fixed number of servers in the autoscale group (for example 10 instances). I would change the AMI associated with the autoscale group to the new AMI, then start terminating a few instances at a time in the autoscale group. So, say I have 10 instances and I terminate two to take it down to 8 instances. The autoscale group is configured to have a minimum of 10 instances, so it will automatically start up two new instances with the new AMI. You can then keep removing instances at whatever rate makes sense for your level of load, so as to not impact the performance of your fleet.
You can obviously do this manually, even without an autoscale group by directly adding/removing instances from the ELB as well.
If you are looking to make this fully automated (i.e. continuous deployment), then you might want to look at using a build system such as Jenkins, which would allow for a commit to kick off a build and then run the necessary AWS commands to make AMI's and deploy them.
I am looking for a solution to the same problem. I came across this post and thought it was an interesting approach
https://gist.github.com/Nilpo/8ed5e44be00d6cf21f22#pc
Go to "Pushing Changes To Multiple Servers"
Basically the idea is to create another remote call it "production" or whatever you want, and then add multiple urls (The ip's of all of the servers) to that remote. This can be done by editing .git/config
Then you can run git push production <branch> and it should push out to all of the urls listed under "production"
One requirement to this approach is that the repo on the servers need to be bare repos and you will need to have a post-receive hook to update the working tree.
Here is an example of how to do that: Setting up post-receive hook for bare repo
How do i get the changes from live to my repo? The files running on the heroku app have changed and now if i push these will be overwritten.
I have my php code running on heroku and storing 'database' things in local files.
{
"id":1,
"date":"12/1/2012",
"topImg":"/img/dates/1.jpg"
.....
So these things are stored in a json object then just saved over.
Don't do this!
Local files are your enemy, because Heroku is a cloud application host that runs applications on multiple anonymous load-balanced nodes.
Perhaps you're running a single dyno right now for development purposes, but if you ever want to make your site go live you'll need at least two dynos (because Heroku free tier service is qualitatively different from their non-free tier service, particularly in that they will spin down a free dyno if it is not being used but they will never do that to a non-free dyno). When you have multiple dynos, using local files for anything other than caching will be totally unmanageable.
Even if you somehow stay with one dyno forever, Heroku dynos are not guaranteed to maintain their local storage -- if for instance there is a hardware failure on the machine your dyno is served from, Heroku will not hesitate to spin down your application, deleting all local storage, and spin it up again with just your application code loaded, because it does not expect your application to be using local storage for anything.
There is no one supported method for getting files off of a dyno, because, again, it's never a good idea to store local files on a dyno. However, if you really, really need to do this, you can use heroku run and run one-off commands to, for instance, open up a shell and upload the files somewhere. Again: do not do this for anything serious, because once you have multiple dynos it'll be nearly impossible to manage files on them.
Totaly agree with #Andrew. Prefer to use something as mongoDB database as a service with heroku : https://addons.heroku.com/catalog/mongolab or elasticsearch, if you want to add search function over those documents: https://addons.heroku.com/catalog/searchbox. There are well designed to store json docs and, with those services, you are sure that your data will be persistent no matter your dynos are.
Now, to get back your heroku local files, I would do something like that :
run the heroku bash with heroku run bash
make a scp -pYourPort yourFile(s) userName#yourDestination:/pathToSaveLocaion
logout from your heroku instance
I hope this will help you.
I’ve been working on a cloud based (AWS EC2 ) PHP Web Application, and I’m struggling with one issue when it comes to working with multiple servers (all under an AWS Elastic Load Balancer). On one server, when I upload the latest files, they’re instantly in production across the entire application. But this isn’t true when using multiple servers – you have to upload files to each of them, every time you commit a change. This could work alright if you don’t update anything very often, or if you just have one or two servers. But what if you update the system multiple times in one week, across ten servers?
What I’m looking for is a way to ‘commit’ changes from our dev or testing server and have it ‘pushed’ out to all of our production servers immediately. Ideally the update would be applied to only one server at a time (even though it just takes a second or two per server) so the ELB will not send traffic to it while files are changing so as not to disrupt any production traffic that may be flowing to the ELB .
What is the best way of doing this? One of my thoughts would be to use SVN on the dev server, but that doesn’t really ‘push’ to the servers. I’m looking for a process that takes just a few seconds to commit an update and subsequently begin applying it to servers. Also, for those of you familiar with AWS , what’s the best way to update an AMI with the latest updates so the auto-scaler always launches new instances with the latest version of the software?
There have to be good ways of doing this….can’t really picture sites like Facebook, Google, Apple, Amazon, Twitter, etc. going through and updating hundreds or thousands of servers manually and one by one when they make a change.
Thanks in advance for your help. I’m hoping we can find some solution to this problem….what has to be at least 100 Google searches by both myself and my business partner over the last day have proven unsuccessful for the most part in solving this problem.
Alex
We use scalr.net to manage our web servers and load balancer instances. It worked pretty well until now. we have a server farm for each of our environments (2 production farms, staging, sandbox). We have a pre configured roles for a web servers so it's super easy to open new instances and scale when needed. the web server pull code from github when it boots up.
We haven't completed all the deployment changes we want to do, but basically here's how we deploy new versions into our production environment:
we use phing to update the source code and deployment on each web service. we created a task that execute a git pull and run database changes (dbdeploy phing task). http://www.phing.info/trac/
we wrote a shell script that executes phing and we added it to scalr as a script. Scalr has a nice interface to manage scripts.
#!/bin/sh
cd /var/www
phing -f /var/www/build.xml -Denvironment=production deploy
scalr has an option to execute scripts on all the instances in a specific farm, so each release we just push to the master branch in github and execute the scalr script.
We want to create a github hook that deploys automatically when we push to the master branch. Scalr has api that can execute scripts, so it's possible.
Have a good look at KwateeSDCM. It enables you to deploy files and software on any number of servers and, if needed, to customize server-specific parameters along the way. There's a post about deploying a web application on multiple tomcat instances but it's language agnostic and will work for PHP just as well as long as you have ssh enabled on your AWS servers.