How do i get the changes from live to my repo? The files running on the heroku app have changed and now if i push these will be overwritten.
I have my php code running on heroku and storing 'database' things in local files.
{
"id":1,
"date":"12/1/2012",
"topImg":"/img/dates/1.jpg"
.....
So these things are stored in a json object then just saved over.
Don't do this!
Local files are your enemy, because Heroku is a cloud application host that runs applications on multiple anonymous load-balanced nodes.
Perhaps you're running a single dyno right now for development purposes, but if you ever want to make your site go live you'll need at least two dynos (because Heroku free tier service is qualitatively different from their non-free tier service, particularly in that they will spin down a free dyno if it is not being used but they will never do that to a non-free dyno). When you have multiple dynos, using local files for anything other than caching will be totally unmanageable.
Even if you somehow stay with one dyno forever, Heroku dynos are not guaranteed to maintain their local storage -- if for instance there is a hardware failure on the machine your dyno is served from, Heroku will not hesitate to spin down your application, deleting all local storage, and spin it up again with just your application code loaded, because it does not expect your application to be using local storage for anything.
There is no one supported method for getting files off of a dyno, because, again, it's never a good idea to store local files on a dyno. However, if you really, really need to do this, you can use heroku run and run one-off commands to, for instance, open up a shell and upload the files somewhere. Again: do not do this for anything serious, because once you have multiple dynos it'll be nearly impossible to manage files on them.
Totaly agree with #Andrew. Prefer to use something as mongoDB database as a service with heroku : https://addons.heroku.com/catalog/mongolab or elasticsearch, if you want to add search function over those documents: https://addons.heroku.com/catalog/searchbox. There are well designed to store json docs and, with those services, you are sure that your data will be persistent no matter your dynos are.
Now, to get back your heroku local files, I would do something like that :
run the heroku bash with heroku run bash
make a scp -pYourPort yourFile(s) userName#yourDestination:/pathToSaveLocaion
logout from your heroku instance
I hope this will help you.
Related
I am currently working with a startup that is in a transitional phase.
We have a PHP web application and utilise continuous integration with the standard unit and regression tests (selenium) run over jenkins. We have a development server which hosts newly committed code and a staging server that holds the build ready for deployment to the production server. The way we deploy to the production server is through a rudimentary script that pulls the latest svn copy and overwrites the changes in the htdocs directory. Any SQL changes are applied via the sync feature from MySQL Workbench.
This setup works fine for a very basic environment but we are now in a transition from single server setups to clusters due to high traffic and I have come up against a conundrum.
My main concern is how exactly do we switch deployment from a single
server to a cluster of servers ? Each server will have its own htdocs
and SQL database and under the current setup I would need to execute
the script on every server which sounds like an abhorrent thing to
do. I was looking into puppet which can be used to automate sysadmin tasks but I am not sure whether it is a formidable approach for deploying new builds to a cluster.
My second problem is to do with the database. Now my assumption is the code changes will be applied immediately, but since we will have a db master/slave replication my concern is the database changes will take longer to propagate and thus introduce inconsistencies during deployment. How can the code AND database be synchronised at the same time ?
My third problem is related to automation of database changes. Does anyone know of any way I can automate the process of updating a DB schema without manually having to run the synchronisation ? At the moment I have to manually run the workbench sync tool, whereas I am really looking for a commit and forget approach. I commit it and DB changes are auto synchronised across the dev and QA setups.
I am running a similar scenario, but I am using a Cloud Provider for my production environment, in order that I do not need to care about replication of DB, multi server instances etc. (I am Using pagodabox, but AWS would also work perfectly fine).
I would recommend you to create real migrations for Database Migrations, in order to track those via svn or something else. In that case, you can also provide information, how to roll back. I am using https://github.com/doctrine/migrations, but mainly because I use doctrine as ORM.
If you have a migration tool, you can easily add a command in your deployment script to run those migrations after deployment.
I don't think that the database synchronisation is a big issue during deployment. That might depend on the actual infrastructure youre using. The cloud providers like pagoda or aws take care of it for you.
I'm looking into using Heroku for a PHP app that uses Redis. I've seen the various addons for redis. With Redis To Go, for example, you can use an environment variable $_ENV['REDISTOGO_URL'] in your PHP code, as the URL of the Redis Server.
Most of these add ons have their own pricing schemes which I'd like to avoid. I'm a little confused about how heroku works. Is there a way that I can just install Redis on my own Dynos without the addons?
Like for example, have one worker dyno that acts as a server, and another that acts as a client? If possible, how would I go about:
Installing and running the redis server on a Dyno? Is this just the same as
installing on any other unix box? Can I just ssh to it and install whatever i want?
Have one Dyno connect to
another with an IP/port via TCP? Do the worker dynos have their own
reference-able IP addresses or named URLS that I can use? Can I get them dynamically from PHP somehow?
The php code for a redis client assumes there is a host and port that you can connect to, but have no idea what it would be?
$redis = new Predis\Client(array(
"scheme" => "tcp",
"host" => $host, //how do i get the host/port of a dyno?
"port" => $port));
Running redis on a dyno is an interesting idea. You will probably need to create a redis buildpack so your dynos can download and run redis. As "redis has no dependencies other than a working GCC compiler and libc" this should be technically possible.
However, here are some problems you may run into:
Heroku dynos don't have a static IP address
"dynos don’t have static IP addresses .. you can never access a dyno directly by IP"
Even if you set up and run Redis on a dyno I am not aware of a way to locate that dyno instance and send it redis requests. This means your Redis server will probably have to run on the same dyno as your web server/main application.
This also means that if you attempt to scale your app by creating more web dynos you will also be creating more local redis instances. Data will not be shared between them. This does not strike me as a particularly scalable design, but if your app is small enough to only require one web dyno it may work.
Heroku dynos have an ephemeral filesystem
"no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted"
By default Redis writes its RDB file and AOF log to disk. You'll need to regularly back these up somewhere so you can fetch and restore after your dyno restarts. See the documentation on Redis persistence.
Heroku dynos are rebooted often
"Dynos are cycled at least once per day, or whenever the dyno manifold detects a fault in the underlying hardware"
You'll need to be able to start your redis server each time the dyno starts and restore the data.
Heroku dynos have 512MB of RAM
"Each dyno is allocated 512MB of memory to operate within"
If your Redis server is running on the same dyno as your web server, subtract the RAM needed for your main app. How much Redis memory do you need?
Here are some questions attempting to estimate and track Redis memory use:
Redis: Database Size to Memory Ratio?
Profiling Redis Memory Usage
--
Overall: I suggest reading up on 12 Factor Apps to understand a bit more about heroku's intended application model.
The short version is that dynos are intended to be independent workers that can be easily created and discarded to meet demand, and that dynos access various resources to read or write data and serve your app. A redis instance is an example of a resource. As you can see from the items above, by using a redis add-on you're getting something that's guaranteed to be static, stable, and accessible.
Reading material:
http://www.12factor.net/ - specifically Processes and Services
The Heroku Process Model
Heroku Blog - The Process Model
redis has a client server architecture you can install it on one machine(in your case dyno) and access it from any client.
for more help on libraries you can refer this link
or you can go through this Redis documentaion which is a simple case study of implementing a twitter clone using Redis ad database and PHP
I have an app on Heroku, using PHP and PostgreSQL. Now I would like to create backup of my database regularly, put it on a folder on the server, or record its S3 urls and download it.
I have been doing research on the topic. It seems that the best is to use pgbackups add-on which I already have and can use on local command line, like: heroku pgbackups:url --app=APP_NAME
I want to automate the process, lets say in a cron job. I see we have workers on Heroku, but I have never used them and this is still a development environment. A free plan does not have workers. Besides, my app doesnt really require background workers. I dont want to buy worker dynos only for automatic database backups. Which way should I go?
If I can create PHP cron jobs on Heroku, then I need to know: How can I run Heroku commands in PHP? I tried exec and passthru, but none of them seems to work on Heroku server. On my localhost, the above command (heroku pgbackups) works pretty well, providing the Heroku toolbelt installed on local server.
For Ruby, they have https://github.com/heroku/ toolkit for server-side commands. But I had no luck in my search for a PHP branch...
The overall purpose is to have the DB backup and store it on the server and download it. (Even though Heroku makes backups itself, we want to see it in our hands :)
How can I make it happen?
Probably the best thing to do is to have a cron job on your backup server to run heroku pgbackups:url and to get a URL to the latest pgbackup, and then download it with curl. Something like this:
curl $(heroku pgbackups:url) > latest_backup
For more info about heroku pgbackups:url, see:
https://devcenter.heroku.com/articles/pgbackups#downloading-a-backup
Doing anything with worker dynos wouldn't really make sense because that wouldn't really help you get the backup to your backup server unless you were downloading and re-uploading it or something. Just running a cron job on your backup server downloading once is a lot more straight forward.
I've got a PHP script on an Amazon EC2 instance. I changed a couple things in it, but the output is the same when I load it in my browser. Does Amazon have some sort of caching in place? I know the East zone was down today, but my instance is running fine now. I've ruled out client-side caching already.
In our ec2 instances this sort of thing happens in 2 scenarios:
1) a bug in our deployment ( for example, puppet or another deploy tool did something funny) or
2) git : a branch was not pushed to head, but the server redeploy happend.
Unless you are using a caching system which you know of, there is no reason to think that ec2 is caching things under the hood - ec2 is controlled and configured quite directly .
I’ve been working on a cloud based (AWS EC2 ) PHP Web Application, and I’m struggling with one issue when it comes to working with multiple servers (all under an AWS Elastic Load Balancer). On one server, when I upload the latest files, they’re instantly in production across the entire application. But this isn’t true when using multiple servers – you have to upload files to each of them, every time you commit a change. This could work alright if you don’t update anything very often, or if you just have one or two servers. But what if you update the system multiple times in one week, across ten servers?
What I’m looking for is a way to ‘commit’ changes from our dev or testing server and have it ‘pushed’ out to all of our production servers immediately. Ideally the update would be applied to only one server at a time (even though it just takes a second or two per server) so the ELB will not send traffic to it while files are changing so as not to disrupt any production traffic that may be flowing to the ELB .
What is the best way of doing this? One of my thoughts would be to use SVN on the dev server, but that doesn’t really ‘push’ to the servers. I’m looking for a process that takes just a few seconds to commit an update and subsequently begin applying it to servers. Also, for those of you familiar with AWS , what’s the best way to update an AMI with the latest updates so the auto-scaler always launches new instances with the latest version of the software?
There have to be good ways of doing this….can’t really picture sites like Facebook, Google, Apple, Amazon, Twitter, etc. going through and updating hundreds or thousands of servers manually and one by one when they make a change.
Thanks in advance for your help. I’m hoping we can find some solution to this problem….what has to be at least 100 Google searches by both myself and my business partner over the last day have proven unsuccessful for the most part in solving this problem.
Alex
We use scalr.net to manage our web servers and load balancer instances. It worked pretty well until now. we have a server farm for each of our environments (2 production farms, staging, sandbox). We have a pre configured roles for a web servers so it's super easy to open new instances and scale when needed. the web server pull code from github when it boots up.
We haven't completed all the deployment changes we want to do, but basically here's how we deploy new versions into our production environment:
we use phing to update the source code and deployment on each web service. we created a task that execute a git pull and run database changes (dbdeploy phing task). http://www.phing.info/trac/
we wrote a shell script that executes phing and we added it to scalr as a script. Scalr has a nice interface to manage scripts.
#!/bin/sh
cd /var/www
phing -f /var/www/build.xml -Denvironment=production deploy
scalr has an option to execute scripts on all the instances in a specific farm, so each release we just push to the master branch in github and execute the scalr script.
We want to create a github hook that deploys automatically when we push to the master branch. Scalr has api that can execute scripts, so it's possible.
Have a good look at KwateeSDCM. It enables you to deploy files and software on any number of servers and, if needed, to customize server-specific parameters along the way. There's a post about deploying a web application on multiple tomcat instances but it's language agnostic and will work for PHP just as well as long as you have ssh enabled on your AWS servers.