I am doing a clean install of OS X Snow Leopard and I was wondering what people think should go into the ideal web developer environment?
Can I get a good rundown of things you would do when setting up a clean system?
My main focus is HTML, CSS, PHP, and Actionscript work. But I would also like to dabble in other languages and frameworks as well (including Ruby and Python, Rails and Django).
The main goals
Cleanly and rationally organized
Efficient to add new frameworks, project source code and packages
Easy to disable components, frameworks and parts of the environment
Ability to host multiple projects simultaneous without too many conflicts or dependencies
Effective source control over all projects and environment configurations
Somewhat idiot proof for a non system administrator, but sufficient power to customize
Easy to isolate and debug environment configurations.
I am aware of MAMP and XAMPP. And I use Git.
I was thinking about using macports for setting up various tools and packages. I also want to cleanly manage frameworks. But I also want to be able to experiment with unknown frameworks without fear of cluttering or breaking my normal day to day working environment.
I know the command line but I am by no means a system administrator. With all that said what would you recommend for managing the hosting environment? Apart from VM are their tools for conveniently sandboxing configurations (different versions of frameworks, databases, webservers, etc) and linking these to a particular client's project code?
It seems like macports by putting everything in /opt is the way to go. I don't have to worry about mucking up the system binaries. And if I make a mistake or want to start over I can blow away /opt anytime I want to.
Are there tools to manage changes to your local web environment? For example if I install a particular package can I conveniently see "what has changed". Basically a way to do a diff on part of or even the whole system to know what implications are in place as I add new a new framework or update a package like PHP, Ruby, Perl, MySQL, Apache, etc. What about snapshots and backups of the configuration? can I version control this? If I setup PHP and Apache in a specific way to work with a specific clients hosting environment needs can I undo that configuration to quickly switch over to a different project?
Feel free to mention or link other related questions or answers if this question has any overlap. And I am not sure if this question should go on serverfault or superuser. But I am mostly interested to hear what other web developers have to offer.
Also, in particular I am curious to learn from people's mistakes. The main gist is, if you were starting over what would you do differently? Knowing what you know now?
For snapshotting entire setups including servers VM's would be the way to go. In Linux I often use chroots to isolate an install, but then the Apache ports etc are still shared.
Related
I just read the Twelve-Factor App, which looks like a pretty comprehensive set of rules to apply in a web-based application. It uses python or rails in its examples, but never php... I was wondering which factors of the manifesto can be applied to PHP projects and how?
Thanks
Short answer:
All points apply to PHP as the twelve factor app manifesto refers specifically for web apps.
PHP has a very hard time complying to twelve factor, in particular in the items 2, 7, 8 9 (as a side effect of 7 and 8) and 12 (partially). Actually that's the first really grounded argument I have heard in the whole "PHP sucks" rant that is common on the Ruby and Python communities (don't get me wrong, I think Ruby and Python are better languages, but I don't hate PHP, and definitively hate the "my language is better" rants.)
Being that said, it may be that your PHP project is not a web app or SaaS, but just a simple website, so you may just deem that twelve factor is not a need.
Long answer: A point-by-point analysis would be:
Codebase: not an issue
Dependencies: the way PEAR works goes quite against this point, as pear dependencies are installed system wide and usually you don't have a consolidated manifesto to declare them. Is also usual for a PHP setup to require you to add packages to your OS installation to get some libraries available. Finally AFAIK there isn't a tool in PHP to provide isolation like "virtualenv", "rbenv" or "rvm" (or if it exists is not popular among the PHP community) Edit: Composer (http://getcomposer.org/) seems to do the right regarding dependencies, it still doesn't isolate the PHP version, but for all the rest it should be fine.
Config: some PHP frameworks are not very well suited to do this, but there are certainly others that do well, so it's not a flaw of the platform itself
Backing Services: shouldn't be much of an issue, despite maybe having to hack some frameworks a little in order to manage the services as resources
Build, release, run: this is totally appliable to PHP as this definitively doesn't concern only "compiling". One may argue that several projects and hosting platforms on the PHP community abuse of direct FTP, etc. but that's not a flaw of PHP itself, and there is no real impediment on doing things right regarding this item.
Processes: This definitively concerns to PHP. PHP is quite capable of running purely stateless processes (the emphasis is on the word stateless), and actually several frameworks make your life easy for it. For example, symfony provides out-of-the-box session management with memcached or database storage instead of regular sessions
Port binding: Over simplyfing it, this point basically demands you to reverse proxy and have the actual webserver embedded on the app instead of being a separated component. This puts PHP in a very hard position to comply. Despite there are ways to do this (see the reply about using PHP as FastCGI) that's definitively not the most common nor the best supported way to serve a PHP app as it is on other communities (e.g. Ruby, Node.js).
Processes: This is not impossible in PHP. However several elements put PHP in a hard position to comply. Namely the lack of good support for items 6 and 7; the fact that the PHP API to spawn new processes isn't really very nice to work with; and specially the way Apache's mod_php handles their workers (which is by far the most common deployment schema for PHP)
Disposability: If you use the right tools there is nothing inherent on PHP to prevent you from creating fast, disposable, tidy web and worker processes. However I believe that since the underlying process model is hard to implement as per points 7 and 8, then 9 becomes a bit cumbersome as a side effect
Dev/prod parity: This is very platform agnostic, and I would say one of the hardest to get done right. PHP is no exception to this rule, but it doesn't have a particular impediment either. Actually most of the tools named on the manifesto can be applied to a PHP project
Logs: Having your app agnostic of the log system on the execution environment is totally doable on PHP
Admin processes: The most important flaw of PHP regarding this point is its lack of a REPL shell. Regarding the rest, several frameworks like Symfony allow you to program admin tasks (e.g Doctine-based database migrations) and run them on the same environment as your "regular" web envionrment.
Since the PHP community is evolving, it may be that it has already righted some of the wrongs mentioned.
Build, release, Run: Applicable to compiled code which is not the case in PHP. So this
point is not something you need to look at.
I don't claim any authority on this 12 factor stuff, but my read of that section is that the author would disagree. It's not just about compiling, it's about managing dependencies both in the small (the snapshot of the code) and in the large (any libraries the code uses).
Imagine you're a new dev and they say, "Okay, this is a custom php app, so...
a) The custom code is really two subprojects, which are in repo A and repo B.
b) You'll need to create a directory layout like so, and then
c) check the code for each subproject out into this subdirectory and this subdirectory.
d) You'll also need these three open source PHP libraries:
version 3.1 of library Foo,
version 2.3 of library Bar, and
version 5.6 of library Bat.
e) download them from their home project sites and unpack them, then copy them into this directory, that directory, and the other directory.
f) then you'll need to set these configurations in the external library,
g) and these configs in our two custom code projects.
h) once that's all done, tar/gzip it all up, upload it up to the QA server and untar it into htdocs.
There's no compiling going in that set of steps, but you can bet there's a lot of building.
Getting all of that set up and working is the build step.
Using tar/gzip to take a snapshot of the working build is the release step.
SCP'ing/unpacking it into the QA server's htdocs directory is the runtime step.
You might say that some of those steps above are unnecessary - the libraries should be deployed at the system level and merely imported. From the 12factors.net site I'd say the author prefers you to import them uniquely and individually for the app. It sidesteps dependency versioning problems at the cost of more disk space (not that anybody cares). There are more hassles in managing all those dependencies as local-to-the-app, but then that's the point of the build/release/runtime scheme.
It might have changed since you read it - there are a few PHP examples now, although a few of them seem like negations of the twelve-factor concept.
One of the ways that normal mod_php sites violate twelve-factor comes with VII. Port binding. From the manifesto:
The twelve-factor app is completely self-contained and does not rely on runtime injection of a webserver into the execution environment to create a web-facing service. The web app exports HTTP as a service by binding to a port, and listening to requests coming in on that port.
However, apache/php can be coaxed into this worldview with something like this:
https://gist.github.com/1398498
Not perfect, but it works for the most part. To test it out, install foreman:
gem install foreman
then run it in the directory you cloned the gist into:
foreman start
then visit
http://localhost:5000/foo
DO NOT TAKE THIS POST AS A REFERENCE, THIS WAS WRITTEN IN 2011, MANY THINGS HAVE CHANGED SINCE THEN... THE WORLD OF PROGRAMMING IS IN CONSTANT EVOLUTION. A POINT OF VIEW OF 2011 IS NOT NECESSARILY STILL VALID IN 2018.
I just read a few lines of each points and here goes my analysis of the document:
Codebase: Everyone, php or not should have a codebase even for the little projects
Dependecies: PHP uses includes and code libraries that you should always be able to port easily by simply copying the code to a server. Sometimes you use PEAR and then if the server doesn't support it, you have to copy and install pear manually. I use Zend Framework most of the time, so it's just copying the code of the framework with the ftp upload.
Config: It is common for php apps to have a writable config file that you store configurations into. Where you store it is your choice as a developer, but it is usually either at the root of your app or in a settings/config folder.
Backing Services: PHP does use backing service most of the time such as MySQL. Other common services depending on your app are twitter and facebook. You use their API to communicate with them and store/retrieve data to work with.
Build, release, Run: Applicable to compiled code which is not the case in PHP. So this point is not something you need to look at.
Processes: HTTP is stateless and is SERVED,thus, you usually have no process apart from the web server. This is not entirely true as a webservice may be bundled with applications you package with or create for it. But, for the sake of standards, you usually don't have to use processes in the web world.
Port binding: PHP doesn't apply to port binding at all because it is not a process that hooks to an address, Apache, NGinx or Lighttpd does that for you. Reading #6/7 makes me understand that a website could never be a Twelve-Factor app.
Concurrency: Again this point treats about processes which do not apply to PHP web pages. This point applies to the server serving the content.
Disposability: This point discusses about how fast a process should be. Obviously, PHP web sites not being a process shouldnt apply but always note that your website should execute pages as fast as possible... Always think about this block or that block of code and see if it is optimized.
Dev/Prod Parity: This point is crucial in any app development. You never want to have a large gap between two app versions or upgrading can become a hassle. Furthermore, never create client specific versions of an app. Always find ways to allow your app to be configured/customized at the template level so you can keep your app as close as possible to all installed versions everywhere.
Logs: Logs are always a good thing to have, they allow you to follow the process of your code without having to output it to screen. My favorite tactic is to "tail -f logfile" inside of a linux console and look at what is happening as i execute my code.
Admin processes: Not applicable, in php, you don't have processes, but you do have pages that you can secure with usernames and passwords.
I've finally committed to really learning the software design process correctly in order to advance my skills and grow my business. This means embracing version control (git), setting up a development-staging-production environment and keeping these environments as similarly configured as possible.
I'm getting really caught up with the last step, in picking a solution to automate and sync my server settings. I've looked into Chef, Puppet & Fabric, but for my purposes they all seem overly complex. I am:
Developing a small web app on a single server
Will be developing in a LAMP environment with intermediate PHP & UNIX skills
Won't be heavily modifying environmental variables (primarily php.ini, apache configs)
I would appreciate any recommendations on solutions that would be easier to implement than mastering the complex Chef environment or learning Python to use Fabric. I can do this if necessary, but am hoping there is a more basic / elegant solution given my very simplistic needs.
In the company I work for, where we have more or less the same needs, we just setup a couple of bash script.
Basically it sets up the git repo (local, and distant bare), install apache2 and PHP5 (and some php extensions), configure the apache's vhost, php.ini, install frameworks and bootstrap project if needed (for us it's symfony).
We have another script, that fire some EC2 instance, run the previously mentionned script, launch the test suite, and download the report of these scripts.
Chef & Puppet works well, but it's a little overkill, unless you have many projects that runs in the same time.
Edit :
If you want to run a script after commiting/pushing (like deploy to staging/pre-production server, launching your continuous integration build, etc), there's a way to do this using git call post-hook, see Deploy a project using Git push
I'd strongly recommend having a look at Ansible for this purpose.
It is a full solution, which means it can handle configuration management, deployment and so forth. However, it is far easier to learn in my experience than Chef or Puppet as you can start by doing basic shell command execution and move on from there.
There's no need to learn a new language; all the configuration and specification you would be doing is done in YAML, which is just structured text.
Overall, Ansible will give you much of what Chef or Puppet will at your level and hopefully you will find it more straightforward to get started with.
If you're serious about professional web development, I would strongly recommend taking a second look at Chef. It works really great for us (me and my co-workers). I know it may seem like overkill, but in my opinion, the advantages far outweigh the learning curve. It's a lot more work to try to maintain different server environments (and local development environments among co-workers). Plus, Chef makes it super easy to install Apache, PHP, and MySQL since there are already cookbooks/recipes available.
Also, make sure you check out Vagrant. It works with Chef and VirtualBox, making it really simple to set up a local development environment.
Also, if you're working on a Zend Framework project, you may be interested in the Zend Framework Boilerplate project which is an all-in-one LAMP development environment which uses Vagrant.
For Simple LAMP Development you don't need anything at all. It is not that simple development can't be automated, it's because for simple development it is usually sufficient (easier and faster) to write some scripts yourself (even in Python).
When you realise that your custom scripts are hard to maintain or not enough, you are ready for tools like Fabric (shell command automation) and/or Chef/Puppet (server configuration management). They are not easy to learn, because system interconnections that they are managing are not simple (which is not your case, obviously).
For your single server, I'd say README + Mercurial (Git if you need GitHub) plus some symlinking should be more than enough to manage configs, sources and server setup. For automation and deployment just write a script that uploads your site to FTP/SSH, restarts server, executes tests, whatever - you decide. That is Simple Deployment for Simple Development. If you'd rather avoid writing PHP for that and don't know shell - then Fabric will save some time for you.
Once your scripts are ready, you already know your problems, you can learn Chef/Puppet in background to see if it is worth complicating things for your environment.
If you choose to try Chef - don't start with Chef Solo - it's a poisonous snack for a starter - use Hosted Chef + Client - it is free for your setup. Can't say anything for Puppet - I chose Chef because my mom said I need to know how to cook.
I started web development some time ago, but have invested essentially all my time in learning PHP, MySQL/SQLite, (X)HTML/XML (XPath, XQuery, XSL), etc. I went with an out-of-the-box XAMPP install, everything was pretty straightforward, and I could focus my learning on languages.
Now, however, I'm looking to learn Ruby and Python (and more Perl) but to my dismay, have discovered that I know essentially nothing about Apache configuration, or any related tasks associated with setting up an environment for embedded PHP, Perl, Python and Ruby.
I posted recently a question that I thought would take me in the correct direction. Having read the mod_wsgi and mod_rails docs, I'm more confused than I was to start.
I'm running Windows 7 x64 with an existing XAMPP installation, just grabbed Ruby and Python, just installed NetBeans; and I'm wondering:
Should I start with a fresh Apache install, or is XAMPP fine to pile additional modules on?
Where can I find some information about configuring Apache to support these modules? I'm thinking step-by-step to build an environment that supports some of, all of, or more than the aforementioned languages.
I suppose my question can be summarized to; What are some suggested reading/learning resources for configuring an Apache based multi-language web application development environment?
I hope this question isn't regarded with high subjectivity or generality; I'm just lost and don't want to waste time staring blankly at configuration files or the like.
I really appreciate any insight, and forgive my noob factor. I am surprised I am as unfamiliar with this territory as I am, and quickly need to change that.
I would recommend the following:
Make your home in the Ubuntu environment within the VM you mentioned
Essentially all of the technologies you mentioned started out in Unix-based systems, and, while the playing field is leveling, there are still significant advantages to developing with these technologies in a Unix-based OS (such as the Ubuntu install you mentioned.)
Additionally, Ubuntu is very well documented online, has active forums, and has very nice install systems (apt-get, aptitude.)
Learn Nano
You can use other IDE's or text editors, but Nano is really, really easy to use, so you can focus on learning the technologies you're interested in, not a complex text editor such as vim or emacs (which are both fantastic, but learning sessions in and of themselves.)
Get yourself a cheap cloud server to pound on
Many companies offer great cloud servers. The nice thing about this is that you can play with the every aspect of the environment, including the networking capabilities, such as getting a firewall running, etc.
For instance, Rackspace Cloud Servers work great and the Wiki pages walk you through a vast number of installs and configurations (I just pointed you to the Ubuntu docs cuz that's what you mentioned already running):
Rackspace Cloud Server Wiki
Build up your web server from scratch (and keep a log of the build steps, so if something doesn't go well, you could redo things quickly.)
Start building (and breaking) your next great app
Start making a cool app using the technology of choice. If you want to focus on Ruby next, crank out an ROR app, or even dig down and use RACK to set up you're own baby framework for handling web requests:
Rack: a Ruby Webserver Interface
Or, if python is your next learning project, try out Django, or even jump down to CherryPy for a low level web server implementation that helps you see all the moving parts:
CherryPy
Join the mailing lists of the technologies you're most interested in
Mailing lists provide great insights into the strengths and weaknesses of technologies. Just joining a list to listen to the daily banter can significantly help your understanding of the technology and the challenges that you'll likely face in the future.
These are all things I've done (different language focuses, but same techniques) and they've helped me greatly. Hopefully, they help you, too :) Happy coding!
Just an additional info, XAMPP has mod_perl included since version 1.7.2.
To setup this quickly on Windows, already prepared stacks built by BitNami can help:
Ruby + Rails
Python + Django
I've spent some time now developing a web application in php. It has mostly been just for the fun of learning as a side project, but the app now has a few users which I don't want to upset by breaking things as I develop.
At the moment, I have a very rudimentary method for managing the development - I use a text editor (ultraedit) to write the code and use its built in ftp to upload the files to the server. In terms of version control I have 2 domains, and only push files to the "live" domain when they work, but that's it. The domains are hosted on a cPanel shared hosting site which I have some doubts about its ability to handle even minor spikes in traffic. I looked at slicehost yesterday for something more scalable but that looks like a bit of a learning curve from where I am now.
I know I could do better than this, but where to start? I think I need advice on three things
1 - code writing tool
2 - version control / management
3 - scalable hosting
I've deliberately asked these in the same question as I'd like to know if one choice impacts another. Is there a good integrated solution?
Thanks in advance as ever.
Each part of your question has been answered before. The list below lists some of the common tools to use and links to appropriate searches on StackOverflow. There is no all in one package and going into details about these tools in one question is out of scope, so I am afraid you have to do some digging:
SVN, Git or Mercurial
UnitTesting (PHPUnit or SimpleTest)
Continuous Integration
Phing (for deployment)
phpqatools.org
Netbeans or Eclipse (for an IDE)
Disclaimer: list is not meant to be complete and order is not important
There is a lot going on here. I'll give you my two cents though.
I used to use ultra edit. Now I use netbeans, its a fully integrated development environment and it makes my life soooo much easier. Its free too. I can't imagine ever going back to UltraEdit. Also, which brings me to number two, netbeans has SVN and CVS integration
I would use subversion for version control. In my experience it does everything you need for version control. Others use ones like git and mercurial, but I think SVN is widely supported and easy enough to set up. As far as pushing code to the server, i've begun using svn for this too. I first ssh into the server and checkout the code into the public_html directory, and then set up an alias to do svn updates through the command line... its way easier than ftping in my opinion. There are other deployment methods, but i've never used them. see this question:
What is your preferred php deployment strategy?
obviously shared hosting is not going to handle traffic as well as a dedicated server. But there are lots of things you can do to improve performance before moving to a dedicated server. Check out this article: http://developer.yahoo.com/performance/rules.html
It seems that you're after a robust deployment strategy as opposed to a development one. But, correct me if I'm wrong.
In terms of 'code writing tool', and
IDE choice is a subjective
discussion. Feel free to work with
the one you are most comfortable
with, for me this is Netbeans.
As for a deployment strategy, I think it was best summed up in this answer.
Your point about scalable hosting is fairly broad. We will need much better forcasted metrics to give better advice. However, for now, if scalable hosting is a worry then maybe look into some sort of Cloud Hosting.
Have you looked at using wamp/xamp/mamp/lamp for development locally? Uploading via ftp for every change is a pain.
You could do that for local and see that everything works, then push it to your test domain and run through it again and then finally push live.
Might want to look at something like SpringLoops for doing your version control - this has the advantage of doing the deploy and then if it goes pear shaped you can revert it (free account gets 3 deploys a day).
I wouldn't worry about scaleable hosting just now, just focus on the site and the coding, you've only got a few users - wait till it starts becoming a problem before moving (however, I suggest looking into options) but don't try and get all cloud ready - might not ever be a problem.
ps, go with Linode over Slicehost.
I think use Aptana Studio ( http://www.aptana.org ). It is an Eclipse based IDE with all the tools you need integrated in it. It has integrated PHP development tools, GIT or SVN for version control.
I've used shared hosting as well. Once an other site on the same server had DoS atacks and my site became unreachable as well. Otherwise it could work in reasonable speed after some optimization. It served 1000-3000 users per day.
Dedicated servers are much better. Or you can use Amazon web services. I know they are more expensive.
1 - code writing tool
Zend Studio. I would recommend Linux as well if you are going to use linux servers.
2 - version control / management
SVN + phing (if you are going to build serious applications).
3 - scalable hosting
Amazon or RackSpace.
For your editor, just use what you are comfortable and productive with. You most definitely should have version control in place. You never know when you need to rollback a version or two.
I always keep at least 3 versions on the production server. I use symlinks to point the web server at the latest version. If there is a problem, you just need to recreate the symlink to point to an older version.
As for shared hosting, you'd be surprised out how much bandwidth you can get. I have a $10/month shared host that gets about 500K page views a month. Generally, it's not that your shared host can't handle the load, it's that the hosting provider puts too many "shares" on the same server. So it depends on how much resources everyone else on the same server is using. If you are having issues, you can always ask to be moved to a different server.
Beginner here, stuck wondering what I need to do to learn development in different web environments. Say, for instance I want to play around in PHP & MySQL. But I also want to try things with Ruby on Rails and maybe even server things with Python. Do I need a different environment for each platform? Am I required to have a virtual machine to do so? Is it easy to uninstall and start over for each endeavor for a Windows 7 machine?
Or, better yet, would it just be easier to throw Linux in a virtual environment?
I started off with php and mysql, it's a lower-level then the rest of the environments like Django and Ruby on Rails; so it's much easier to understand what is really happening.
If you want to get into web development, php is a solid foundation and has easy bundle installers such as WAMP, and has a massive community.
However if understanding the core of web applications is not your goal and you want some magic to make your application quicker and speed through development, you would want to use a web application framework environment, my favorite here is Ruby on Rails.
Ruby on Rails is a little difficult for the newbie to setup, it typically sucks running in Windows, so you'd be better off with an install on top of a UNIX environment such as Linux, Mac OS, BSD.
If you do want to go the rails route, I suggest installing Ruby with a ruby manager called RVM which is very helpful for playing around with Ruby environments.
Django is a good solid framework but it's playing catch up with Rails and in my honest opinion, doesn't have all those power magic that Rails has such as database migrations.
It is theoretically possible to install all of these on a single web server instance, but I imagine it might be an awfully hard thing to do. (Update: As Col. Shrapnel says, it is not impossible though. See the comments.)
You could take a look into the BitNami stack. It consists of installable packages that promise modularity (i.e. the possibility to add languages like PHP and Ruby to one server instance) and pre-configured virtual machines for the most popular development environments.
I would go the vitual road (VMware or virtual box). Multiple enviroments have a lot of dependencies, so its just much much easier with vitual hosts.