I've finally committed to really learning the software design process correctly in order to advance my skills and grow my business. This means embracing version control (git), setting up a development-staging-production environment and keeping these environments as similarly configured as possible.
I'm getting really caught up with the last step, in picking a solution to automate and sync my server settings. I've looked into Chef, Puppet & Fabric, but for my purposes they all seem overly complex. I am:
Developing a small web app on a single server
Will be developing in a LAMP environment with intermediate PHP & UNIX skills
Won't be heavily modifying environmental variables (primarily php.ini, apache configs)
I would appreciate any recommendations on solutions that would be easier to implement than mastering the complex Chef environment or learning Python to use Fabric. I can do this if necessary, but am hoping there is a more basic / elegant solution given my very simplistic needs.
In the company I work for, where we have more or less the same needs, we just setup a couple of bash script.
Basically it sets up the git repo (local, and distant bare), install apache2 and PHP5 (and some php extensions), configure the apache's vhost, php.ini, install frameworks and bootstrap project if needed (for us it's symfony).
We have another script, that fire some EC2 instance, run the previously mentionned script, launch the test suite, and download the report of these scripts.
Chef & Puppet works well, but it's a little overkill, unless you have many projects that runs in the same time.
Edit :
If you want to run a script after commiting/pushing (like deploy to staging/pre-production server, launching your continuous integration build, etc), there's a way to do this using git call post-hook, see Deploy a project using Git push
I'd strongly recommend having a look at Ansible for this purpose.
It is a full solution, which means it can handle configuration management, deployment and so forth. However, it is far easier to learn in my experience than Chef or Puppet as you can start by doing basic shell command execution and move on from there.
There's no need to learn a new language; all the configuration and specification you would be doing is done in YAML, which is just structured text.
Overall, Ansible will give you much of what Chef or Puppet will at your level and hopefully you will find it more straightforward to get started with.
If you're serious about professional web development, I would strongly recommend taking a second look at Chef. It works really great for us (me and my co-workers). I know it may seem like overkill, but in my opinion, the advantages far outweigh the learning curve. It's a lot more work to try to maintain different server environments (and local development environments among co-workers). Plus, Chef makes it super easy to install Apache, PHP, and MySQL since there are already cookbooks/recipes available.
Also, make sure you check out Vagrant. It works with Chef and VirtualBox, making it really simple to set up a local development environment.
Also, if you're working on a Zend Framework project, you may be interested in the Zend Framework Boilerplate project which is an all-in-one LAMP development environment which uses Vagrant.
For Simple LAMP Development you don't need anything at all. It is not that simple development can't be automated, it's because for simple development it is usually sufficient (easier and faster) to write some scripts yourself (even in Python).
When you realise that your custom scripts are hard to maintain or not enough, you are ready for tools like Fabric (shell command automation) and/or Chef/Puppet (server configuration management). They are not easy to learn, because system interconnections that they are managing are not simple (which is not your case, obviously).
For your single server, I'd say README + Mercurial (Git if you need GitHub) plus some symlinking should be more than enough to manage configs, sources and server setup. For automation and deployment just write a script that uploads your site to FTP/SSH, restarts server, executes tests, whatever - you decide. That is Simple Deployment for Simple Development. If you'd rather avoid writing PHP for that and don't know shell - then Fabric will save some time for you.
Once your scripts are ready, you already know your problems, you can learn Chef/Puppet in background to see if it is worth complicating things for your environment.
If you choose to try Chef - don't start with Chef Solo - it's a poisonous snack for a starter - use Hosted Chef + Client - it is free for your setup. Can't say anything for Puppet - I chose Chef because my mom said I need to know how to cook.
Beginner here, stuck wondering what I need to do to learn development in different web environments. Say, for instance I want to play around in PHP & MySQL. But I also want to try things with Ruby on Rails and maybe even server things with Python. Do I need a different environment for each platform? Am I required to have a virtual machine to do so? Is it easy to uninstall and start over for each endeavor for a Windows 7 machine?
Or, better yet, would it just be easier to throw Linux in a virtual environment?
I started off with php and mysql, it's a lower-level then the rest of the environments like Django and Ruby on Rails; so it's much easier to understand what is really happening.
If you want to get into web development, php is a solid foundation and has easy bundle installers such as WAMP, and has a massive community.
However if understanding the core of web applications is not your goal and you want some magic to make your application quicker and speed through development, you would want to use a web application framework environment, my favorite here is Ruby on Rails.
Ruby on Rails is a little difficult for the newbie to setup, it typically sucks running in Windows, so you'd be better off with an install on top of a UNIX environment such as Linux, Mac OS, BSD.
If you do want to go the rails route, I suggest installing Ruby with a ruby manager called RVM which is very helpful for playing around with Ruby environments.
Django is a good solid framework but it's playing catch up with Rails and in my honest opinion, doesn't have all those power magic that Rails has such as database migrations.
It is theoretically possible to install all of these on a single web server instance, but I imagine it might be an awfully hard thing to do. (Update: As Col. Shrapnel says, it is not impossible though. See the comments.)
You could take a look into the BitNami stack. It consists of installable packages that promise modularity (i.e. the possibility to add languages like PHP and Ruby to one server instance) and pre-configured virtual machines for the most popular development environments.
I would go the vitual road (VMware or virtual box). Multiple enviroments have a lot of dependencies, so its just much much easier with vitual hosts.
I've been contacted to see about updating an old legacy web application that was built using ASP and Access. The server is running Windows 2000 Advanced Server and I believe IIS 5.0 (I am trying to get confirmation on that, but the company isn't technical so I highly doubt Apache is running on the server).
What languages would be viable for updating this web app on the above platform? I've never touched classic ASP much less done any web development work against Windows 2000/IIS 5. There are no plans on updating the server to anything new due to budget concerns.
I'm leaning at the moment to moving to an SQLite-based database (customer isn't too keen on installing MySQL at the moment but I'm still in planning stages and this is a relatively low-traffic website) but what language would I pair with that? Does ASP.NET work well under IIS 5? Does PHP perform worth anything under this kind of setup?
I have a similar situation, did it about a year ago, and ended up using asp.net 2.0.
Generally ok, but the machine is showing it's age, I usually need to get someone to give it the 3 fingered salute every month or so, and it blew a psu recently.
If it's only low volume, you might be able to install sql express, which will make your life a lot easier than something like SQLlite, as dotnet plays nicest with other MS stuff, and there is a lot of labour saving goodness built in.
You would also be able to use the access to sql migration tools if you use sql express.
Would also suggest that you look at something like subsonic or nhibernate, which will take care of a lot of the boring and error prone stuff for you.
It really depends on where your experience lies, and how big the project is, if you've never used dotnet before, then start on something small, this may or may not be the one.
Apparently php performs well on win 2008, but as for 2000, never tried. Did have apache on a 2k box many years ago, but wasn't using php.
If the company is concerned with cost, I would be very conservative making changes. Concentrate on why they want to update- do they want to add new functionality? What are their mid-to-long term plans for the site? Are they having trouble maintaining the site? Going to a custom .NET solution may only complicate things further unless they are willing to make some ongoing investment in development.
If it's a relatively simple site, they may want to consider a platform like DotNetNuke. There are hosts out there that sell ready-to-configure sites that can do quite a lot with a minimum of configuration. That combined with a profressionally developed DotNetNuke UI template (TemplateMonster.com offers them) may be a good solution.
If they do want to go with a custom solution, ASP.NET runs fine on IIS 5.0. I believe you can run the .NET Framework up to at least 2.0, not sure about 3.0 or 3.5. Language won't make a difference to functionality, so C# or VB.NET are fine, all things being equal.
In this scenario, I would probably go with ASP.NET. Since you're running on a microsoft server, there will be plenty of documentation from MS on installing, configuring, and running the site. It's a lot easier to support something when all the components are "in the same family" so to speak. Asp.net will run fine under IIS 5. It doesn't have a lot of the security and scalability upgrades that IIS 6 does, but it will do the trick.
I was able to get a bit more information. The box is running IIS 5.0 and the IT guy handling it is more than happy to let me install whatever I need. From googling and responses below it seems like my best bet will be to convert the site to ASP.NET 2.0 with SQL Server Express 2005 running as the DB.
Is the LAMP (Linux, Apache, MySQL, PHP / Ruby / Python) stack appropriate for Enterprise use?
To be clear, by "Enterprise", I mean a large or very large company, where security, robustness, availability of skill sets, Total Cost of Ownership (TCO), scalability, and availability of tools are key considerations. Said another way, a company that looks for external adoption of frameworks / architecture - Something ubiquitous will be seen as more "valid" than something exotic / esoteric in this kind of environment.
I've seen use cases where Oracle, IBM, and Sun have implemented systems on the LAMP stack for various Enterprises. I've also seen examples where websites like yellowpages.com (Ruby on rails) and Facebook (php) are built on it. However, none of these examples are exactly what I'm looking for.
I'm really trying to find examples where it is an Enterprise standard at a very large bank (I.e., Citigroup), Telecom company (I.e., AT&T), or manufacturer (I.e., Proctor and Gamble). Just to be clear, I'm not looking for an example where it's used in a limited sense (Like at JPMorgan Chase), but where it's a core platform for systems like CRM, manufacturing systems, or HR management, as well as for internal and external websites.
The perception I've seen so far is that applications built on the LAMP stack perform slower and are less flexible. Some of the arguments I've heard are:
Linux is seen as not as well supported as Unix, Solaris, or Windows Servers.
Apache is harder to configure and maintain than web servers like BEA WebLogic or IIS.
MySQL is a "not ready for prime time" DB for hobbyists, and not a competitor for SQL Server or Oracle (Although PostgreSQL seems to have a reputation for being more robust).
PHP / Ruby on rails are optimized for CRUD (Create, Read, Update and Delete operations). Although this is an advantage when building CRUD-intensive web aplications, both perform slower than Java/Java EE or C# (which are both common Enterprise standards). Furthermore, a lot of applications and systems (like manufacturing systems) have a lot of non-CRUD functionality that may be harder to build with PHP or Ruby, or even Python.
Can anyone please provide arguments to support or refute the idea of the LAMP stack being appropriate for the Enterprise?
Thanks!
KA
UPDATE: Some times the LAMP Stack is Appropriate for Enterprise Use: Externally-Facing Blogs
"but where it's a core platform for systems like CRM and HR, as well as for internal and external websites"
First, find a LAMP CRM or HR application.
Then find a customer for the LAMP CRM or HR application.
Sadly, there aren't a lot of examples of item 1. Therefore, your case is proven. It can't be used for enterprise applications because -- currently -- there aren't any of the applications you call "enterprise".
Your other points, however, are very interesting.
Linux is seen as not as well supported as Unix, Solaris, or Windows Servers. I think Red Hat would object strongly to this. Give them a call. I think they'll make a very persuasive sales pitch. Read their success stories.
Apache is harder to configure and maintain than web servers like BEA WebLogic or IIS. By whom? Apache web site managers? Or IIS web site managers? This is entirely subjective.
MySQL is a "not ready for prime time" DB. Take it up with Sun Microsystems. I think they'd object strongly to this. Give them a call. I think they'll make a very persuasive sales pitch. Read their success stories.
PHP / Ruby on rails are optimized for CRUD, and both are slowly performing. Could be true. Java and Python might be faster. PHP and Ruby aren't the last word in LAMP.
Something ubiquitous will be seen as more "valid" than something exotic / esoteric in this kind of environment.
Although I personally wouldn't recommend PHP due to the many flaws in the language, it's most certainly ubiquitous. With the advent of phusion passenger, Rails support amongst shared-hosting companies is growing pretty quickly too. I give it another year or 2 at most before 90+% of shared-hosting accounts support rails out of the box. If that's not ubiquitous, what is?
Linux is seen as not as well supported as Unix, Solaris, or Windows Servers.
If this bothers you, purchase support from RedHat, or install Solaris and purchase support from Sun. Both of those will give you just as good support as Microsoft is likely to
Apache is harder to configure and maintain than web servers like BEA WebLogic or IIS.
I can't speak for BEA WebLogic, but having configured both Apache, IIS, and Tomcat, Apache is the easiest both to understand, and to find examples and documentation for by a long way.
MySQL is a "not ready for prime time" DB for hobbyists, and not a competitor for SQL Server or Oracle.
Oh really?. You should make it your mission to tell NASA, Google, CERN, Reuters etc that they're all using a hobbyist database that isn't ready for prime-time.
PHP / Ruby on rails are optimized for CRUD, and both perform slower than Java/Java EE or C# (which are both common Enterprise standards).
There are 2 things here:
Optimized for CRUD - This is totally irrelevant.
Rails and some of the python/php frameworks are optimized for CRUD apps. Many of the C#/Java frameworks are also optimized for CRUD apps. However, if the app you're building is a CRUD app (and 99% of web applications are), isn't this a Good Thing?
If you're not building a CRUD app, there are plenty of non-crud-optimized frameworks in ruby/python/php/java/C#. Net win: Nobody (hence it's irrelevant)
Perform slower than Java/C# - This is undoubtedly true, but it also doesn't matter. For a low-traffic site the performance difference isn't going to amount to anything, and for a high-traffic site your bottleneck will be the database, whether it be MySQL, oracle, or whatever.
What you trade-off for all of this is development time.
Once you've used all this advice to convince your boss that you won't lose out on anything by using LAMP, If you crunch the numbers and show your them that it is going to take 6 man-months to build the site in Java, and only 3 to build it in ruby/python then that's really what it comes down to.
If you hire idiots to implement it, C++ & Oracle will fail to scale.
If you hire people who are smart and get things done, PHP & MySQL will scale just fine.
Same argument goes for security & robustness.
Facebook, Digg, portions of Yahoo run on PHP.
Of course, they hire lots of PhD programmers.
Just thought I'd add another website to the list of those that run on LAMP - Wikipedia. Seventh biggest website in the world, written entirely in PHP and runs off MySQL, and they only have two or three paid developers. Of course, they have some assistance from volunteers, but it's not a lot, and it's scaled just fine. Don't know if you'd really call them 'enterprise', but for such a huge and popular website they seem to have done alright for themselves.
Linux is seen as not as well supported
as Unix, Solaris, or Windows Servers.
As others have said above, give Red Hat a call and I'm sure they'll beg to differ. And the amount of support out there for Linux absolutely free is astonishing.
Apache is harder to configure and
maintain than web servers like BEA
WebLogic or IIS.
That depends who you're asking. People who usually administer IIS servers will probably view it this way. People who usually administer Apache won't. It depends on who you hire, and if your stack is LAMP you won't want to be hiring people with no Apache experience anyway.
I just want to add that I've witnessed many times that clients only feel comfortable once they dish out serious $$$ for some solution, even if it makes enterprise integration even harder, despite what arguments you bring to the table.
I think the first criteria should be your team's skill level, comfort level jut to make sure what ever platform decisions are made works well with them. Whatever you decide think of scalability and maintainability of your code. Tools are awesome no matter what stack you choose.
I personally would break it down into 3 stacks-
The Java Stack where you have Solaris or Enterprise Linux like ( RedHat ) with Weblogic/Websphere/Tomcat etc and Java Enterprise along with Hibernate,Spring etc technologies. Most would opt for Oracle as DB.
The Microsoft Stack with some Open Source if needed Win Server - IIS - .net/C# (ASP.net etc) - NHibernate, NUnit (unit testing) etc. Most likely you would want to use SQL Server as DB
None of the above stack with Enterprise Linux running a whole buffet of open source stuff like MySQL (now under Sun's domain so can be looked at seriously), Apache (there are apache gurus out there), Ruby ( not my personal choice)/ PHP (good luck) / Python (I like it because its a mature language). I would advocate python or ruby from the managing code point of view. Maybe for some it could be PHP..i am not into it.
strictly a subjective opinion but I personally find MySQL and to a lesser extent PHP to be a bit of a weakness, but certainly there's plenty of people who disagree and big companies who went LAMP.
I'd prefer to see postgres or even SQLite take chunks out of the MySQL market, and I'd like to see mono or jsp or cocoon based apps more. I guess LAMP is a bit too specific for an umbrella term. :)
Linux/Apache are hardened, lean and each comes with plenty of people(for the right price of course) who will provide support, plenty of useful tools, many at exceptionally high levels of utility which work with them and which have been built upon them.
Not sure about the other two, however. In particular MySQL seems to have taken a strange turn for the worse since their being acquired by Sun, contrary to the posts in this thread suggesting that Sun may be a good influence:
http://www.reddit.com/r/programming/comments/7gb8j/oops_we_did_it_again_mysql_51_released_as_ga_with/
The reason for not finding Enterprise applications built on LAMP is not because they aren't enterprise level but something entirely different in my opinion. A lot of the big players use LAMP or similar--Facebook and MySpace immediately come to mind. So its clearly not an issue of scale and perf.
That said, the reason I find that there aren't any enterprise apps built on LAMP is because of their intrinsic open nature. I don't want to build an actuarial module as a PHP file because anyone can steal the logic. On the other hand if I have a DLL I can retain control. You don't find a lot of 30-trial apps built on PHP for this very reason but it's much easier to achieve that kind of protection with say ASP.NET.
You have some real bad myths in your posting:
JavaEE Myths:
-App Servers easier to configure than apache, nope apache is easier.
-You imply that only JavaEE full solution is enterprise, nope.
CRUD Myths:
-CRUD is slower than JavaEE? WTF? POJO and EJB is using CRUD.
The limiting factor is not crud, its server throughput
There are 3 limiting bottleneck areas no matter what technology even MS..server implementation, persistence layer, and app layer..the technology chosen is not the speed factor as you can exchange advantages in one layer for disadvantages in the other layer.
Fro example we could spee dup Java by using document store instead of normal DB..
Most new Rails implementations use non apache servers that are faster by a factor of 3 to 5 than Apache..even a well tunned Apache server can outperform some javaEE stacks..just ask yahoo as they use Symfony on some of their properties..
I think you will find that many enterprises use Linux servers, often supported by Redhat, Novell or IBM, and that Apache is also commonly used.
But many enterprises tend to use databases like Oracle or IBM DB2 instead of open source offerings - although there are many enterprises that don't really need the kind of power those systems provide and could get away with MySQL or PostgreSQL.
And for the web-server language, I think you can use just about anything. However, if you use Apache it is probably easier to use PHP, Ruby or Python, whereas if you use IIS or Weblogic or Domino it will be easier to do it in Java / C#.
IMO there are no good general arguments against Linux and Apache; You can certainly get enterprise-level support for Linux if you're prepared to pay for it (and a good approximation of it for free if you're willing to play by the community's rules). And Apache is not that hard to configure unless you need its more complex features, which is unlikely in an application server.
You can certainly make a case against MySQL since some of the most important features in regard to data safety have been added only recently. If you're concerned about that, use PostgreSQL instead.
As for the language you write your app in: PHP has definitely proven to be able to run extremely large and complex systems; I'd be more concerned about maintainability than performance. And Ruby on Rails is "optimized for CRUD" only in asmuch as a simple CRUD webapp can be written in nearly no time (literally minutes), but that does not mean it is somehow less suited to more complex apps, just that it will take much more time (still less than with many other languages)
I suppose that large commercial CRM and HR applications might be biased toward delivering large commercial RDBMS products as the foundation for their products. If nothing else they will I'm sure prefer to unite against a common threat.
And they have a harder time justifying license and support fees if they integrate products that don't have them.
My 2c:
Linux: Since kernel 2.6 came out, I would say it is definitelly a high-quality OS. Version 2.4 wasn't quite there and 2.2 was a joke, but 2.6 is really good. Be careful with a choice of distribution, though. In my experience, RedHat/CentOS is very good, and apparently Debian (original, not Ubuntu!) can be set up nicely if you have a good admin. My experience with OpenSUSE was not very good.
Apache: Haven't used it, but I don't see why it would be a problem.
MySQL: This is the weakest point of the stack. I am not going to go into details here - look into comments at reddit.programming if you are interested. Better look at PostgreSQL.
PHP/Perl/Ruby/Python: I have worked with Perl and to a lesser extent with Python. They are probably OK for web-based applications where the bulk of the work is done by the web server and DBMS anyway. However, I do prefer static type system and would rather pick Java/C# for a business application and C++ for system programming.
I would like to suggest that we identify the scalability requirements of Enterprise systems and how they differ compared to Web Applications. Look at some of the most scalable systems like Wikipedia, Flickr, Wordpress, Facebook, MySpace and a host of others. You will see LAMP stack there. I am more of a Python fan (since I feel that the language has a cleaner feel) but I listen to experts like Cal Henderson (Flickr) who wrote a book on scalability talking about how he scaled a bank of MySQL servers.
What are the essential features of an enterprise system?
Support, availability of expertise, stability of the platform/language probably count.
But LAMP has other features like faster development, easier extensibility, lots of available libraries for reuse, several documented stories of scalability, maturing web frameworks.
Here are a couple of pointers to building Scalable systems (I am talking about Web Scale). I always wondered in the light of all this evidence, why the perception of LAMP as not being ready for Enterprise apps keep popping up.
As for Apache, every Netcraft study shows a very different adoption story. By the sheer number of servers, there may be more people with knowledge to configure, tune and extend the web server.
Scalable Web Architectures
Please Look at Market Share of all Servers Aug 1995 to Jan 2009
Linux is used a lot.
Apache and Tomcat are used a lot.
MySQL may be robust now. I'd use PostgreSQL instead. Banks will use Oracle, but there's good support for Java and Tomcat there.
PHP is used a lot, but many big companies would prefer Java.
You're best off arguing for a Linux, (possibly commercially supported version of) Tomcat, Java, Tomcat|Oracle|MSSQL solution, in my opinion.
You'll need a Linux sysadmin, especially as the number of servers ramps up, although I'm sure you can get a part time one in before that time arises. If the company already has Windows sysadmins then arguing for Linux is going to be tough.
I believe it's not that the technology is premature or something which keeps biggies like AT&T to go ahead with a full implementation at enterprise level. These companies have such a big budget for IT spends that the last thing they would have on mind is to spend more on the customization and enhancement required on the open source techs to suit their business needs.
So what they look for (which comes from my consulting experience) is buy and run product pack and don't have to spend more on the research and hack part. Companies which use open source build have developed their own support groups globally to cater to any support demands, which large enterprises are not much willing to do. They need thing done fast and for sure and they can pay.
There are two main issues for large enterprises using LAMP stacks:
TCO: taking into consideration that LAMP basically comes free, enterprises still achieve a lower total cost of operation with other commercial solutions
Supportability: enterprises have no problem paying the extra buck to get around-the-clock professional support from their commercial vendors
Redhat and IBM give full support for Linux, Sun bought MySQL, Yahoo uses Php, numerous companies use a LAMP stack, but many use parts.
I personally don't see Linux as being less well supported than the other OS mentioned; in fact hardware vendors typically DO support Linux over any other OS (except for Windows, which they do generally support quite well provided you use maintream distributions).
Provided you don't use a bizarre flavour (Tip: Just use RHEL or Centos which is its free equivalent), Linux is very well supported.
MySQL may have some shortcomings, but in my opinion it has many strengths; we use it at a large scale in ways not intended, but it still works quite well generally (most of the problems are due to our versions being out of date or badly configured).
What "P" stands for in LAMP is debatable. I feel that PHP is not enterprise-ready, because it has so many individual shortcomings (e.g. poor unicode handling, no namespaces, inconsistent APIs, inconsistent syntax, poor version backwards compatibility, duplicated/obsolete functionality) that they add up to making it difficult to implement a maintainable system.
But given an appropriately experienced team, even if you choose PHP it can be used to make an extremely high quality application.
If it's good enough for Google, trust me, it's good enough for you.