Tips for building a site like stackoverflow - php

Not as whimsical a question as you might expect - If you have an average web developer (we'll say his name is "Bob") with a fair amount of php/mysql/jquery/ajax/etc knowledge under his belt, and you wanted to recreate a site like as dynamic and as complex as SO from scratch, what are some best practices, and how long (hours or days) would you expect it to take to get it to a fairly stable beta?
I ask because I'm working on building a site (admittedly completely different, but the workload I imagine will end up being comparable), and it just seems like it's taking forever. It's been about a month working it no less than a few hours a night and it's coming along, but development just seems to escalate exponentially. Every new idea or button spawns a whole range of other work projects.
I try to use pre-made libraries and open source projects when I can, and I've looked into using CMS's like Drupal/Joomla, but honestly so much of custom sites are, well custom, that using a visual CMS just seemed to hold me back.
Being a paycheck to paycheck kind of person, I couldn't afford to pay someone else to help, so beyond volunteer work I'm more less on my own here to make the site. (We do have a microscopic community of willing testers)
Because I know it, because we aren't planning to charge any fees I've been building the site on top of PHPBB3. (provides the user registration/authentication, a database API plus functions for security and a full featured templating system and a plethora of other features.. Oh, and a FORUM, heh.)
I use libraries like jQuery+UI for the client side interactions.
I try to make things as modular as possible, so if I need to rewire the whole thing it wont be a terrible chore, but I worry that my lack of experience in major projects like this is going to be fatal to the site's success.
What lessons can folks who have been here offer, and what kind of timeline should I realistically expect to be looking at here?

but development just seems to escalate exponentially. Every new idea or button spawns a whole range of other work projects
This is what's causing it to take forever. You need to one or two things, preferably both.
Figure out your desired feature set, and stop there. Resist the urge to expand your workload with every new idea. It's hard, I know.
If you can't manage #1, then you need this even more. Release quickly and iterate. It's better to get something working done, and then consider the rest of it as improvements rather than original features. Even if the workload doesn't change, your mindset will, and you'll feel a lot better about it and more motivated. As an added plus, if you have users, they'll help you figure out which new features are actually worthwhile.

I wouldn't recommend a CMS. I'd recommend going with a framework instead, perhaps something like CakePHP or Codeigniter. They already include a ton of libraries built in and ready to be used. Makes things go way faster once you get the hang of it. You may still need a JS library or two, but as long as you use the framework as much as possible, you shouldn't have any trouble if proper practices are followed.

My advice: instead of building a new site from scratch, propose a new site within the Stack Exchange network, at http://area51.stackexchange.com.

Related

Should I use a simpler framework or CMS to meet deadline?

TL;DR:
Boss needs site up and running yesterday. BackEndGuy1 uses zend framework 2 and is going way too slow. BackEndGuy2 (this is me) was hired to help BackEndGuy1 meet deadline. FrontEndGuy and BackEndGuy2 decide that using Zend will take forever, so they want to switch to an easier framework or a CMS. What should they choose? Expression Engine? Codeigniter? Concrete5? Something else?
A friend of mine and I have a major decision problem. We’re working on a site that has to be up and running as soon as possible. My friend works on the front end along with an artist, and I work on the back end along with another guy. Actually, I was only recently hired because the boss and the rest of the team decided that the other back end guy needed a speed boost. That other guy thought that using zend framework 2 for this job was a good idea. As a result, I’ve spent the last couple of weeks trying to learn zf2 (which is very hard to do, believe me) and doing php and javascript patchwork on existing code. Plus, I have the boss frequently hovering over my head and asking “What do you think? Are we going to be up before Christmas?”, to which I try to respond in a diplomatic way like “Anything is possible with hard work and determination!”, but my honest opinion is “Sorry, this is impossible. At this rate it’ll probably take a month or two…”.
Bearing in mind that (a) the other back end guy practically stopped working on the site after I was hired (we only talk on the phone when I need him to explain parts of his code) and (b) there is still a considerable amount of work to be done on the back end, my friend and I decided that a switch to a different tool will probably give us the boost we need. I’ll try to give a brief but comprehensive description of what we are trying to build, and I’d like you to help us find the best option we have.
Ok, so the site we’re building will be a place where people will publish *candies* for sale and other people will browse published *candies*, and if they find one they like, they will be able to contact the publisher. It is important to note that no transactions will take place over the site. We will only provide a means for publishers to show off their product and customers to contact the publishers. Roughly, the pages / functionality we need are:
home page
*candy* search based on *candy* properties (with pagination, filtering, sorting… etc)
individual *candy* page (as viewed by publisher (editable) and customer)
publisher page with contact info and product list (as viewed by publisher (editable) and customer)
login and registration functionality for publishers
maybe some static helper pages I forgot to mention
Now, many of them are already working (e.g. the search page, with all the desired features, is ready), but there are many things left that have to be done using zf2 and I have no idea how to do them…
What we need now is something that (1) is easy to pick up, (2) is fast to create stuff with and (3) has as much out-of-the-box functionality as possible. My friend (the front end guy) is leaning towards Expression Engine (money is not a problem), because being it a CMS he will be more comfortable with it too, and also because he noticed that we will be needing a lot of its built-in features.
His only concern is that it may not be as customizable as a framework. The framework solution we are considering is Codeigniter, as it fulfills criteria (1) and (2). Another option I am considering is Concrete5. I just watched a couple of introductory videos and I was amazed by the in-place page editing functionality and the block system they use to dynamically add content to pages.
What is your advice? What would you do in our position?
There is quite a big difference between a CMS and a framework. I think you should make that main decision first, before going to details like which one is better.
Also, using a different tool might get you up to speed, but the fact that you need to ask, suggests that you are not very experienced in those other tools as well. Other tools also have a learning curve, even the easy ones, and besides, you will have to start over completely, not being able to take along the work that is already finished.
So I would recommend to stick with zf for now. If you are going to make a switch, maybe Drupal or even a CMS like WordPress would be better and easier. Also, I think it is important to tell the boss what the status is and what problems you have. You can then decide on the best strategy, and let the boss define the most important features, so there will be at least something if not everything before Christmas. Better to have some functionalities working and being usable than having nothing at all.
If you're going to be doing all the work, then switch to a framework or CMS that you already know.
If you don't know any, either stay with Zend & take whatever time it takes (making it clear to the boss what's happening), or get out now.

Is there anything that PHP with CodeIgniter can do, which Ruby on Rails 3 can't do?

I and my partner are trying to develop a website, and we are arguing which language to use to build a website. We both have some experienced with PHP with Codeigniter 1.6++ as well as RoR, although my partner used rails when it was in RoR1, which now is RoR3.
He wants to use PHP with CodeIgniter because he knows whats going around more explicitly, while RoR does not seem to satisfy him.
I want to use RoR 3, because it takes less time, and there are many gems which I can use (devise for example).
He is kind of worrying that Ruby on Rails won't be easy to change some configuration in db or codes once the websites gets bigger and bigger.
I just hate to think about writing lines and lines of codes from the scratch with Codeigniter within 2 months.. although I think it was not easy to manage db tables, once things got settled in rails..
So, I have been wondering.. is there any big advantage one from the other?
As an avid user of both I would not say that this is a question of which framework is better or worse as they are apples and oranges.
CodeIgniter has not changed all that much since 1.6.x both of your experience is still valid and you will be able to code right from the get-go.
Rails 3 is wonderful but quite a lot has changed since Rails 1 (not that I was using it back then). I think your partner would have too much time scratching his head unless you are going to be there for him to constantly answer the WTF's that will inevitably come up.
To explain, I am a CodeIgniter man and have been for years. There is not much to it, no conventions and what code is there is simple, easy to extend and easy to ignore if you don't like it.
Recently starting working with Rails has been a great series of highs and lows but it's not always quicker. I spent 2 hours implementing a fully functional user system with fb, twitter, etc and had most of my controllers built, but since then I have spent hours trying to get various date formats to play nicely with the ActiveRecord, or trying to override create_at, manually set id's and all the other stuff that nobody ever mentions you are not allowed to do.
If you need to pump out some code fast then Rails might be an option. If you want fine grained control over everything and have a really lightweight base to build your application on then CodeIgniter really would be a better choice.
Or, look at FuelPHP which is a framework me and a few others are working on to combine the two into one Configuration over Convention framework with a command line utility to bring in some of Rails best features. Saves a fair bit of code and you'll feel right at home.
You can make most languages do most things at the end of the day, just a matter of how long it'll take you.
Hopefully you know your project well enough to know what the major stumbling blocks are liable to be and then match them against your combined knowledge to solve them within both languages. Which ever you both feel more comfortable with overall should win. You will be able to do it in either, just a matter of familiarity and confidence with the language (as well as availability of libraries for complex tasks).
Incidentally, this question would be better on https://softwareengineering.stackexchange.com/
From a management perspective, I can tell you that if you plan on growing in the future, you will pay twice as much and have to look twice as long for RoR programmers. PHP programmers on the other hand are easy to find and hire since the language is ubiquitous. I have known of companies that were bought and their product completely folded just to recruit the Ruby talent because it is so hard to find.

How can I evaluate how and whether to change php frameworks?

Need some guidance and ideally some first-hand experience.
We committed to a php framework which, shortly after we built the first rev of the product, stopped all development on the framework for about a year, forked twice, and doesn't really have a big community to begin with, meaning no plugins, tutorials, etcetera.
For another project we developed on rails and it has been night and day: a robust, continually developed framework and a healthy ecosystem of great plugins and a community that is active, growing, smart and helpful.
But the thought of junking all the sunk time and costs into the framework has been a huge hurdle that I'm not sure we're ready to cross, to go from php to rails. However, trying to work with this framework/s has had various level of frustration and investment.
Are there some ideas on how such a port could be less painful (staying in php but a similar OO framework that is growing/healthy?)
Suggestions on how we can continue to plow ahead with what we have?
Ideally someone who maybe found themselves in a similar situation would be super helpful for us to get our heads wrapped around it. The internal conversations we keep coming back to and I'd like to find a direction and move forward.
Thanks for some suggestions, or even questions, that will help us build a decision-matrix around it.
PS: The two or three people I've met on SO who actually have used this framework have been awesome, so I don't want it to be a neg on that. Size (of community) at least from our perspective does matter, and I think we just are seeing the comparison with Rails (perhaps that's an unfair comparison) So thank you!
No matter how far down the wrong road you've went, turn back. Sunk costs are already sunk.
I'd suggest Zend framework if you're going to stay using PHP. Make sure that you build unit tests as you start to refactor so you can be assured that your new code does the same as the old.
It really depends on the cost of changing vs. the cost of maintaining what you have. I don't think anyone here at stackoverflow can make that judgment call for you. I would suggest though, that it's easy to focus on the bad things and forget the things that work. People have a tendency to underestimate how much work they have to put into a port from one framework/technology to another. So if you plan to go that route, try to make it in as small steps as possible; Eg. take the smaller projects first, to get a feel for what it means to port a project over. This will give you time to pull the brake if it turns out to be unmanageable. It will also give you time to adjust to the new platform (Eg. you say that you aren't quite ready to make the step from php to ruby yet).
I can't answer all of your questions, but I was in a similar situation about 6 months ago. Long story short, I gave up on my own framework and moved to Symfony. I hated the idea of abandoning what I had worked on for so long and was so accustomed to, but I couldn't ignore the community aspect. Aside from plugins, I needed to be able to ask other people things about the framework - something that just wouldn't have been possible had I stuck with my own framework. The learning curve sucked (even though I knew it would be inevitable), but in the end I have no regrets after switching. I feel a lot more confident in my products now that I use a mature framework with a healthy community. I would suggest looking at the big name PHP frameworks, and seeing which fits your development style the best.
I would suggest taking a look at the Akelos framework, it is supposed to be a port of Rails to PHP and could significantly ease your framework transition.
As far as moving forward with your current implementation is concerned I would agree with the thought that it's time to switch if your current framework does not have a healthy user base and solid forward progress, this is a key factor for me in choosing a long-term framework solution. The other key factor I hold just as high is how close the framework's implementation, support features and eventual goals line up with the project I want to apply it to. With so many PHP frameworks available you now have the option of being very selective and should take full advantage of that.

Best starting point for a website with user management functionality

I'm about to start creating a new website that has standard user management (customers login and handling (change customer details etc) + my own functionality. I'm looking for the most efficient way to do it. I know PHP/CSS/Jquery quite well.
I have looked into Drupal as a starting point and found it too cumbersome for my needs.
CodeIgniter and PHPcake seems not to be efficient because I'll spend time learning the platform instead of developing (which I would love to do, but not currently).
It seems that what I need is a skeleton of PHP site that simply handles users functionality. Surprisingly I couldn't find one.
Could you recommend a starting point such as an open source website code that I can easily cut the user management part from? Or another option which is more streightforward than learning a new platform/framework?
To be honest, to get started in a framework like CodeIgniter you shouldn't need more than 5 to 15 minutes of learning time (a CI "skeleton" is extremely easy to do).
Yes, it may have plenty of tools/helpers/libraries but for the most part the learning curve is extremely shallow.
As to the users functionality, there are a couple of user-made libraries that may suit your needs - a comprehensive list with detailed functionality can be found here: what-code-igniter-authentication-library-is-best
Quite honestly, if you are going to use one of the existing platforms out there you are going to have to put the effort in to learning the architecture of it and then adapting to it to further develop on it.
Also, user management is a pain but really shouldn't take you THAT long to implement. If that's all you want, I'd say roll your own because then you are going to be that much more familiar with it. Anything that someone else has written you are going to have to learn about.
If all you want is authorization, start with Pear::Auth. It's probably a little less than you're looking for, but that may be preferable to a solution that's heavier than you desire.

Why do I need to use a popular framework?

I've been a PHP developer for many years now, with many tools under my belt; tools that I've either developed myself, or free-to-use solutions that I have learned to trust.
I looked into CodeIgniter recently, and discovered that they have many classes and helper routines to aid with development, yet saw nothing in the examples that I couldn't do just as easily with my own tools. Simple things like DB abstractions, Email helpers, etc. There was some interesting code relating to routes - mapping urls to the right controllers; but even that's not particularly difficult to code yourself if you've ever written an MVC style web app with pretty urls.
Even after looking through some of the other popular frameworks, I still see nothing that would be that much of a time-saver. Even looking at the forums, I see people struggling to get the tools to work for them. I do understand how they would be more useful for junior developers, since full system design skills take a while to understand and appreciate fully.
Yet, I'm often told that I should use an off-the-shelf framework to produce my solutions, but I still remain unconvinced. What's the real benefit to someone like myself? Am I just being elitist, or is this a common opinion?
Edit: Looking at some of the answers here, should I perhaps consider packaging up my toolset as its very own framework, writing some documentation and posting tutorials? If I'm hesitant to take on other's frameworks, would opening it up and getting more eyes on it help to improve my own skills/tools?
Frameworks have several advantages:
You don't have to write everything. In your case, this is less of a help because you have your own framework which you have accumulated over the years.
Frameworks provide standardized and tested ways of doing things. The more users there are of a given framework, the more edge cases that have been encountered and coded for. Your own code may, or may not, be battle hardened in the same way.
Others can be recruited onto a project with a standard framework and have access to the documentation, examples and experience with that framework. Your own snippets may or may not be fully documented or have examples of use... but isn't much chance that others are comfortable with them initially.
EDIT:
With regards to your idea of packaging up your own framework, the benefit of cleaning it up for public consumption can be larger than the benefit of getting others to use it.
The reason is simple: you will have to re-evaluate your assumptions about each component, how they fit together and how clear each piece is to understand. Once you publish your framework, your success will be strongly dependent on how easy it is to get up and running with.
Big wins with little effort are essential for adoption (those wins will encourage people to delve further into the framework). Ruby on Rails in an example of a framework that gives such big wins with little effort, and then has hidden layers of features that would have overwhelmed someone just getting started. (The question of the quality of RoR apps is not the point, the point is about adoption speed).
After people adopt a framework, it is about the ease of continued use. Little details like consistent parameter use patterns make all the difference here. If one class has many parameters on every method, while another has setters that are expected to be called before invoking methods, you will lose users because they can't get a "feel" for what is expected in a given case without resorting to the documents.
If both ease-of-adoption and ease-of-living-with issues are addressed properly, you only have to get lucky for people to adopt your framework. If those issues are not addressed properly, even an initial interest in the framework will wane quickly. The reason is that there are many frameworks: you will need to stand out to gain the advantages of having others using your kit (as they rightfully are as wary of your framework as you are of others).
Here's another reason not to create your own framework. Linus' Law - "Given enough eyeballs, all bugs are shallow". In other words, the more people who use a given framework, the more solid and bug-free it is likely to be.
Have you seen how many web frameworks there are for Java? Back in the day, it was fashionable for any half-decent developer/architect to write their own custom web framework. And at the end of the day, 95% of them looked like a custom implementation of Struts (the most popular Java web framework at the time). So they basically created a Struts clone that was: 1) proprietary; and 2) not as well documented and tested.
Let's face it - writing our own customer framework is fun, but what happens next? It becomes a maintenance burden to keep up with the framework yourself (or the poor soul who replaces you). And maintaining software is much, much more costly, especially when it comes to custom frameworks. Is the company in business to solve domain problems or in the business of maintaining frameworks?
I forget who said it, but I once heard a great quote - "The first rule to creating your own framework is: don't". Somebody else has probably gone through the effort of doing so and probably done the same work you would have done. Save yourself the time, effort, and testing.
There's many comments here as to the advantages of using a framework, and certainly I think in a good many cases they are perfectly correct.
HOWEVER
All frameworks come with the downside that they have a domain of problems that can be fitted into them. If your problem is well inside the scope of the domain then using a framework isn't an issue, and most of the time it's readily apparent if your problem is well outside the domain so you don't give it a thought. Issues arise when you try to force a problem into a framework that it just doesn't quite fit into or has some unusual non-standard feature - in which case you complete 90% of the code really fast then spend all the time you've saved figuring out how to bend or extend the framework so it can accomplish some obscure requirement. Because in these case your solution/extension has to plug into the framework it can often be more difficult to code than if you'd come to it independently.
In the wrong circumstances this can actually be disastrous. For example if a client asks for a project that you believe will fit into a framework solution and you quote accordingly, then after completing 90% you find the gotcha then you can be really up the creek, especially if it's some feature that the client is insistent upon (and it always is). These issues tend to arise because it is not always apparent from the word go where the gotchas might lie, particularly if you're using a framework you are less familiar with (and you have to from time to time).
This is really the same problem as arises with deploying any third party software in a project. Myself from experience I have no qualms about using frameworks or similar, but given the choice I will always go for the lightest, thinnest, wrapper I can find that will do what I need. That way I gain the advantages, whilst knowing that if issues do arise (and they are generally less likely to with a thinner wrapper) then figuring out how to work around them is likely to be simpler than learning an extensive code-base to the point where I can safely modify it.
The framework code is likely to be well-tested and relatively free of bugs. By using it you save yourself time testing/maintaining your own code to do the same thing.
And any time saved is good. Laziness pays off in programming.
One thing that you will be missing out on is all of the Validation that goes into a popular framework.
Your routines simply don't have the same exposure that the popular libraries have.
You may have a point.... however I wouldn't underestimate the power of many, as an example phpBB is as far as i'm concerned the bb solution to use. Why? Because there are many, many thousands of posts on their support boards and many people using it who are knowledgeable and can help people solve problems.
Therefore the only reason in your case to use a popular framework is the many others that use it, report bugs against it, fix it and support it. It'll be tricky to get the same coverage on your own libraries.
I would go against the grain here, and say, you should use your own custom framework, If the software you are building is the core of your business. As Joel would say, "Find the dependencies - and eliminate them". If you are just putting up a little website for your company, and you business isn't maintaining websites, then go ahead and use a framework. But when that website is your business, then you shouldn't depend on a framework from somebody else to let you get the job done.
I think the main reason is that when you use a common framework, there are a lot of people who are instantly familiar with your product.
Apart from that I think it's most important that whatever tools you use actually get the job done. If it happens to be familiar to other people, then that's a bonus.
I agree you should use your own custom framework. Not only is it easier for you to understand, but it provides the ultimate in job security!
Three reasons I can think of immediately:
A well-known framework will be familiar to other developers who may have to work on your project
A well-known framework will have resources like books, discussion boards, and other experts that can be used for finding out more information
Managers will often have a "don't reinvent the wheel" philosophy. In fact, existing solutions have probably solved the same problems that you'd discover if you create your own solution.
All of that said, there may still be a place for your own solutions. We wouldn't have so many frameworks (or scripting languages) to choose from if no one started something new.
Any experienced developer can build a framework -- the challenging part is convincing others that it's worth using. You'll need to create documentation and tutorials for it for those who plan to use or maintain it. You'll probably need to create a demo site to prove that its useful and actually works like it's supposed to.
That alone can be a considerable investment, not including bugs that could pop up in between. When its all said it done, it could be worth spending time learning another framework instead of making your own.
You mentioned CodeIgniter -- I personally feel like that's a pretty framework -- it doesn't get much more barebones than that.
What you essentially have is your own framework. So, it isn't a time-saver FOR YOU, because you have already spent the time to develop the framework. If you didn't have that to build from, it would certainly be easier and faster to use an existing framework than to roll your own.
What you need to look at is whether or not your framework is better than other options out there, and whether your familiarity with your own code outweighs having other eyes looking at it, and other people using it in enough different ways that the likelihood of any problems being found and corrected is much higher.
Also, if your framework is so much better than everyone elses', you might consider opening yours up to the community ;)
As you probably know: "time is money". So by using a popular framework with a lot of helpers, a lot of code examples on web and a big community you do more in less time.
Sometimes it if ok to use frameworks because you become more productive, but in some advanced and difficult projects it may happen so that the framework stays in your way and you have to find workarounds.
I think there is no definitive answer. You should put in balance the pros and cons and take the right decision for your project.
Usually I adopt popular frameworks very fast but not in critical parts of the projects and in time I extend their usage.
I think that if you don't see a need to use a framework then don't.
The reason I use a framework for example Django for python or Rails for Ruby or Webforms and MVC for ASP.net is because they make it easier and faster to write applications for them. In the case of Ruby and Python not using a framework for me would make me go crazy.
If you have something that works and don't see a need to use a framework I would say stick with what you feel is best. But, I would still keep up to date with frameworks.
I think they are more useful if you are starting from scratch and don't have the time to write your own. If you already have a codebase you developed over the years, they may be much less useful, but it may still be useful to take a look and see what they did.
For example, I am sure major game development shops are not using third-party tools, engines and frameworks, not because they are not sufficient, but they already have built their own since the 80's or whatever.
Plus, if you are using an off-the-shelf component, there is no way to exceed it in its particular area. If you need to be a market leader in a particular dimension, you should be building your own solution in that dimension, so you can lead. If you don't need this capability, using a third-party component that is just as good as yours may be a good solution as long as it is an easy transition. Time to train in the new tool and living with its idiosyncrasies may or may not be worth it though.
The other thing to consider is that if you can build something, you truly understand it. Otherwise, you don't. Now, you don't need to fully understand stuff to use them, so long as they "just work", but we all know how that goes... :)
Can you solve the problems you are given have with your code faster and more reliably than public frameworks?
If yes, then keep using your own.
If no, then find the framework that does a better job and run with it for that project.
It all comes down to which codebase gets the job done better(for the value of better given by the client. ;) )
Disadvantages.
Most frame works are not object orientated. (code igniter does show some promiss)
Most of the code is done via includes. trying to track down the problem is like pulling on a thread on a sweater, and having to unravel the entire garment to fully understand the creation.
Most frame works have poorly written documentation.
Most frame works try to do many many many things.
I find from my experience developing with frame works that it takes a good 3-6 months to get on top of the code base. And its only after that period of time that you will find out weather you are trying to fit a square peg into a round hole. Given that most php projects want to be finished before that period has elapsed, it will cost employers more to get any project using a big "frame work" to fruition.
Many of the php Frame works were written for php 4, and were written in a different enviroment. They have been extended greatly, but are showing their origins. The use of global constraints is particularly prevalent. I am hoping that php 6 puts most of them to death. Code igniter escapes most of this, but it is new, and has object orientated parts.
Some frame works have written code that is not needed, and causes problems.. eg: CAKE has a most excellent model view controller, but its session handling is a disaster. Unfortunately frame works are not written in a modular way. Often its an all or nothing option.
MOst programers "hack" the frame work to get it to do what they want. This leaves future programers sractching their heads. It also makes "upgrading" the frame work a impossibility.
I have yet to see a frame work that implements unit testing. (how do you know that you have not broken it).
give me a well written object any time. At least them you know the scope right off the bat.
Advantages are that it's already written and tested by multiple people therefore less likely to be bug prone.
Disadvantages are that it's not built specifically for your application and therefore will most likely perform worse.
All in all, I can't really see much reason to use one considering you already have your own...although it may be worth releasing that open source so others are able to bug check and recommend improvements.

Categories