Is there currently any technology that would separate storage from business logic and allow me, to easily switch from MySQL do MongoDB? (I assume I migrate the data myself, or start with an empty database after the switch).
I would like the change to be as easy as changing the configuration, the driver, and the db connection data.
I understand it is possible for PHP with Doctrine, switching between different RDBMS, but I'm interested in switching from any RDBMS to a NoSQL system.
I am focusing on PHP now - but if you know any solutions for other programming languages - I will be happy to learn about them.
I am assuming not a complex database, no transactions, no complex relations.
More background/details
I am writing a simple crawler that will visit websites, and read some data and save it to the DB. It is super simple and I might go with pdo_mysql for PHP only. I am considering an extra layer only to cover the situation in case I want to switch from MySQL to MongoDB one day - and I asking if this is even possible.
Update
I think that Laravel with its Eloquent supports MySQL out of the box, and with an extra plugin: https://github.com/jenssegers/laravel-mongodb supports MongoDB - I will check if this is truly transparent from the programmer's perspective. Unless someone has experience and knows right away?
A typical approach here is the DAO or Repository design pattern. In this pattern you provide a library which is responsible for persistence and returns business objects. The objects do not have persistence logic and get stashed in the Repository or retrieved from the repository.
This being said, that only works for business objects. The problems will come up when you want to redo reporting or the like.... Here the differences between an RDBMS and a NoSQL solution will bite you hard.
To be sure I don't know what a use case is given that "what do you want to do with your data" is a massive concern in selecting between these.
Related
I'm staring to build a system for working with native languages, tags and such data in Yii Framework.
I already choose MongoDB for storing my data as I think it feets nicelly and will get better performance with less costs (the database will have huge amounts of data).
My question regards user authentication, payments, etc... This are sensitive bits of information and areas where I think the data is relational.
So:
1. Would you use two different db systems? Should I need them or I'm I complicating this?
2. If you recommend the two db approach how would I achieve that in Yii?
Thanks for your time!
PS: I do not intend this question to be another endless discussion between the relational vs non-relational folks. Having said that I think that my data feets mongo but if you have something to say about that go ahead ;)
You might be interested in this presentation on OpenSky's infrastructure, where MongoDB is used alongside MySQL. Mongo was utilized mainly for CMS-type data where a flexible schema was useful, and they relied upon MySQL for transactions (e.g. customer orders, payments). If you end up using the Doctrine library, you'll find that the ORM (for SQL databases) and MongoDB ODM share a similar API, which should make the experimentation process easier.
I wouldn't shy away from using MongoDB to store user data, though, as that's often a record that can benefit from embedded document storage (e.g. storing multiple billing/shipping addresses within a single user document). If anything, Mongo should be flexible enough to enable you to develop your application without worrying about schema changes due to evolving product requirements. As those requirements become more clear, you'll be able to make a decision based on the app's performance needs and types of database queries you end up needing.
There is no harm in using multiple databases (if you really need), many big websites are using multiple databases so go a head and start your project.
I am in the process of picking a PHP framework for a web application I am starting. I have never really used a framework in the past but with this project there is a great need.
I have been debating between the usual suspects; CakePHP, Zend Framework and Symfony. I have been going back and forth about which framework will work best for me and this project. I am leaning towards CakePHP but I am still researching.
My question is not what framework is best. I know there is no real answer to that and there are tons of posts related to this subject. My question is related to the Model and ORM. I have read a lot about ORMs being slow and I am concerned about speed. I am very comfortable writing SQL and in the past have tried to keep all of my database interactions in stored procedures.
I am looking for some feedback about using CakePHP's ORM or Doctrine with Zend or Symfony as apposed to keeping everything in stored procedures. I know stored procedures are going to be faster but what else will I loose if I do not use an ORM? I understand that an ORM will give me database abstraction but in my mind that just helps people who do not write SQL. I also know that I do not know enough about ORMs.
If anyone can give me some feedback about this and which framework might be best based on using or not using an ORM.
Thanks for any help in advance.
0) Bang for effort
The key advantage with an ORM is that you don't spend as much time dealing with the persistence layer. A pre-written ORM ( I have worked with EclipseLink) will provide a ton of things you probably won't get in custom written stored procs. I think it's worth thinking about how much time you want to spend writing your persistence layer.
1) Caching
All the major ORMs provide multi-level distributed caches. Combined with Named/Predefined queries you can get SQL queries that don't actually have to go to the database. This can give you excellent performance.
2) Abstraction
ORMs allow you to define your table layout in one location and then they manage all the painful mapping between columns/tables and objects. Some will allow you to remap column, table and schema names without changing any code at all. If you work with people who like to change things around this can really simplify things.
3) Speed
Some ORMs can have bad performance, but it really is based on how you use it. I find that you tend to end up over-querying for things. On the other hand, you get things like built-in query profilers. You can write custom SQL for queries if you find you aren't getting the performance you need.
Mark Robinson gives a great response. I'm just going to back up what he says by giving our experience with Doctrine2.
I chose to use Doctrine2 as our ORM with Zend Framework a little while back. Our project is still being developed, but choosing D2 has been a decision we've not regretted one bit. Whilst you still need to give a lot of thought to your data architecture, D2 gives you the flexibility to be able to modify that model at a later date if needs be. It allowed us to try things out quickly in the early stages and the room to grow and change later when we decided that things weren't quite right - it happens.
In relation to Mark's point about abstraction. One of the other things I love about D2 is that we're working with plain old PHP objects. Don't underestimate the power of being able to think in terms of objects - both for the people responsible for modelling the data and the developers who work with the data - it'll make your life easier, trust me. Also, having inline documentation of the ORM mapping (if you choose the docblock approach) is nice.
Right, performance. As Mark says, there are ways and means to speed things up - but there's always going to be some overhead. Whenever you introduce another software layer, there'll be some performance hit, but it's a tradeoff. For us, the tradeoff - the advantages of using the ORM vs performance - is worth it. We'd spend more time debugging code and not getting things done without the ORM.
Anyway, D2 can help you with caching for queries, results and metadata. Whilst you probably just want an array cache during development, it's great that the facility for things like APC, memcache etc. is there when you go to test and deploy. You could even develop your own if you're brave.
http://www.doctrine-project.org/docs/orm/2.0/en/reference/caching.html
Hope that helps, I've probably missed stuff, but if you have any questions just fire them in and I'll do my best.
A framework implements mainly three kinds of features :
the flow between "getting a request" and "rendering a page". That's were you put things like MVC, router, etc...
the way to manage your model and it's persistence. That's where you see acronyms like ORM, DBAL, DAO
Components. Features, often working also standalone. Like Xml parsing, i18n handling, pdf generation...
When you choose your framework, it in facts means that you choose 1). It's the thing you will certainly have to stick with, it's the flow of your application. 2) and 3) ? You can integrate those you prefer. As an example, i'm on Zend Framework with most of it's components, but use Doctrine ORM 2 and Symfony's Dependency injection. A friend of mine is on Symfony 2, uses Doctrine ORM too, but does it's pdf generation and mail management with Zend's related components.
The other thing you need to know if that currently there is a "second generation" of php frameworks/orm's, (re)written to take advantage of the new php 5.3 features, and/or to solve the general performance/coupling issues they (nearly) all had. Some of them are production ready, some are still under development :
Doctrine ORM 2 (production ready)
Symfony 2 (production ready)
CakePhp 2 (in RC 2 currently, but by the time your project is ready it should be stable)
Zend Framework 2 (still under active development, but normally not for so long)
FLOW 3 (beta2, should be ready soon too)
For the ORM part, i'll recommend using one, especially Doctrine's. #Mark and #iainp999 explained perfectly why.
ORMs are for programmers who don't "grock" SQL!
Yes ORMs make it easier (or at least require less lines of code) to write simple CRUD stuff, but when you get to more complex requirements its like trying to write SQL with a piece of wet spaghetti from ten feet away.
So stick with SQL.
Its worth looking at something like "SQLMap" which is ORM starting from the "R"elational side of the mapping (most try to map an "O"bject on to a table). This will allow you to write the SQL yourself and generate the appropriate "helper" classes to easily access the results in your program.
We are developing a substantial PHP web application using the Zend framework.
At this time, the product is starting to stabilize and we are moving away from quick-and-dirty setups in order to avoid regressions. In particular, we are now working on top of a single shared database that everyone edits. We want to get rid of this ASAP.
The Zend framework does not seem to support the classic RAD (Djangoish) mechanism where you define your data models and then it creates the tables for you. So we are thinking of using an external ORM tool that will do this.
We could have our schemas and initial fixtures defined in plain SQL but this is a) verbose b) error prone c) too low level and d) problematic because we must maintain different versions for every supported database backend.
So we are thinking of using an ORM like Doctrine or Propel to define our models and create tables with their initial data using the chosen framework's dialect. The application uses the Zend tool for ORMing so consistency between both tools would have to be maintained manually, but since changes are more gradual now this doesn't seem like much of a problem.
So, far, we are evaluating Doctrine and Propel for this task. Any suggestions about other ORMs that we missed? Maybe a different approach altogether for the task at hand?
Thanks!
Gonzalo
I really love RedBean. You keep your database models in pure php and it keeps track of everything.
RedBeanPHP is an open source ORM tool
for PHP. It focuses on simplicity and
ease of use. What makes RedBean unique
is that it creates your database
schema on-the-fly. It scans your data
and adjusts the column types to fit
your object properties. If your models
are stabilized you can freeze the
database. This way RedBean is easy to
develop with but is also extremely
fast on production servers.
Since I've found this ORM, I don't use doctrine any more.
http://redbeanphp.com/
I have a class that helps me to handle users.
For example:
$user = new User("login","passw");
$name = $user->getName();
$surname = $user->getSurname();
$table = $user->showStats();
All these methods have SQL queries inside. Some actions require only one sql queries, some - more than one. If database structure changes - it will be difficult to change all queries (class is long). So I thought to keep SQL queries away from this class. But how to do this?
After reading this question I've known about Stored Procedures. Does it mean, that now one action requires only one SQL query (call of Stored Procedure)? But how to organize separation sql from php? Should i keep sql-queries in an array? Or may be it should be an sql-queries class. If yes, how to organise this class (maybe what pattern I should learn)
This is a surprisingly large topic, but I have a few suggestions to help you on your way:
You should to look into object-relational mapping, in which an object automatically generates SQL queries. Have a look at the Object-Relational Mapping and Active Record articles for an overview. This will keep your database code minimal and make it easier if your table structure changes.
But there is no silver bullet here. If your schema changes you will have to change your queries to match. Some people prefer to deal with this by encapsulating their query logic within database views and stored procedures. This is also a good approach if you are consistent, but keep in mind that once you start writing stored procedures, they are going to be tied heavily to the particular database you are using. There is nothing wrong with using them, but they are going to make it much more difficult for you to switch databases down the road - usually not an issue, but an important aspect to keep in mind.
Anyway, whatever method you choose, I recommend that you store your database logic within several "Model" classes. It looks like you are doing something similar to this already. The basic idea is that each model encapsulates logic for a particular area of the database. Traditionally, each model would map to a single table in the DB - this is how the Ruby on Rails active record class works. It is a good strategy as it breaks down your database logic into simple little "chunks". If you keep all of the database query logic within a single file it can quickly grow out of control and become a maintenance nightmare - trust me, I've been there!
To get a better understanding of the "big picture", I recommend you spend some time reading up on web Model-View-Controller (MVC) architecture. You will also want to look at the established PHP MVC frameworks, such as CodeIgniter, Kohaha, CakePHP, etc. Even if you do not use one - although I recommend you do - it would be helpful to see how these frameworks organize your code.
I would say you should look into implementing the "repository" design pattern in your code.
A good answer to how to implement this would be too long for this space, so I'll post a couple of PHP-oriented references:
travis swicegood -- Repository Pattern in PHP
Jon Lebensold -- A Repository Pattern in PHP
You are on the right lines if you use separation of concerns to separate your business logic from your data access logic you will be in a better place.
Judging by your "there are already 2K lines of code" statement, you're either maintaining something, or midway through developing something.
Both Faust and Justin Ethier make good recommendations - "how should I separate my database access from my application code" is one of the oldest, and most-answered, questions in web development.
Personally, I like MVC - it's pretty much the default paradigm for web development, it balances maintainability with productivity, and there are a load of frameworks to support you while you're doing it.
You may, of course, decide that re-writing your app from scratch is too much effort - in which case the repository pattern is a good halfway house.
Either way, you need to read up on refactoring - getting from where you are to where you want to be is going to be tricky. I recommend the book by Fowler, as a starter.
Could you explain more about why your database schema may change? That's usually a sign of trouble ahead.....
I am thinking about writing a quick chat application for a client to help them solve some of their communication needs. Clearly, writing a simple chat is no brainer, but the company have serious scaling needs, so it is probably a good idea to build the service on a noSQL storage from the beginning.
Besides the obvious lack of transactions, which isn't one of our concerns, is it a good idea to use a noSQL storage for a chat?
MongoDB should be good enough if you're after scalability and performance. Most SQL engines would be overkill for this stuff. I doubt if you need complex data aggregation and other queries for chat data. Even with that, MongoDB has map-reduce capability to help you along.
NoSQL ist used if you have no fixed data model, this applies to document oriented applications where you have to store objects and documents where each one may have a different structure.
I don't think this is the case in your situation, since a chat log has a well defined fixed data model for example (user, time, text). I think a traditional SQL database may be the right fit for you. If used on client side only, SQLite will be the best fit, since there is no need to install or configure, simply redistribute the SQLite dll. Also the footprint is very small.
I would say no. SQLite is included in PHP... why not just use that? Or better still, why not use one of the hundreds of chat applications that already exist, and save yourself a whole load of development time.