Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I would like to know whether Redbean ORM can be used for performance oriented scenarios like social networking web apps or no and is it stable even if thousands of data are pulled by multiple users at same time. Also I'd like to know whether Redbean consumes more memory space.
Can anyone offer a comparison study of Doctrine-Propel-Redbean?
I feel Tereško's answer is not quite right.
Firstly it does not address the original question. It's indeed a case against ORMs, and I agree with the problems described in his answer. That's why I wrote RedBeanPHP. Just because most ORMs fail to make your life a bit easier does not mean the concept of an object relational mapping system is flawed. Most ORMs try to hide SQL, which is why JOINs get so complex; they need to re-invent something similar in an object oriented environment. This is where RedBeanPHP differs, as it does not hide SQL. It creates readable, valid SQL tables that are easy to query. Instead of a fabricated query language RedBeanPHP uses plain old SQL for record and bean retrieval. In short; RedBeanPHP works with SQL rather than against it. This makes it a lot less complex.
And yes, the performance of RedBeanPHP is good. How can I be so sure? Because unlike other ORMs, RedBeanPHP distinguishes between development mode and production mode. During the development cycle the database is fluid; you can add entries and they will be added dynamically. RedBeanPHP creates the columns, indexes, guesses the data types etc. It even stretches up columns if you need more bytes (higher data type) after a while. This makes RedBeanPHP extremely slow, but only during development time when speed should not be an issue. Once you are done developing you use freeze the database with a single mode specifier R::freeze() and no more checks are done. What you are left with is a pretty straight forward database layer on your production server. And because not much is done, performance is good.
Yes, I know, I am the author of RedBeanPHP so I am biased. However I felt like my ORM was being viewed in the same light as the other ORMs, which prompted me to write this. If you want to know more, feel free to consult the RedBeanPHP website, and here is a discussion on performance.
At our company we use RedBeanPHP for embedded systems as well as financial business systems, so it seems to scale rather well.
Together, me and the RedBeanPHP community are sincerely trying to make the ORM world a better place; you can read the mission statement here.
Good luck with your project and I hope you find the technical solution you are looking for.
#tereško if tis possible, can you give the pros and cons of orm with respect to pure sql according to your experience and also i will google the topic at same time. – Jaison Justus
Well .. explaining this in 600 characters would be hard.
One thing I must clarify: this is about ORMs in PHP, though i am pretty sure it applies to some Ruby ORMs too and maybe others.
In brief, you should avoid them, but if you have to use an ORM, then you will be better of with Doctrine 2.x , it's the lesser evil. (Implements something similar to DataMapper instead of ActiveRecord).
Case against ORMs
The main reason why some developers like to use ORMs is also the worst thing about them: it is easy to do simple thing in ORM, with very minor performance costs. This is perfectly fine.
1. Exponential complexity
The problem originates in people to same tool for everything. If all you have is a hammer (..) type of issue. This results in creating a technical debt.
At first it is easy to write new DB related code. And maybe, because you have a large project, management in first weeks (because later it would case additional issues - read The Mythical Man-Month, if interested in details) decides to hire more people. And you end up preferring people with ORM skills over general SQL.
But, as project progresses, you will begin to use ORM for solving increasingly complex problems. You will start to hack around some limitations and eventually you may end up with problems which just cannot be solved even with all the ORM hacks you know ... and now you do not have the SQL experts, because you did not hire them.
Additionally most of popular ORMs are implementing ActiveRecord, which means that your application's business logic is directly coupled to ORM. And adding new features will take more and more time because of that coupling. And for the same reason, it is extremely hard to write good unit-tests for them.
2. Performance
I already mentioned that even simple uses of ORM (working with single table, no JOIN) have some performance costs. It is due to the fact that they use wildcard * for selecting data. When you need just the list of article IDs and titles, there is no point on fetching the content.
ORMs are really bad at working with multiple tables, when you need data based on multiple conditions. Consider the problem:
Database contains 4 tables: Projects, Presentations, Slides and Bulletpoints.
Projects have many Presentations
Presentations have many Slides
Slides have many Bulletpoitns
And you need to find content from all the Bulletpoints in the Slides tagged as "important" from 4 latest Presentations related to the Projects with ids 2, 4 and 8.
This is a simple JOIN to write in pure SQL, but in any ORM implementation, that i have seen, this will result in 3-level nested loop, with queries at every level.
P.S. there are other reasons and side-effects, but they are relatively minor .. cannot remember any other important issues right now.
I differ from #tereško here - ORMs can make database queries easier to write and easier to maintain. There is some great work going into Propel and Doctrine, in my opinion - take advantage of them! There are a number of performance comparisons on the web, and check out NotORM as well (I've not used it but they do some comparisons to Doctrine, if I recall correctly).
If you get to a point where your throughput requires you to do raw SQL then optimise at that point. But in terms of reducing your bug count and increasing your productivity, I think that your savings will fund a better server anyway. Of course, your mileage may vary.
I don't know RedBean, incidentally, but I am mildly of the view that Propel is faster than Doctrine in most cases, since the classes are pre-generated. I used Propel when it was the only option and have stuck with it, though I certainly wouldn't be averse to using Doctrine.
2018 update
Propel 2 is still in alpha after a number of years, and is in need of a number of large refactoring projects, which sadly were not getting done. Although the maintainers say that this alpha is good to use in production, since it has good test coverage, they have now started on Propel 3. Unfortunately, this has not actually had any releases yet, at the time of my writing this, despite the repository being a year old.
While I think Propel was a great project, I wonder if it is best to use something else for the time being. It could yet rise from the ashes!
I would go with "Horses for Courses" situation that utilizes a mix and match of both the worlds. I have built few large scale applications using RedBean, so my comment will focus purely on RedBean and not on other ORMs.
IS RedBean ORM SLOW?
Well, it depends on how you use it. In certain scenarios, it's faster than traditional query because RedBean cache the result for few seconds. Reusing the query will produce result faster. Have a look at the log using R::debug(true); It always shows
"SELECT * FROM `table` -- keep-cache"
Scenario 1: Fetching All (*)
In RedBean if you query
$result = R::findOne('table', ' id = ?', array($id));
This is represented as
$result= mysql_query("Select * from TABLE where id =".$id);
You may argue that if the table is having multiple columns why should you query (*).
Scenario 2: Single column
Fetching a single column
R::getCol( 'SELECT first_name FROM accounts' );
Like i mentioned "Horses for Courses", developers should not simply rely on FindOne, FindAll, FindFirst, FindLast but also carefully draft what they really need.
Scenario 3: Caching
When you don't need caching, you can disable at application level which isn't an ideal situation
R::$writer->setUseCache(true);
RedBean suggests that if you don't want to disable caching at the application level you should use traditional query with no-cache parameter like $result = R::exec("SELECT SQL_NO_CACHE * FROM TABLE");
This perfectly solves the problem of fetching real-time data from table by completely discarding query cache.
Scenario 4: Rapid Development
Using ORM makes your application development really fast, developers can code using ORM 2-3x faster than writing SQL.
Scenario 5: Complex Queries & Relationships
RedBean presents a really nice way of implementing complex queries and one-to-many or many-to-many relationships
Plain SQL for complex queries
$books = R::getAll( 'SELECT
book.title AS title,
author.name AS author,
GROUP_CONCAT(category.name) AS categories FROM book
JOIN author ON author.id = book.author_id
LEFT JOIN book_category ON book_category.book_id = book.id
LEFT JOIN category ON book_category.category_id = category.id
GROUP BY book.id
' );
foreach( $books as $book ) {
echo $book['title'];
echo $book['author'];
echo $book['categories'];
}
OR RedBean way of handling many-t-to-many relationships
list($vase, $lamp) = R::dispense('product', 2);
$tag = R::dispense( 'tag' );
$tag->name = 'Art Deco';
//creates product_tag table!
$vase->sharedTagList[] = $tag;
$lamp->sharedTagList[] = $tag;
R::storeAll( [$vase, $lamp] );
Performance Issues
The arguments like ORMs are typically slow, consumes more memory and tends to make an application slow. I think they are not talking about RedBean.
We have tested it with MySQL and Postgres both, trust me performance was never a bottleneck.
There is no denying that ORMs adds up little overhead and tend to make your application slower ( just a little ). Using ORM is primarily a trade-off between developer time and slightly slower runtime performance. My strategy is to first build the application end-to-end with the ORM then based on test cases, tweak the speed critical modules to use straight data access.
Related
I'm not a fan of ORMs. The great SO contributor Bill Karwin's thoughts represent my feelings fairly well. See here and here (and read his specific responses on Glenn Block's list of advantages of an ORM).
I think just using good solid SQL in a repository pattern or/and DAL is the best approach for creating an app that will do more than just the small examples (with simple databases) I see in tutorial after tutorial. (and I never want to see another Posts::model()->findAll(); example please!). It seems all the energy is spent on how to do a cool one-liner, instead of real scenario after scenario. Try complex joins, self joins, heavy aliasing, concatenation, nested or's/ands, aggregate functions on results or in sorting, sub-queries, composite keys, table prefixes, and you start to get my point.
However I know they have become better over the years and I like Laravel 4 so much that I figured I finally need to give them another try. But after evaluating my needs, it appears many of my queries will need to be ran "raw"DB::select(DB::raw because Eloquent can't handle them properly or it becomes more complex to express it as opposed to using SQL directly.
What are the advantages to using Eloquent/Query Builder when many of the queries must be ran "raw", as opposed to using SQL purely and adding any methods in the repository or DAL to cover for all your data access needs? Because it seems to me that raw queries in Laravel are the same as pure SQL and I don't get any of the advantages the ORM offers in those situations. I would be forced to create my own methods for things I'd be already using in the ORM like accessors, mutators, timestamps, soft deleting, etc for those DB::raw situations. Also the advantage of swapping out database engines compatible with Eloquent would be lost.
I'm just ramping up on laravel, but here are my thoughts...
I view the strength in an ORM is in substituting it in lieu of executing raw sql queries. My feeling is that if you are already having to write raw sql queries, shelf the ORM and do it all yourself. It's likely the model is complex enough that you'll run into efficiency problems long term anyway.
The second issue that comes to mind is the context switching between using the ORM and the raw queries. Karwin, in his blog posts, mentions that most people use ORMs "inexpertly". My guess is that if you are using it for very light data access/queries you won't develop a deep understanding of it and will fall into that category. ORMs can be very complex and it's tough to get a good feel for how to use them appropriately.
Note: To date, I've not found an ORM that I liked enough to implement on a wide scale, but I've got a simple project that I'm going to try and use one with as I work with Laravel.
Consider the creation of high traffic PHP web-site with many parallel users. Which is the best possible MySQL abstraction (ORM or OODBMS) in terms of effectiveness (15-20 database tables with sum of about 100000 items and JOIN queries between no more than 4 tables)?
Somewhere I heard that Doctrine libraries are appropriate or I should use framework like Zend? Which of these database solutions are build over PDO and don't require much learning (at this time I'm using pure PHP)?
Regardless of the DB solution you should look at using a system like MemCached. With the proper caching strategy you will significantly reduce the load your databases are putting on your server.
There is a PHP API for memcached here
ORM or any data modeling layer will never get you better performance. Their sole purposes is to make your development time faster and easier to maintain. They are notoriously bad at decision making when it comes to actually using relationships appropriately and end up querying all tables in order to find the correct data. At that level of complex queries you are not going to be able to abstract away these relationships without sacrificing performance.
MySQL is fine for up to a couple million records at least (I've used it for over 100 million in a single table). For performance sake you generally want to have at least a master/slave setup and some method of distributing reads between them. The database will almost always be the limiting factor in performance. You can always add in more web servers and get a load balance in front of them to solve the other side of things but the database setup is always a little harder to maintain.
You have to think about why you want to use an ORM. If its for development reasons, that's fine, but be coginiscent that your performance will suffer. Otherwise stick to queries. An ORM adds a third layer of code to deal with and learn. If you know PHP and MySQL, do you need to learn a 3rd language to use them effectively? Most often the answer is no.
You have many options to choose from but be aware that at some point the framework/ORM you choose will not behave the way you want it to and to get it to behave to your desires you will have to do a lot of searching and digging through code. It's the classic problem - save time up front and pay for it later or spend time up front with no possible payoff later.
ORM solutions will be able to optimize some aspects, if you cache query data and use the object API in a planned and deliberate way.
Column / document[nosql : hbase,mongo] databases will improve performance if you have lots (millions+) of records, and are still growing.
Memcached will help if you have a lot of spare memory and especially if there are a lot of repetitious queries being run.
I am in the process of picking a PHP framework for a web application I am starting. I have never really used a framework in the past but with this project there is a great need.
I have been debating between the usual suspects; CakePHP, Zend Framework and Symfony. I have been going back and forth about which framework will work best for me and this project. I am leaning towards CakePHP but I am still researching.
My question is not what framework is best. I know there is no real answer to that and there are tons of posts related to this subject. My question is related to the Model and ORM. I have read a lot about ORMs being slow and I am concerned about speed. I am very comfortable writing SQL and in the past have tried to keep all of my database interactions in stored procedures.
I am looking for some feedback about using CakePHP's ORM or Doctrine with Zend or Symfony as apposed to keeping everything in stored procedures. I know stored procedures are going to be faster but what else will I loose if I do not use an ORM? I understand that an ORM will give me database abstraction but in my mind that just helps people who do not write SQL. I also know that I do not know enough about ORMs.
If anyone can give me some feedback about this and which framework might be best based on using or not using an ORM.
Thanks for any help in advance.
0) Bang for effort
The key advantage with an ORM is that you don't spend as much time dealing with the persistence layer. A pre-written ORM ( I have worked with EclipseLink) will provide a ton of things you probably won't get in custom written stored procs. I think it's worth thinking about how much time you want to spend writing your persistence layer.
1) Caching
All the major ORMs provide multi-level distributed caches. Combined with Named/Predefined queries you can get SQL queries that don't actually have to go to the database. This can give you excellent performance.
2) Abstraction
ORMs allow you to define your table layout in one location and then they manage all the painful mapping between columns/tables and objects. Some will allow you to remap column, table and schema names without changing any code at all. If you work with people who like to change things around this can really simplify things.
3) Speed
Some ORMs can have bad performance, but it really is based on how you use it. I find that you tend to end up over-querying for things. On the other hand, you get things like built-in query profilers. You can write custom SQL for queries if you find you aren't getting the performance you need.
Mark Robinson gives a great response. I'm just going to back up what he says by giving our experience with Doctrine2.
I chose to use Doctrine2 as our ORM with Zend Framework a little while back. Our project is still being developed, but choosing D2 has been a decision we've not regretted one bit. Whilst you still need to give a lot of thought to your data architecture, D2 gives you the flexibility to be able to modify that model at a later date if needs be. It allowed us to try things out quickly in the early stages and the room to grow and change later when we decided that things weren't quite right - it happens.
In relation to Mark's point about abstraction. One of the other things I love about D2 is that we're working with plain old PHP objects. Don't underestimate the power of being able to think in terms of objects - both for the people responsible for modelling the data and the developers who work with the data - it'll make your life easier, trust me. Also, having inline documentation of the ORM mapping (if you choose the docblock approach) is nice.
Right, performance. As Mark says, there are ways and means to speed things up - but there's always going to be some overhead. Whenever you introduce another software layer, there'll be some performance hit, but it's a tradeoff. For us, the tradeoff - the advantages of using the ORM vs performance - is worth it. We'd spend more time debugging code and not getting things done without the ORM.
Anyway, D2 can help you with caching for queries, results and metadata. Whilst you probably just want an array cache during development, it's great that the facility for things like APC, memcache etc. is there when you go to test and deploy. You could even develop your own if you're brave.
http://www.doctrine-project.org/docs/orm/2.0/en/reference/caching.html
Hope that helps, I've probably missed stuff, but if you have any questions just fire them in and I'll do my best.
A framework implements mainly three kinds of features :
the flow between "getting a request" and "rendering a page". That's were you put things like MVC, router, etc...
the way to manage your model and it's persistence. That's where you see acronyms like ORM, DBAL, DAO
Components. Features, often working also standalone. Like Xml parsing, i18n handling, pdf generation...
When you choose your framework, it in facts means that you choose 1). It's the thing you will certainly have to stick with, it's the flow of your application. 2) and 3) ? You can integrate those you prefer. As an example, i'm on Zend Framework with most of it's components, but use Doctrine ORM 2 and Symfony's Dependency injection. A friend of mine is on Symfony 2, uses Doctrine ORM too, but does it's pdf generation and mail management with Zend's related components.
The other thing you need to know if that currently there is a "second generation" of php frameworks/orm's, (re)written to take advantage of the new php 5.3 features, and/or to solve the general performance/coupling issues they (nearly) all had. Some of them are production ready, some are still under development :
Doctrine ORM 2 (production ready)
Symfony 2 (production ready)
CakePhp 2 (in RC 2 currently, but by the time your project is ready it should be stable)
Zend Framework 2 (still under active development, but normally not for so long)
FLOW 3 (beta2, should be ready soon too)
For the ORM part, i'll recommend using one, especially Doctrine's. #Mark and #iainp999 explained perfectly why.
ORMs are for programmers who don't "grock" SQL!
Yes ORMs make it easier (or at least require less lines of code) to write simple CRUD stuff, but when you get to more complex requirements its like trying to write SQL with a piece of wet spaghetti from ten feet away.
So stick with SQL.
Its worth looking at something like "SQLMap" which is ORM starting from the "R"elational side of the mapping (most try to map an "O"bject on to a table). This will allow you to write the SQL yourself and generate the appropriate "helper" classes to easily access the results in your program.
I know there already are a lot of posts floating on the web regarding this topic.
However, many people tend to focus on different things when talking about it. My main goal is to create a scalable web application that is easy to maintain. Speed to develop and maintain is far more appreciated BY ME than raw performance (or i could have used Java instead).
This is because i have noticed that when a project grows in code size, you must have maintainable code. When I first wrote my application in the procedural way, and without any framework it became a nightmare only after 1 month. I was totally lost in the jungle of spaghetti code lines. I didn't have any structure at all, even though i fought so badly to implement one.
Then I realized that I have to have structure and code the right way. I started to use CodeIgniter. That really gave me structure and maintainable code. A lot of users say that frameworks are slowing things down, but I think they missed the picture. The code must be maintainable and easy to understand.
Framework + OOP + MVC made my web application so structured so that adding features was not a problem anymore.
When i create a model, I tend to think that it is representing a data object. Maybe a form or even a table/database. So I thought about ORM (doctrine). Maybe this would be yet another great implementation into my web application giving it more structure so I could focus on the features and not repeating myself.
However, I have never used any ORM before and I have only learned the basics of it, why it's good to use and so on.
So now Im asking all of you guys that just like me are striving for maintainable code and know how important that is, is ORM (doctrine) a must have for maintainable code just like framework+mvc+oop?
I want more life experience advices than "raw sql is faster" advices, cause if i would only care about raw performance, i should have dropped framework+mvc+oop in the first place and kept living in a coding nightmare.
It feels like it fits so good into a MVC framework where the models are the tables.
Right now i've got like 150 sql queries in one file doing easy things like getting a entry by id, getting entry by name, getting entry by email, getting entry by X and so on. i thought that ORM could reduce these lines, or else im pretty sure that this will grow to 1000 sql lines in the future. And if i change in one column, i have to change all of them! what a nightmare again just thinking about it. And maybe this could also give me nice models that fits to the MVC pattern.
Is ORM the right way to go for structure and maintainable code?
Ajsie,
My vote is for an ORM. I use NHibernate. It's not perfect and there is a sizable learning curve. But the code is much more maintainable, much more OOP. Its almost impossible to create an application using OOP without an ORM unless you like a lot of duplicate code. It will definitely eliminate probably the vast majority of your SQL code.
And here's the other thing. If you're are going to build an OOP system, you'll end up writing your own O/R Mapper anyway. You'll need to call dynamic SQL or stored procs, get the data as a reader or dataset, convert that to an object, wire up relationships to other objects, turn object modifications into sql inserts/updates, etc. What you write will be slower and more buggy than NHibernate or something that's been in the market for a long while.
Your only other choice really is to build a very data centric, procedural application. Yes it may perform faster in some areas. I agree that performance IS important. But what matters is that its FAST ENOUGH. If you save a few milliseconds here and there doing procedural code, your users will not notice the performance increase. But you 'll notice the crappy code.
The biggest performance bottle-necks in an ORM are in the right way to pre-fetch and lazy-load objects. This gets into the n-query problems with ORMs. However, these are easily solved. You just have to performance tune your object queries and limit the number of calls to the database, tell it when to use joins, etc. NHibernate also supports a rich caching mechanism so you don't hit the database at all at times.
I also disagree with those that say performance is about users and maintenance is about coders. If your code is not easily maintained, it will be buggy and slow to add features. Your users will care about that.
I wont say every application should have an ORM, but I think most will benefit. Also don't be afraid to use native SQL or stored procedures with an ORM every now and then where necessary. If you have to do batch updates to millions of records or write a very complex report (hopefully against a separate, denormalized reporting database) then straight SQL is the way to go. Use ORMs for the OOP, transactional, business logic and C.R.U.D. stuff, and use SQL for the exceptions and edge cases.
I'd recommend reading Jeffrey Palermo's stuff on NHibernate and Onion Architecture. Also, take his agile boot camp or other classes to learn O/R Mapping, NHibernate and OOP. Thats what we use: NHibernate, MVC, TDD, Dependency Injection.
A lot of users say that frameworks are
slowing things down, but I think they
missed the big picture. The code MUST
BE MAINTAINABLE and EASY TO
UNDERSTAND.
A well-structured, highly-maintainable system is worthless if its performance is Teh Suck!
Maintability is something which benefits the coders who construct an application. Raw performance benefits the real people who use the app for their work (or whatever). So, whose concerns ought to be paramount: those who build the system or those who pay for it?
I know it's not as simple as that, because the customer will eventually pay for a poorly structured system - perhaps more bugs, certainly more time to fix them, more time to implement enhancements to the application. As is usually the case, everything is a trade-off.
I've started developing like you, without orm tools.
Then i worked for companies where software development was more industrialized, and they all use some kind of orm mapping tool (with more or less features). The development is far easier, faster, produce more maintainable code, etc.
But i've also seen the drawbacks of these tools : very slow performance. But it was mostly misuses of the tool (hibernate in that case).
Orm tool are very complex tool, so it is easy to misuse them, but if you have experience with them, you should be able to get nearly the same performances as with raw sql. I would have three advices for you :
If performance is not critical, use an orm tool (choose a good one, i am not developing with php, so i can't give you a name)
Be sure for each feature you add, to check the sql that the orm tool produce and send to the database (thanks to a logging facility for example). Think if it is the way you would have written your queries. Most of the inefficiencies of orm tools come from unwanted data that are gathered from the db, unique request split in multiple ones, etc. Slowness rarely comes from the tool in itself
Do not use the tool for everything. Choose wisely when not to use it (you reduce maintainability each time you do raw db access), but sometimes, it isn't just worst trying to make the orm tool do something it was not developed for.
Edit:
Orm tool are most useful with very complex model : many relationships between entities. Which is most of the time encountered in configuration part of the application, or in complex business part of the application.
So it is less useful if you have only few entities, and if there is less chance they get changed (refactored).
The limit between few entities and many is not clear. I would say more that 50 differents Types (sql tables, without join tables) is many, and less than 10 is few.
I don't know what was used to build stackoverflow but it must have been very carefully performance tested before.
If you want to build a web site that will get such a heavy load, and if you don't have experience with that, try to get someone in your team that have already worked on such sites (performance testing with a real set of data and a representative number of concurrent users is not an easy and fast task to implement). Having someone that have experience with it will greatly speed up the process.
Its very important to have a maintainabilty that is high. Ive developed large scaled web application with lowlevel super high preformance. The big disadvantage was maintaining the system, that is, developing new features. If you'r to slow developing the customers will look for other systems/applications.. Its a trade of. Most of the orms has features if you need to do optmized queries direct to sql. The orm itself isnt the bottleneck. Ill say its more about a good db design.
I think you missed the picture. Performance is everyday for your users, they care not at all about maintainability. You are being ethnocentric, you are concerned only for your personal concerns and not those of the the people who pay for the system. It isn't all about your convenience.
Perhaps you should sit down with the users and watch them use your system for day or two. Then you should sit down at a PC that is the same power as the ones they use (not a dev machine) and spend an entire week doing nothing but using your system all day long. Then you might understand their point.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
What's your experience with doctrine?
I've never been much of an ORM kind of guy, I mostlymanaged with just some basic db abstraction layer like adodb.
But I understood all the concepts and benifits of it. So when a project came along that needed an ORM I thought that I'd give one of the ORM framework a try.
I've to decide between doctrine and propel so I choose doctrine because I didn't want to handle the phing requirement.
I don't know what I did wrong. I came in with the right mindset. And I am by no means a 'junior' php kiddie. But I've been fighting the system each step of the way. There's a lot of documentation but it all feels a little disorganize. And simple stuff like YAML to db table creation just wouldn;t work and just bork out without even an error or anything. A lot of other stuff works a little funky require just that extra bit of tweaking before working.
Maybe I made some soft of stupid newbie assumption here that once I found out what it is I'll have the aha moment. But now I'm totally hating the system.
Is there maybe some tips anyone can give or maybe point me to a good resource on the subject or some authoritative site/person about this? Or maybe just recommend another ORM framework that 'just works"?
I have mixed feelings. I am a master at SQL only because it is so easy to verify. You can test SELECT statements quickly until you get the results right. And to refactor is a snap.
In Doctorine, or any ORM, there are so many layers of abstraction it almost seems OCD (obsessive/compulsive). In my latest project, in which I tried out Doctrine, I hit several walls. It took me days to figure out a solution for something that I knew I could have written in SQL in a matter of minutes. That is soooo frustrating.
I'm being grumpy. The community for SQL is HUGE. The community/support for Doctrine is minuscule. Sure you could look at the source and try to figure it out ... and those are the issues that take days to figure out.
Bottom line: don't try out Doctrine, or any ORM, without planning in a lot of time for grokking on your own.
I think mtbikemike sums it up perfectly: "It took me days to figure out a solution for something that I knew I could have written in SQL in a matter of minutes." That was also my experience. SAD (Slow Application Development) is guaranteed. Not to mention ugly code and limitations around every corner. Things that should take minutes take days and things that would normally be more complicated and take hours or days are just not doable (or not worth the time). The resulting code is much more verbose and cryptic (because we really need another query language, DQL, to make things even less readable). Strange bugs are all around and most of the time is spent hunting them down and running into limitations and problems. Doctrine (I only used v2.x) is akin to an exercise in futility and has absolutely no benefits. It's by far the most hated component of my current system and really the only one with huge problems. Coming into a new system, I'm always going back and forth from the db to the entity classes trying to figure out which name is proper in different places in the code. A total nightmare.
I don't see a single pro to Doctrine, only cons. I don't know why it exists, and every day I wish it didn't (at least in my projects).
we have been using Propel with Symfony for 2 years and Doctrine with Symfony for more than 1 year. I can say that moving to ORM with MVC framework was the best step we've made. I would recommend sticking with Doctrine eventhough it takes some time to learn how to work with it. In the end you'll find your code more readable and flexible.
If you're searching for some place where to start, I would recommend Symfony Jobeet tutorial http://www.symfony-project.org/jobeet/1_4/Doctrine/en/ (chapters 3, 6 covers the basics) and of course Doctrine documentation.
As I wrote above we have been using Doctrine for some time now. To make our work more comfortable we developed a tool called ORM Designer (www.orm-designer.com) where you can define DB model in a graphical user interface (no more YAML files :-), which aren't btw bad at all). You can find there also some helpful tutorials.
My experiences sound similar to yours. I've only just started using doctrine, and have never used Propel. However I am very disapointed in Doctrine. It's documentation is terrible. Poorly organised, and quite incomplete.
Propel and Doctrine uses PDO. PDO has a lot of open bugs with the Oracle Database. All of them related with CLOB fields. Please keep this in mind before starting a new project if you are working with Oracle. The bugs are open since years ago. Doctrine and PDO will crash working with Oracle and CLOBs
I'm using Doctrine in a medium sized project where I had to work from pre-existing databases I don't own. It gives you alot of built in features, but I have one major complaint.
Since I had to generate my models from the databases and not vice-versa, my models are too close to the database: the fields have very similar names to the database columns, to get objects you have to query in what is essential sql (where do I put that code, and how do I test it?), etc.
In the end I had to write a complex wrapper for doctrine that makes me question if it wouldn't have been easier to just use the old dao/model approach and leave doctrine out of the picture. The jury is still out on that. Good luck!
Using Doctrine 2.5 in 2015. It was seemingly going well. Until I wanted to use two entities (in a JOIN). [it's better now after I got a hang of DQL]
Good:
generating SQL for me
use of Foreign Keys and Referential Integrity
InnoDB generation by default
updates made to SQL with doctrine command line tool
Okay:
being hyper-aware of naming and mapping and how to name and how to map entities to actual tables
The Bad
takes a lot of time - learning custom API of query builder. Or figuring out how to do a simple JOIN, wondering if better techniques are out there.. Simple JOINs seem to require writing custom functions if you want to do object oriented queries.
[update on first impression above] -- I chose to use DQL as it is most similar to SQL
It seems to me that the tool is great in concept but its proper execution desires much of developer's time to get onboard. I am tempted to use it for entity SQL generation but then use PDO for actual Input/Output. Only because I didn't learn yet how to do Foreign Key and Referential Integrity with SQL. But learning those seems to be much easier task than learning Doctrine ins and outs even with simple stuff like a entity equivalent of a JOIN.
Doctrine in Existing Projects
I (am just starting to) use Doctrine to develop new features on an existing project. So instead of adding new mysql table for example for the feature, I have added entities (which created the tables for me using Doctrine schema generation). I reserve not using Doctrine for existing tables until I get to know it better.
If I were to use it on existing tables, I would first ... clean the tables up, which includes:
adding id column which is a primary/surrogate key
using InnoDb/transaction-capable table
map it appropriately to an entity
run Doctrine validate tool (doctrine orm:validate-schema)
This is because Doctrine makes certain assumptions about your tables. And because you are essentially going to drive your tables via code. So your code and your tables have to be in as much as 1:1 agreement as possible. As such, Doctrine is not suitable for just any "free-form" tables in general.
But then, you might be able to, with some care and in some cases, get away with little things like an extra columns not being accounted for in your entities (I do not think that Doctrine checks unless you ask it to). You will have to construct your queries knowing what you are getting away with. i.e. when you request an "entity" as a whole, Doctrine requests all fields of the entity specifically by column name. If your actual schema contains more column names, I don't think Doctrine will mind (It does not, as I have verified by creating an extra column in my schema).
So yes it is possible to use Doctrine but I'd start small and with care. You will most likely have to convert your tables to support transactions and to have the surrogate index (primary key), to start with. For things like Foreign Keys, and Referential Integrity, you'll have to work with Doctrine on polishing your entities and matching them up perfectly. You may have to let Doctrine re-build your schema to use its own index names so that it can use FK and RI properly. You are essentially giving up some control of your tables to Doctrine, so I believe it has to know the schema in its own way (like being able to use its own index names, etc).
A little late for the party, but let me throw my two cents here. I will make connections with Laravel, because that is the framework I use.
Active Record vs. Data Mapping vs. Proper OOP
Laravel and many other frameworks love Active Record. It might be great for simple applications, and it saves you time for trivial DB management. However, from the OOP perspective it is a pure anti-pattern. SoC (Separation of Concerns) just got killed. It creates a coupling between the model attributes and SQL column names. Terrible for extensions and future updates.
As your project growths (and yes, it will!), ActiveRecord will be more and more of pain. Don't even think of updating SQL structure easily. Remember, you have the column names all over your PHP code.
I was hired for a project that aims to be quite big down the road. I saw the limits of ActiveRecord. I sat back for 3 weeks and rewrote everything using a Data Mapper, which separates DB from the layers above.
Now, back to the Data Mapper and why I didn't choose Doctrine.
The main idea of Data Mapper is, that it separates your database from your code. And that is the correct approach from the OOP perspective. SoC rules! I reviewed Doctrine in detail, and I immediately didn't like several aspects.
The mapping. Why in a world would anyone use comments as commands? I consider this to be an extremely bad practise. Why not just use a PHP Class to store the mapping relations?
Yaml or XML for the map. Again, Why?? Why wasting time parsing text files, when a regular PHP Class can be used. Plus, a class can be extended, inhereted, can contain methods, not just data. Etc.
If we have a mapper and a model carrying data, then it should be the mapper storing the model. Methods such as $product->save() ar just not good. Model handles data, it should not care about storing anything to the DB. It is a very tight coupling. If we spend time building a mapper, then why not having $mapper->save($product). By definition, it shall be the mapper knowing how to save the data.
Tools such as Doctrine or Eloquent save time at the beginning, no doubt about it. But here is the tricky question for everyone individually. What is the right compromise between /development time/future updates/price/simplicity/following OOP principles/? In the end, it is up to you to answer and decide properly.
My own DataMapper instead of Doctrine
I ended up developing my own DataMapper and I have already used it for several of my small projects. It works very nicely, easy to extend and reuse. Most of the time we just set up parameters and no new code is required.
Here are the key principles:
Model carries data, similar to Laravel's model. Example variable $model for the following examples.
ModelMap contains a field that maps the attributes of the Model to the columns of the table in the SQL database. ModelMaps knows the table name, id, etc. It knows which attributes should be tranfromed to json, which attributes should be hidden (e.g. deleted_at). This ModelMap contains aliases for columns with the same name (connected tables). Example variable: $modelMap.
ModelDataMapper is a class that accepts Model and ModelMap in the controller and provides the store/getById/deleteById functionalities. You simply call $modelMapper->store($model) and that's all.
The base DataMapper also handles pagination, search ability, converting arrays to json, it adds time stamps, it checks for soft deletes, etc. For simple usages, the base DataMapper is enough. For anything more complex, it is easy to extend it using inheritance.
Using Doctrine ORM in 2016 with Approx experience ~2 - 2.5 years.
Inherent Inconsistency
SELECT i, p
FROM \Entity\Item i
JOIN i.product p
WHERE ...
Assume entities are Item and Product. They are connected via Item.product_id to Product.id, and Product contains Product.model that we want to display along with Item.
Here is retrieval of same "product.model" from database, using the above SQL but varying SQL parameters:
//SELECT i, p
$ret[0]->getProduct()->getModel();
//SELECT i as item, p as product
$ret[0]['item']->getProduct()->getModel();
//SELECT i as item, p.model as model
$ret[0]['model'];
Point I am making is this:
Output ResultSet structure can change drastically depending on how you write your DQL/ORM SELECT statement.
From array of objects to array of associative array of objects, to array of associative array, depending on how you want to SELECT. Imagine you have to make a change to your SQL, then imagine having to go back to your code and re-do all the code associated with reading data from the result set. Ouch! Ouch! Ouch! Even if it's a few lines of code, you depend on the structure of result set, there is no full decoupling/common standard.
What Doctrine is good at
In some ways it removes dealing with SQL, crafting and maintaining your own tables. It's not perfect. Sometimes it fails and you have to go to MySQL command line and type SQL to adjust things to the point where Doctrine and you are happy, to where Doctrine sees column types as valid and to where you are happy with column types. You don't have to define your own foreign keys or indices, it is done for you auto-magically.
What Doctrine is bad at
Whenever you need to translate any significantly advanced SQL to DQL/ORM, you may struggle. Separately from that, you may also deal with inconsistencies like one above.
Final thoughts
I love Doctrine for creating/modifying tables for me and for converting table data to Objects, and persisting them back, and for using prepared statements and other checks and balances, making my data safer.
I love the feeling of persistent storage being taken care of by Doctrine from within the object oriented interface of PHP. I get that tingly feeling that I can think of my data as being part of my code, and ORM takes care of the dirty stuff of interacting with the database. Database feels more like a local variable and I have gained an appreciation that if you take care of your data, it will love you back.
I hate Doctrine for its inconsistencies and tough learning curve, and having to look up proprietary syntax for DQL when I know how to write stuff in SQL. SQL knowledge is readily available, DQL does not have that many experts out in the wild, nor an accumulated body of knowledge (compared to SQL) to help you when you get stuck.
I'm not an expert with Doctrine - just started using it myself and I have to admit it is a bit of a mixed experience. It does a lot for you, but it's not always immediately obvious how to tell it to do this or that.
For example when trying to use YAML files with the automatic relationship discovery the many-to-many relationship did not translate correctly into the php model definition. No errors as you mention, because it just did not treat it as many-to-many at all.
I would say that you probably need time to get your head around this or that way of doing things and how the elements interact together. And having the time to do things one step at a time would be a good thing and deal with the issues one at a time in a sort of isolation. Trying to do too much at once can be overwhelming and might make it harder to actually find the place something is going wrong.
After some research into the various ORM libraries for PHP, I decided on PHP ActiveRecord (see phpactiverecord). My decision came down to the little-to-no configuration, light-weight nature of the library, and the lack of code generation. Doctrine is simply too powerful for what I need; what PHP ActiveRecord doesn't do I can implement in my wrapper layer. I would suggest taking a moment and examining what your requirements are in an ORM and see if either a simple one like PHP ActiveRecord offers what you need or if a home-rolled active record implementation would be better.
For now I'm using Symfony framework with Doctrine ORM,
how about using Doctrine together with plain queries?
For e.g. from knpuniversity, I can create custom repository method like:
public function countNumberPrintedForCategory(Category $category)
{
$conn = $this->getEntityManager()
->getConnection();
$sql = '
SELECT SUM(fc.numberPrinted) as fortunesPrinted, AVG(fc.numberPrinted) as fortunesAverage, cat.name
FROM fortune_cookie fc
INNER JOIN category cat ON cat.id = fc.category_id
WHERE fc.category_id = :category
';
$stmt = $conn->prepare($sql);
$stmt->execute(array('category' => $category->getId()));
return $stmt->fetch();
... lines 30 - 37
}
I'm just use Doctrine Entities for e.g. creating an processing forms,
When I need more complex query I just make plain statement and take values I need, from this example I can also pass Entity as variable and take it values for making query. I think this solution is easy understand and it takes less time for building forms, passing data for them and writing complex queries is not as hard as writing them with Doctrine.