The problem is next - I want to execute simple query (e.g. 10 rows from one table)
In Doctrine this operation takes 0.013752s
Here is DQL:
$q = Doctrine_Query::create()
->update('TABLE')
->set('FIELD', 1)
->where('ID = ?', $id);
$rows = $q->execute();
But when i use plain sql and mysql_query() it takes only 0.003298s
What's wrong? Is Doctrine realy 4x slower?
John,
Nothing is wrong. Doctrine introduces considerable overhead compared to a straight SQL query. But you gain the convenience of a nice object oriented interface to the database as well as many other benefits. If raw performance is really important then you might not want to use Doctrine.
For queries where I need performance over convenience (hundreds of thousands of inserts for example) I use PDO to avoid the overhead that gets introduced by the ORM.
Related
I am using Doctrine 2.5.x and I am having problems with getting the LIMIT clause to work for UPDATE queries. It always updates all matched records (i.e. it seems to ignore the LIMIT clause).
setMaxResults() seems to have no effect when used together with UPDATE queries.
As a quick workaround I am using a native MySQL query but that cannot be the best solution.
I tried these examples but none are working:
Doctrine update query with LIMIT
https://recalll.co/app/?q=doctrine2%20-%20Doctrine%20update%20query%20with%20LIMIT
QueryBuilder with setMaxResults() (does not work):
$qb = $em->createQueryBuilder();
$query = $qb->update('\Task\Entity', 't')
->set('t.ClaimedBy', 1)
->where('t.Claimed IS NULL')
->getQuery();
$query->setMaxResults(20);
$this->log($query->getSQL());
Hope someone can help in finding a better solution than a native query. It takes away the whole benefit of the ORM.
Is it even possible to use a LIMIT clause in an UPDATE statement?
In short, no, because the SQL specification does not support UPDATE ... LIMIT ..., so none of the ORM trying to achieve portability should allow you to do it.
Please also have a look at MySQL Reference Manual itself stating that UPDATE ... LIMIT ... is not a standard SQL construction:
MySQL Server supports some extensions that you probably will not find in other SQL DBMSs. Be warned that if you use them, your code will not be portable to other SQL servers. In some cases, you can write code that includes MySQL extensions, but is still portable, by using comments of the following form:
SQL statement syntax
The ORDER BY and LIMIT clauses of the UPDATE and DELETE statements.
So by essence because what you are trying to achieve is not standard SQL the ORM will not have a portable way to implement it and will probably not implement it at all.
Sorry, but what you are trying to achieve is not possible through DQL, because:
Ocramius commented on Sep 2, 2014
DQL doesn't allow limit on UPDATE queries, as it is not portable.
As suggested in this issue of DoctrineBundle repository by its owner, Marco Pivetta (he also happen to be the owner of the ORM repository).
Further information, although it might needs a good link to the right ISO specification documentation that is sadly not freely available:
The ISO standard of UPDATE instruction do not allow LIMIT in an UPDATE, where SELECT is, of course, an instruction that does allow it.
As you were raising it by yourself, the purpose of an ORM is to not write pure SQL in order to have it cross DBMS compatible. If there is no possibility to make it that way, then it makes sense that the ORM does not implement it.
Also note that on other SQL variant than MYSQL, the limit is actually part of the SELECT clause:
select * from demo limit 10
Would translate in a SQL Server to
select top 10 from demo
Or in Orcale to
select * from demo WHERE rownum = 1
Also see: https://stackoverflow.com/a/1063937/2123530
As b.enoit.be already stated in his answer, this is not possible in Doctrine because using LIMIT's in an UPDATE statement is not portable (only valid in MySQL).
Hope someone can help in finding a better solution than a native query. It takes away the whole benefit of the ORM.
I would argue that you are mixing business rules with persistence (and the ORM does not play well with that, luckily).
Let me explain:
Updating an entity's state is not necessarily a business rule. Updating max. 20 entities is (where does that 20 come from?).
In order to fix this, you should properly separate your business rules and persistence by separating it into a service.
class TaskService
{
private $taskRepository;
public function __construct(TaskRepository $taskRepository)
{
$this->taskRepository = $taskRepository;
}
public function updateClaimedBy()
{
$criteria = ['Claimed' => null];
$orderBy = null;
// Only update the first 20 because XYZ
$limit = 20;
$tasks = $taskRepository->findBy($criteria, $orderBy, $limit);
foreach($tasks as $task) {
$task->setClaimedBy(1)
}
}
}
Is there any advantages of using Laravel Eloquent all the time instead of raw SQL?
I have a habit of writing SQL first in phpMyAdmin to check the relationship and then translate it Eloquent ORM.
Sometime translating to Eloquent ORM is painful and time consuming especially translating from long complicated SQL query. I am able to write fast in SQL than using Eloquent ORM.
The most important benefit of using the Query Builder is abstraction, which usually leads to less code. Also, because the builder is database agnostic, it allows to seamlessly switch the RDBMS, for example from MySQL to PostgreSQL (but this only applies in some cases as there are some things that are database specific and cannot be abstracted).
Using it in conjunction with Eloquent offers the added benefit of having the results converted into Eloquent models, which means you can use relations, mutators, accessors and all the other benefits that Eloquent models offer. For example:
$users = DB::select('select * from users');
Will return an array of stdClass objects, while the following:
$users = User::all();
Will return a collection of Eloquent models on which you can get relations:
foreach ($users as $user) {
$user->projects();
}
Or make changes and save the entry:
$user->name = 'Bob Dylan';
$user->save();
These things can be done with the raw query approach, by manually creating a collection of models like so:
// Replace the stdClass items with User models
foreach ($users as &$user) {
$user = new User($user);
}
// Create a Collection with the results
$users = new Illuminate\Support\Collection($users);
But it adds complexity which is already implemented by Eloquent.
A good example of how abstraction leads to less code would be this:
User::whereIn('id', [1, 7, 100]);
Which is less code than the equivalent:
DB::select('select * from users where id in (?)', [implode(',', [1, 7, 100]);
The important thing to take in consideration when using raw queries that make use of user input, is to always use bindings to avoid leaving yourself open to SQL Injection.
That being said, there are cases where, as you said, it's a pain to convert queries to use the Query Builder, because the builder has limitations given that it's database agnostic.
There is no problem with using raw queries. I usually use a combination of both Query Builder for simpler queries that need Eloquent models and raw queries for more complex operations, where it just makes sense not to use DB::raw allover the place just to use the Query Builder. So taking into account the things I said above, it just boils down to preference really.
There's no problem and you should use raw SQL if that suits you better. Eloquent's goal is to simplify queries.
The only thing you have to take care of is to prepare everything.
Recently I tried new way to filter/order SQL queries: instead filter / order the results in SQL query I pull all data I need "as is" then doing the filters / orders with php code.
For example : I want only events with name like "test" order by Date .
Table struct :
id eventDate eventName
New way :
$query = mysqli_query($GLOBALS["link"], "SELECT `eventDate`, `eventName` FROM `tablename` WHERE `id`='X' ");
while($data = mysqli_fetch_array($query))array_push($this->events,$data);
Then I'm using array_fillter and usort /array_multisort and array_values in php...
Old way :
$query = mysqli_query($GLOBALS["link"], "SELECT `eventDate`, `eventName` FROM `tablename` WHERE `id`='X' AND (`eventName` LIKE '%test%') ORDER by `eventDate` DESC ") ;
while($data = mysqli_fetch_array($query))array_push($this->events,$data);
So what is better ? complex sql queries or pull all the data "as is" then make the filters and orders in php ?
The above example is very simple. I'm talking about much more complex filtering...
Please answer if you are absolutely sure !
Thanks :)
The answer will depend on a number of factors, including:
The size of your database
How the database is indexed
The size of the result row
Pros for filtering with PHP:
Less Complex SQL Queries
Potentially Simpler Code
Cons for filtering with PHP:
Higher RAM usage, and with a large dataset, this can be a real deal breaker
Slower (Unless you have a table which is not indexed properly)
Pros for filtering with SQL:
Typically much faster, especially on properly indexed tables
Less RAM Usage
Less Data to Parse
Cons for filtering with SQL:
SQL queries can become unreadable if taken too far
Moving more logic to the query can make database interpretability more challenging (mySQL, SQLite, etc.)
Your milage my vary. Everyone has their own opinions, but in my personal experience, I've found that using native SQL filtering is typically the better choice. Remember the ultimate goal should be clean, maintainable code. And using an SQL Formatter goes a long way in making SQL more readable.
Logicaly mysql will be faster in all cases (avoid data transfer to php, mysql is written on c++), but it is the subject to test.
True power reveals with usage of indexes. In your case you use "like '%value%'" which disables index. But in easily solvable with fulltext index. With MySql 5.6 Fulltext indexes supported in InnoDB engine as well.
Do queries with no excessive spaces and small table aliases stress the MySQL server less or is the difference negligible?
SELECT my_table.a, my_table.b, my_table.c, my_table.d
FROM table_containing_data as my_table
WHERE my_table.a = 1 AND my_table.b = 1 AND my_table.c = 1 AND my_table.d = 1
vs.
SELECT t.a,t.b,t.c,t.d
FROM table_containing_data t
WHERE t.a=1 AND t.b=1 AND t.c=1 AND t.d=1
I am fully aware of the readability of each of them. This question is purely hypothetical. If the difference (when running queries like these hundreds of thousands of times a day) is significant, I might change how my query engine works.
Even for billions of queries, whitespace in your queries is not going to make the difference.
Databases cache queries and execution plans anyway, so if you end up sending the same query over and over again (why aren't you using prepared statements?) any decent database will not have to parse the whole thing again. Of course that only applies if you don't mix SQL and data in the same string.. which is usually done with MySQL. So here's another reason to use prepared statements!
Also, compared to actually retrieving data from the disk (which, by the way, should happen rarely if your database is not huge - you want LOTS of RAM in your database machines to have as much as possible in memory) any parsing etc. is negligible.
Consider 2 ways of querying the database:
With a framework (Yii):
$user = Yii::app()->db->createCommand()
->select('id, username, profile')
->from('tbl_user u')
->join('tbl_profile p', 'u.id=p.user_id')
->where('id=:id', array(':id'=>$id))
->queryRow();
With string concatenation (separating individual parts of a SQL statement):
$columns = "id,username,profile"; // or =implode(",",$column_array);
//you can always use string functions to wrap quotes around each columns/tables
$join = "INNER JOIN tbl_profile p ON u.id=p.user_id";
$restraint = "WHERE id=$id ";//$id cleaned with intval()
$query="SELECT $columns FROM tbl_user u {$restraint}{$join}";
//use PDO to execute query... and loop through records...
Example with string concatenation for pagination:
$records_per_page=20;
$offset = 0;
if (isset($_GET['p'])) $offset = intval($_GET['p'])*$records_per_page;
Squery="SELECT * FROM table LIMIT $offset,$records_per_page";
Which method has better performance?
PHP's PDO allows code to be portable to different databases
2nd method can be wrapped in a function so no code is ever repeated.
String concatenation allows building complex SQL statements programmatically (by manipulating strings)
Use which is right for you and your project team. Frameworks are written for a reason, so use them if it suits, but if it doesn't (and there are reasons they don't) then fall away.
I don't know Yii, but if you look at a lot of frameworks, all they do is build a string query from the parts at the end of the day, hopefully taking advantage of parametization but not always. So, regarding speed, string concat is probably "fastest" - but you're unlikely to really see the difference with a stop watch (you could benchmark if you needed with 1000 queries, but other features such as better error checking or caching may unfairly slow or speed up hte results).
But one advantage frameworks have is they can add context-sensitive caching and know when you update table X that you query caches for A, D and F need to be deleted, but queries B, C and E are all good.
You also have "easy to read" and "debug" and "functionality" to worry about. The top example is much easier to read, which is important in a shared project.
You also need to consider prepared statements - does the framework use them? If so, does it allow you to re-use them (as opposed to merely using them for syntax purposes).
But can the framework do sub-selects? Can it do parametization inside the "JOIN ON"? If not, string concatination with PDO may be more appropriate.
It's not a hard and fast answer - but hopefully provides all the points you need to consider.
Recommendation: use framework unless you really notice it being slow, using too much memory or there is some other good reason not to.