In the Laravel Eloquent Model::all() function, what is the default ordering for the returned query? I ask this because I'm pretty sure it's in ascending order by primary key which defaults do 'id' when you make the model through
php artisan make:model Model -m
However, when I call it like this:
return $users = User::all();
I get the following results in the browser:
The results seem to be in no particular order by any of the attributes.
I am fully aware I can order them by id by doing
return $users = User::orderBy('id', 'asc')->get();
But just a few days ago they were being ordered automatically. What gives?
The default sort order for laravel is simply nothing. It does not apply a default "ORDER BY" clause, which means that it follows PostgreSQL rules for unordered results. From some similar answers here and here:
Do not depend on order when ORDER BY is missing.
Always specify ORDER BY if you want a particular order -- in some situations the engine can eliminate the ORDER BY because of how it
does some other step.
GROUP BY forces ORDER BY. (This is a violation of the standard. It can be avoided by using ORDER BY NULL.)
SELECT * FROM tbl -- this will do a "table scan". If the table has
never had any DELETEs/REPLACEs/UPDATEs, the records will happen to be
in the insertion order, hence what you observed.
If you had done the same statement with an InnoDB table, they would
have been delivered in PRIMARY KEY order, not INSERT order. Again,
this is an artifact of the underlying implementation, not something to
depend on.
With a great agreement with FrankerZ's answer, I would like to add, whenever you wonder what query laravel is structuring under the ORM, DB listener will be quickest option to use :
<?php
\DB::listen(function($sql) {
dump($sql);
});
W.r.t. this question it would give select * from users when you do User::all().
Note: Check listen function as per the laravel version you are using, the parameter count are different in older and newer versions.
Reference : Documentation Link
Related
What is the default order of a query when no ORDER BY is used?
There is no such order present. Taken from http://forums.mysql.com/read.php?21,239471,239688#msg-239688
Do not depend on order when ORDER BY is missing.
Always specify ORDER BY if you want a particular order -- in some situations the engine can eliminate the ORDER BY because of how it
does some other step.
GROUP BY forces ORDER BY. (This is a violation of the standard. It can be avoided by using ORDER BY NULL.)
SELECT * FROM tbl -- this will do a "table scan". If the table has
never had any DELETEs/REPLACEs/UPDATEs, the records will happen to be
in the insertion order, hence what you observed.
If you had done the same statement with an InnoDB table, they would
have been delivered in PRIMARY KEY order, not INSERT order. Again,
this is an artifact of the underlying implementation, not something to
depend on.
There's none. Depending on what you query and how your query was optimised, you can get any order. There's even no guarantee that two queries which look the same will return results in the same order: if you don't specify it, you cannot rely on it.
I've found SQL Server to be almost random in its default order (depending on age and complexity of the data), which is good as it forces you to specify all ordering.
(I vaguely remember Oracle being similar to SQL Server in this respect.)
MySQL by default seems to order by the record structure on disk, (which can include out-of-sequence entries due to deletions and optimisations) but it often initially fools developers into not bother using order-by clauses because the data appears to default to primary-key ordering, which is not the case!
I was surprised to discovere today, that MySQL 5.6 and 4.1 implicitly sub-order records which have been sorted on a column with a limited resolution in the opposite direction. Some of my results have identical sort-values and the overall order is unpredictable. e.g. in my case it was a sorted DESC by a datetime column and some of the entries were in the same second so they couldn't be explicitly ordered. On MySQL 5.6 they select in one order (the order of insertion), but in 4.1 they select backwards! This led to a very annoying deployment bug.
I have't found documentation on this change, but found notes on on implicit group order in MySQL:
By default, MySQL sorts all GROUP BY col1, col2, ... queries as if you specified ORDER BY col1, col2, ... in the query as well.
However:
Relying on implicit GROUP BY sorting in MySQL 5.5 is deprecated. To achieve a specific sort order of grouped results, it is preferable to use an explicit ORDER BY clause.
So in agreement with the other answers - never rely on default or implicit ordering in any database.
The default ordering will depend on indexes used in the query and in what order they are used. It can change as the data/statistics change and the optimizer chooses different plans.
If you want the data in a specific order, use ORDER BY
I have a table with several million records.
The table has a long string as a key, and searching for records, especially new ones, is very slow.
I have tried to narrow the search to whereDate >= this week, or whereID >= 999999 but this doesn't give a huge performance boost.
We are using Laravel. Is there some kind of feature to search a table from the bottom up or in reverse, instead of searching incrementally from the top down? Something like Model::findBackwards($key) that would start looking at the most recent record and process backwards would be ideal in this situation.
How would you approach this problem?
Edit - here is our current query. It's a job running on horizon
if (DB::table('big_table_with_a_stupid_key')
->whereDate('created_at', '>=', '2019-11-10')
->where('hash', $this->hash)
->increment('counter')
);
else {
DB::table('big_table_with_a_stupid_key')->where(['hash' => $this->hash])->increment('counter');
}
If you want to reverse order of the result, you can use latest() function.
Laravel documentation says: https://laravel.com/docs/6.x/queries
latest / oldest
The latest and oldest methods allow you to easily order results by date. By default, result will be ordered by the created_at column. Or, you may pass the column name that you wish to sort by:
$user = DB::table('users')->latest()->first(); // order by created_at
$user = DB::table('users')->latest('id')->first(); // order by id
You will want to do this by using the ORDER BY clause in your underlying database query. Laravel (of course) exposes this with ->orderBy()
https://www.tutorialsplane.com/laravel-order-by-asc-desc/
The SQL engine will place the result rows in the requested sequence before returning them to you.
Then, to address your performance issues, be sure that any column that you routinely use for searching has an index. This is the secret to maintaining good performance as the number of rows increases.
What is the default order of a query when no ORDER BY is used?
There is no such order present. Taken from http://forums.mysql.com/read.php?21,239471,239688#msg-239688
Do not depend on order when ORDER BY is missing.
Always specify ORDER BY if you want a particular order -- in some situations the engine can eliminate the ORDER BY because of how it
does some other step.
GROUP BY forces ORDER BY. (This is a violation of the standard. It can be avoided by using ORDER BY NULL.)
SELECT * FROM tbl -- this will do a "table scan". If the table has
never had any DELETEs/REPLACEs/UPDATEs, the records will happen to be
in the insertion order, hence what you observed.
If you had done the same statement with an InnoDB table, they would
have been delivered in PRIMARY KEY order, not INSERT order. Again,
this is an artifact of the underlying implementation, not something to
depend on.
There's none. Depending on what you query and how your query was optimised, you can get any order. There's even no guarantee that two queries which look the same will return results in the same order: if you don't specify it, you cannot rely on it.
I've found SQL Server to be almost random in its default order (depending on age and complexity of the data), which is good as it forces you to specify all ordering.
(I vaguely remember Oracle being similar to SQL Server in this respect.)
MySQL by default seems to order by the record structure on disk, (which can include out-of-sequence entries due to deletions and optimisations) but it often initially fools developers into not bother using order-by clauses because the data appears to default to primary-key ordering, which is not the case!
I was surprised to discovere today, that MySQL 5.6 and 4.1 implicitly sub-order records which have been sorted on a column with a limited resolution in the opposite direction. Some of my results have identical sort-values and the overall order is unpredictable. e.g. in my case it was a sorted DESC by a datetime column and some of the entries were in the same second so they couldn't be explicitly ordered. On MySQL 5.6 they select in one order (the order of insertion), but in 4.1 they select backwards! This led to a very annoying deployment bug.
I have't found documentation on this change, but found notes on on implicit group order in MySQL:
By default, MySQL sorts all GROUP BY col1, col2, ... queries as if you specified ORDER BY col1, col2, ... in the query as well.
However:
Relying on implicit GROUP BY sorting in MySQL 5.5 is deprecated. To achieve a specific sort order of grouped results, it is preferable to use an explicit ORDER BY clause.
So in agreement with the other answers - never rely on default or implicit ordering in any database.
The default ordering will depend on indexes used in the query and in what order they are used. It can change as the data/statistics change and the optimizer chooses different plans.
If you want the data in a specific order, use ORDER BY
I am using CodeIgniter 2.1.3 and PHP 5.4.8, and two PostgreSQL servers (P1 and P2).
I am having a problem with list_fields() function in CodeIgniter. When I retrieve fields from P1 server, fields are in the order that I originally created the table with. However, if I use the exact same codes to retrieve fields from P2 server, fields are in reverse order.
If fields from P1 is array('id', 'name', 'age'),
fields from P2 becomes array('age', 'name', 'id')
I don't think this is CodeIgniter specific problem, but rather general database configuration or PHP problem, because codes are identical.
This is the code that I get fields with.
$fields = $this->db->list_fields("clients");
I have to clarify something. #muistooshort claims in a comment above:
Technically there is no defined order to the columns in a table.
#mu may be thinking of the order or rows, which is arbitrary without ORDER BY.
It is completely incorrect for the order of columns, which is well defined and stored in the column pg_attribute.attnum. It's used in many places, like INSERT without column definition list or SELECT *. It is preserved through a dump / restore cycle and has significant bearing on storage size and performance.
You cannot simply change the order of columns in PostgreSQL, because it has not been implemented, yet. It's deeply wired into the system and hard to change. There is a Postgres Wiki page and it's on the TODO list of the project:
Allow column display reordering by recording a display, storage, and
permanent id for every column?
Find out for your table:
SELECT attname, attnum
FROM pg_attribute
WHERE attrelid = 'myschema.mytable'::regclass
AND NOT attisdropped -- no dropped (dead) columns
AND attnum > 0 -- no system columns
ORDER BY attnum;
It is unwise to use SELECT * in some contexts, where the columns of the underlying table may change and break your code. It is explicitly wise to use SELECT * in other contexts, where you need all columns (in default order).
As to the primary question
This should not occur. SELECT * returns columns in a well defined order in PostgreSQL. Some middleware must be messing with you.
I suspect you are used to MySQL which allows you to reorder columns post columns post-table creation. PostgreSQL does not let you do this, so when you:
ALTER TABLE foo ADD bar int;
It puts this on the end of the table always and there is no way to change the order.
On PostgreSQL you should not assume that the order of the columns is meaningful because these can differ from server to server based on the order in which the columns were defined.
However the fact that these are reversed is odd to me.
If you want to see the expected order on the db, use:
\d foo
from psql
If these are reversed then the issue is in the db creation (this is my first impression). That's the first thing to look at. If that doesn't show the problem then there is something really odd going on with CodeIgniter.
I'm trying to sort a collection by attribute_id. I thought it would be easy, but I think I'm not using it correctly:
$attributes = Mage::getResourceModel('eav/entity_attribute_collection')
->setOrder('attribute_id');
echo $attributes->getSelect();
Result:
SELECT `main_table`.* FROM `eav_attribute` AS `main_table`
Why isn't there any order by?
You're actually doing it the right way. However, since Magento uses EAV, it needs to apply tricks to help performance.
One of these tricks is the timing used to build the eventual SQL string. Usually it's lazily loaded at the last minute and it's not until you actually indicate you want to access a collection's data, that you can see the full SQL used to produce the collection. For example running your code, but prompting magento to actually construct and load the collection, produces the expected output.
$attributes = Mage::getResourceModel('eav/entity_attribute_collection')
->setOrder('attribute_id');
$attributes->count(); // forces the collection to load
echo $attributes->getSelect()->assemble();
This results in the SQL:
SELECT `main_table`.* FROM `eav_attribute` AS `main_table` ORDER BY attribute_id DESC
So you were on the right path, just Magento was doing its level best to confuse you. It's very good at that.
Use this instead of $attributes->getSelect();:
$attributes->getSelect()->order('main_table.attribute_id ASC');
Don't ask why.