I want to retrieve the last 100 values, I've used this query:
SELECT * FROM values WHERE ID BETWEEN max(ID)-100 and max(ID);
but I receive this message:
ERROR 1111 (HY000): Invalid use of group function
Order by the ID in descending order and take only the first 100 records of the result
SELECT * FROM values
order by id desc
limit 100
This is the more reliable version since there can be gaps in the ID sequence which would make your query inaccurate (besides of it being wrong syntactically).
Your question is not very clear
What is last 100 values? Last 100 ids inserted? Or last 100 rows updated?
Assuming that you are looking for the last 100 roes inserted, you approach has issues. First know that the IDs are not sequentially committed to DB.
For, example ID of a row can be 5 at sometime and ID of a row inserted later can be 4. How this happens is beyond the scope, but just know that it is possible.
Coming to solution
Just do
SELECT TOP 100 * from VALUES ORDER BY ID DESC
Related
I have this MariaDB table:
id, name
The id column has these attributes: Primary, auto_increment, unique.
The table has 40,000 rows.
I'm using this PHP & MariaDB to load rows from this table.
This is the PHP code:
$get_rows = $conn->prepare("SELECT * FROM my_table where id> 0 ORDER BY id ASC LIMIT 30 OFFSET ?");
$get_rows->bind_param('i', $offset);
//etc.
The query returned everything correctly at the first time, but in the next query (made through AJAX), I received the next 30 rows with a gap of one row between the current result and the next one. And this goes on and on.
In the table, the row #1 had been deleted. So, I restored it, and now the query works. However, I will definitely have to delete more rows in the future. (I don't have the option of soft-deleting).
Is there any way I can keep deleting rows, and have these queries return correct results (without skipping any row)?
EDIT
Here's an example of the range of the ids in the first 2 queries:
Query 1:
247--276
Query 2:
278--307
(277 is missing)
NB I asked ChatGPT, but it couldn't help. :')
LIMIT and OFFSET query rows by position, not by value. So if you deleted a row in the first "page," then the position of all subsequent rows moves down by one.
One solution to ensure you don't miss a row is to define pages by the greatest id value on the preceding page, instead of by the offset.
$get_rows = $conn->prepare("
SELECT * FROM my_table where id> ?
ORDER BY id ASC LIMIT 30");
$get_rows->bind_param('i', $lastId);
This only works if your previous query viewed the preceding page, so you can save the value of the last id in that page.
I would like to know how to update my database table with new data but having the latest data be at the top with the unique id starting at 1. Here's what I mean:
First insert query, for example, inserts 3 article topics:
id article
1 firstNews
2 secondNews
3 thirdNews
Then the next time I run a Cron job to check for new articles, for example two new articles appear, I want the two new articles to be in the beginning of the table, like this:
id article
1 NewArticle1
2 NewArticle2
3 firstNews
4 secondNews
5 thirdNews
Hope that made sense. This might be a bad way to do it, I guess I could have a column with insert date() and then get the data out OrderBy date but it has to be an easier way to do this. I think this would be the easiest to output the most recent data from the table...
If I do ORDER BY DESC, it would output NewArticle2 before NewArticle1, which would defeat the purpose...
id article
1 firstNews
2 secondNews
3 thirdNews
4 NewArticle1
5 NewArticle2
So by DESC, id 5 would be the first one output...I was really hoping there was a way around this...
You should never do this. Just insert at the end, and to get the latest articles, use a query:
SELECT * FROM articles ORDER BY id DESC;
In general: Don't fit the data to your query, fit the query to your data. The id is not a position or number, it uniquely identifies that row of data.
You could use a Date columns and sort on that. Or just sort by ID descending.
It's rarely a good idea to change a PK in place. Not least, you mah have child tables using that PK and maybe history tables too
Note that there is no implicit order to a SQL table: you always need an ORDER BY to guarantee the resultset order
You should add another column specifically for sorting, e.g. a date. and add an INDEX on it.
Then the query becomes:
SELECT *
FROM news
ORDER BY newsdate DESC
Latest news comes first
If there are two news items that could be posted at exactly the same time, you may wish to include a secondary sort:
ORDER BY newsdate DESC, id ASC
It shouldn't be your concern at all.
ID is not a number nor a position by any means. It's just an unique identifier.
Means every database row sticks to it's id forever. While you want to assign your "unique" id to the every newcoming news, which makes no sense.
So, just have usual autoincremented id and select it using ORDER BY id DESC to have latest entries first.
I'm trying to get 4 random results from a table that holds approx 7 million records. Additionally, I also want to get 4 random records from the same table that are filtered by category.
Now, as you would imagine doing random sorting on a table this large causes the queries to take a few seconds, which is not ideal.
One other method I thought of for the non-filtered result set would be to just get PHP to select some random numbers between 1 - 7,000,000 or so and then do an IN(...) with the query to only grab those rows - and yes, I know that this method has a caveat in that you may get less than 4 if a record with that id no longer exists.
However, the above method obviously will not work with the category filtering as PHP doesn't know which record numbers belong to which category and hence cannot select the record numbers to select from.
Are there any better ways I can do this? Only way I can think of would be to store the record id's for each category in another table and then select random results from that and then select only those record ID's from the main table in a secondary query; but I'm sure there is a better way!?
You could of course use the RAND() function on a query using a LIMIT and WHERE (for the category). That however as you pointed out, entails a scan of the database which takes time, especially in your case due to the volume of data.
Your other alternative, again as you pointed out, to store id/category_id in another table might prove a bit faster but again there has to be a LIMIT and WHERE on that table which will also contain the same amount of records as the master table.
A different approach (if applicable) would be to have a table per category and store in that the IDs. If your categories are fixed or do not change that often, then you should be able to use that approach. In that case you will effectively remove the WHERE from the clause and getting a RAND() with a LIMIT on each category table would be faster since each category table will contain a subset of records from your main table.
Some other alternatives would be to use a key/value pair database just for that operation. MongoDb or Google AppEngine can help with that and are really fast.
You could also go towards the approach of a Master/Slave in your MySQL. The slave replicates content in real time but when you need to perform the expensive query you query the slave instead of the master, thus passing the load to a different machine.
Finally you could go with Sphinx which is a lot easier to install and maintain. You can then treat each of those category queries as a document search and let Sphinx randomize the results. This way you offset this expensive operation to a different layer and let MySQL continue with other operations.
Just some issues to consider.
Working off your random number approach
Get the max id in the database.
Create a temp table to store your matches.
Loop n times doing the following
Generate a random number between 1 and maxId
Get the first record with a record Id greater than the random number and insert it into your temp table
Your temp table now contains your random results.
Or you could dynamically generate sql with a union to do the query in one step.
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
UNION
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
UNION
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
UNION
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
Note: my sql may not be valid, as I'm not a mySql guy, but the theory should be sound
First you need to get number of rows ... something like this
select count(1) from tbl where category = ?
then select a random number
$offset = rand(1,$rowsNum);
and select a row with offset
select * FROM tbl LIMIT $offset, 1
in this way you avoid missing ids. The only problem is you need to run second query several times. Union may help in this case.
For MySQl you can use
RAND()
SELECT column FROM table
ORDER BY RAND()
LIMIT 4
I'm using auto-increment to assign a id to every new entry in my database.
Also, i've got a php-script which is selecting 10 entrys from this database, according to the current pagenumber.
For example, if the current page is 2, the script should select every entry between id=20 and id=29.
Now, if i delete an entry, the id is lost. Therefore the script will only show 9 entrys when id28 has been deleted.
Is there a way to reassign the auto-increment values after e record has been deleted?
This is usually considered desirable: if entry number 28 is deleted, there will never again be an entry 28. If someone ever tries to refer to it, they're clearly trying to refer to the one that's been deleted, and you can report an error.
If you went back and reassigned 28 to some new entry, now you have no idea whether the person meant the old 28 or the new record.
Let's take a step back and revisit what you want to do. Your goal is not to show ten entries between 20 and 30. Your goal is to show ten entries that meet your criteria. For that, you can use the built-in LIMIT and OFFSET terms:
SELECT ...
FROM ...
WHERE ...
OFFSET 29
LIMIT 10
The OFFSET tells MySQL where to start counting and LIMIT tells it how many to count. MySQL will put together the whole result set, then give you 10 entries beginning at the 30th one. That's your fourth page of results. It does not remotely matter now what IDs they happen to have.
You should change the way you select entries for a page.
By using the LIMIT .. OFFSET clause, you will be able to select 10 entries, starting at Nth entry:
SELECT *
FROM table
ORDER BY id
LIMIT 10
OFFSET 19
LIMIT 10 means return only 10 rows
OFFSET 19 means return rows after 19th row
This way, it doesn't matter if some id has been removed and ids are not sequential.
Just change the OFFSET:
Page 1 => OFFSET 0
Page 2 => OFFSET 9
Page 3 => OFFSET 19
Page N => OFFSET (N-1)*10-1
Your question suggests a query like
SELECT columns FROM table WHERE id BETWEEN 20 AND 29;
or less elegant
SELECT columns FROM table WHERE id >= 20 AND id <= 29;
If so, I suggest you read up on the LIMIT clause and do somethong along the lines of
SELECT columns FROM table ORDER BY id LIMIT 20, 10;
You can but you shouldn't. If id=29 exists are you reset the auto-increment to 28, then there will be problems when auto-increment wants to use id=29 but the record already exists.
You'd be better off writing a query like so:
select * from table order by ID LIMIT n,10;
Where n is the page number*10
Reassign auto-increment values it's bad practice, because id, may used in relation with other tables.Use offset / limit construction, and forget about reassign auto-increment :)
As a direct answer to your question (auto_increment), you can change its value with the following query:
ALTER TABLE `your_table` AUTO_INCREMENT = 200
The next created item would have its next AI collumn value set to 200. This is useful in some cases, although I agree you would be better off using the LIMIT offsets.
I have got table with 300 000 rows. There is specially dedicated field (called order_number) in this table to story the number, which is later used to present the data from this table ordered by order_number field. What is the best and easy way to assign the random number/hash for each of the records in order to select the records ordered by this numbers? The number of rows in the table is not stable and may grow to 1 000 000, so the rand method should take it into the account.
Look at this tutorial on selecting random rows from a table.
If you don't want to use MySQL's built in RAND() function you could use something like this:
select max(id) from table;
$random_number = ...
select * from table where id > $random_number;
That should be a lot quicker.
UPDATE table SET order_number = sha2(id)
or
UPDATE table SET order_number = RAND(id)
sha2() is more random than RAND().
I know you've got enough answers but I would tell you how we did it in our company.
The first approach we use is with additional column for storing random number generated for each record/row. We have INDEX on this column, allowing us to order records by it.
id, name , ordering
1 , zlovic , 14
2 , silvas , 8
3 , jouzel , 59
SELECT * FROM table ORDER BY ordering ASC/DESC
POS: you have index and ordering is very fast
CONS: you will depend on new records to keep the randomization of the records
Second approach we have used is what Karl Roos gave an his answer. We retrieve the number of records in our database and using the > (greater) and some math we retrieve rows randomized. We are working with binary ids thus we need to keep autoincrement column to avoid random writings in InnoDB, sometimes we perform two or more queries to retrieve all of the data, keeping it randomized enough. (If you need 30 random items from 1,000,000 records you can run 3 simple SELECTs each for 10 items with different offset)
Hope this helps you. :)