Let's see, I'm making a mess with the cursor pagination, based on an Id in my case ULID, I want to return an array with the results, next_cursor and prev_cursor.
To obtain the NextCursor is very easy, I only have to add one more to the Limit, that is to say, if I have a limit of 10, I request 11 records and if I get 11 records then the NextCursor is the result 11. But for the PrevCursor the only thing I can think of is to do an additional Query to the one I am already doing. Example:
$limit = 10;
$result = 'SELECT * FROM Table WHERE id <= $cursor ORDER BY id DESC LIMIT $limit+1'
$results = array_slice($result, 0, $limit);
$nextCursor = array_slice($result, $limit, 1);
And now to get the Prev Cursor, I do as I said before an additional query
$prevCursor = 'SELECT * FROM Table WHERE id > $cursor ORDER BY id ASC LIMIT 1'
That way my API can return the following array to the frontend
return [
'data' => $results,
'next_cursor' => $nextCursor,
'prev_cursor' => $prevCursor
];
Now I rephrase the same question again, is there any way to do this without having to do additional Mysql query to get the Prev Cursor, I mean in a same Query or in some other way, I don't know, it's the first time I do this, and I'm a bit lost.
Thanks very much!
Indexing column allows you to quickly find specific row by its ULID and scan nearest rows forward and backward, but obviously, scanning forward and backward internally are two different operations, so if you insist on having the result of the both, you are doomed to perform two different operations.
There's some SQL syntactic sugar to help you hide those two internally executed operations inside one query, but first let me clarify around your objective a little.
What you are trying to build here is actually a window, not a page.
Pagination is when you have an ordered list of rows split over in pages of some size and user references an index of a page which she wants to browse. E.g. page #0, page #1, ... etc. Last page might have less items than a page size, and also if the total number of rows is less than a page, then first page and last page would be the same page and it's OK.
LIMIT and OFFSET operators are here to support that use case. A link to previous page is simply min(0,current_page-1) and a link to next page is min(max_pages,current_page+1).
On the opposite side, windowing is when you have an ordered list of rows, and when user references some specific row by its ULID, you fetch him a few rows behind and/or after queried row. It's like grep -C 10 in bash.
You can emulate window using sub-selects and UNION.
$limit = 10;
// Fetch a limit+1 of results after and including id AND
// a limit of results before id
$result = 'SELECT * FROM (
(
SELECT * from Table
WHERE id >= $cursor ORDER BY id ASC LIMIT $limit+1
) UNION (
SELECT * from Table
WHERE id < $cursor ORDER BY id DESC LIMIT $limit
)
) TableAlias ORDER by id;'
// as we actually fetching some rows before cursor
// we should find its position in the result set ...
$cursor_index = array_search($cursor, array_column($result, 'id'));
// ... and throw away the rest rows
$results = array_slice($result, $cursor_index, $limit);
// here cursors are always first and last items of the result set
$prevCursor = array_slice($result, 0, 1)['id'];
$nextCursor = array_slice($result, -1, 1)['id'];
return [
'data' => $results,
'prev_cursor' => $prevCursor,
'next_cursor' => $nextCursor
];
Since MySQL 8.0 you have a whole set of windowing functions, for your case, LEAD() and LAG() can help you move away all cursor calculations and slicing to your MySQL server.
$limit = 10;
// Wrap same query as above into sub-select, because LAG/LEAD work after WHERE, so we still need UNION to fetch previous cursor
$result = '
WITH
tab1 AS (SELECT * FROM Table where id >= $cursor ORDER BY id ASC LIMIT $limit+1),
tab2 AS (SELECT * FROM Table WHERE id < $cursor ORDER BY id DESC LIMIT $limit),
tab3 AS (SELECT * FROM tab1
UNION ALL
SELECT * FROM tab2 ORDER BY id),
tab4 AS (SELECT
*,
LAG(id, 4) OVER (order by id) as prevcursor,
LEAD(id, 4) OVER (order by id) as nextcursor
FROM tab3)
SELECT * FROM tab4
WHERE id >= $cursor LIMIT $limit'
// here calculated cursor ids are always on first row of result set
$prevCursor = $result[0]['prevcursor'];
$nextCursor = $result[0]['nextcursor'];
// (optionally) strip unwanted columns from result
$results = array_map(function ($a) { return array_diff_key($a, array_flip(array('prevcursor', 'nextcursor'))); }, $result);
return [
'data' => $results,
'prev_cursor' => $prevCursor,
'next_cursor' => $nextCursor
];
That should work well if you don't ever delete rows.
Now, consider following. Both pagination and windowing do not work well with unstable lists.
E.g. if new rows are sequentially adding to the end of the set, then the last page is constantly moving forward. So when one user is opening 'last' page and sees, say, three items there, another user might add another bunch of items and his view of what 'last' page would be different.
What's worse is that if your table usage allows deleting rows, the whole set of pages after and including the page where deleted row was is now rearranged. This leads to very nasty user experience when user is clicking 'next' page and accidentally skips some items, or is clicking 'previous' and sees some of the items he has already seen before.
To overcome those deficiencies you might want to redesign your API such that querying 'previous' page and 'next' page be semantically clearly different from querying 'current' page.
That is, you would need three API endpoints:
query a row by ULID (and a set of up to N rows after it) - initial user entry point from, e.g. search or catalog tree.
query a set of N rows before specific ULID. You pass ULID of the first row in the window user is currently looking at. If there are no rows before given one, result set is empty, you might either display a notification message to user or silently redirect them to first endpoint.
query a set of N rows after specific ULID. You pass ULID of the last row in the window user is currently looking at. If there are no rows after given one, result set is empty, you might either display a notification message to user or silently redirect them back.
If you design your API that way, you would have following benefits:
all three API implemented by only simple ORDER and LIMIT
no need in second query neither explicit nor implicit
your next/previous window results would not ever have any misses or duplicates comparing to previously seen windows.
The only drawback here is that original row user is referring to can be deleted as well. To overcome this, you might want to add a boolean deleted flag to your table schema and set it to false instead of actual row deletion.
After reading #shomeax 's comment and thinking a little more I can suggest to encode cursor in base-64 and make it to contain prev cursor additionally. For example:
[$prevCursor, $curCursor] = explode(':', base64_decode($request['cursor']));
$limit = 10;
$prevPlusCurPagesLimit = $limit * 2;
$ulids = 'SELECT ulid FROM Table WHERE ulid <= $prevCursor ORDER BY ulid DESC LIMIT $prevPlusCurPagesLimit+1';
$resultUlids = array_slice($ulids, $limit, $limit);
$nextCursor = array_slice($ulids, $prevPlusCurPagesLimit, 1);
$prevPrevCursor = reset($ulids);
$response = [
'data' => $resultUlids,
'prevCursor' => base64_encode("$curCursor:$nextCursor"),
'nextCursor' => base64_encode("$prevPrevCursor:$prevCursor"),
];
I didn't try such approach myself but it is partially based on this article https://slack.engineering/evolving-api-pagination-at-slack/ and looks like working
You don’t need to request more, only use < and > rather than <= and >=.
Then you can use the last id in $results for next and the first id for previous.
Assuming that
the "cursor" is the ID from which the next portion starts
and IDs are sequential and without gaps,
and limit is not changed from one query to another
you could get the prev cursor by substracting/adding (depending on sorting) $limit from/to current cursor: $prevCursor = $results[0]['id'] - $limit
And if the ID column has gaps, I suppose there is no reasonable way to implement ability to get prev cursor without additional query. You could only turn it into sub-query or use UNION, but this does not make a big difference.
Consider this...
You fetch the 5 rows for the current page. Then the "previous" page ends before the first id on this page and the "next" page starts after the last id on the current page.
Example: The current page contains ids 65, 67, 71, 82, 91. This finds the 5 rows for the previous page:
SELECT ...
WHERE id < 65
ORDER BY id DESC
LIMIT 5;
(They will be in reverse order, but that is easy to fix.) For the "next" page (in proper order):
SELECT ...
WHERE id > 91
ORDER BY id ASC
LIMIT 5;
As another tip: fetching an extra row (6 instead of 5), lets you cheaply discover whether you are at the "end", thereby being able to suppress the [Next] or [Prev] button.
More: Pagination
Granted, this technique does not deliver the 5 rows for the next/previous page, but do you really need that? Since the Selects should be quite efficient, I don't necessarily see a drawback of doing more than one Select or combining selects with UNION.
I am going to delete my previous answer, despite its upvote, because it is clearly the wrong approach.
Update
Given that you are retrieving on a column, id, that is presumably unique and indexed, then when a "page" of N rows is returned (where N is 10), you need to pass up either the id of the first row (the one with the greater value since we are sorting by descending id) or the last row as query parameter lastId along with a direction flag parameter directionFlag that is either F for forward or B for backward to give the direction of "paging." It should then be possible to directly seek to the correct rows as follows (I am assuming that we are using PDO for MySql Access):
define(PAGE_SIZE, 10);
$limit = PAGE_SIZE;
// lastId parameter specified? It will not be present on the initial request:
if (isset($_REQUEST('lastId')) {
$lastId = $_REQUEST['lastId'];
$direction = $_REQUEST['direction']; 'F'orward or 'B'ackward
if ($direction == 'F') {
$sql = "SELECT * FROM Table ORDER BY id DESC where id < :id LIMIT $limit";
}
else {
// paging backward:
$sql = "SELECT * from (SELECT * FROM Table ORDER BY id ASC where id > :id LIMIT $limit) sq ORDER BY id DESC";
}
$params = [':id' => $lastId];
}
else {
// this is the initial request:
$sql = "SELECT * FROM Table ORDER BY id DESC LIMIT $limit";
$params = [];
}
$stmt = $pdo->prepare($sql);
$stmt->execute($params);
$rows = $stmt->fetchAll(PDO::FETCH_ASSOC);
// The id from $rows[0] will be passed back as lastId with direction flag 'B' for paging backward
// And the id from $rows[PAGE_SIZE-1] will be passed back as lastId with direction flag 'F' for paging forward
hello guys I have problem with mysql like and limit command when used together they return 0 null I tried this in my php then I go to test it inside phpmyadmin I get the same result here is the code
//mysql
SELECT * FROM `items` WHERE name like '%php%' LIMIT 9,9; //this return null
SELECT * FROM `items` WHERE name like '%php%' LIMIT 9 OFFSET 9; //this return null
SELECT * FROM `items` WHERE name like '%php%';//this return some results
//this is weird please help here is the php code that I use
//php
$db = DB::getInstance();
$results = $db->query("SELECT * FROM `items` WHERE name LIKE ? LIMIT {$start},9",array("'%{$search}%'"));//the query function does the bindvalue
//$start come form $_GET['page'];
//$start equation // $start = ($_GET['page'] * $perpage) - $perpage
//the $perpage is always 9 cause I want to display 9 always or less
//and for sure I check for them if they were set , empty , and escape them etc...
$results = $results->results();
printItems($results);//a function that print items in a special way
You do not say how many results you get when you have no limit set, but I assume it's less than 9?
The LIMIT 9,9 or LIMIT 9 OFFSET 9 sets a limit of 9 results total, but starts looking from result number 9. If there are less than 9 results, you will get 0 results returned. I'm guessing you have misunderstood the offset-part of the limit clause (which aren't needed).
Remove the OFFSET-part, and you'll get a result back, even with LIMIT present.
Ensure you have enough data that satisfy LIMIT 9, 9
LIMIT 9, 9 means retrieve rows from 10 to 18
hello all i am a php developer and right now i am having a simple issue like i want to get the last 10 data out of my database table and then show random 5 out of them .
like see the code below.
$random=rand(0,18);
mysqli_query(
$connection,
"SELECT something1,something2
FROM `table`
where (id !='0')
ORDER BY time DESC
LIMIT 4 OFFSET $random");
this query selects last 4 updated fields randomely out of 18 but i want a something better solution to this
First, define "the last 10"; for now let's assume the last ten when ordered by time. So, that is in fact the first ten when I order them the order way (I guess you got that):
SELECT * FROM `table` WHERE (id != '0') ORDER BY time DESC LIMIT 10;
then we subselect that to randomize and limit:
SELECT * FROM (
SELECT * FROM `table` WHERE (id != '0') ORDER BY time DESC LIMIT 10;
) A ORDER BY RAND() LIMIT 5
Remember, your database is extremely powerful and fast, use it for any data-processing.
Not sure there is a great solution with Select. Your sample take 4 line in a row with an OFFSET which is not a truly random selection. You should limit your select to 10 and then get 5 row of the 10 by another php code.
$rows = mysqli_query($connection,
"SELECT something1,something2 FROM `table` where (id !='0')
ORDER BY time DESC LIMIT 10");
//then, get 5 random key from your array of row from the mysql select
$rand_keys = array_rand($rows, 5);
In case you prefer randomizing in PHP, you can use PHP's shuffle and array_slice.
<?php
$last_10_rows = mysqli_query($connection, "simple query to get last 10 rows");
shuffle($last_10_rows); // randomize the array
$random_5_rows = array_slice($last_10_rows, 0, 5); // take first 5
?>
For
When I run this query on phpmyadmin
SELECT *
FROM `product_stock`
WHERE `product_warehouse_id` =5
LIMIT 100000
it loads the data by section 50 by 50 till its stops. But when i do it in a normal php page it takes a while to load and sometimes i get server error. How can I aproach something similar to the way phpmyadmin loads the result?
http://www.tutorialspoint.com/php/mysql_paging_php.htm
this is tutorial how to add paging on mysql long queries.
The MySQL LIMIT takes a couple of possible arguments. A single value specifies to return all of those results to the caller. If you pass two values:
LIMIT 0, 50
Then you're passing the start row and the page size. The 10000 in your example is really just a short form for:
LIMIT 0, 10000
See the MySQL help documents on SELECT for some more detail.
I recommend if you're not going to use all data for all columns, do not use:
SELECT * FROM tablename WHERE 1 AND fieldkey = 'value'
If the data size is very large, causes performance problems in Mysql.
Use:
SELECT field1, field2, field3 FROM tablename WHERE 1 AND fieldkey = 'value'
In the SELECT statement, use comma separated names of the columns you want to display, this will help you the server to respond faster without problems and you can paginate through the results easyly.
Also verify that the data on "fieldkey" are indexed, this helps to the query to work faster.
You can paginate the results of your query with something like this:
php- Paginate data from array
This is what I wanted I had to do it with logic at the end it works, below the sample code:
while ($limit <= 1000){
$q1 = "SELECT *
FROM product_stock
WHERE product_warehouse_id = 5
LIMIT $start, $limit";
$r1 = mysql_query($q1) or die(mysql_error());
while($stock_info = mysql_fetch_assoc($r1)){
echo $stock_info[product_stock_id]."<br />";
}
$start = $limit + 1;
$limit += 50;
}
I'm working with php and I want to do a next button that's step to the next 10 records (like when you browse question with stackoverflow)
I don't know how to do that but I'm thinking to do it with Top n record ? Do you think that's a good idea ? Any suggestion?
As for doing it in PHP, you can easily make the button send a POST or GET request for the starting amount. For instance, a user would make the initial request and that is just yoursite.com/search.php, and the next button would send them to the same page with the same search criteria only send an additional field of "start", (i.e. yoursite.com/search.php?start=10). And in the code, you can simply check for it:
if(isset($_POST['start'])) {
//code to add to the search string to start at $_POST['start']
}
Edit 1: This article is the best I could find as to how to replicate MySQL's LIMIT function. Also, this one has a more definitive query to reference, but it's the same idea.
I know in MySQL you can use LIMIT X, Y where X is the lower bound of the return and Y is the upper bound. So if you wanted to return 10-20 you would use LIMIT 10, 20. Not sure what the MS-SQL equivalent is.
doesn't mssql have something like LIMIT in mysql? so you could do:
select xxx from yyy LIMIT 0,10
for first 10 results, then do LIMIT 10,20 for next 10 results etc.
You can use MySQL's limit
Set a variable called:
$limit = 10;
$pl = $_GET["page"] * $limit;
if(!isset($_GET["page"]) || $_GET["page"] == 1)
{
$pl = 0;
}
and in your query do
$sql = sprintf("SELECT * FROM table LIMIT %d,%d"
mysql_real_escape_string($pl),
mysql_real_escape_string($limit));
Btw this is from memory but i think it works.
Would this be any help to you?
$count=$_POST[page]*10;
for MySQL:
$rowsPerPage = 10;
$offset = ((int)$_GET['page'] - 1) * $rowsPerPage;
$result = mysql_query(
sprintf('select xxx from yyy LIMIT %d,%d', $offset, $rowsPerPage)
);