Related
Let's see, I'm making a mess with the cursor pagination, based on an Id in my case ULID, I want to return an array with the results, next_cursor and prev_cursor.
To obtain the NextCursor is very easy, I only have to add one more to the Limit, that is to say, if I have a limit of 10, I request 11 records and if I get 11 records then the NextCursor is the result 11. But for the PrevCursor the only thing I can think of is to do an additional Query to the one I am already doing. Example:
$limit = 10;
$result = 'SELECT * FROM Table WHERE id <= $cursor ORDER BY id DESC LIMIT $limit+1'
$results = array_slice($result, 0, $limit);
$nextCursor = array_slice($result, $limit, 1);
And now to get the Prev Cursor, I do as I said before an additional query
$prevCursor = 'SELECT * FROM Table WHERE id > $cursor ORDER BY id ASC LIMIT 1'
That way my API can return the following array to the frontend
return [
'data' => $results,
'next_cursor' => $nextCursor,
'prev_cursor' => $prevCursor
];
Now I rephrase the same question again, is there any way to do this without having to do additional Mysql query to get the Prev Cursor, I mean in a same Query or in some other way, I don't know, it's the first time I do this, and I'm a bit lost.
Thanks very much!
Indexing column allows you to quickly find specific row by its ULID and scan nearest rows forward and backward, but obviously, scanning forward and backward internally are two different operations, so if you insist on having the result of the both, you are doomed to perform two different operations.
There's some SQL syntactic sugar to help you hide those two internally executed operations inside one query, but first let me clarify around your objective a little.
What you are trying to build here is actually a window, not a page.
Pagination is when you have an ordered list of rows split over in pages of some size and user references an index of a page which she wants to browse. E.g. page #0, page #1, ... etc. Last page might have less items than a page size, and also if the total number of rows is less than a page, then first page and last page would be the same page and it's OK.
LIMIT and OFFSET operators are here to support that use case. A link to previous page is simply min(0,current_page-1) and a link to next page is min(max_pages,current_page+1).
On the opposite side, windowing is when you have an ordered list of rows, and when user references some specific row by its ULID, you fetch him a few rows behind and/or after queried row. It's like grep -C 10 in bash.
You can emulate window using sub-selects and UNION.
$limit = 10;
// Fetch a limit+1 of results after and including id AND
// a limit of results before id
$result = 'SELECT * FROM (
(
SELECT * from Table
WHERE id >= $cursor ORDER BY id ASC LIMIT $limit+1
) UNION (
SELECT * from Table
WHERE id < $cursor ORDER BY id DESC LIMIT $limit
)
) TableAlias ORDER by id;'
// as we actually fetching some rows before cursor
// we should find its position in the result set ...
$cursor_index = array_search($cursor, array_column($result, 'id'));
// ... and throw away the rest rows
$results = array_slice($result, $cursor_index, $limit);
// here cursors are always first and last items of the result set
$prevCursor = array_slice($result, 0, 1)['id'];
$nextCursor = array_slice($result, -1, 1)['id'];
return [
'data' => $results,
'prev_cursor' => $prevCursor,
'next_cursor' => $nextCursor
];
Since MySQL 8.0 you have a whole set of windowing functions, for your case, LEAD() and LAG() can help you move away all cursor calculations and slicing to your MySQL server.
$limit = 10;
// Wrap same query as above into sub-select, because LAG/LEAD work after WHERE, so we still need UNION to fetch previous cursor
$result = '
WITH
tab1 AS (SELECT * FROM Table where id >= $cursor ORDER BY id ASC LIMIT $limit+1),
tab2 AS (SELECT * FROM Table WHERE id < $cursor ORDER BY id DESC LIMIT $limit),
tab3 AS (SELECT * FROM tab1
UNION ALL
SELECT * FROM tab2 ORDER BY id),
tab4 AS (SELECT
*,
LAG(id, 4) OVER (order by id) as prevcursor,
LEAD(id, 4) OVER (order by id) as nextcursor
FROM tab3)
SELECT * FROM tab4
WHERE id >= $cursor LIMIT $limit'
// here calculated cursor ids are always on first row of result set
$prevCursor = $result[0]['prevcursor'];
$nextCursor = $result[0]['nextcursor'];
// (optionally) strip unwanted columns from result
$results = array_map(function ($a) { return array_diff_key($a, array_flip(array('prevcursor', 'nextcursor'))); }, $result);
return [
'data' => $results,
'prev_cursor' => $prevCursor,
'next_cursor' => $nextCursor
];
That should work well if you don't ever delete rows.
Now, consider following. Both pagination and windowing do not work well with unstable lists.
E.g. if new rows are sequentially adding to the end of the set, then the last page is constantly moving forward. So when one user is opening 'last' page and sees, say, three items there, another user might add another bunch of items and his view of what 'last' page would be different.
What's worse is that if your table usage allows deleting rows, the whole set of pages after and including the page where deleted row was is now rearranged. This leads to very nasty user experience when user is clicking 'next' page and accidentally skips some items, or is clicking 'previous' and sees some of the items he has already seen before.
To overcome those deficiencies you might want to redesign your API such that querying 'previous' page and 'next' page be semantically clearly different from querying 'current' page.
That is, you would need three API endpoints:
query a row by ULID (and a set of up to N rows after it) - initial user entry point from, e.g. search or catalog tree.
query a set of N rows before specific ULID. You pass ULID of the first row in the window user is currently looking at. If there are no rows before given one, result set is empty, you might either display a notification message to user or silently redirect them to first endpoint.
query a set of N rows after specific ULID. You pass ULID of the last row in the window user is currently looking at. If there are no rows after given one, result set is empty, you might either display a notification message to user or silently redirect them back.
If you design your API that way, you would have following benefits:
all three API implemented by only simple ORDER and LIMIT
no need in second query neither explicit nor implicit
your next/previous window results would not ever have any misses or duplicates comparing to previously seen windows.
The only drawback here is that original row user is referring to can be deleted as well. To overcome this, you might want to add a boolean deleted flag to your table schema and set it to false instead of actual row deletion.
After reading #shomeax 's comment and thinking a little more I can suggest to encode cursor in base-64 and make it to contain prev cursor additionally. For example:
[$prevCursor, $curCursor] = explode(':', base64_decode($request['cursor']));
$limit = 10;
$prevPlusCurPagesLimit = $limit * 2;
$ulids = 'SELECT ulid FROM Table WHERE ulid <= $prevCursor ORDER BY ulid DESC LIMIT $prevPlusCurPagesLimit+1';
$resultUlids = array_slice($ulids, $limit, $limit);
$nextCursor = array_slice($ulids, $prevPlusCurPagesLimit, 1);
$prevPrevCursor = reset($ulids);
$response = [
'data' => $resultUlids,
'prevCursor' => base64_encode("$curCursor:$nextCursor"),
'nextCursor' => base64_encode("$prevPrevCursor:$prevCursor"),
];
I didn't try such approach myself but it is partially based on this article https://slack.engineering/evolving-api-pagination-at-slack/ and looks like working
You don’t need to request more, only use < and > rather than <= and >=.
Then you can use the last id in $results for next and the first id for previous.
Assuming that
the "cursor" is the ID from which the next portion starts
and IDs are sequential and without gaps,
and limit is not changed from one query to another
you could get the prev cursor by substracting/adding (depending on sorting) $limit from/to current cursor: $prevCursor = $results[0]['id'] - $limit
And if the ID column has gaps, I suppose there is no reasonable way to implement ability to get prev cursor without additional query. You could only turn it into sub-query or use UNION, but this does not make a big difference.
Consider this...
You fetch the 5 rows for the current page. Then the "previous" page ends before the first id on this page and the "next" page starts after the last id on the current page.
Example: The current page contains ids 65, 67, 71, 82, 91. This finds the 5 rows for the previous page:
SELECT ...
WHERE id < 65
ORDER BY id DESC
LIMIT 5;
(They will be in reverse order, but that is easy to fix.) For the "next" page (in proper order):
SELECT ...
WHERE id > 91
ORDER BY id ASC
LIMIT 5;
As another tip: fetching an extra row (6 instead of 5), lets you cheaply discover whether you are at the "end", thereby being able to suppress the [Next] or [Prev] button.
More: Pagination
Granted, this technique does not deliver the 5 rows for the next/previous page, but do you really need that? Since the Selects should be quite efficient, I don't necessarily see a drawback of doing more than one Select or combining selects with UNION.
I am going to delete my previous answer, despite its upvote, because it is clearly the wrong approach.
Update
Given that you are retrieving on a column, id, that is presumably unique and indexed, then when a "page" of N rows is returned (where N is 10), you need to pass up either the id of the first row (the one with the greater value since we are sorting by descending id) or the last row as query parameter lastId along with a direction flag parameter directionFlag that is either F for forward or B for backward to give the direction of "paging." It should then be possible to directly seek to the correct rows as follows (I am assuming that we are using PDO for MySql Access):
define(PAGE_SIZE, 10);
$limit = PAGE_SIZE;
// lastId parameter specified? It will not be present on the initial request:
if (isset($_REQUEST('lastId')) {
$lastId = $_REQUEST['lastId'];
$direction = $_REQUEST['direction']; 'F'orward or 'B'ackward
if ($direction == 'F') {
$sql = "SELECT * FROM Table ORDER BY id DESC where id < :id LIMIT $limit";
}
else {
// paging backward:
$sql = "SELECT * from (SELECT * FROM Table ORDER BY id ASC where id > :id LIMIT $limit) sq ORDER BY id DESC";
}
$params = [':id' => $lastId];
}
else {
// this is the initial request:
$sql = "SELECT * FROM Table ORDER BY id DESC LIMIT $limit";
$params = [];
}
$stmt = $pdo->prepare($sql);
$stmt->execute($params);
$rows = $stmt->fetchAll(PDO::FETCH_ASSOC);
// The id from $rows[0] will be passed back as lastId with direction flag 'B' for paging backward
// And the id from $rows[PAGE_SIZE-1] will be passed back as lastId with direction flag 'F' for paging forward
I want to read the last 3 rows of my table seperate and then place them in 3 different div's of a slider. The problem is that i cant use 'where id=xxx' because i insert rows dynamically every time that i make a post item.
if i use query('select * from news order by id desc limit 3') and then a loop while ($result->fetch_assoc()) then i have the last 3 rows.
My problem is that i want to place every row in a different div so that i will have 3 divs.
I suppose i must do 3 different queries for that but i dont know how.
I have this one right now.
$result = $myDb->query('select * from news order by id desc');
while ($nI = $result->fetch_assoc()) {
$title = $nI['title'];
$date = $nI['date'];
$author = $nI['author'];
$mainobjective = $nI['mainobjective'];
$contents = $nI['contents'];
$keywords = $nI['keywords'];
and then i have my html where with the use of echo i place every variable in the div i want.
It sounds like the problem you are describing is more with your PHP code, that you haven't posted, than your MySQL. Don't read them separately. Use a single query to get all 3 and then iterate through them separately with PHP.
You can use the query you already had:
$result = $myDb->query('SELECT * FROM news ORDER BY id DESC LIMIT 0,3');
Make sure you are placing the results in separate containers in PHP:
foreach($result as $row)
{
echo "<div>".$row."</div>";
}
How can I make a limit of showing the results? I need to limit it for 100 views.
In DB I have:
ID|NAME|PAGE|COUNT|DATE
In count I want to count untill 100 and then stop showing that ID. I could do it with count < 100. And then update the specific ID. I could get records with less than 100 views, but I couldn't manage to update count on the specific ID.
Row is showed with:
php code:
foreach($bannerGroups[0] as $ban) {
echo '<li class="right1">'.$ban->html().'</li>';
}
But I just don't know where to put the update in there. I tried, but all I got was to update only one ID. But it shows 4 on one page and randomizes them on refresh. So I don't know what to do.
Also I would like to say I am only learning php. Sorry for all the mess.
Code at http://pastebin.com/A9hJTPLE
If I understand correctly, you want to show all banners that have been previously-displayed less than 100 times?
If that's right, you can just add that to your WHERE clause:
$bannerResult = mysql_query("SELECT * FROM table WHERE page='cat' WHERE `COUNT` < 100");
To update them all, you can either run a query while displaying each individual banner, or "record" the id of each and run a single query at the end, like:
$ids = array();
foreach($bannerGroups[0] as $ban) {
$ids[] = $ban['ID']; // record the ID; don't know how Banner
// class works, assuming uses indexes; maybe ID() method?
echo '<li class="right1">'.$ban->html().'</li>';
}
...
mysql_query('UPDATE table SET `COUNT` = `COUNT` + 1 WHERE ID IN (' . join(',', $ids) . ')');
UPDATE:
Based off of a comment, your Banner class doesn't have a method to retrieve the individual banner's ID. In this case, you can record the ID values when you're building your banners array:
$ids = array();
while($row=mysql_fetch_assoc($bannerResult)) {
$banners[] = new Banner($row);
$ids[] = $row['ID']; // record the ID
}
// update the `count` on each record:
mysql_query('UPDATE table SET `COUNT` = `COUNT` + 1 WHERE ID IN (' . join(',', $ids) . ')');
sorry, but I got your question wrong...
first you have to insert a new sql-column like "viewcount" to the db...
on every read, you have to increment the value in viewcount...
for that behaviour (because, mysql does not allow sub-selects on update-clause on the same table), you have to fetch the results from db, as you do that, and pass all the primary-keys of the records to an array...
after the view-logic you have to fire up a query like:
UPDATE foo SET viewcount = viewcount + 1 WHERE id IN (1,2,3,4,5,6...,100);
where the IN-clause can be easily generated using your primary-keys-array with "implode(',', $arr);"
hope this helps.
$bannerResult = mysql_query("SELECT * FROM table WHERE page='cat' AND `count`<100");
#newfurniturey figured it out. in each foreach($banneruGroups added: $ids = $ban->getValue('id'); and then mysql_query("UPDATE dataa SET COUNT = COUNT + 1 WHERE id = '$ids'"); but is there any way to update them by adding query only once? And if the id is showed already 100 times i get Warning: Invalid argument supplied for foreach() in. Any idea how to fix it? I have 4 ids in DB . If one of them already have 100 views (count) then i get error!
Try to limit your data source for 100 items.
It's like OFFSET x LIMIT 100 in MySQL/PostgreSQL query or TOP 100 in MSSQL.
I know things are escaped incorrectly, but I am trying to write a previous/next query to get the next id and previous id within an order, but my output does not match the phpmyadmin sorting, which it should. I just have the 'next' written, and would write the previous with reverse conditions. This query seems to give me the next highest ID value number within the order...I need the next or previous in the order index...can anyone help?
$db = JFactory::getDBO();
$query = "SELECT image_id FROM #__jxgallery_images WHERE image_id >".$currentid."' ORDER BY '".$ordering." ".$direction." LIMIT 1";
// Executes the current SQL query string.
$db->setQuery($query);
// returns the array of database objects
$list = $db->loadObjectList();
// create the list of ids
foreach ($list as $item) {
$next = $item->image_id;
echo $next.'<br/>';
}
echo 'The next ID in this sort state is '.$next.'<br />';
?>
This is phpmyadmin and it is right...
SELECT * FROM `jos_jxgallery_images`
ORDER BY `jos_jxgallery_images`.`hits` DESC
LIMIT 0 , 30
I have now matched this query in my code to get the same results. My variables fill in the ($ordering) 'hits' field and the 'desc' ($direction) within my clause. That works fine. The image_ids and hits aren't special...just numbers. When hits are ordered, the image_ids are resored to match. I don't need the next value of image_id as to what is in the field. I need the next row or previous row, regardless of value, based on the current image_id I plugin.
These are actual image_ids LIMIT 5, and these are Ordered by the hits field Descending:
52791
52801
52781
52771
52581`
Now if the current image I'm looking at has an id of 52791, then previous should be nothing and next should be 52801. What my query is doing I think is giving me an image_id of a higher valued number as 'next' because that is the next highest VALUED image_id, not the next row. I can see why in the query, I am asking for greater than...but I just need the next row
I think the problem is with your WHERE condition: WHERE image_id >".$currentid."'
If I understand what you're trying to do, one way to do it is this:
$query = "SELECT image_id FROM `#__jxgallery_images` ORDER BY ".$ordering." ".$direction." LIMIT 2";
$db->setQuery($query);
$list = $db->loadObjectList();
$next_item = $list[1]->image_id;
Notice that the WHERE condition is removed from the query, and I also changed LIMIT 1 to LIMIT 2. This way, the query basically returns your top value and the one with the next highest "hits" value.
I hope this helps.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
What is the SQL for 'next' and 'previous' in a table?
I'm trying to find a better way to get the next or previous record from a table. Let's say I have a blog or news table:
CREATE TABLE news (
news_id INT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
news_datestamp DATETIME NOT NULL,
news_author VARCHAR(100) NOT NULL,
news_title VARCHAR(100) NOT NULL,
news_text MEDIUMTEXT NOT NULL
);
Now on the frontend I want navigation buttons for the next or previous records, if i'm sorting by news_id, I can do something rather simple like:
SELECT MIN(news_id) AS next_news_id FROM news WHERE news_id > '$old_news_id' LIMIT 1
SELECT MAX(news_id) AS prev_news_id FROM news WHERE news_id < '$old_news_id' LIMIT 1
But the news can be sorted by any field, and I don't necessarily know which field is sorted on, so this won't work if the user sorts on news_author for example.
I've resorted to the rather ugly and inefficient method of sorting the entire table and looping through all records until I find the record I need.
$res = mysql_query("SELECT news_id FROM news ORDER BY `$sort_column` $sort_way");
$found = $prev = $next = 0;
while(list($id) = mysql_fetch_row($res)) {
if($found) {
$next = $id;
break;
}
if($id == $old_news_id) {
$found = true;
continue;
}
$prev = $id;
}
There's got to be a better way.
Edit, clarifications:
Why dont I use limit? I would need to know the position in the result set of the current record, which I don't. If this were your typical pagination, and I had query strings like ?startpage=n then yes that would work, I could just increment $startpage and add LIMIT $startpage,1 to the query. But I have urls like news/news_id-news-title-here that are rewritten to ?news_id=n, so I don't know what the startpage is. Even if I did, what if the user gets there via an external link? What if new posts are added while the user is reading the current page?
Don't get too stuck on the specifics of the example above, the real question is this:
Given a unique record id and an arbitrary sort column, is there a way to determine which records fall immediately before and after that specific record, without looping through the entire record set.
why don't you use the same way for the news_author you used for the news_id?
EDIT:
There is one pitfall: news_author is not unique for sure. So, you will need to order news_author query by 2 fields : news_author, news_id. So, you will need 2 conditions to get next author
Why you dont use limit ?
e.g. you listing 10 items per page. Limit 1,10
the next 10 items: Limit 11,10
and so on.
no matter if you display one or 100 items per page, the system is always the same.
EDIT: more explanation needed, if you have a detail page, you most certainly will come to it via a list, the list was created with a certain query, so you know the position in this list, and can give this param to the detail page. so when you there have prev/next links you can call the query with a limit of 1 and the offset from your params.
EDIT 2: You can use a query to find the position of the row based on your order but this will only work with DESC:
SELECT COUNT(*)+1 FROM news ma JOIN news mp ON (mp.$sort_column, mp.news_id) > (ma.$sort_column, ma.news_id) WHERE ma.news_id = $news_id;
in pseudo code:
$currentValue = "SELECT $sort_column FROM news WHERE news_id = $current_news_id LIMIT 1"
$operator = ($sort_way == 'ASC')?' > ':' < ';
$nextNewsId = "SELECT news_id FROM news WHERE $sort_column $operator $currentValue ORDER BY $sort_column $sort_way LIMIT 1"
no?