Let's see, I'm making a mess with the cursor pagination, based on an Id in my case ULID, I want to return an array with the results, next_cursor and prev_cursor.
To obtain the NextCursor is very easy, I only have to add one more to the Limit, that is to say, if I have a limit of 10, I request 11 records and if I get 11 records then the NextCursor is the result 11. But for the PrevCursor the only thing I can think of is to do an additional Query to the one I am already doing. Example:
$limit = 10;
$result = 'SELECT * FROM Table WHERE id <= $cursor ORDER BY id DESC LIMIT $limit+1'
$results = array_slice($result, 0, $limit);
$nextCursor = array_slice($result, $limit, 1);
And now to get the Prev Cursor, I do as I said before an additional query
$prevCursor = 'SELECT * FROM Table WHERE id > $cursor ORDER BY id ASC LIMIT 1'
That way my API can return the following array to the frontend
return [
'data' => $results,
'next_cursor' => $nextCursor,
'prev_cursor' => $prevCursor
];
Now I rephrase the same question again, is there any way to do this without having to do additional Mysql query to get the Prev Cursor, I mean in a same Query or in some other way, I don't know, it's the first time I do this, and I'm a bit lost.
Thanks very much!
Indexing column allows you to quickly find specific row by its ULID and scan nearest rows forward and backward, but obviously, scanning forward and backward internally are two different operations, so if you insist on having the result of the both, you are doomed to perform two different operations.
There's some SQL syntactic sugar to help you hide those two internally executed operations inside one query, but first let me clarify around your objective a little.
What you are trying to build here is actually a window, not a page.
Pagination is when you have an ordered list of rows split over in pages of some size and user references an index of a page which she wants to browse. E.g. page #0, page #1, ... etc. Last page might have less items than a page size, and also if the total number of rows is less than a page, then first page and last page would be the same page and it's OK.
LIMIT and OFFSET operators are here to support that use case. A link to previous page is simply min(0,current_page-1) and a link to next page is min(max_pages,current_page+1).
On the opposite side, windowing is when you have an ordered list of rows, and when user references some specific row by its ULID, you fetch him a few rows behind and/or after queried row. It's like grep -C 10 in bash.
You can emulate window using sub-selects and UNION.
$limit = 10;
// Fetch a limit+1 of results after and including id AND
// a limit of results before id
$result = 'SELECT * FROM (
(
SELECT * from Table
WHERE id >= $cursor ORDER BY id ASC LIMIT $limit+1
) UNION (
SELECT * from Table
WHERE id < $cursor ORDER BY id DESC LIMIT $limit
)
) TableAlias ORDER by id;'
// as we actually fetching some rows before cursor
// we should find its position in the result set ...
$cursor_index = array_search($cursor, array_column($result, 'id'));
// ... and throw away the rest rows
$results = array_slice($result, $cursor_index, $limit);
// here cursors are always first and last items of the result set
$prevCursor = array_slice($result, 0, 1)['id'];
$nextCursor = array_slice($result, -1, 1)['id'];
return [
'data' => $results,
'prev_cursor' => $prevCursor,
'next_cursor' => $nextCursor
];
Since MySQL 8.0 you have a whole set of windowing functions, for your case, LEAD() and LAG() can help you move away all cursor calculations and slicing to your MySQL server.
$limit = 10;
// Wrap same query as above into sub-select, because LAG/LEAD work after WHERE, so we still need UNION to fetch previous cursor
$result = '
WITH
tab1 AS (SELECT * FROM Table where id >= $cursor ORDER BY id ASC LIMIT $limit+1),
tab2 AS (SELECT * FROM Table WHERE id < $cursor ORDER BY id DESC LIMIT $limit),
tab3 AS (SELECT * FROM tab1
UNION ALL
SELECT * FROM tab2 ORDER BY id),
tab4 AS (SELECT
*,
LAG(id, 4) OVER (order by id) as prevcursor,
LEAD(id, 4) OVER (order by id) as nextcursor
FROM tab3)
SELECT * FROM tab4
WHERE id >= $cursor LIMIT $limit'
// here calculated cursor ids are always on first row of result set
$prevCursor = $result[0]['prevcursor'];
$nextCursor = $result[0]['nextcursor'];
// (optionally) strip unwanted columns from result
$results = array_map(function ($a) { return array_diff_key($a, array_flip(array('prevcursor', 'nextcursor'))); }, $result);
return [
'data' => $results,
'prev_cursor' => $prevCursor,
'next_cursor' => $nextCursor
];
That should work well if you don't ever delete rows.
Now, consider following. Both pagination and windowing do not work well with unstable lists.
E.g. if new rows are sequentially adding to the end of the set, then the last page is constantly moving forward. So when one user is opening 'last' page and sees, say, three items there, another user might add another bunch of items and his view of what 'last' page would be different.
What's worse is that if your table usage allows deleting rows, the whole set of pages after and including the page where deleted row was is now rearranged. This leads to very nasty user experience when user is clicking 'next' page and accidentally skips some items, or is clicking 'previous' and sees some of the items he has already seen before.
To overcome those deficiencies you might want to redesign your API such that querying 'previous' page and 'next' page be semantically clearly different from querying 'current' page.
That is, you would need three API endpoints:
query a row by ULID (and a set of up to N rows after it) - initial user entry point from, e.g. search or catalog tree.
query a set of N rows before specific ULID. You pass ULID of the first row in the window user is currently looking at. If there are no rows before given one, result set is empty, you might either display a notification message to user or silently redirect them to first endpoint.
query a set of N rows after specific ULID. You pass ULID of the last row in the window user is currently looking at. If there are no rows after given one, result set is empty, you might either display a notification message to user or silently redirect them back.
If you design your API that way, you would have following benefits:
all three API implemented by only simple ORDER and LIMIT
no need in second query neither explicit nor implicit
your next/previous window results would not ever have any misses or duplicates comparing to previously seen windows.
The only drawback here is that original row user is referring to can be deleted as well. To overcome this, you might want to add a boolean deleted flag to your table schema and set it to false instead of actual row deletion.
After reading #shomeax 's comment and thinking a little more I can suggest to encode cursor in base-64 and make it to contain prev cursor additionally. For example:
[$prevCursor, $curCursor] = explode(':', base64_decode($request['cursor']));
$limit = 10;
$prevPlusCurPagesLimit = $limit * 2;
$ulids = 'SELECT ulid FROM Table WHERE ulid <= $prevCursor ORDER BY ulid DESC LIMIT $prevPlusCurPagesLimit+1';
$resultUlids = array_slice($ulids, $limit, $limit);
$nextCursor = array_slice($ulids, $prevPlusCurPagesLimit, 1);
$prevPrevCursor = reset($ulids);
$response = [
'data' => $resultUlids,
'prevCursor' => base64_encode("$curCursor:$nextCursor"),
'nextCursor' => base64_encode("$prevPrevCursor:$prevCursor"),
];
I didn't try such approach myself but it is partially based on this article https://slack.engineering/evolving-api-pagination-at-slack/ and looks like working
You don’t need to request more, only use < and > rather than <= and >=.
Then you can use the last id in $results for next and the first id for previous.
Assuming that
the "cursor" is the ID from which the next portion starts
and IDs are sequential and without gaps,
and limit is not changed from one query to another
you could get the prev cursor by substracting/adding (depending on sorting) $limit from/to current cursor: $prevCursor = $results[0]['id'] - $limit
And if the ID column has gaps, I suppose there is no reasonable way to implement ability to get prev cursor without additional query. You could only turn it into sub-query or use UNION, but this does not make a big difference.
Consider this...
You fetch the 5 rows for the current page. Then the "previous" page ends before the first id on this page and the "next" page starts after the last id on the current page.
Example: The current page contains ids 65, 67, 71, 82, 91. This finds the 5 rows for the previous page:
SELECT ...
WHERE id < 65
ORDER BY id DESC
LIMIT 5;
(They will be in reverse order, but that is easy to fix.) For the "next" page (in proper order):
SELECT ...
WHERE id > 91
ORDER BY id ASC
LIMIT 5;
As another tip: fetching an extra row (6 instead of 5), lets you cheaply discover whether you are at the "end", thereby being able to suppress the [Next] or [Prev] button.
More: Pagination
Granted, this technique does not deliver the 5 rows for the next/previous page, but do you really need that? Since the Selects should be quite efficient, I don't necessarily see a drawback of doing more than one Select or combining selects with UNION.
I am going to delete my previous answer, despite its upvote, because it is clearly the wrong approach.
Update
Given that you are retrieving on a column, id, that is presumably unique and indexed, then when a "page" of N rows is returned (where N is 10), you need to pass up either the id of the first row (the one with the greater value since we are sorting by descending id) or the last row as query parameter lastId along with a direction flag parameter directionFlag that is either F for forward or B for backward to give the direction of "paging." It should then be possible to directly seek to the correct rows as follows (I am assuming that we are using PDO for MySql Access):
define(PAGE_SIZE, 10);
$limit = PAGE_SIZE;
// lastId parameter specified? It will not be present on the initial request:
if (isset($_REQUEST('lastId')) {
$lastId = $_REQUEST['lastId'];
$direction = $_REQUEST['direction']; 'F'orward or 'B'ackward
if ($direction == 'F') {
$sql = "SELECT * FROM Table ORDER BY id DESC where id < :id LIMIT $limit";
}
else {
// paging backward:
$sql = "SELECT * from (SELECT * FROM Table ORDER BY id ASC where id > :id LIMIT $limit) sq ORDER BY id DESC";
}
$params = [':id' => $lastId];
}
else {
// this is the initial request:
$sql = "SELECT * FROM Table ORDER BY id DESC LIMIT $limit";
$params = [];
}
$stmt = $pdo->prepare($sql);
$stmt->execute($params);
$rows = $stmt->fetchAll(PDO::FETCH_ASSOC);
// The id from $rows[0] will be passed back as lastId with direction flag 'B' for paging backward
// And the id from $rows[PAGE_SIZE-1] will be passed back as lastId with direction flag 'F' for paging forward
Whilst populating a table based on ids and labels from different tables, it appeared apparent there must potentially be a better way of achieving the same result with less code and a more direct approach using LEFT JOIN but i am puzzled after trying to work out if its actually capable of achieving the desired result.
Am i correct in thinking a LEFT JOIN is usable in this instance?
Referencing two tables against one another where one lists id's related to another table and that other table has the titles allocated for each reference?
I know full well that if theres independent information for each row LEFT JOIN is suitable, but where theres in this case only several ids to reference for many rows, i just am not clicking with how i could get it to work...
The current way i am achieving my desired result in PHP/MySQL
$itemid = $row['item_id'];
$secid = mysql_query(" SELECT * FROM item_groups WHERE item_id='$itemid' ");
while ($secidrow = mysql_fetch_assoc($secid)) {
//echo $secidrow["section_id"]; //testing
$id = $secidrow["section_id"];
$secnameget = mysql_query(" SELECT * FROM items_section_list WHERE item_sec_id='$id' ");
while ($secname = mysql_fetch_assoc($secnameget)) {
echo $secname["section_name"];
}
}
Example of the data
Item groups
:drink
:food
:shelf
Item List
itemId, groupId
Group List
groupId, groupTitle
The idea so outputting data to a table instead of outputting "Item & Id Number, in place of the ID Number the title actually appears.
I have achieved the desired result but i am always interested in seeking better ways to achieve the desired result.
If I've deciphered your code properly, you should be able to use the following query to get both values at the same time.
$itemid = $row['item_id'];
$secid = mysql_query("
SELECT *
FROM item_groups
LEFT JOIN items_section_list
ON items_section_list.item_sec_id = item_groups.section_id
WHERE item_id='$itemid'
");
while ($secidrow = mysql_fetch_assoc($secid)) {
//$id = $secidrow["section_id"];
echo $secidrow["section_name"];
}
How can I make a limit of showing the results? I need to limit it for 100 views.
In DB I have:
ID|NAME|PAGE|COUNT|DATE
In count I want to count untill 100 and then stop showing that ID. I could do it with count < 100. And then update the specific ID. I could get records with less than 100 views, but I couldn't manage to update count on the specific ID.
Row is showed with:
php code:
foreach($bannerGroups[0] as $ban) {
echo '<li class="right1">'.$ban->html().'</li>';
}
But I just don't know where to put the update in there. I tried, but all I got was to update only one ID. But it shows 4 on one page and randomizes them on refresh. So I don't know what to do.
Also I would like to say I am only learning php. Sorry for all the mess.
Code at http://pastebin.com/A9hJTPLE
If I understand correctly, you want to show all banners that have been previously-displayed less than 100 times?
If that's right, you can just add that to your WHERE clause:
$bannerResult = mysql_query("SELECT * FROM table WHERE page='cat' WHERE `COUNT` < 100");
To update them all, you can either run a query while displaying each individual banner, or "record" the id of each and run a single query at the end, like:
$ids = array();
foreach($bannerGroups[0] as $ban) {
$ids[] = $ban['ID']; // record the ID; don't know how Banner
// class works, assuming uses indexes; maybe ID() method?
echo '<li class="right1">'.$ban->html().'</li>';
}
...
mysql_query('UPDATE table SET `COUNT` = `COUNT` + 1 WHERE ID IN (' . join(',', $ids) . ')');
UPDATE:
Based off of a comment, your Banner class doesn't have a method to retrieve the individual banner's ID. In this case, you can record the ID values when you're building your banners array:
$ids = array();
while($row=mysql_fetch_assoc($bannerResult)) {
$banners[] = new Banner($row);
$ids[] = $row['ID']; // record the ID
}
// update the `count` on each record:
mysql_query('UPDATE table SET `COUNT` = `COUNT` + 1 WHERE ID IN (' . join(',', $ids) . ')');
sorry, but I got your question wrong...
first you have to insert a new sql-column like "viewcount" to the db...
on every read, you have to increment the value in viewcount...
for that behaviour (because, mysql does not allow sub-selects on update-clause on the same table), you have to fetch the results from db, as you do that, and pass all the primary-keys of the records to an array...
after the view-logic you have to fire up a query like:
UPDATE foo SET viewcount = viewcount + 1 WHERE id IN (1,2,3,4,5,6...,100);
where the IN-clause can be easily generated using your primary-keys-array with "implode(',', $arr);"
hope this helps.
$bannerResult = mysql_query("SELECT * FROM table WHERE page='cat' AND `count`<100");
#newfurniturey figured it out. in each foreach($banneruGroups added: $ids = $ban->getValue('id'); and then mysql_query("UPDATE dataa SET COUNT = COUNT + 1 WHERE id = '$ids'"); but is there any way to update them by adding query only once? And if the id is showed already 100 times i get Warning: Invalid argument supplied for foreach() in. Any idea how to fix it? I have 4 ids in DB . If one of them already have 100 views (count) then i get error!
Try to limit your data source for 100 items.
It's like OFFSET x LIMIT 100 in MySQL/PostgreSQL query or TOP 100 in MSSQL.
I try to build a variable that integrates some other variable.
one of that will be the number of an auto-increment-field where later on an insert-query will happens.
I tried to use:
$get_num = $db/*=>mysqli*/->query("SELECT COUNT (*) auto_increment_column FROM table1");
$num = $query->fetch_assoc($get_num);
$end = $num + 1;
I don't have any update/insert query before that so I can't use
$end = $db->insert_id;
that's why i thought i can just count the numbers of the auto_increment rows and have my last variable that is necessary to build my new variable.
for a reason this wonT count the entries and outputs 0. i dont understand why this happens.
i really would appreciate if there is someone who could tell me what am i doing wrong. thanks a lot.
UPDATE
For everyone who likes to know about what's the goal:
I like to create a specific name or id for a file that later on will be created by the input of the fields from the insert query. I like to have an unique key. this key consists of an user_id and a timestamp. at the end of this generated variable it should be placed the auto_increment nr. of the query that will be placed in the table. so the problem is, that I create an variable before the insert query happens so that this variable will be part of the insert query like:
$get_num = $db->query("SELECT COUNT (*) FROM tableA");
$num = $query->fetch_assoc();
$end = $num + 1;
$file_id = $id .".". time() .".". $end;
$insert = $db->query("INSERT INTO tableA ( file_id, a, b, c) VALUES('".$file_id."','".$a."','".$b."','".c."')");{
hope now, it will be clear what I like to approach.
If you need an auto-incrementing column in MySQL then you should use AUTO_INCREMENT. It implements it all for you and avoids race conditions. The manual way you are trying to implement it has a couple of flaws, namely
If two scripts are trying to insert concurrently they might both get the same COUNT (say 10) and hence both try to insert with ID 11. One will then fail (or else you will have duplicates!)
If you add 10 items but then delete item 1, the COUNT will return 9 but id 10 will already exist.
try
SELECT COUNT(*) FROM table1
I have the following 3 tables in the database.
Programs_Table
Program_ID (Primary Key)
Start_Date
End_Date
IsCompleted
IsGoalsMet
Program_type_ID
Programs_Type_Table(different types of programs, supports a dropdown list in the form)
Program_type_ID (Primary Key)
Program_name
Program_description
Client_Program_Table
Client_ID (primary key)
Program_ID (primary key)
What is the best way to find out how many clients are in a specific program (program type)?
Would the following SQL statement be the best way, or even plausible?
SELECT Client_ID FROM Client_Program_Table
INNER JOIN Programs_Table
ON Client_Program_Table.Program_ID = Programs_Table.Program_ID
WHERE Programs_Table.Program_type_ID = "x"
where "x" is the Program_type_ID of the specific program we're interested in.
OR is the following a better way?
$result = mysql_query("SELECT Program_ID FROM Programs_Table
WHERE Program_type_ID = 'x'");
$row = mysql_fetch_assoc($result);
$ProgramID = $row['Program_ID'];
$result = mysql_query("SELECT * FROM Client_Program_Table
WHERE Program_ID = '$ProgramID'");
mysql_num_rows($result) // returns how many rows of clients we pulled.
Thank you in advance, please excuse my inexperience and any mistakes that I've made.
Here is how you can do it:
<?php
// always initialize a variable
$number_of_clients = 0;
// escape the string which will go in an SQL query
// to protect yourself from SQL injection
$program_type_id = mysql_real_escape_string('x');
// build a query, which will count how many clients
// belong to that program and put the value on the temporary colum "num_clients"
$query = "SELECT COUNT(*) `num_clients` FROM `Client_Program_Table` `cpt`
INNER JOIN `Programs_Table` `pt`
ON `cpt`.`Program_ID` = `pt`.`Program_ID`
AND `pt`.`Program_type_ID` = '$program_type_id'";
// execute the query
$result = mysql_query($query);
// check if the query executed correctly
// and returned at least a record
if(is_resource($result) && mysql_num_rows($result) > 0){
// turn the query result into an associative array
$row = mysql_fetch_assoc($result);
// get the value of the "num_clients" temporary created column
// and typecast it to an intiger so you can always be safe to use it later on
$number_of_clients = (int) $row['num_clients'];
} else{
// query did not return a record, so we have no clients on that program
$number_of_clients = 0;
}
?>
If you want to know how many clients are involved in a program, you'd rather want to use COUNT( * ). MySQL (with MyISAM) and SQL Server have a fast way to retrieve the total number of lines. Using a SELECT(*), then mysql_num_rows leads to unnecessary memory ressources and computing time. To me, this is the fastest, though not the "cleanest" way to write the query you want:
SELECT
COUNT(*)
FROM
Client_Program_Table
WHERE
Program_ID IN
(
SELECT
Program_ID
FROM
Programs_Table
WHERE
Program_type_ID = 'azerty'
)
Why is that?
Using JOIN make queries more readable, but subqueries often prove to be computed faster.
This returns a count of the clients in a specific program type (x):
SELECT COUNT(cpt.Client_ID), cpt.Program_ID
FROM Client_Program_Table cpt
INNER JOIN Programs_Table pt ON cpt.Program_ID=pt.Program_ID
WHERE pt.Program_type_ID = "x"
GROUP BY cpt.Program_ID