I've a classic pagination system using LIMIT startrecord, endrecord and I want to figure out in what page number an X record is located.
The only idea I've right now is to seek recursively all the records to find it out. But I'm looking for a much more "economic" method!
Any ideas ?
You could use a sub query to create a table with the results and their position, then query that for the specific entry you are looking at:
SET #rank=0;
SELECT rank, record
FROM (
SELECT
#rank:=#rank+1 AS rank,
record
FROM table
) as subquery
WHERE record = x;
The returned table would show the record an the rank it appeared in the original query. You can the divide the rank by the number of results per page... Or build it into the query. Hope this helps.
Cout the number of records that are prior to the one you are looking for. This requires you to assume an order for your query which is natural.
SELECT COUNT(id) AS c
FROM tbl
WHERE sort_field < ((SELECT sort_field FROM tbl WHERE id = 18))
OR (sort_field = ((SELECT sort_field FROM tbl WHERE id = 18)) AND id < 18);
Then just retrieve the c and calculate ceilling(c/page_size). This will give you the page number that your record will fall in. The only important thing to remember is that you need to sort the records in the same order as you would in your query with limit.
To describe what the query does, it counts the number records that stand before the record with id 18. The only tricky part is with records with the same value as for their sort_field in which MySQL will make use of primary key and in our case the id. And that's why we have the OR part in our condition. In my answer I'm assuming you are sorting your original query (with limit statement in it) ascending, but if you are sorting descending then you need to change all of < to >.
Use something like this with your query as part of the s subselect
SELECT s.row, s.RECORD, YOUR_OTHER_FIELDS...
FROM (SELECT #row := 0) cnt
JOIN (SELECT #row := #row + 1 row, RECORD, ...YOUR QUERY WITH ORDER BY ...) s
WHERE s.RECORD = <desired record number>
and divide row by the pagesize from your pagination.
Concrete but nonsensical example:
SELECT p.row, p.id
FROM (SELECT #row := 0) cnt
JOIN (SELECT #row := #row + 1 row, id FROM products ORDER BY id desc) p
WHERE p.id = 485166
As intended, the value of row changes with the order you use in the subselect.
It folds the variable initialization into the query so this is only one statement.
It also does not depend on a natural order or distribution of rows - as long as the order they ARE returned in stays the same for whatever ORDER you specify (or leave out).
if this is something that you will use often, i think it is a good idea to create an stored procedure or a function. We can use a cursor inside, to iterate through the results and get the position of the desired item. I think this will be faster, it wont have to iterate to all the records, and dont need a subquery (for all this i would say that it is more economic) and you can use order, join, and whatever you need.
DELIMITER $$
CREATE FUNCTION position ( looking_for INT )
RETURNS INT
READS SQL DATA
BEGIN
-- First we declare all the variables we will need
DECLARE id INT;
DECLARE pos INT;
SET pos=0;
-- flag which will be set to true, when cursor reaches end of table
DECLARE exit_loop BOOLEAN;
-- Declare the sql for the cursor
DECLARE pos_cursor CURSOR FOR
SELECT id
FROM your_table
--you can use where, join, group by, order and whatever you need
--end of query
-- Let mysql set exit_loop to true, if there are no more rows to iterate
DECLARE CONTINUE HANDLER FOR NOT FOUND SET exit_loop = TRUE;
-- open the cursor
OPEN example_cursor;
-- marks the beginning of the loop
example_loop: LOOP
-- read the id from next row into the variable id
FETCH pos_cursor INTO id;
-- increment the pos var
SET pos=pos+1;
-- check if we found the desired item,
-- if it has been set we close the cursor and exit
-- the loop
IF id=looking_for THEN
CLOSE example_cursor;
LEAVE example_loop;
END IF;
-- check if the exit_loop flag has been set by mysql,
-- if it has been set we close the cursor and exit
-- the loop
IF exit_loop THEN
CLOSE example_cursor;
LEAVE example_loop;
END IF;
END LOOP example_loop;
RETURN pos;
END $$
DELIMITER ;
You create the function just once, and for using it, you just need to use this sql:
CALL position(ID_OF_THE_ITEM_YOU_ARE_LOOKING_FOR);
and it returns the position of the item, in the position [0][0] of the returned rowset.
Of course instead of the id you can create a function that compares the name, or any other field, or even more than one.
If the query is always diferent, then you cannot use a function, but you can still use the cursor (the syntax will be the same). You can build the cursor in your PHP, let pos be a System variable (using #pos), and in any case just add the specific sql of the query (the part between DECLARE pos_cursor CURSOR FOR and --end of query)
You can't really create an "economic" way. You have to get the full list of records from the DB since there is no way to know the position of a record from MySQL.
Depending on your sorting, the frequency at which the data changes, you could assign the record its position in a column: add column position to the table you are querying. That might not be feasible in all cases.
Related
I would like to create a MySql function that will return an incremental row count as long as the given id is the same and if the id changes function would reset the count starting from 1.
Below is a result I am looking for, where you can see as long as the itemId (on left column) remains the same, the Count on right column will increments, and when itemId changes the Count will restart from 1.
In my mind, the MySql function like the one below would do the incremental counting and resetting, but unfortunately it returns 1 for each row. My thought was to provide the current itemId to the function and the function would compare the sent in id to to the one saved in #n session variable from last row, and as long as the id's are the same the function would return incremented row count, else it would reset to 1.
Can anybody guide me to why this function is not working? Or is there a better way to achieves the result I am looking for?
CREATE FUNCTION `nth`(id int) RETURNS tinyint(4)
BEGIN
declare ln tinyint;
if #saved_id = id then
set #n := #n+1;
set ln = #n;
else
set #saved_id := id;
set #n := 1;
set ln = #n;
end if;
RETURN ln;
END
The Mysql version I am using is 5.7
Here is the example query I am using, the itemId is foreign key
select id, itemId, started_at 'Start', stopped_at Stop, nth(started_at) 'Count'
from events
order by itemId, stopped_at
You don't need to define a UDF for this. You can achieve this within a SELECT query itself. In newer versions of MySQL (8.0.2 and above), it is achievable using ROW_NUMBER() OVER (PARTITION BY itemId ORDER BY id)
In older version, we can use the user-defined variables. In a Derived Table (subquery inside the FROM clause), we order our data such that all the rows having same itemId values come together, with further sorting between them based on id.
Now, we use this result-set and use conditional CASE..WHEN expressions to evaluate the numbering ("count"). It will be like a Looping technique (which we use in application code, eg: PHP). We would store the previous row values in the User-defined variables, and then check the current row's value(s) against the previous row. Eventually, we will assign row number ("Count") accordingly.
SELECT
dt.id,
dt.Start,
dt.Stop,
#rn := CASE WHEN dt.itemId = #itm THEN #rn + 1
ELSE 1
END AS Count,
#itm := dt.itemId AS itemId
FROM
(
SELECT
id,
itemId,
started_at AS Start,
stopped_at AS Stop
FROM events
ORDER BY itemID, id
) AS dt
CROSS JOIN (SELECT #itm := 0, #rn := 0) AS user_init_vars
I'm trying to sum columns x through x+n in an SQL table. Essentially, I have multiple tables that contain grades in them and a user_id. I want to sum all the grades to come up with a total grade column without specifying the column names as the names and number of columns changes with each table. For instance, one table might have columns (user_id, calculations, prelab, deductions) while another might have (user_id, accuracy, precision, graphs, prelab, deductions).
I could rename my columns col1, col2, col3, col4, col5, etc., but I can't figure out how to get around the varying number of columns.
As far as I know, there is no way to sum groups of columns without actually specifying the column names directly in SQL. It seems to me like this is a badly designed schema, but that's a separate topic.
In any your case, you're going to need to create a new column in each table that contains the sum of all the grades in that particular table, say called total, and then, do something like this:
select user_id, sum(table1.total, table2.total, table3.total)
from table1, table2, table3
where table1.user_id = table2.user_id
and table2.user_id = table3.user_id
group by user_id
1) You could write some pl/sql to go and hit the data dictionary and get the columns and then construct dynamic sql to do the work of adding them up correctly.
2) Or you could create views on top of the tables that contain the user_id and the sum of the interesting columns (the views themselves could be constructed programmatically - but that only needs to happen once rather than every time you want the totals).
But either of the above is probably over-kill compared to simply fixing your schema.
The following procedure would likely do the trick.
It will look for all column names for the given tableName in the INFORMATION_SCHEMA.COLUMNS table (excluding 'userid' - This may be subject to change if the name you use is different).
The procedure also creates a temporary table (this is also subject to improvement - it would probably be better to do a 'drop if exists before the create) to store the sum up to a point.
The items inside the loop is just building an SQL UPDATE statement with the given tableName argument and the columnName from the cursor and doing the math.
To test this (after creation):
call myProcedure('tableName');
DELIMITER //
DROP PROCEDURE IF EXISTS myProcedure //
CREATE PROCEDURE
myProcedure( tableName varchar(32) )
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE columnName varchar(64);
DECLARE cur1 CURSOR FOR SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = tableName and COLUMN_NAME <> 'userid';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur1;
CREATE TEMPORARY TABLE intermediateresults(userid integer, sumOfScores integer);
SET #st1 = CONCAT('INSERT INTO intermediateresults (SELECT DISTINCT userid, 0 FROM ',tableName,' )' );
PREPARE stmt3 FROM #st1;
EXECUTE stmt3;
looping: LOOP
FETCH cur1 into columnName;
IF done THEN
LEAVE looping;
END IF;
SET #st1 = CONCAT('UPDATE intermediateresults set sumOfScores = sumOfScores + COALESCE( (SELECT ', columnName, ' FROM ',tableName, ' t WHERE t.userid=intermediateresults.userid) , 0)' );
PREPARE stmt3 FROM #st1;
EXECUTE stmt3;
DEALLOCATE PREPARE stmt3;
END LOOP;
CLOSE cur1;
SELECT * FROM intermediateresults;
DROP table intermediateresults;
END
//
DELIMITER ;
What might be of interest when doing this kind of thing:
INFORMATION_SCHEMA also has data on:
DATA_TYPE: which can be used to test if a specific column has the actual type you are expecting - a condition such as DATA_TYPE='int' can be added to the cursor definition to make sure that it is in fact an int (assuming that the columns to be summed are in fact INTs)
ORDINAL_POSITION: which can be used if you know in which order the columns are supposed to arrive (for cases where the last four are housekeeping, for instance)
TABLE_SCHEMA: the procedure above rather assumes that the table is only present in the current default schema. Using this would require an additional parameter in the procedure and a slight change in the constructed SQL statements.
I have a stored procedure that is outputting my results in XML format; all is good with the output.
I need to however include the row count it is outputting if possible along with it.
SELECT A.[id],
A.[petName],
A.[petCaption],
B.[petType],
C.[FirstName] as ownerFirstName,
C.[LastName] as ownerLastName,
A.[imageName],
(
SELECT CONVERT(varchar(20),sum(transactionAmount), 1) as totalRaised
FROM petContestTransactionsDonations
WHERE submissionID = A.[id]
FOR XML PATH ('transactionDetails'), TYPE, ELEMENTS
)
FROM petContestSubmissions as A
LEFT OUTER JOIN petContestTypes as B
ON A.[petType] = B.[id]
LEFT OUTER JOIN EmpTable as C
ON A.[empID] = C.EmpID
LEFT OUTER JOIN petContestTransactionsEntries as E
ON E.[submissionID] = A.[id]
WHERE E.[transactionStatus] = 'completed'
ORDER BY A.[id]
OFFSET #offset ROWS
FETCH NEXT #rows ROWS ONLY
FOR XML PATH ('submission'), TYPE, ELEMENTS, ROOT ('root');
I am passing offset and rows to the stored procedure as I am using it with pagination. It is getting records for that page limiting to X rows.
Even though I say get me the next 10 records, it may only have 8 left. That's what I need to return; the total records it found in the select statement.
Is it best to do that in the stored procedure or in my php that is looping over the records?
If you want the row count on each row, then include:
count(*) over () as RowCount
in the outermost select clause.
If you want to read all the data into the application first, then do it in the application layer.
EDIT:
If you want the number of rows returned by the query, then you might as well do it in the application layer. You can also do:
select t.*, count(*) over () as RowCOunt
from (<your query here>) t;
Maybe this will meet your needs. Not in the same query, but you can try
SELECT ##ROWCOUNT AS RowsAffected
immediately after your query, you should get what you are looking for. ##ROWCOUNT is per session, so as long as you don't end your connection, should be ok.
I need to be able to safely insert a row with first available ID. I read alot answers about PRIMARY_KEY and AUTO_INCREMENT and all this stuff, but this is something else. I need to keep and be able to work on database with fixed ID range from 1 to 60000. Is there any way to do that with MySQL? Writing own function that check what is nearest "free" ID, is not an option cause there may be conflicts on multiuser usage.
In the best case scenario,MySQL would somehow work as with PRIMARY_KEY, but reusing keys.
Follow these steps:
1) Create a sequence table with columns id, rowstate.
2) Insert ids 1-60000 to that with the rowstate 1
3) Whenever you want to insert in your main table, search for the lowest id from the sequence table having rowstate=1 and update the sequence to -1.
When you want to delete a record from the main table, set the rowstate of the id to 1.
you are write, you need to concider the the concurrency issues
hence you need to implement a table lock mechnism
1) Lock mysql table
2) Insert the record, you can just use the auto_increment property since no two values would be added at the same time (i dont think you have to lock tables if this is used anyway)
3) If you dont want to use auto_increment, any of the above suggested code will work
You can try like this:
INSERT INTO tableName (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE id=(Select max(id)+1 from tableName);
For more info: http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
OR
Getting highest id from table and increament it and insert your row.
Select max(id) from tableName;
You will get the id. Than add 1 it and insert into table. Additionally you can check it should be less than 60000.
I know this is not the best answer, but can be consider as second alternative.
To get the first id that is free you can do
select id - (id-ra) as lowest
from
(
select id, #r := #r + 1 as ra
from t, (select #r := 0) rank
order by id
)
x
where ra <> id
limit 1
SQLFiddle demo
You can put that in a procedure where you lock the table during the operation
delimiter |
CREATE PROCEDURE create_save ()
BEGIN
LOCK TABLES your_table WRITE;
set #lowid := 0;
select #lowid := id - (id-ra)
from
(
select id, #r := #r + 1 as ra
from your_table, (select #r := 0) rank
order by id
)
x
where ra <> id
limit 1;
if #lowid between 1 and 59999
then
insert into your_table (id, othercolumn)
select #lowid, 12345;
end if;
UNLOCK TABLES;
end
|
delimiter ;
Just working with a database and some tests were done recently which checked the integrity of the setup.
As a result, a lot of test entries were added which were then deleted. However, when new entries are added, the ID number value continues from after the entries added.
What I want:
ID increases by one from where it left off before the additional rows were added:
4203, 4204, 4205, 4206 etc.
What is happening:
ID increases by one from after the additional rows ID:
4203, 4204, 6207, 6208 6209 etc.
Not sure where to fix this...whether in phpmyadmin or in the PHP code. Any help would be appreciated. Thanks!
I have ran into this before and I solve it easily with phpMyAdmin. Select the database, select the table, open the operations tab, and in the Table Options set the AUTO_INCREMENT to 1 then click GO. This will force mysql to look for the last auto incremented value and then set it to the value directly after that. I do this on a manually basis that way I know that when a row is skipped that it was not from testing but a deletion because when I test and delete the rows I fix the AI value.
I don't think there's a way to do this with an auto-incrementing ID key.
You could probably do it by assigning the ID to (select max(id) + 1 from the_table)
You could drop the primary key then recreate it, but this would reassign all the existing primary keys so could cause issues with relationships (although if you don't have any gaps in your primary key you may get away with it).
I would however say that you should accept (and your app should reflect) the possibility of missing IDs. For example in a web app if someone links to a missing ID you would want a 404 returned not a different record.
There should be no need to "reset" the id values; I concur with the other comments concerning this issue.
The behavior you observe with AUTO_INCREMENT is by design; it is described in the MySQL documentation.
With all that said, I will describe an approach you can use to change the id values of those rows "downwards", and make them all contiguous:
As a "stepping stone" first step, we will create a query that gets a list of the id values that we need changed, along with a proposed new id value we are going to change it to. This query makes use of a MySQL user variable.
Assuming that 4203 is the id value you want to leave as is, and you want the next higher id value to be reset to 4204, the next higher id to be reset to 4205, etc.
SELECT s.id
, #i := #i + 1 AS new_id
FROM mytable s
JOIN (SELECT #i := 4203) i
WHERE s.id > 4203
ORDER BY s.id
(Note: the constant value 4203 appears twice in the query above.)
Once we're satisfied that this query is working, and returning the old and new id values, we can use this query as an inline view (MySQL calls it a derived table), in a multi-table UPDATE statement. We just wrap that query in a set of parentheses, and give assign it an alias, so we can reference it like a regular table. (In an inline view, MySQL actually materializes the resultset returned by the query into a MyISAM table, which probably explains why MySQL refers to it as a "derived table".)
Here's an example UPDATE statement that references the derived table:
UPDATE ( SELECT s.id
, #i := #i + 1 AS new_id
FROM mytable s
JOIN (SELECT #i := 4203) i
WHERE s.id > 4203
ORDER BY s.id
) n
JOIN mytable t
ON t.id = n.id
SET t.id = n.new_id
ORDER BY t.id
Note that the old id value from the inline view is matched to the id value in the existing table (the ON clause), and the "new_id" value generated by the inline view is assigned to the id column (the SET clause.)
Once the id values are assigned, we can reset the AUTO_INCREMENT value on the table:
ALTER TABLE mytable AUTO_INCREMENT = 1;
NOTE: this is just an example, and is provided with the caveat that this should not be necessary to reassign id values. Ideally, primary key values should be IMMUTABLE i.e. they should not change once they have been assigned.