Related
The requirement is to find the first available identifier where an identifier is an alphanumeric string, such as:
ABC10000
ABC10345
ABC88942
ABC90123
The database table has a structure such as:
id, user, identifier
Note that the alpha component ABC is consistent throughout and won't change. The numeric component should be between 10000 and 99999.
How best to tackle this? It does not seem like an overly complex problem - looking for simplest solution using either just MySQL or a combination of SQL and PHP. The current solution pulls each record from the database into an array and then loops from 10000 onwards, prepending ABC and checking availability, which seems like it could be improved significantly.
Edit: Original question was not clear enough in that a certain amount of identifiers have been assigned already, and I am looking to fill in the gaps. From the short list I provided, the next available would be ABC10001. Eventually, however, it would be ABC10346 and then ABC88943 and so on
Edit: Sorry for a poorly structured question. To avoid any further confusion, here is the actual table structure:
CREATE TABLE IF NOT EXISTS `User_Loc` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user` int(11) DEFAULT NULL,
`value` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `UNIQ_64FB41DA17323CBC` (`user`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=4028 ;
You have to self join the table and look for the first NULL value in the joined table
SELECT CONCAT('ABC', SUBSTRING(t1.value, 4)+1) AS next_value
FROM test t1
LEFT JOIN test t2 on SUBSTRING(t1.value, 4)+1 = SUBSTRING(t2.value, 4)
WHERE ISNULL(t2.value)
ORDER BY t1.value ASC
LIMIT 1
http://sqlfiddle.com/#!2/d69105/22
edit
With the comment about some 'specialities' at ncatnow. There are slight adjusments to make with the help of subselects for ridding the 'ABC' and UNION for having a default value
SELECT
CONCAT('ABC', t1.value+1) AS next_value
FROM
((SELECT '09999' AS value) UNION (SELECT SUBSTRING(value, 4) AS value FROM test)) t1
LEFT JOIN
((SELECT '09999' AS value) UNION (SELECT SUBSTRING(value, 4) AS value FROM test)) t2
ON t1.value+1 = t2.value
WHERE
ISNULL(t2.value)
AND t1.value >= '09999'
ORDER BY t1.value ASC
LIMIT 1
http://sqlfiddle.com/#!2/28acf6/50
Similar to the above reply by #HerrSerker, but this will cope with existing identifiers which have the numeric part starting with a zero.
SELECT CONCAT('ABC',SUBSTRING(CONCAT('00000', CAST((CAST(SUBSTRING(a.identifier, 4) AS SIGNED) + 1) AS CHAR)), -5)) AS NextVal
FROM SomeTable a
LEFT OUTER JOIN SomeTable b
ON b.identifier = CONCAT('ABC',SUBSTRING(CONCAT('00000', CAST((CAST(SUBSTRING(a.identifier, 4) AS SIGNED) + 1) AS CHAR)), -5))
WHERE b.identifier IS NULL
ORDER BY NextVal
LIMIT 1
what comes to my mind is one table with all indentifiers and use this sql
SELECT identifier FROM allIdentifiersTable WHERE identifier NOT IN (SELECT identifier FROM yourTable) LIMIT 1
Reconsidering this from your edit, you should go the PHP route and add another table or other means to store the last filled id:
$identifier = 0;
$i = mysql_query("SELECT identifier FROM last_identifier");
if ($row = mysql_fetch_assoc($i)) $identifier = $row["identifier"];
if ($identifier < 10000) $identifier = 10000;
do {
$identifier += 1;
$result = mysql_query("
INSERT IGNORE INTO table (id, user, identifier)
VALUES ('[...]', '[...]',
'ABC" . str_pad($identifier, 5, "0", STR_PAD_LEFT) . "'
)");
if (mysql_affected_rows($result) < 1) continue;
} while (false);
mysql_query("UPDATE last_identifier SET identifier = '$identifier'");
Of course, you need to add a UNIQUE index on the identifier field.
so I'm trying to create a ranking system for my website, however as a lot of the records have same number of points, they all have same rank, is there a way to avoid this?
currently have
$conn = $db->query("SELECT COUNT( * ) +1 AS 'position' FROM tv WHERE points > ( SELECT points FROM tv WHERE id ={$data['id']} )");
$d = $db->fetch_array($conn);
echo $d['position'];
And DB structure
`id` int(11) NOT NULL,
`name` varchar(150) NOT NULL,
`points` int(11) NOT NULL,
Edited below,
What I'm doing right now is getting records by lets say
SELECT * FROM tv WHERE type = 1
Now I run a while loop, and I need to make myself a function that will get the rank, but it would make sure that the ranks aren't duplicate
How would I go about making a ranking system that doesn't have same ranking for two records? lets say if the points count is the same, it would order them by ID and get their position? or something like that? Thank you!
If you are using MS SQL Server 2008R2, you can use the RANK function.
http://msdn.microsoft.com/en-us/library/ms176102.aspx
If you are using MySQL, you can look at one of the below options:
http://thinkdiff.net/mysql/how-to-get-rank-using-mysql-query/
http://www.fromdual.ch/ranking-mysql-results
select #rnk:=#rnk+1 as rnk,id,name,points
from table,(select #rnk:=0) as r order by points desc,id
You want to use ORDER BY. Applying on multiple columns is as simple as comma delimiting them: ORDER BY points, id DESC will sort by points and if the points are the same, it will sort by id.
Here's your SELECT query:
SELECT * FROM tv WHERE points > ( SELECT points FROM tv WHERE id ={$data['id']} ) ORDER BY points, id DESC
Documentation to support this: http://dev.mysql.com/doc/refman/5.0/en/sorting-rows.html
Many Database vendors have added special functions to their products to do this, but you can also do it with straight SQL:
Select *, 1 +
(Select Count(*) From myTable
Where ColName < t.ColName) Rank
From MyTable t
or to avoid giving records with the same value of colName the same rank, (This requires a key)
Select *, 1 +
(Select Count(Distinct KeyCol)
From myTable
Where ColName < t.ColName or
(ColName = t.ColName And KeyCol < t.KeyCol)) Rank
From MyTable t
I have a query that gets all the info I need for a messaging system's main page (including unread message count, etc)... but it currently retrieves the original threads message. I would like to augment the below query to grab the most recent message in each thread instead.
This query is very close, however my mediocre SQL skills are keeping me from wrapping things up...
$messages = array();
$unread_messages_total = 0;
$messages_query = "
SELECT m.*
, COUNT(r.id) AS num_replies
, MAX(r.datetime) AS reply_datetime
, (m.archived NOT LIKE '%,".$cms_user['id'].",%') AS message_archive
, (m.viewed LIKE '%,".$cms_user['id'].",%') AS message_viewed
, SUM(r.viewed NOT LIKE '%,".$cms_user['id'].",%') AS unread_replies
, CASE
WHEN MAX(r.datetime) >= m.datetime THEN MAX(r.datetime)
ELSE m.datetime
END AS last_datetime
FROM directus_messages AS m
LEFT JOIN directus_messages as r ON m.id = r.reply
WHERE m.active = '1'
AND (m.to LIKE '%,".$cms_user['id'].",%' OR m.to = 'all' OR m.from = '".$cms_user['id']."')
GROUP BY m.id
HAVING m.reply = '0'
ORDER BY last_datetime DESC";
foreach($dbh->query($messages_query) as $row_messages){
$messages[] = $row_messages;
$unread_messages_total += (strpos($row_messages['archived'], ','.$cms_user['id'].',') === false && ( (strpos($row_messages['viewed'], ','.$cms_user['id'].',') === false && $row_messages['unread_replies'] == NULL) || ($row_messages['unread_replies']>0 && $row_messages['unread_replies'] != NULL) ) )? 1 : 0;
}
Thanks in advance for any help you can provide!
EDIT: (Database)
CREATE TABLE `cms_messages` (
`id` int(10) NOT NULL auto_increment,
`active` tinyint(1) NOT NULL default '1',
`subject` varchar(255) NOT NULL default '',
`message` text NOT NULL,
`datetime` datetime NOT NULL default '0000-00-00 00:00:00',
`reply` int(10) NOT NULL default '0',
`from` int(10) NOT NULL default '0',
`to` varchar(255) NOT NULL default '',
`viewed` varchar(255) NOT NULL default ',',
`archived` varchar(255) NOT NULL default ',',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
EDIT 2: (Requirements)
Return all parent messages for a specific user_id: $cms_user['id']
Return the number of replies for that parent message: num_replies
Return the number of unread replies for that parent message: unread_replies
Return the date of the parent message or it's most recent reply: last_datetime
Return whether the message is in the archive: message_archive
Return whether the message has been viewed: message_viewed
Return all messages in DESC datetime order
Return the newest message, from the parent or replies if there are some (like gmail)
If you have only 2 levels of messages (i.e., only parent messages and direct answers), you might try this query:
select
root_message.id,
root_message.active,
root_message.subject,
case
when max_reply_id.max_id is null then
root_message.message
else
reply_message.message
end as message,
root_message.datetime,
root_message.reply,
root_message.from,
root_message.to,
root_message.viewed,
root_message.archived
from
-- basic data
cms_messages as root_message
-- ID of last reply for every root message
left join (
select
max(id) as max_id,
reply as parent_id
from
cms_messages
where
reply <> 0
group by
reply
) as max_reply_id on max_reply_id.parent_id = root_message.id
left join cms_messages as reply_message on reply_message.id = max_reply_id.max_id
where
root_message.reply = 0
It uses subquery max_reply_id as source of data to select ID of the latest answer. If it exists (i.e., if there are answers), reply_message.message is used. If it does not exist (no answer has been found for root message), then root_message.message is used.
You should also think about structure of table. E.g., it would make more sense if reply contained either NULL, if it is parent message, or ID of existing message. Currently, you set it to 0 (ID of non-existent message), which is wrong. Types of viewed and archived are also weird.
Edit: you should also avoid using having clause. Use where instead, when possible.
Here's a new query that should fulfil your requirements. If there is any problem with it (i.e., if it returns wrong data), let me know.
Like the first query, it:
uses subquery reply_summary to accumulate data about replies (ID of last reply, number of replies and number of unread replies);
joins this subquery to the base table;
joins cms_messages as reply_message to the subquery, based on reply_summary.max_reply_id, to get data about the last reply (message, datetime).
I've simplified the way how you determine last_datetime - it now takes either time of last reply (if there is any reply), or time of original post (when no replies are found).
I have not filtered replies by from and to fields. If it is necessary, where clause of reply_summary subquery should be updated.
select
parent_message.id,
parent_message.subject,
parent_message.message,
parent_message.from,
parent_message.to,
coalesce(reply_summary.num_replies, 0) as num_replies,
last_reply_message.datetime as reply_datetime,
(parent_message.archived NOT LIKE '%,{$cms_user['id']},%') AS message_archive,
(parent_message.viewed LIKE '%,{$cms_user['id']},%') AS message_viewed,
reply_summary.unread_replies,
coalesce(last_reply_message.message, parent_message.message) as last_message,
coalesce(last_reply_message.datetime, parent_message.datetime) as last_datetime
from
cms_messages as parent_message
left join (
select
reply as parent_id,
max(id) as last_reply_id,
count(*) as num_replies,
sum(viewed not like '%,{$cms_user['id']},%') as unread_replies
from
cms_messages
where
reply <> 0 and
active = 1
group by
reply
) as reply_summary on reply_summary.parent_id = parent_message.id
left join cms_messages as last_reply_message on last_reply_message.id = reply_summary.last_reply_id
where
parent_message.reply = 0 and
parent_message.active = 1 and
(parent_message.to like '%,{$cms_user['id']},%' or parent_message.to = 'all' or parent_message.from = '{$cms_user['id']}')
order by
last_datetime desc;
your problem is that you are fetching only m records no matter what the order of the r records.
try adding
SELECT m.*, r.*
or
SELECT r.*, m.*
if you are using PDO::FETCH_ASSOC as your PDO fetch mode (assuming you are using PDO to access your database), the result will be an associative array where if the result set contains multiple columns with the same name, PDO::FETCH_ASSOC returns only a single value per column name. not sure which order takes presidence, so you would have to try both.
if your columns are defined in the right order, they will return the r.* value if one exists, or the m.* value if no r records exist. does this make sense? this way your result set will contain the latest record no matter which table (m or r) contains them.
http://www.php.net/manual/en/pdo.constants.php
I am afraid that you wont be able to solve this problem with a single query. Either you have to use more queries and gather the informations in the surrounding code or you will have to redesign the database structure for your messaging system a litte (tables: threads, posts, etc.). If you decide to redesign the database structure, you should also take care of the way you handle the viewed and archived fields. The way you use the fields (varchar 255 only!) might work for some users, but as soon as there are more users and higher user IDs your message system will break down.
I have 3 queries. I was told that they were potentially inefficient so I was wondering if anyone who is experienced could suggest anything. The logic is somewhat complex so bear with me.
I have two tables: shoutbox, and topic. Topic stores all information on topics that were created, while shoutbox stores all comments pertaining to each topic. Each comment comes with a group labelled by reply_chunk_id. The earliest timestamp is the first comment, while any following with the same reply_chunk_id and a later timestamp are replies. I would like to find the latest comment for each group that was started by the user (made first comment) and if the latest comment was made this month display it.
What I have written achieves that with one problem: all the latest comments are displayed in random order. I would like to organize these groups/latest comments. I really appreciate any advice
Shoutbox
Field Type
-------------------
id int(5)
timestamp int(11)
user varchar(25)
message varchar(2000)
topic_id varchar(35)
reply_chunk_id varchar(35)
Topic
id mediumint(8)
topic_id varchar(35)
subject_id mediumint(8)
file_name varchar(35)
topic_title varchar(255)
creator varchar(25)
topic_host varchar(255)
timestamp int(11)
color varchar(10)
mp3 varchar(75)
custom_background varchar(55)
description mediumtext
content_type tinyint(1)
Query
$sql="SELECT reply_chunk_id FROM shoutbox
GROUP BY reply_chunk_id
HAVING count(*) > 1
ORDER BY timestamp DESC ";
$stmt16 = $conn->prepare($sql);
$result=$stmt16->execute();
while($row = $stmt16->fetch(PDO::FETCH_ASSOC)){
$sql="SELECT user,reply_chunk_id, MIN(timestamp) AS grp_timestamp
FROM shoutbox WHERE reply_chunk_id=? AND user=?";
$stmt17 = $conn->prepare($sql);
$result=$stmt17->execute(array($row['reply_chunk_id'],$user));
while($row2 = $stmt17->fetch(PDO::FETCH_ASSOC)){
$sql="SELECT t.topic_title, t.content_type, t.subject_id,
t.creator, t.description, t.topic_host,
c1.message, c1.topic_id, c1.user, c1.timestamp AS max
FROM shoutbox c1
JOIN topic t ON (t.topic_id = c1.topic_id)
WHERE reply_chunk_id = ? AND c1.timestamp > ?
ORDER BY c1.timestamp DESC, c1.id
LIMIT 1";
$stmt18 = $conn->prepare($sql);
$result=$stmt18->execute(array($row2['reply_chunk_id'],$month));
while($row3 = $stmt18->fetch(PDO::FETCH_ASSOC)){
Make the first query:
SELECT reply_chunk_id FROM shoutbox
GROUP BY reply_chunk_id
HAVING count(*) > 1
ORDER BY timestamp DESC
This does the same, but is faster.
Make sure you have an index on reply_chunk_id.
The second query:
SELECT user,reply_chunk_id, MIN(timestamp) AS grp_timestamp
FROM shoutbox WHERE reply_chunk_id=? AND user=?
The GROUP BY is unneeded, because only one row gets returned, because of the MIN() and the equality tests.
The third query:
SELECT t.topic_title, t.content_type, t.subject_id,
t.creator, t.description, t.topic_host,
c1.message, c1.topic_id, c1.user, c1.timestamp AS max
FROM shoutbox c1
JOIN topic t ON (t.topic_id = c1.topic_id)
WHERE reply_chunk_id = ? AND c1.timestamp > ?
ORDER BY c1.timestamp DESC, c1.id
LIMIT 1
Doing it all in one query:
SELECT
t.user,t.reply_chunk_id, MIN(t.timestamp) AS grp_timestamp,
t.topic_title, t.content_type, t.subject_id,
t.creator, t.description, t.topic_host,
c1.message, c1.topic_id, c1.user, c1.timestamp AS max
FROM shoutbox c1
INNER JOIN topic t ON (t.topic_id = c1.topic_id)
LEFT JOIN shoutbox c2 ON (c1.id = c2.id and c1.timestamp < c2.timestamp)
WHERE c2.timestamp IS NULL AND t.user = ?
GROUP BY t.reply_chunk_id
HAVING count(*) > 1
ORDER BY t.reply_chunk_id
or the equivalent
SELECT
t.user,t.reply_chunk_id, MIN(t.timestamp) AS grp_timestamp,
t.topic_title, t.content_type, t.subject_id,
t.creator, t.description, t.topic_host,
c1.message, c1.topic_id, c1.user, c1.timestamp AS max
FROM shoutbox c1
INNER JOIN topic t ON (t.topic_id = c1.topic_id)
WHERE c1.timestamp = (SELECT max(timestamp) FROM shoutbox c2
WHERE c2.reply_chunk_id = c1.reply_chunk_id)
AND t.user = ?
GROUP BY t.reply_chunk_id
HAVING count(*) > 1
ORDER BY t.reply_chunk_id
How does this work?
The group by selects one entry per topic.reply_chunk_id
The left join (c1.id = c2.id and c1.`timestamp` < c2.`timestamp`) + WHERE c2.`timestamp` IS NULL selects only those items from shoutbox which have the highest timestamp. This works because MySQL keeps increasing c1.timestamp to get c2.timestamp to be null as soon as that is true, it c1.timestamp will have reached its maximum value and will select that row within the possible rows to choose from.
If you don't understand point 2, see: http://dev.mysql.com/doc/refman/5.0/en/example-maximum-column-group-row.html
Note that the PDO is autoescaping the fields with backticks
Sounds like most of it should be directly from your ShoutBox table. Prequery to find all "Chunks" the user replied to... of those chunks (and topic_ID since each chunk is always the same topic), get their respective minimum and maximum. Using the "Having count(*) > 1" will force only those that HAVE a second posting by a given user (what you were looking for).
THEN, re-query to the chunks to get the minimum regardless of user. This prevents the need of querying ALL chunks. Then join only what a single user is associated with back to the Topic.
Additionally, and I could be incorrect and need to adjust (minimally), but it appears that the SOUNDBOX table ID column would be an auto-increment column, and just happens to be time-stamped too at time of creation. That said, for a given "Chunk", the earliest ID would be the same as the earliest timestamp as they would be stamped at the same time they are created. Also makes easier on subsequent JOINs and sub query too.
By using STRAIGHT_JOIN, should force the "PreQuery" FIRST, come up with a very limited set, then qualify the WHERE clause and joins afterwords.
select STRAIGHT_JOIN
T.topic_title,
T.content_type,
T.subject_id,
T.creator,
T.description,
T.topic_host,
sb2.Topic_ID
sb2.message,
sb2.user,
sb2.TimeStamp
from
( select
sb1.Reply_Chunk_ID,
sb1.Topic_ID,
count(*) as TotalEntries,
min( sb1.id ) as FirstIDByChunkByUser,
min( sbJoin.id ) as FirstIDByChunk,
max( sbJoin.id ) as LastIDByChunk,
max( sbJoin.timestamp ) as LastTimeByChunk
from
ShoutBox sb1
join ShoutBox sbJoin
on sb1.Reply_Chunk_ID = sbJoin.Reply_Chunk_ID
where
sb1.user = CurrentUser
group by
sb1.Reply_Chunk_ID,
sb1.Topic_ID
having
min( sb1.id ) = min( sbJoin.ID ) ) PreQuery
join Topic T on
PreQuery.Topic_ID = T.ID
join ShoutBox sb2
PreQuery.LastIDByChunk = sb2.ID
where
sb2.TimeStamp >= YourTimeStampCriteria
order by
sb2.TimeStamp desc
EDIT ---- QUERY EXPLANATION -- with Modified query.
I've changed the query from re-reading (as was almost midnight when answered after holiday weekend :)
First, "STRAIGHT_JOIN" is a MySQL clause telling the engine to "do the query in the way / sequence I've stated". Basically, sometimes an engine will try to think for you and optimize in ways that may appear more efficient, but if based on your data, you know what will retrieve the smallest set of data first, and then join to other lookup fields next might in fact be better. Second the "PreQuery". If you have a "SQL-Select" statement (within parens) as Alias "From" clause, The "PreQuery" is just the name of the alias of the resultset... I could have called it anything, just makes sense that this is a stand-alone query of it's own. (Ooops... fixed to ShoutBox :) As for case-sensitivity, typically Column names are NOT case-sensitive... However, table names are... You could have a table name "MyTest" different than "mytest" or "MYTEST". But by supplying "alias", it helps shorten readability (especially with VeryLongTableNamesUsed ).
Should be working after the re-reading and applying adjustments.. Try the first "Prequery" on its own to see how many records it returns. On its own merits, it should return... for a single "CurrentUser" parameter value, every "Reply_Chunk_ID" (which will always have the same topic_id", get the first ID the person entered (min()). By JOINing again to Shoutbox on the chunk id, we (only those qualified as entered by the user), get the minimum and maximum ID per the chunk REGARDLESS of who started or responded. By applying the HAVING clause, this should only return those where the same person STARTED the topic (hence both have the same min() value.)
Finally, once those have been qualified, join directly to the TOPIC and SHOUTBOX tables again on their own merits of topic_id and LastIDByChunk and order the final results by the latest comment response timestamp descending.
I've added a where clause to further limit your "timestamp" criteria where the most recent final timestamp is on/after the given time period you want.
I would be curious how this query's time performance works compared to your already accepted answer too.
I have a mysql table with items in relation to their order.
CREATE DATABASE IF NOT EXISTS `sqltest`;
USE `sqltest`;
DROP TABLE IF EXISTS `testdata`;
CREATE TABLE `testdata` (
`orderID` varchar(10) DEFAULT NULL,
`itemID` varchar(10) DEFAULT NULL,
`qtyOrdered` int(10) DEFAULT NULL,
`sellingPrice` decimal(10,2) DEFAULT NULL
)
INSERT INTO `testdata`(`orderID`,`itemID`,`qtyOrdered`,`sellingPrice`)
values ('1','a',1,'7.00'),('1','b',2,'8.00'),('1','c',3,'3.00'),('2','a',1,'7.00'),('2','c',4,'3.00');
Intended Result:
A = (1+1)2
B = 2
C = (2+4)6 <- most popular
How do I add up all the qty's for each item and result the highest one?
It should be fairly strait forward but I'm new to SQL and I can't work this one out :S
Solution needs to be mysql and or php.
I guess there needs to be some sort of temporary tally variable for each item ID,
but that seems like it could get messy with too many items.
ANSWER:
(thanks nuqqsa)
SELECT itemID, SUM(qtyOrdered) AS total FROM testdata GROUP BY itemID ORDER BY total DESC LIMIT 1;
How about this:
SELECT itemID, SUM(qtyOrdered) AS total FROM testdata GROUP BY itemID ORDER BY total DESC;
SELECT itemID, SUM(qtyOrdered) as blah FROM sqltest GROUP BY itemID ORDER BY blah DESC should do it
SELECT *
FROM testdata
ORDER BY SUM(gtyOrdered) DESC
GROUP BY itemID
SELECT SUM( qtyOrdered ) AS sum_ordered, itemID
FROM testdata
GROUP BY itemID
ORDER BY sum_ordered
select count(qtyOrdered), qtyOrdered from testdata group by qtyOrdered