I am currently coding a web application which I will need to create unique reference for quotes and invoices.
I wanted to create a reference that included the year and month then an reference number. i.e YYYY-MM-001.
The web application will be multi tenant and several users will be using it at the same time. One of my concerns is, how would I generate my reference without it be duplicated at the same time if there is multiple users doing the same request at the same time?
What would be the best way for me to approach this?
I am using PHP 8 and a MySQL database.
You can use the primary key id or any unique column, then don't worry about the uniqueness of the data.
Use that column/id to generate a unique reference as follows :
update invoices
set reference = concat(date_format(now(), "%Y-%m-00"), unique_id)
where unique_id = 1;
If you're sure you won't have more than 999 quotes/invoices in one month, you could do something like the following...
CREATE TABLE quotes (
id INT(14) NOT NULL AUTO_INCREMENT,
YMFqid char(11) NOT NULL,
PRIMARY KEY (id),
UNIQUE KEY (YMFqid));
CREATE TABLE YMnext (
YM char(7) NOT NULL,
next INT(3) NOT NULL DEFAULT 0,
PRIMARY KEY (YM));
Then to get the next values, you'll need some queries like...
SELECT next AS latest FROM YMnext WHERE YM = '2022-12';
INSERT INTO YMnext VALUES ('2022-12', 0);
UPDATE YMnext SET next = next + 1 WHERE YM = '2022-12' AND next = latest;
If the SELECT fails, then you insert, otherwise you update AND check that one and only one row was updated. If there's a chance you could have more than 999 quotes in a month, then you can alter the column sizes accordingly.
Related
I am developing an web application where I have to give every new registrant a serial number. The main issue is with 'how to ensure uniqueness?'. I have searched through the different functions available with mysql and found mysql_insert_id() to be the fittest solution here. But before I run towards it, I need to know whether this function is thread-free. To more precise, say there are two users sitting at two different terminals and submits the registration form synchronously. Will they both get the same id out of the execution of the function mysql_insert_id()? Otherwise, my project will spoil. Please help. If I could not clear my point, please comment. Thanks in advance.
here is detailed solution
CREATE TABLE Persons
(
ID int NOT NULL AUTO_INCREMENT,
LastName varchar(255) NOT NULL,
FirstName varchar(255),
Address varchar(255),
City varchar(255),
PRIMARY KEY (ID)
)
By default, the starting value for AUTO_INCREMENT is 1, and it will increment by 1 for each new record.
To insert a new record into the "Persons" table, we will NOT have to specify a value for the "ID" column (a unique value will be added automatically):
If you have an id column on your table in your database and that column is set to be the primary key that will be enough. Even if 2 people will submit the form at the same the ids will be unique.
id column could be defined like this
ADD PRIMARY KEY (`id`)
id` int(11) NOT NULL AUTO_INCREMENT
Alternatively, you can use the UUID() function in mysql.
A UUID is designed as a number that is globally unique in space and
time. Two calls to UUID() are expected to generate two different
values, even if these calls are performed on two separate computers
that are not connected to each other.
mysql> SELECT UUID();
-> '6ccd780c-baba-1026-9564-0040f4311e29'
For further details : http://dev.mysql.com/doc/refman/5.0/en/miscellaneous-functions.html#function_uuid
I have come up with a total of three different, equally viable methods for saving data for a graph.
The graph in question is "player's score in various categories over time". Categories include "buildings", "items", "quest completion", "achievements" and so on.
Method 1:
CREATE TABLE `graphdata` (
`userid` INT UNSIGNED NOT NULL,
`date` DATE NOT NULL,
`category` ENUM('buildings','items',...) NOT NULL,
`score` FLOAT UNSIGNED NOT NULL,
PRIMARY KEY (`userid`, `date`, `category`),
INDEX `userid` (`userid`),
INDEX `date` (`date`)
) ENGINE=InnoDB
This table contains one row for each user/date/category combination. To show a user's data, select by userid. Old entries are cleared out by:
DELETE FROM `graphdata` WHERE `date` < DATE_ADD(NOW(),INTERVAL -1 WEEK)
Method 2:
CREATE TABLE `graphdata` (
`userid` INT UNSIGNED NOT NULL,
`buildings-1day` FLOAT UNSIGNED NOT NULL,
`buildings-2day` FLOAT UNSIGNED NOT NULL,
... (and so on for each category up to `-7day`
PRIMARY KEY (`userid`)
)
Selecting by user id is faster due to being a primary key. Every day scores are shifted down the fields, as in:
... SET `buildings-3day`=`buildings-2day`, `buildings-2day`=`buildings-1day`...
Entries are not deleted (unless a user deletes their account). Rows can be added/updated with an INSERT...ON DUPLICATE KEY UPDATE query.
Method 3:
Use one file for each user, containing a JSON-encoded array of their score data. Since the data is being fetched by an AJAX JSON call anyway, this means the file can be fetched statically (and even cached until the following midnight) without any stress on the server. Every day the server runs through each file, shift()s the oldest score off each array and push()es the new one on the end.
Personally I think Method 3 is by far the best, however I've heard bad things about using files instead of databases - for instance if I wanted to be able to rank users by their scores in different categories, this solution would be very bad.
Out of the two database solutions, I've implemented Method 2 on one of my older projects, and that seems to work quite well. Method 1 seems "better" in that it makes better use of relational databases and all that stuff, but I'm a little concerned in that it will contain (number of users) * (number of categories) * 7 rows, which could turn out to be a big number.
Is there anything I'm missing that could help me make a final decision on which method to use? 1, 2, 3 or none of the above?
If you're going to use a relational db, method 1 is much better than method 2. It's normalized, so it's easy to maintain and search. I'd change the date field to a timestamp and call it added_on (or something that's not a reserved word like 'date' is). And I'd add an auto_increment primary key score_id so that user_id/date/category doesn't have to be unique. That way, if a user managed to increment his building score twice in the same second, both would still be recorded.
The second method requires you to update all the records every day. The first method only does inserts, no updates, so each record is only written to once.
... SET buildings-3day=buildings-2day, buildings-2day=buildings-1day...
You really want to update every single record in the table every day until the end of time?!
Selecting by user id is faster due to being a primary key
Since user_id is the first field in your Method 1 primary key, it will be similarly fast for lookups. As first field in a regular index (which is what I've suggested above), it will still be very fast.
The idea with a relational db is that each row represents a single instance/action/occurrence. So when a user does something to affect his score, do an INSERT that records what he did. You can always create a summary from data like this. But you can't get this kind of data from a summary.
Secondly, you seem unwontedly concerned about getting rid of old data. Why? Your select queries would have a date range on them that would exclude old data automatically. And if you're concerned about performance, you can partition your tables based on row age or set up a cronjob to delete old records periodically.
ETA: Regarding JSON stored in files
This seems to me to combine the drawbacks of Method 2 (difficult to search, every file must be updated every day) with the additional drawbacks of file access. File accesses are expensive. File writes are even more so. If you really want to store summary data, I'd run a query only when the data is requested and I'd store the results in a summary table by user_id. The table could hold a JSON string:
CREATE TABLE score_summaries(
user_id INT unsigned NOT NULL PRIMARY KEY,
gen_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
json_data TEXT NOT NULL DEFAULT '{}'
);
For example:
Bob (user_id=7) logs into the game for the first time. He's on his profile page which displays his weekly stats. These queries ran:
SELECT json_data FROM score_summaries
WHERE user_id=7
AND gen_date > DATE_SUB(CURDATE() INTERVAL 1 DAY);
//returns nothing so generate summary record
SELECT DATE(added_on), category, SUM(score)
FROM scores WHERE user_id=7 AND added_on < CURDATE() AND > DATE_SUB(CURDATE(), INTERVAL 1 WEEK)
GROUP BY DATE(added_on), category; //never include today's data, encode as json with php
INSERT INTO score_summaries(user_id, json_data)
VALUES(7, '$json') //from PHP, in this case $json == NULL
ON DUPLICATE KEY UPDATE json_data=VALUES(json_data)
//use $json for presentation too
Today's scores are generated as needed and not stored in the summary. If Bob views his scores again today, the historical ones can come from the summary table or could be stored in a session after the first request. If Bob doesn't visit for a week, no summary needs to be generated.
method 1 seems like a clear winner to me . If you are concerned about size of single table (graphData) being too big you could reduce it by creating
CREATE TABLE `graphdata` (
`graphDataId` INT UNSIGNED NOT NULL,
`categoryId` INT NOT NULL,
`score` FLOAT UNSIGNED NOT NULL,
PRIMARY KEY (`GraphDataId'),
) ENGINE=InnoDB
than create 2 tables because you obviosuly need to have info connecting graphDataId with userId
create table 'graphDataUser'(
`graphDataId` INT UNSIGNED NOT NULL,
`userId` INT NOT NULL,
)ENGINE=InnoDB
and graphDataId date connection
create table 'graphDataDate'(
`graphDataId` INT UNSIGNED NOT NULL,
'graphDataDate' DATE NOT NULL
)ENGINE=InnoDB
i think that you don't really need to worry about number of rows some table contains because most of dba does a good job regarding number of rows. Its your job only to get data formatted in a way it is easly retrived no matter what is the task for which data is retrieved. Using that advice i think should pay off in a long run.
I have the following schema with the following attributes:
USER(TABLE_NAME)
USER_ID|USERNAME|PASSWORD|TOPIC_NAME|FLAG1|FLAG2
I have 2 questions basically:
How can I make an attribute USER_ID as primary key and it should
automatically increment the value each time I insert the value into
the database.It shouldn't be under my control.
How can I retrieve a record from the database, based on the latest
time from which it was updated.( for example if I updated a record
at 2pm and same record at 3pm, if I retrieve now at 4pm I should get
the record that was updated at 3pm i.e. the latest updated one.)
Please help.
I'm assuming that question one is in the context of MYSQL. So, you can use the ALTER TABLE statement to mark a field as PRIMARY KEY, and to mark it AUTOINCREMENT
ALTER TABLE User
ADD PRIMARY KEY (USER_ID);
ALTER TABLE User
MODIFY COLUMN USER_ID INT(4) AUTO_INCREMENT; -- of course, set the type appropriately
For the second question I'm not sure I understand correctly so I'm just going to go ahead and give you some basic information before giving an answer that may confuse you.
When you update the same record multiple times, only the most recent update is persisted. Basically, once you update a record, it's previous values are not kept. So, if you update a record at 2pm, and then update the same record at 3pm - when you query for the record you will automatically receive the most recent values.
Now, if by updating you mean you would insert new values for the same USER_ID multiple times and want to retrieve the most recent, then you would need to use a field in the table to store a timestamp of when each record is created/updated. Then you can query for the most recent value based on the timestamp.
I assume you're talking about Oracle since you tagged it as Oracle. You also tagged the question as MySQL where the approach will be different.
You can make the USER_ID column a primary key
ALTER TABLE <<table_name>>
ADD CONSTRAINT pk_user_id PRIMARY KEY( user_id );
If you want the value to increment automatically, you'd need to create a sequence
CREATE SEQUENCE user_id_seq
START WITH 1
INCREMENT BY 1
CACHE 20;
and then create a trigger on the table that uses the sequence
CREATE OR REPLACE TRIGGER trg_assign_user_id
BEFORE INSERT ON <<table name>>
FOR EACH ROW
BEGIN
:new.user_id := user_id_seq.nextval;
END;
As for your second question, I'm not sure that I understand. If you update a row and then commit that change, all subsequent queries are going to read the updated data (barring exceptionally unlikely cases where you've set a serializable transaction isolation level and you've got transactions that run for multiple hours and you're running the query in that transaction). You don't need to do anything to see the current data.
(Answer based on MySQL; conceptually similar answer if using Oracle, but the SQL will probably be different.)
If USER_ID was not defined as a primary key or automatically incrementing at the time of table creation, then you can use:
ALTER TABLE tablename MODIFY USER_ID INT NOT NULL PRIMARY KEY AUTO_INCREMENT;
To issue queries based on record dates, you have to have a field defined to hold date-related datetypes. The date and time of record modifications would be something you would manage (e.g. add/change) based on the way in which you are accessing the records (some PHP-related way? it's unclear what scripts you have in play, based on your question.) Once you have dates in your records you can ORDER BY the date field in your SELECT query.
Check this out
For your AUTOINCREMENT, Its a question already asked here
For your PRIMARY KEY use this
ALTER TABLE USER ADD PRIMARY KEY (USER_ID)
Can you provide more information. If the value gets updated you definitely do NOT have your old value that you entered at 2pm present in the dB. So querying for it will be fine
You can use something like this:
CREATE TABLE IF NOT EXISTS user (
USER_ID unsigned int(8) NOT NULL AUTO_INCREMENT,
username varchar(25) NOT NULL,
password varchar(25) NOT NULL,
topic_name varchar(100) NOT NULL,
flag1 smallint(1) NOT NULL DEFAULT 0,
flag2 smallint(1) NOT NULL DEFAULT 0,
update_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (uid)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1 ;
For selection use query:
SELECT * from user ORDER BY update_time DESC
I am creating a ticketing system that will keep track of tickets that a customer creates. The ticket's basic information will be stored in a table 'tickets' who's structure is as follows:
Primary Key (int 255)
Ticket_Key (varchar)
Ticket Number (varchar 500)
Label
Date Created
Delete
and so on..
The issue is that there will eventually be a large amount of tickets and we need a more uniform way of identifying tickets. I would like PHP to create a Ticket Number in the ticket number that will contain mixed values. The date (in format 20111107), followed by a auto incremented value 1001. 1002, 1003, ...). So the Ticket Number will be 201111071001 for an example.
The issue is how do I program this in PHP to insert to the MySQL database? Also, how do I prevent the possibility of duplicate values in the Unique Id in PHP? There will be a very large amount of customers using the table to insert records.
What about using an auto-increment and combining this with the date field to generate a sequence number for that date and hence a ticketId.
So your insert process would be something like this:
INSERT INTO table (...ticket info...)
You would then retrieve the auto-increment for this row and run a query like this
UPDATE table SET sequence = (SELECT ($id-MAX(auto_increment)) FROM table WHERE date_created=DATE_SUB(CURDATE(),INTERVAL 1 DAY)) WHERE auto_increment=$id
You could then easily create a ticketId of format YYYMMDDXXXX. Assuming you never retro-add tickets in the past this would only ever require these two queries even under heavy usage.
[EDIT] Actually, after looking into this there is a much better way to do this natively in MySQL. If you define two columns (date and sequence) and make them a primary key (both columns) with the sequence field as an auto-increment then MySQL will update the sequence column as an auto-increment per date (i.e. it will start with value 1 for each date).
[EDIT] A table structure along these lines would do the job for you:
CREATE TABLE IF NOT EXISTS `table` (
`created_date` date NOT NULL,
`ticket_sequence` int(11) NOT NULL auto_increment,
`label` varchar(100) NOT NULL,
[other fields as required]
PRIMARY KEY (`created_date`,`ticket_sequence`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;
When retrieving the data you could then do something like
SELECT CONCAT( DATE_FORMAT(created_date,'%Y%m%d'),LPAD(ticket_sequence,4,'0')) AS ticket_number, other fields.... FROM table
as i understand that you want to make one result of two different fields like datefield and ticketnumfield
in mysql you do this through the command:
SELECT concat( datefield, ticketnumfeild ) FROM `tbl_name`
this query return the result like 201111071001
I did something like this before where I wanted to refresh the counter for each new day. Unfortunately I do not speak PHP so you will have to settle for explanation and maybe some pseudo code.
Firstly, create a couple of fields in a config file to keep track of your counter. This should be a date field and a number fields...
LastCount (Number)
LastCountDate (Date)
Then you make sure that your ticket number field in your database table is set to only unique values, so it throws an error if you try to insert a duplicate.
Then in your code, you load your counter values (LastCount and LastCountDate) and you process them like so...
newCount = LastCount;
if LastCountDate == Today
increment newCount (newCount++)
else
reset newCount (newCount = 1)
you can then use newCount to create your ticket number.
Next, when you try to insert a row, if it is successful, then great. If it fails, then you need to increment newCount again, then try the insert again. Repeat this until the insert is successful (put it in a loop)
Once you have successfully inserted the row, you need to update the database with the Count Values you just used to generate the ticket number - so they are ready for use the next time.
Hope that helps in some way.
First, I apologize if this has been asked before - indeed I'm sure it has, but I can't find it/can't work out what to search for to find it.
I need to generate unique quick reference id's, based on a company name. So for example:
Company Name Reference
Smiths Joinery smit0001
Smith and Jones Consulting smit0002
Smithsons Carpets smit0003
These will all be stored in a varchar column in a MySQL table. The data will be collected, escaped and inserted like 'HTML -> PHP -> MySQL'. The ID's should be in the format depicted above, four letters, then four numerics (initially at least - when I reach smit9999 it will just spill over into 5 digits).
I can deal with generating the 4 letters from the company name, I will simply step through the name until I have collected 4 alpha characters, and strtolower() it - but then I need to get the next available number.
What is the best/easiest way to do this, so that the possibility of duplicates is eliminated?
At the moment I'm thinking:
$fourLetters = 'smit';
$query = "SELECT `company_ref`
FROM `companies`
WHERE
`company_ref` LIKE '$fourLetters%'
ORDER BY `company_ref` DESC
LIMIT 1";
$last = mysqli_fetch_assoc(mysqli_query($link, $query));
$newNum = ((int) ltrim(substr($last['company_ref'],4),'0')) + 1;
$newRef = $fourLetters.str_pad($newNum, 4, '0', STR_PAD_LEFT);
But I can see this causing a problem if two users try to enter company names that would result in the same ID at the same time. I will be using a unique index on the column, so it would not result in duplicates in the database, but it will still cause a problem.
Can anyone think of a way to have MySQL work this out for me when I do the insert, rather than calculating it in PHP beforehand?
Note that actual code will be OO and will handle errors etc - I'm just looking for thoughts on whether there is a better way to do this specific task, it's more about the SQL than anything else.
EDIT
I think that #EmmanuelN's suggestion of using a MySQL trigger may be the way to handle this, but:
I am not good enough with MySQL, particularly triggers, to get this to work, and would like a step-by-step example of creating, adding and using a trigger.
I am still not sure whether this will will eliminate the possibility of two identical ID's being generated. See what happens if two rows are inserted at the same time that result in the trigger running simultaneously, and produce the same reference? Is there any way to lock the trigger (or a UDF) in such a way that it can only have one concurrent instance?.
Or I would be open to any other suggested approaches to this problem.
If you are using MyISAM, then you can create a compound primary key on a text field + auto increment field. MySQL will handle incrementing the number automatically. They are separate fields, but you can get the same effect.
CREATE TABLE example (
company_name varchar(100),
key_prefix char(4) not null,
key_increment int unsigned auto_increment,
primary key co_key (key_prefix,key_increment)
) ENGINE=MYISAM;
When you do an insert into the table, the key_increment field will increment based on the highest value based on key_prefix. So insert with key_prefix "smit" will start with 1 in key_inrement, key_prefix "jone" will start with 1 in key_inrement, etc.
Pros:
You don't have to do anything with calculating numbers.
Cons:
You do have a key split across 2 columns.
It doesn't work with InnoDB.
How about this solution with a trigger and a table to hold the company_ref's uniquely. Made a correction - the reference table has to be MyISAM if you want the numbering to begin at 1 for each unique 4char sequence.
DROP TABLE IF EXISTS company;
CREATE TABLE company (
company_name varchar(100) DEFAULT NULL,
company_ref char(8) DEFAULT NULL
) ENGINE=InnoDB
DELIMITER ;;
CREATE TRIGGER company_reference BEFORE INSERT ON company
FOR EACH ROW BEGIN
INSERT INTO reference SET company_ref=SUBSTRING(LOWER(NEW.company_name), 1, 4), numeric_ref=NULL;
SET NEW.company_ref=CONCAT(SUBSTRING(LOWER(NEW.company_name), 1, 4), LPAD(CAST(LAST_INSERT_ID() AS CHAR(10)), 4, '0'));
END ;;
DELIMITER ;
DROP TABLE IF EXISTS reference;
CREATE TABLE reference (
company_ref char(4) NOT NULL DEFAULT '',
numeric_ref int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (company_ref, numeric_ref)
) ENGINE=MyISAM;
And for completeness here is a trigger that will create a new reference if the company name is altered.
DROP TRIGGER IF EXISTS company_reference_up;
DELIMITER ;;
CREATE TRIGGER company_reference_up BEFORE UPDATE ON company
FOR EACH ROW BEGIN
IF NEW.company_name <> OLD.company_name THEN
DELETE FROM reference WHERE company_ref=SUBSTRING(LOWER(OLD.company_ref), 1, 4) AND numeric_ref=SUBSTRING(OLD.company_ref, 5, 4);
INSERT INTO reference SET company_ref=SUBSTRING(LOWER(NEW.company_name), 1, 4), numeric_ref=NULL;
SET NEW.company_ref=CONCAT(SUBSTRING(LOWER(NEW.company_name), 1, 4), LPAD(CAST(LAST_INSERT_ID() AS CHAR(10)), 4, '0'));
END IF;
END;
;;
DELIMITER ;
Given you're using InnoDB, why not use an explicit transaction to grab an exclusive row lock and prevent another connection from reading the same row before you're done setting a new ID based on it?
(Naturally, doing the calculation in a trigger would hold the lock for less time.)
mysqli_query($link, "BEGIN TRANSACTION");
$query = "SELECT `company_ref`
FROM `companies`
WHERE
`company_ref` LIKE '$fourLetters%'
ORDER BY `company_ref` DESC
LIMIT 1
FOR UPDATE";
$last = mysqli_fetch_assoc(mysqli_query($link, $query));
$newNum = ((int) ltrim(substr($last['company_ref'],4),'0')) + 1;
$newRef = $fourLetters.str_pad($newNum, 4, '0', STR_PAD_LEFT);
mysqli_query($link, "INSERT INTO companies . . . (new row using $newref)");
mysqli_commit($link);
Edit: Just to be 100% sure I ran a test by hand to confirm that the second transaction will return the newly inserted row after waiting rather than the original locked row.
Edit2: Also tested the case where there is no initial row returned (Where you would think there is no initial row to put a lock on) and that works as well.
Ensure you have an unique constraint on the Reference column.
Fetch the current max sequential reference the same way you do it in your sample code. You don't actually need to trim the zeroes before you cast to (int), '0001' is a valid integer.
Roll a loop and do your insert inside.
Check affected rows after the insert. You can also check the SQL state for a duplicate key error, but having zero affected rows is a good indication that your insert failed due to inserting an existing Reference value.
If you have zero affected rows, increment the sequential number, and roll the loop again. If you have non-zero affected rows, you're done and have an unique identifier inserted.
Easiest way to avoid duplicate values for the reference column is to add a unique constraint. So if multiple processes try to set to the same value, MySQL will reject the second attempt and throw an error.
ALTER TABLE table_name ADD UNIQUE KEY (`company_ref`);
If I were faced with your situation, I would handle the company reference id generation within the application layer, triggers can get messy if not setup correctly.
A hacky version that works for InnoDB as well.
Replace the insert to companies with two inserts in a transaction:
INSERT INTO __keys
VALUES (LEFT(LOWER('Smiths Joinery'),4), LAST_INSERT_ID(1))
ON DUPLICATE KEY UPDATE
num = LAST_INSERT_ID(num+1);
INSERT INTO __companies (comp_name, reference)
VALUES ('Smiths Joinery',
CONCAT(LEFT(LOWER(comp_name),4), LPAD(LAST_INSERT_ID(), 4, '0')));
where:
CREATE TABLE `__keys` (
`prefix` char(4) NOT NULL,
`num` smallint(5) unsigned NOT NULL,
PRIMARY KEY (`prefix`)
) ENGINE=InnoDB COLLATE latin1_general_ci;
CREATE TABLE `__companies` (
`comp_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`comp_name` varchar(45) NOT NULL,
`reference` char(8) NOT NULL,
PRIMARY KEY (`comp_id`)
) ENGINE=InnoDB COLLATE latin1_general_ci;
Notice:
latin1_general_ci can be replaced with utf8_general_ci,
LEFT(LOWER('Smiths Joinery'),4) would better become a function in PHP