So I am working on a fairly small script for the company I work at the help us manage our servers better. I don't use MySQL too often though so I am a bit confused on what would be the best path to take.
I am doing something like this...
$sql = "CREATE TABLE IF NOT EXISTS Servers
(
MachineID int NOT NULL AUTO_INCREMENT,
PRIMARY KEY(MachineID),
FirstName varchar(32),
LastName varchar(32),
Datacenter TINYINT(1),
OperatingSystem TINYINT(1),
HostType TINYINT(1)
)";
$result = mysql_query($sql,$conn);
check ($result);
$sql = "CREATE TABLE IF NOT EXISTS Datacenter
(
DatacenterID int NOT NULL AUTO_INCREMENT,
PRIMARY KEY(DatacenterID),
Name varchar(32),
Location varchar(32)
)";
$result = mysql_query($sql,$conn);
check ($result);
Basically inside the Servers table, I will be storing the index of an entry in another table. My concern is that if we remove one of those entries, it will screw up the auto incremented indexes and potentially cause a lot of incorrect data.
To explain better, lets say the first server I add, the Datacenter value is 3 (which is the DatacenterID), and we remove the id 1 (DatacenterID) Datacenter at a later time.
Is there a good way to do this?
Auto increment only has an effect on inserting new rows into a table. So you insert three records into database and they get assigned ids of 1, 2, and 3. Later you delete id 1, but the records at id 2 and 3 are unchanged. This says nothing of any of the server records that might be trying to reference the database id 1 record though that is now missing.
as said by paul, it is safe to remove old row and add another new one later. The auto incremental index won't be affected by deletion.
But I would suggest instead of removing them, simply add a column 'status' and set 0, implying they are no longer in use, to keep any possible record in db.
Servers.Datacenter should be an INT, too, as you would store the DataCenterID in this field. Then, nothing will be mixed up when you remove some Datacenter from the second table.
If you remove in the way you suggest nothing will happen. auto-increment will just add the next highest number in sequence whenever a new record is added and so will not affect any previous records.
Hope that helps.
Related
i have two tables(innodb) in MYSQL data base both share a similar column the account_no column i want to keep both columns as integers and still keep both free from collusion when inserting data only.
there are 13 instances of this same question on stackoverflow i have read all. but in all, the recommended solutions where:
1) using GUID :this is good but am trying to keep the numbers short and easy for the users to remember.
2) using sequence :i do not fully understand how to do this but am thinking it involves making a third table that has an auto_increment and getting my values for the the two major tables from it.
3) using IDENTITY (1, 10) [1,11,21...] for the first table and the second using IDENTITY (2, 10) [2,12,22...] this works fine but in the long term might not be such a good idea.
4) using php function uniqid(,TRUE) :not going to work its not completely collision free and the columns in my case have to be integers.
5) using php function mt_rand(0,10): might work but i still have to check for collisions before inserting data.
if there is no smarter way to archive my goal i would stick with using the adjusted IDENTITY (1, 10) and (2, 10).
i know this question is a bit dumb seeing all the options i have available but the most recent answer on a similar topic was in 2012 there might have been some improvements in the MYSQL system that i do not know about yet.
also am using php language to insert the data thanks.
Basically, you are saying that you have two flavors of an entity. My first recommendation is to try to put them in a single table. There are three methods:
If most columns overlap, just put all the columns in a single table (accounts).
If one entity has more columns, put the common columns in one table and have a second table for the wider entity.
If only some columns overlap, put those in a single table and have a separate table for each subentity.
Let met assume the third situation for the moment.
You want to define something like:
create table accounts (
AccountId int auto_increment primary key,
. . . -- you can still have common columns here
);
create table subaccount_1 (
AccountId int primary key,
constraint foreign key (AccountId) references accounts(AccountId),
. . .
);
create table subaccount_2 (
AccountId int primary key,
constraint foreign key (AccountId) references accounts(AccountId),
. . .
);
Then, you want an insert trigger on each sub-account table. This trigger does the following on insert:
inserts a row into accounts
captures the new accountId
uses that for the insert into the subaccount table
You probably also want something on accounts that prevents inserts into that table, except through the subaccount tables.
A big thank you to Gordon Linoff for his answer i want to fully explain how i solved the problem using his answer to help others understand better.
original tables:
Table A (account_no, fist_name, last_name)
Table B (account_no, likes, dislikes)
problem: need account_no to auto_increment across both tables and be unique across both tables and remain a medium positive integer (see original question).
i had to make an extra Table_C to which will hold all the inserted data at first, auto_increment it and checks for collisions through the use of primary_key
CREATE TABLE Table_C (
account_no int NOT NULL AUTO_INCREMENT,
fist_name varchar(50),
last_name varchar(50),
likes varchar(50),
dislikes varchar(50),
which_table varchar(1),
PRIMARY KEY (account_no)
);
Then i changed MySQL INSERT statement to insert to Table_C and added an extra column which_table to say which table the data being inserted belong to and Table_C on insert of data performs auto_increment and checks collision then reinsert the data to the desired table through the use of triggers like so:
CREATE TRIGGER `sort_tables` AFTER INSERT ON `Table_C` FOR EACH ROW
BEGIN
IF new.which_table = 'A' THEN
INSERT INTO Table_A
VALUES (new.acc_no, new.first_name, new.last_name);
ELSEIF new.which_table = 'B' THEN
INSERT INTO Table_B
VALUES (new.acc_no, new.likes, new.dislikes);
END IF;
END
I am developing an web application where I have to give every new registrant a serial number. The main issue is with 'how to ensure uniqueness?'. I have searched through the different functions available with mysql and found mysql_insert_id() to be the fittest solution here. But before I run towards it, I need to know whether this function is thread-free. To more precise, say there are two users sitting at two different terminals and submits the registration form synchronously. Will they both get the same id out of the execution of the function mysql_insert_id()? Otherwise, my project will spoil. Please help. If I could not clear my point, please comment. Thanks in advance.
here is detailed solution
CREATE TABLE Persons
(
ID int NOT NULL AUTO_INCREMENT,
LastName varchar(255) NOT NULL,
FirstName varchar(255),
Address varchar(255),
City varchar(255),
PRIMARY KEY (ID)
)
By default, the starting value for AUTO_INCREMENT is 1, and it will increment by 1 for each new record.
To insert a new record into the "Persons" table, we will NOT have to specify a value for the "ID" column (a unique value will be added automatically):
If you have an id column on your table in your database and that column is set to be the primary key that will be enough. Even if 2 people will submit the form at the same the ids will be unique.
id column could be defined like this
ADD PRIMARY KEY (`id`)
id` int(11) NOT NULL AUTO_INCREMENT
Alternatively, you can use the UUID() function in mysql.
A UUID is designed as a number that is globally unique in space and
time. Two calls to UUID() are expected to generate two different
values, even if these calls are performed on two separate computers
that are not connected to each other.
mysql> SELECT UUID();
-> '6ccd780c-baba-1026-9564-0040f4311e29'
For further details : http://dev.mysql.com/doc/refman/5.0/en/miscellaneous-functions.html#function_uuid
I have come up with a total of three different, equally viable methods for saving data for a graph.
The graph in question is "player's score in various categories over time". Categories include "buildings", "items", "quest completion", "achievements" and so on.
Method 1:
CREATE TABLE `graphdata` (
`userid` INT UNSIGNED NOT NULL,
`date` DATE NOT NULL,
`category` ENUM('buildings','items',...) NOT NULL,
`score` FLOAT UNSIGNED NOT NULL,
PRIMARY KEY (`userid`, `date`, `category`),
INDEX `userid` (`userid`),
INDEX `date` (`date`)
) ENGINE=InnoDB
This table contains one row for each user/date/category combination. To show a user's data, select by userid. Old entries are cleared out by:
DELETE FROM `graphdata` WHERE `date` < DATE_ADD(NOW(),INTERVAL -1 WEEK)
Method 2:
CREATE TABLE `graphdata` (
`userid` INT UNSIGNED NOT NULL,
`buildings-1day` FLOAT UNSIGNED NOT NULL,
`buildings-2day` FLOAT UNSIGNED NOT NULL,
... (and so on for each category up to `-7day`
PRIMARY KEY (`userid`)
)
Selecting by user id is faster due to being a primary key. Every day scores are shifted down the fields, as in:
... SET `buildings-3day`=`buildings-2day`, `buildings-2day`=`buildings-1day`...
Entries are not deleted (unless a user deletes their account). Rows can be added/updated with an INSERT...ON DUPLICATE KEY UPDATE query.
Method 3:
Use one file for each user, containing a JSON-encoded array of their score data. Since the data is being fetched by an AJAX JSON call anyway, this means the file can be fetched statically (and even cached until the following midnight) without any stress on the server. Every day the server runs through each file, shift()s the oldest score off each array and push()es the new one on the end.
Personally I think Method 3 is by far the best, however I've heard bad things about using files instead of databases - for instance if I wanted to be able to rank users by their scores in different categories, this solution would be very bad.
Out of the two database solutions, I've implemented Method 2 on one of my older projects, and that seems to work quite well. Method 1 seems "better" in that it makes better use of relational databases and all that stuff, but I'm a little concerned in that it will contain (number of users) * (number of categories) * 7 rows, which could turn out to be a big number.
Is there anything I'm missing that could help me make a final decision on which method to use? 1, 2, 3 or none of the above?
If you're going to use a relational db, method 1 is much better than method 2. It's normalized, so it's easy to maintain and search. I'd change the date field to a timestamp and call it added_on (or something that's not a reserved word like 'date' is). And I'd add an auto_increment primary key score_id so that user_id/date/category doesn't have to be unique. That way, if a user managed to increment his building score twice in the same second, both would still be recorded.
The second method requires you to update all the records every day. The first method only does inserts, no updates, so each record is only written to once.
... SET buildings-3day=buildings-2day, buildings-2day=buildings-1day...
You really want to update every single record in the table every day until the end of time?!
Selecting by user id is faster due to being a primary key
Since user_id is the first field in your Method 1 primary key, it will be similarly fast for lookups. As first field in a regular index (which is what I've suggested above), it will still be very fast.
The idea with a relational db is that each row represents a single instance/action/occurrence. So when a user does something to affect his score, do an INSERT that records what he did. You can always create a summary from data like this. But you can't get this kind of data from a summary.
Secondly, you seem unwontedly concerned about getting rid of old data. Why? Your select queries would have a date range on them that would exclude old data automatically. And if you're concerned about performance, you can partition your tables based on row age or set up a cronjob to delete old records periodically.
ETA: Regarding JSON stored in files
This seems to me to combine the drawbacks of Method 2 (difficult to search, every file must be updated every day) with the additional drawbacks of file access. File accesses are expensive. File writes are even more so. If you really want to store summary data, I'd run a query only when the data is requested and I'd store the results in a summary table by user_id. The table could hold a JSON string:
CREATE TABLE score_summaries(
user_id INT unsigned NOT NULL PRIMARY KEY,
gen_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
json_data TEXT NOT NULL DEFAULT '{}'
);
For example:
Bob (user_id=7) logs into the game for the first time. He's on his profile page which displays his weekly stats. These queries ran:
SELECT json_data FROM score_summaries
WHERE user_id=7
AND gen_date > DATE_SUB(CURDATE() INTERVAL 1 DAY);
//returns nothing so generate summary record
SELECT DATE(added_on), category, SUM(score)
FROM scores WHERE user_id=7 AND added_on < CURDATE() AND > DATE_SUB(CURDATE(), INTERVAL 1 WEEK)
GROUP BY DATE(added_on), category; //never include today's data, encode as json with php
INSERT INTO score_summaries(user_id, json_data)
VALUES(7, '$json') //from PHP, in this case $json == NULL
ON DUPLICATE KEY UPDATE json_data=VALUES(json_data)
//use $json for presentation too
Today's scores are generated as needed and not stored in the summary. If Bob views his scores again today, the historical ones can come from the summary table or could be stored in a session after the first request. If Bob doesn't visit for a week, no summary needs to be generated.
method 1 seems like a clear winner to me . If you are concerned about size of single table (graphData) being too big you could reduce it by creating
CREATE TABLE `graphdata` (
`graphDataId` INT UNSIGNED NOT NULL,
`categoryId` INT NOT NULL,
`score` FLOAT UNSIGNED NOT NULL,
PRIMARY KEY (`GraphDataId'),
) ENGINE=InnoDB
than create 2 tables because you obviosuly need to have info connecting graphDataId with userId
create table 'graphDataUser'(
`graphDataId` INT UNSIGNED NOT NULL,
`userId` INT NOT NULL,
)ENGINE=InnoDB
and graphDataId date connection
create table 'graphDataDate'(
`graphDataId` INT UNSIGNED NOT NULL,
'graphDataDate' DATE NOT NULL
)ENGINE=InnoDB
i think that you don't really need to worry about number of rows some table contains because most of dba does a good job regarding number of rows. Its your job only to get data formatted in a way it is easly retrived no matter what is the task for which data is retrieved. Using that advice i think should pay off in a long run.
First, I apologize if this has been asked before - indeed I'm sure it has, but I can't find it/can't work out what to search for to find it.
I need to generate unique quick reference id's, based on a company name. So for example:
Company Name Reference
Smiths Joinery smit0001
Smith and Jones Consulting smit0002
Smithsons Carpets smit0003
These will all be stored in a varchar column in a MySQL table. The data will be collected, escaped and inserted like 'HTML -> PHP -> MySQL'. The ID's should be in the format depicted above, four letters, then four numerics (initially at least - when I reach smit9999 it will just spill over into 5 digits).
I can deal with generating the 4 letters from the company name, I will simply step through the name until I have collected 4 alpha characters, and strtolower() it - but then I need to get the next available number.
What is the best/easiest way to do this, so that the possibility of duplicates is eliminated?
At the moment I'm thinking:
$fourLetters = 'smit';
$query = "SELECT `company_ref`
FROM `companies`
WHERE
`company_ref` LIKE '$fourLetters%'
ORDER BY `company_ref` DESC
LIMIT 1";
$last = mysqli_fetch_assoc(mysqli_query($link, $query));
$newNum = ((int) ltrim(substr($last['company_ref'],4),'0')) + 1;
$newRef = $fourLetters.str_pad($newNum, 4, '0', STR_PAD_LEFT);
But I can see this causing a problem if two users try to enter company names that would result in the same ID at the same time. I will be using a unique index on the column, so it would not result in duplicates in the database, but it will still cause a problem.
Can anyone think of a way to have MySQL work this out for me when I do the insert, rather than calculating it in PHP beforehand?
Note that actual code will be OO and will handle errors etc - I'm just looking for thoughts on whether there is a better way to do this specific task, it's more about the SQL than anything else.
EDIT
I think that #EmmanuelN's suggestion of using a MySQL trigger may be the way to handle this, but:
I am not good enough with MySQL, particularly triggers, to get this to work, and would like a step-by-step example of creating, adding and using a trigger.
I am still not sure whether this will will eliminate the possibility of two identical ID's being generated. See what happens if two rows are inserted at the same time that result in the trigger running simultaneously, and produce the same reference? Is there any way to lock the trigger (or a UDF) in such a way that it can only have one concurrent instance?.
Or I would be open to any other suggested approaches to this problem.
If you are using MyISAM, then you can create a compound primary key on a text field + auto increment field. MySQL will handle incrementing the number automatically. They are separate fields, but you can get the same effect.
CREATE TABLE example (
company_name varchar(100),
key_prefix char(4) not null,
key_increment int unsigned auto_increment,
primary key co_key (key_prefix,key_increment)
) ENGINE=MYISAM;
When you do an insert into the table, the key_increment field will increment based on the highest value based on key_prefix. So insert with key_prefix "smit" will start with 1 in key_inrement, key_prefix "jone" will start with 1 in key_inrement, etc.
Pros:
You don't have to do anything with calculating numbers.
Cons:
You do have a key split across 2 columns.
It doesn't work with InnoDB.
How about this solution with a trigger and a table to hold the company_ref's uniquely. Made a correction - the reference table has to be MyISAM if you want the numbering to begin at 1 for each unique 4char sequence.
DROP TABLE IF EXISTS company;
CREATE TABLE company (
company_name varchar(100) DEFAULT NULL,
company_ref char(8) DEFAULT NULL
) ENGINE=InnoDB
DELIMITER ;;
CREATE TRIGGER company_reference BEFORE INSERT ON company
FOR EACH ROW BEGIN
INSERT INTO reference SET company_ref=SUBSTRING(LOWER(NEW.company_name), 1, 4), numeric_ref=NULL;
SET NEW.company_ref=CONCAT(SUBSTRING(LOWER(NEW.company_name), 1, 4), LPAD(CAST(LAST_INSERT_ID() AS CHAR(10)), 4, '0'));
END ;;
DELIMITER ;
DROP TABLE IF EXISTS reference;
CREATE TABLE reference (
company_ref char(4) NOT NULL DEFAULT '',
numeric_ref int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (company_ref, numeric_ref)
) ENGINE=MyISAM;
And for completeness here is a trigger that will create a new reference if the company name is altered.
DROP TRIGGER IF EXISTS company_reference_up;
DELIMITER ;;
CREATE TRIGGER company_reference_up BEFORE UPDATE ON company
FOR EACH ROW BEGIN
IF NEW.company_name <> OLD.company_name THEN
DELETE FROM reference WHERE company_ref=SUBSTRING(LOWER(OLD.company_ref), 1, 4) AND numeric_ref=SUBSTRING(OLD.company_ref, 5, 4);
INSERT INTO reference SET company_ref=SUBSTRING(LOWER(NEW.company_name), 1, 4), numeric_ref=NULL;
SET NEW.company_ref=CONCAT(SUBSTRING(LOWER(NEW.company_name), 1, 4), LPAD(CAST(LAST_INSERT_ID() AS CHAR(10)), 4, '0'));
END IF;
END;
;;
DELIMITER ;
Given you're using InnoDB, why not use an explicit transaction to grab an exclusive row lock and prevent another connection from reading the same row before you're done setting a new ID based on it?
(Naturally, doing the calculation in a trigger would hold the lock for less time.)
mysqli_query($link, "BEGIN TRANSACTION");
$query = "SELECT `company_ref`
FROM `companies`
WHERE
`company_ref` LIKE '$fourLetters%'
ORDER BY `company_ref` DESC
LIMIT 1
FOR UPDATE";
$last = mysqli_fetch_assoc(mysqli_query($link, $query));
$newNum = ((int) ltrim(substr($last['company_ref'],4),'0')) + 1;
$newRef = $fourLetters.str_pad($newNum, 4, '0', STR_PAD_LEFT);
mysqli_query($link, "INSERT INTO companies . . . (new row using $newref)");
mysqli_commit($link);
Edit: Just to be 100% sure I ran a test by hand to confirm that the second transaction will return the newly inserted row after waiting rather than the original locked row.
Edit2: Also tested the case where there is no initial row returned (Where you would think there is no initial row to put a lock on) and that works as well.
Ensure you have an unique constraint on the Reference column.
Fetch the current max sequential reference the same way you do it in your sample code. You don't actually need to trim the zeroes before you cast to (int), '0001' is a valid integer.
Roll a loop and do your insert inside.
Check affected rows after the insert. You can also check the SQL state for a duplicate key error, but having zero affected rows is a good indication that your insert failed due to inserting an existing Reference value.
If you have zero affected rows, increment the sequential number, and roll the loop again. If you have non-zero affected rows, you're done and have an unique identifier inserted.
Easiest way to avoid duplicate values for the reference column is to add a unique constraint. So if multiple processes try to set to the same value, MySQL will reject the second attempt and throw an error.
ALTER TABLE table_name ADD UNIQUE KEY (`company_ref`);
If I were faced with your situation, I would handle the company reference id generation within the application layer, triggers can get messy if not setup correctly.
A hacky version that works for InnoDB as well.
Replace the insert to companies with two inserts in a transaction:
INSERT INTO __keys
VALUES (LEFT(LOWER('Smiths Joinery'),4), LAST_INSERT_ID(1))
ON DUPLICATE KEY UPDATE
num = LAST_INSERT_ID(num+1);
INSERT INTO __companies (comp_name, reference)
VALUES ('Smiths Joinery',
CONCAT(LEFT(LOWER(comp_name),4), LPAD(LAST_INSERT_ID(), 4, '0')));
where:
CREATE TABLE `__keys` (
`prefix` char(4) NOT NULL,
`num` smallint(5) unsigned NOT NULL,
PRIMARY KEY (`prefix`)
) ENGINE=InnoDB COLLATE latin1_general_ci;
CREATE TABLE `__companies` (
`comp_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`comp_name` varchar(45) NOT NULL,
`reference` char(8) NOT NULL,
PRIMARY KEY (`comp_id`)
) ENGINE=InnoDB COLLATE latin1_general_ci;
Notice:
latin1_general_ci can be replaced with utf8_general_ci,
LEFT(LOWER('Smiths Joinery'),4) would better become a function in PHP
Hi all is there any SQL function to reset all auto_increment ids?
I mean , i have this situation (id =1 , id2 = 2, id3 =3)
once i delete id i would like to be returned this situation (id2 = 1, id3 = 2) and so on.
Need i a script to do that?
You can reset the AUTO_INCREMENT for the whole table:
ALTER TABLE table AUTO_INCREMENT=1;
But you cannot really go back to fill in values in individual rows that have been deleted. The auto_increment values are intended to be unique, but aren't guaranteed to remain sequential if deletions happen or new rows are inserted with an explicit value set for the auto_increment column.
If you needed to modify them to be sequential again, it would have to be done in code, probably looping over all rows and performing UPDATE statements individually.
Id is Id which means it must never change! At least in good database design. Id is something you are born with and you die with.