I have the following function which seems to run forever. Its creating a random string then checking if its in the database. If it is it should run again and again until it has one that is new. It should then return the value to me
public function checkPromo(){
$continue = true;
while ($continue){
$promo = $this->getRandString(6);
$query = sprintf("SELECT * FROM table WHERE field=%s",
$this->db->cleanCode($promo, "text"));
$result = $this->db->query($query);
if($this->db->num_rows($result) >= 1){
$continue = false;
}
}
return $promo;
}
Your specific problem is that your check is backwards. You're looping until you find a duplicate, which will likely take quite a while or forever.
Even if you fixed it, it's a bad algorithm. It will take exponentially longer to generate a new unique code the more codes you have in your database, since the chance of duplicates increases. Especially if your codes are only 6 characters long the chance for duplication increases a lot.
Note that you're also prone to race conditions with your code (think it through from that perspective).
There are two approaches to generating unique ids:
use an incrementing counter, which means you're using a central generator which keeps track of existing ids
use a decentralised approach where you're not keeping track of your ids, but you're using an algorithm that is random enough and has a large enough space that collisions are so unlikely as to be irrelevant in practice
You're combining the worst of both worlds: you're using a central system to keep track of your ids, but you're using random id generation with it. Use one or the other, not both.
If you're going to use a database anyway, just use a standard auto_increment id. If you want it to look a bit random, hash it with MD5 or such.
Alternatively, simple pre-generate all possible codes (just 6 characters isn't a lot), and pick one at random using a method which is not prone to race conditions. Something along the lines of this:
UPDATE codes
SET claimed_user_id = %d
WHERE claimed_user_id IS NULL
ORDER BY RAND()
LIMIT 1
Otherwise, if you want decentralised random ids, use an appropriate algorithm, which is pretty much UUID.
while (true) is a dangerous thing. As you noted, its defaulted to run again and again creating an infinite loop.
You'd be better offer utilizing an auto-incremented value(link assumes MySQL) or generate a pseudo unique id with PHP (such as uniqid()).
You could expand on these values further by prefixing or suffixing a common word, say promo, or padding or hashing them to create a standard length.
Assuming you simply want to generate a promocode that was not recorded in the database and taking into account the context, I advise you change the attitude a little bit:
public function generatePromo(){
$promo = $this->getRandString(6);
$query = sprintf("SELECT * FROM table WHERE field=%s", $this->db->cleanCode($promo, "text"));
$result = $this->db->query($query);
if($this->db->num_rows($result) > 0){ # in case a record with this "text" already exists, run this method again
$this->generatePromo();
} else {
return $this->promo = $promo; # otherwise return the value/store it in the object
}}
If you expect your table to contain only unique values, then I suggest you make it UNIQUE. That will significantly reduce query time.
Related
I am trying to come up with a solution to generate a Unique Id preferably on the Fly. Usage scope could be Order, Product or Plan Id, where there is no security involved.
I don't like idea of generating a random number and then querying the db to check its uniqueness and repeating the process if it is not in this case where security isn't an issue.
Also I don't prefer using Auto Increment id since it looks so simple.
My initial thought is to use a combination of Auto Increment id + timestamp converting the decimal value to hex so it looks like a random string. And then finally prefixing and suffixing it with 2 digit random string.
function generateUID($num){
$str = dechex(time()+ intval($num));
$prefix = dechex(rand(1,15));
$suffix = dechex(rand(1,15));
return strtoupper($suffix.$str.$prefix);
}
Where $num is the auto_increment id
Returns something like E53D42A046
Is this the right way to go about doing this, are there collision issues ?
I thank all responses..!
I acknowledge the usefulness of uniqid() but in this context to be genuinely unique Auto_Increment need to play a significant part so how will it do so in uniqid. Passing it as a prefix would result in a Product id which vary greatly in size. (153d432da861fe, 999999953d432f439bc0).
To expand the scope further, Ideally we want a unique code which looks random with fairly consistent length and could be reversed to the auto_increment id from which it was created.
Such a function already exists - uniqid()
http://php.net/manual/en/function.uniqid.php
It works based on the timestamp down to the microsecond - you can add a prefix based on the process ID to further refine it. There are a couple more robust versions out there as well - see PHP function to generate v4 UUID
i made a small code that generates different type of code, but i'll make it simpler,
i have a registration form submitted while submitting i collect some info about the user and i create for him a random, but i want this random to be unique for this user.
so i have 3 cases :
$code_random = rand(1000,9999);
if($code_random < 0){
$code_random = -$code_random;
}
$random = $fname.$code_random; //case 1
$random = $lname.$code_random; //case 2
$random = $fname.lname.$code_random; //case 3
But i want to create case 1 check if this random exist in the database, if it does use the second case if it does use the third case, before submitting the form and without displaying anything for the user.
Thanks for your help in advance.
Don't reinvent the wheel - SQL databases have two great ways of assigning unique IDs to every row.
1) Auto-incrementing primary key - goes up by one for every new row. Managed by the database, guaranteed to not use the same value for two rows by mistake. Nice and small and simple. http://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html
2) GUIDs (also known as UUIDs) - The algorithm used to generate GUIDs means that you'll never see the same one twice, ever. Over auto-incrementing integers, they have the advantage of being unpredictable, being generateable outside of the database and being meaningful outside of their database table context. http://www.php.net/manual/en/function.uniqid.php#94959 http://dev.mysql.com/doc/refman/5.0/en/miscellaneous-functions.html#function_uuid
If you really want to use a random integer, you can use the MySQL - rand() function for this
insert into users (id, ...) values (FLOOR(1000 + RAND() * (9999 – 1000)), ...)
There is a simple way of making infos unique: Create a unique index on the database column.
Then simply insert what you want and check if the database complains about violating the unique index. If this is the case, use one of your alternative queries and check again.
What if the last query still does not work?
Hi I'm Integrating memcache in my codeigniter application,
My query change on the user selected values. some thing like this
$sql='select * from user where user_name="'.$name.'" and location='".$location."'";
$result = $this->memcached_library->get(md5($sql));
if(!$result ){
/* execute query and get $result */
$this->memcached_library->add(md5($sql), $result );
}else{
return $result;
}
This is my approach to handle the keys for each different query. But I have heard some where that md5() is not always unique.?
1> Is md5() always unique?
2> if md5() is not unique than what shoud be the other option....
3> what about crc32() is that unique??
Thanks....
A. Your SQL is wrong it should be
$sql = "select * from user where user_name = '{$name}' and location = '{$location}'";
B. You code might have errors since MD5 is case sensitive
See
var_dump(md5("A"),md5("a"));
Output
string '7fc56270e7a70fa81a5935b72eacbe29' (length=32)
string '0cc175b9c0f1b6a831c399e269772661' (length=32)
Better Approach us using strtolower
$result = $this->memcached_library->get(md5(strtolower($sql)));
C, Is MD5 unique
MD5 cannot guarantee total uniqueness, however there are approximately 3.402823669209387e+38 different values in a 32 digit hex value (16^32). That means that, assuming the math behind the algorithm gives a good distribution, your odds are phenomenally small that there will be a duplicate.
D. Better option is using sha1
$betterKey = "user" . sha1(strtolower($sql));
^- identify request for each table
MD5 is unique enough in your case.
But:
I used this technic for a while before dropping it. Why?
md5 is considered to be too fast for password hashing. But it is still a hashing function and when called each time you have to make a DB query, it might slow down your code. I've seen in the past that hashing the SQL query to generate a key was responsible 20% of the PHP execution time, which was huge.
If you need to delete or update a specific key, for example in your case if a user want to change his location and you want this change to be reflected as soon as possible, you'll have to rebuild the same DB query, hash it, to retrieve your key.
The solution I prefer:
Create simple, short, useful keys instead. Typically, use the method name from your model. If the method in your example is User::getUser($name, $location), make your key:
$key = "User::getUser($name, $location)"
You won't need hashing at all, it will be clearer, and easier to manage keys.
You are refering to the Collision vulnerabilities of MD5. For practical, simple use you can forget about those. So MD5( uniqid() ) is unique. See http://php.net/manual/en/function.uniqid.php
For generating hashes in use like SessionID, MD5() is fine.
This is for a file sharing website. In order to make sure a "passcode", which is unique to each file, is truely unique, I'm trying this:
$genpasscode = mysql_real_escape_string(sha1($row['name'].time())); //Make passcode out of time + filename.
$i = 0;
while ($i < 1) //Create new passcode in loop until $i = 1;
{
$query = "SELECT * FROM files WHERE passcode='".$genpasscode."'";
$res = mysql_query($query);
if (mysql_num_rows($res) == 0) // Passcode doesn't exist yet? Stop making a new one!
{
$i = 1;
}
else // Passcode exists? Make a new one!
{
$genpasscode = mysql_real_escape_string(sha1($row['name'].time()));
}
}
This really only prevents a double passcode if two users upload a file with the same name at the exact same time, but hey better safe than sorry right? My question is; does this work the way I intend it to? I have no way to reliably (read: easily) test it because even one second off would generate a unique passcode anyway.
UPDATE:
Lee suggest I do it like this:
do {
$query = "INSERT IGNORE INTO files
(filename, passcode) values ('whatever', SHA1(NOW()))";
$res = mysql_query($query);
} while( $res && (0 == mysql_affected_rows()) )
[Edit: I updated above example to include two crucial fixes. See my answer below for details. -#Lee]
But I'm afraid it will update someone else's row. Which wouldn't be a problem if filename and passcode were the only fields in the database. But in addition to that there's also checks for mime type etc. so I was thinking of this:
//Add file
$sql = "INSERT INTO files (name) VALUES ('".$str."')";
mysql_query($sql) or die(mysql_error());
//Add passcode to last inserted file
$lastid = mysql_insert_id();
$genpasscode = mysql_real_escape_string(sha1($str.$lastid.time())); //Make passcode out of time + id + filename.
$sql = "UPDATE files SET passcode='".$genpasscode."' WHERE id=$lastid";
mysql_query($sql) or die(mysql_error());
Would that be the best solution? The last-inserted-id field is always unique so the passcode should be too. Any thoughts?
UPDATE2: Apperenatly IGNORE does not replace a row if it already exists. This was a misunderstanding on my part, so that's probably the best way to go!
Strictly speaking, your test for uniqueness won't guarantee uniqueness under a concurrent load. The problem is that you check for uniqueness prior to (and separately from) the place where you insert a row to "claim" your newly generated passcode. Another process could be doing the same thing, at the same time. Here's how that goes...
Two processes generate the exact same passcode. They each begin by checking for uniqueness. Since neither process has (yet) inserted a row to the table, both processes will find no matching passcode in database, and so both processes will assume that the code is unique. Now as the processes each continue their work, eventually they will both insert a row to the files table using the generated code -- and thus you get a duplicate.
To get around this, you must perform the check, and do the insert in a single "atomic" operation. Following is an explanation of this approach:
If you want passcode to be unique, you should define the column in your database as UNIQUE. This will ensure uniqueness (even if your php code does not) by refusing to insert a row that would cause a duplicate passcode.
CREATE TABLE files (
id int(10) unsigned NOT NULL auto_increment PRIMARY KEY,
filename varchar(255) NOT NULL,
passcode varchar(64) NOT NULL UNIQUE,
)
Now, use mysql's SHA1() and NOW() to generate your passcode as part of the insert statement. Combine this with INSERT IGNORE ... (docs), and loop until a row is successfully inserted:
do {
$query = "INSERT IGNORE INTO files
(filename, passcode) values ('whatever', SHA1(NOW()))";
$res = mysql_query($query);
} while( $res && (0 == mysql_affected_rows()) )
if( !$res ) {
// an error occurred (eg. lost connection, insufficient permissions on table, etc)
// no passcode was generated. handle the error, and either abort or retry.
} else {
// success, unique code was generated and inserted into db.
// you can now do a select to retrieve the generated code (described below)
// or you can proceed with the rest of your program logic.
}
Note: The above example was edited to account for the excellent observations posted by #martinstoeckli in the comments section. The following changes were made:
changed mysql_num_rows() (docs) to mysql_affected_rows() (docs) -- num_rows doesn't apply to inserts. Also removed the argument to mysql_affected_rows(), as this function operates on the connection level, not the result level (and in any case, the result of an insert is boolean, not a resource number).
added error checking in the loop condition, and added a test for error/success after loop exits. The error handling is important, as without it, database errors (like lost connections, or permissions problems), will cause the loop to spin forever. The approach shown above (using IGNORE, and mysql_affected_rows(), and testing $res separately for errors) allows us to distinguish these "real database errors" from the unique constraint violation (which is a completely valid non-error condition in this section of logic).
If you need to get the passcode after it has been generated, just select the record again:
$res = mysql_query("SELECT * FROM files WHERE id=LAST_INSERT_ID()");
$row = mysql_fetch_assoc($res);
$passcode = $row['passcode'];
Edit: changed above example to use the mysql function LAST_INSERT_ID(), rather than PHP's function. This is a more efficient way to accomplish the same thing, and the resulting code is cleaner, clearer, and less cluttered.
I'd personally would have write it on a different way but I'll provide you a much easier solution: sessions.
I guess you're familiar with sessions? Sessions are server-side remembered variables that timeout at some point, depending on the server configuration (the default value is 10 minutes or longer). The session is linked to a client using a session id, a random generated string.
If you start a session at the upload page, an id will be generated which is guaranteed to be unique as long the session is not destroyed, which should take about 10 minutes. That means that when you're combining the session id and the current time you'll never have the same passcode. A session id + the current time (in microseconds, milliseconds or seconds) are NEVER the same.
In your upload page:
session_start();
In the page where you handle the upload:
$genpasscode = mysql_real_escape_string(sha1($row['name'].time().session_id()));
// No need for the slow, whacky while loop, insert immediately
// Optionally you can destroy the session id
If you do destroy the session id, that would mean there's a very slim chance that another client can generate the same session id so I wouldn't advice that. I'd just allow the session to expire.
Your question is:
does this work the way I intend it to?
Well, I'd say... yes, it does work, but it could be optimized.
Database
To make sure to not have the same value in the field passcode on the database layer, add a unique key to this:
/* SQL */
ALTER TABLE `yourtable` ADD UNIQUE `passcode` (`passcode`);
(duplicate key handling has to be taken care of than ofc)
Code
To wait a second until a new Hash is created, is ok, but if you're talking heavy load, then a single second might be a tiny eternity. Therefore I'd rather add another component to the sha1-part of your code, maybe a file id from the same database record, userid or whatever which makes this really unique.
If you don't have a unique id at hand, you still can fall back to a random number rand-function in php.
I don't think mysql_real_escape_string is needed in this context. The sha1 returns a 40-character hexadecimal number anyway, even if there are some bad characters in your rows.
$genpasscode = sha1(rand().$row['name'].time());
...should suffice.
Style
Two times the passcode generation code is used in your code sample. Start cleaning this up in moving this into a function.
$genpasscode = gen_pc(row['name']);
...
function gen_pc($x)
{
return sha1($row[rand().$x.time());
}
If I'd do it, I'd do it differently, I'd use the session_id() to avoid duplicates as good as possible. This way you wouldn't need to loop and communicate with your database in that loop possibly several times.
You can add unique constraint to your table.
ALTER TABLE files ADD UNIQUE (passcode);
PS: You can use microtime or uniqid to make the passcode more unique.
Edit:
You make your best to generate a unique value in php, and unique constraint is used to guarantee that at database side. If your unique value is very unique, but in very rare case it failed to be unique, just feel free to give the message like The system is busy now. Please try again:).
Is there any way I could make my model ID (primary key) generated into random unique 8 digits containing only numbers instead of the default auto increment?
A client requested this specific 8-digits-number-only feature, so I can't argue much about the reasons.
I want to use the PHP uniqid but it's 13 digits and contains alphabets as well.
Any idea?
Thanks.
Update
I forgot to tell that I need the ID randomly generated each new record being saved.
Just want to ask the mechanism on generating the ID and then saving the ID (also the attributes). Do I have to check the database first for the randomly generated ID whether another same key already exists and then save the attributes or what?
Why dont you keep the auto increment but set it to start from 10000000 on your primary key instead of 1?
ALTER TABLE some_table AUTO_INCREMENT=10000000
Yes you can. I assume you are on MySQL, when talking about AI. Just do not set it as auto increment and insert the value as for the other columns. You can create a function or method, which will take up to 8 numbers randomly or in specific order (algorithm).
INSERT INTO model (id, name, value, etc) VALUES (87654321, 'My selected name', 'some price or text', 'etc').
Consider that INT(11) value may accept from -2147483648 to 2147483647. Which will fit you for numbers with 8 digits. If at later time the client request bigger numbers you may need to switch to BIGINT.
I use to set the Primary Keys as unsigned, which allows you to fit numbers between 0 and 4294967295.
For php function - generator of 8 digits:
<?php
mt_srand();
$id = mt_rand(10000000, 99999999);
?>
You can read more about mt_srand() and mt_rand() on php documentation. It is said that they are better than the srand() and rand().
Keep the ID, but pad it.
$id = 6;
$padded_id = sprintf("%013d", $id);
// This will print 0000000000006
That'll pad the $id so that it's 13 digits long.
Every time you need to display the ID use a function to convert it, like this.
function padId($id){
return sprintf("%013d", $id);
}
Or you could make a row in your table called pad-id, then run this function when you create a record (along with mysql_insert_id (to get the ID just inserted)).
The best approach depends on a subtle aspect of your client's randomness requirement --
When they say random do they mean completely unpredictable or just hard to predict? I don't mean to sound like Clinton at the Lewinski trial, but what your client intends when they say random affects whether it will even be possible for you to meet the requirement.
If the client wants to hide user IDs (for some perceived security benefit) and make them virtually impossible to predict or reverse-engineer, then that is very difficult. If the client would be satisfied with just "hard" to predict (which I suspect), then you can do something simple, similar to the md5 approach (#Dotty). But md5 is not collision resistant. And even with the best, provably unique hash algorithms (which md5 is not), you'll have a collision problem if the number of users is large compared to the number of digits you are allowed for user ID's (8). You have about 27 bits to work with in the 8 decimal digits allowed. Which means you're likely to get a collision after 2^N/2 = 2^(27/2) which is about 10K users. So if your client's user list approaches 10K users, then even the best hash algorithm will spend a lot of time filtering out all the collisions.
To solve this without filters and nondeterministic algorithms, just use a simple "Full Cycle" algorithm. Some will produce pseudo-random numbers (PRNs) that are guaranteed to be unique and guaranteed to fully span whatever range you're trying to cover (e.g., the set of all 8-digit positive integers). And if you ever need to reverse engineer the user registration sequence just rerun the full cycle PRN generator again with whatever initial value you used. And you can keep this initial value a secret, like a private key, if your client wants to make it slightly more difficult than easy for a hacker to reverse-engineer your user ID sequence.
Another question for your client is whether leading zeros are allowed in the user id. If so, (and the client's randomness requirements are liberal) then the simple Full Cycle algorithm on Wikipedia will work nicely for you. It could be distilled to 2 lines of PHP.
Whatever algorithm you use, it might be good to actually generate the list of official 8-digit semi-random user IDs in a separate table, and then just "pop" the value from the top of the table (deleting that row) whenever you add a new user. The database memory requirements shouldn't be prohibitive and it will streamline the user experience, eliminating any delays and memory gobbling caused by sophisticated, nondeterministic, random number generators and uniqueness filters. Trying to create the user ID online, live, it's conceivable you could get into a perpetual loop with some hash algorithms stalling your user registration indefinitely. And this stall (due to perpetual collision) might not occur until user 1000 or 10000. In contrast, with the offline lookup table approach, you can easily add additional client-prescribed filters like eliminating IDs with leading zeros; in case the client never wants to see a user with the ID 1 (00000001). And you'd know in advance whether everything is going to always work, without any hangs.