I'm working on a site that generates a random puzzle and the exact puzzle can be recreated using this number. So i give them the url to the puzzle in case they want to share it with a friend or solve it later etc. somepuzzlesite.com/4233312409408127365 would generate a unique puzzle that is always the same if they use that link/number
What I don't want is to expose how the puzzle is generated. The 9th digit, for example, can be 0 to 3, and defines the rotation of the puzzle.
If I just use it "as is" then a user could change a single digit in the url, see what changes, and eventually discover how I make my puzzle. I also wouldn't mind if my number were smaller, since I don't need all the way to 9:
digits 1st to 8th [possible values 0 to 5]
digit 9 [value 0 to 3]
digits 11th to 20th represent the arrangement of 10 objects in order.
I could just specify the first 9 objects in order, and then the unmentioned item is assumed to be last. (that gets me down to 9 digits used)
I could change the base, or use alpha characters in my URL in addition to digits, but some alpha characters are always trouble - lowercase "L" and "1" get mixed up easily, and "o" and zero can too.
But to keep the question simple, I'd just like to make it so that changing a single digit would represent a totally different number, and thereby create a totally different puzzle, rather than the minor difference that would result if I only changed one factor.
Let's see... a rather naive approach would be this: Assign each value so many bits as is necessary to hold it. That is, you'd have eight 3-bit values, one 2-bit value, and ten 4-bit values. That's 8*3+2+10*4=66 bits. Well, if you skip that last one, you'll get 62 bits. You can get it even smaller, but that gets unnecessarily complicated.
Anyway.
Just take any standard encryption algorithm and apply it to these 62 bits. The industry-standard AES (aka Rijndael) operates on 128-bit blocks, which might be a bit too lengthy - or maybe not, depending on your preferences. 3DES won't be any worse for your purposes, and works on 64-bit blocks, which is just perfect.
When you've got your encrypted 64 or 128 bits, just hex-encode them and make that the URL. If it's 64 bits, you'll have 16 hex characters. Not too much. And you'd be hard pressed to go lower anyway. Plus, it uses only 0-9, A-F, and there is little chance of mix-ups when calling over the phone. Not that people often share links vocally these days. :P
Your number is about 18 digits or about 61-62 bits in size. That means that it will fit nicely in a single DES block (8 bytes, or 64 bits). If you encrypt it in ECB mode you would retrieve a 64 bit value, which looks like a random value. You can leave the key on the server. A single 8 byte DES key should be enough for obfuscation, but you could also use 16/24 byte key for DESede encryption.
So: when generating a new random puzzle: create your number, convert it into a byte array with a length of 8 bytes (or N * 8 bytes if your number gets too big) then encrypt it with a single key kept on the server (8, 16 or 24 randomly generated bytes) and on some backup. The result will be 8 bytes again, which you can convert to a number of about 20 digits. If the user supplies a previously generated number, you can decrypt it with the key on the server, revert the resulting bytes back into the number used to create the puzzle.
Note that if the user just enters some random number, it will still decrypt, so you may want to check the resulting number for validity (e.g. test if a digit is indeed 0..3 and not something else).
Another approach to solve this would be to save the puzzles internally and bind the puzzle to an unique ID.
Related
I'm getting hexadecimal digits to generate unique random activators link, such as:
hostname/account/confirm/$randomHex
By searching, actually, my random hex using PHP could be:
bin2hex( openssl_random_pseudo_bytes(16) )
Above generates a string with 32 hex digits and i would appreciate to use a less length as 12 hex digits.
Considering the power processing of computers, what's the more secure minimum size of the hexadecimal that i'm able to use?
Considering the power processing of computers, what's the more secure minimum size of the hexadecimal that i'm able to use?
This is actually an easy number to calculate, if you have a threat model in place.
Based on the URL you provided, it seems you're generating a URL for email ownership verification. This is decidedly needy than, say, a password reset URL.
If you rate limit bad attempts (i.e. block their IP address from being able to attempt again for 24 hours), you can get by with 8 hex characters (32 bits) sheer chance means they'll be able to guess a valid confirmation link after 65,536 tries with 50% probability. (The Birthday paradox.) Pulling this off would also require 65,536 IP addresses just to blindly confirm someone's email address (probably not their own).
HOWEVER!
As stated above, if you are using this for, e.g. a recovery feature (I forgot my password), don't skimp out on string length. 128 bits (32 hex, 16 raw binary) should be considered a lower bound. I'd say shoot for 256 bits just to be safe.
Above generates a string with 32 hex digits and i would appreciate to use a less length as 12 hex digits.
If you want to increase the security of a string given a fixed length, the only way to do so is to increase the number of possible values for each character in the string.
Even if you were using raw binary, which you're not, the upper limit of 11 characters is 88 bits of entropy. Specifying hex cuts you down to 44 (but most likely 40, since you'd probably write bin2hex(random_bytes(5)) here).
If you want to securely generate an fixed-size string with an arbitrary alphabet, check out this StackOverflow answer.
16 randomly generated bytes gives 128bits of entropy. A key with 128bits of entropy is uncrackable using an offline brute force attack. Even with every computer in the world working on cracking it.
However, you are looking to prevent an online brute force attack, which is much slower. If you wanted 12 hex characters, this would be 6 bytes and therefore 48 bits of entropy. This gives you 281,474,976,710,656 possibilities. If your site takes 0.25* seconds to respond, this would take 2^47 * 0.25 = 35,184,372,088,832 seconds to brute force on average by making requests to your site (1.116 million years).
You're safe with 48 bits.
*In reality this would be a parallel attack, so the attacker would not have to wait for a response if all they're trying to do is validate an account. However, there will be a rate limit to any system, slowing the attack. Adjust the figures to suit your system as necessary.
What is the main purpose for the maximum limit for number in php and mysql?
Does this means that we can/cannot process or store numbers that is larger than the limit?
I echo PHP_INT_MAX and it shows 2147483647.
I multiply by 1000000000000000000000 and get answers like 2.147483647E+30 which i think is already over the limit?
Please advise.
Many thanks for guidance.
This question arise when I'm thinking about validating the user form input. I want to make sure that the user input is according to our defined number format and not something else.
Then i do some online search for best practices and come to aware of this "limits" but do not know how to handle it correctly when using PHP & MYSQL. Pls advise:
Step 1: trim and convert user form input "string number" to number.
Step 2: validate the number is in positive integer format.
Step 3: validate the number does not exceed my "max limit".
Since php limit (2,147,483,647) is smaller than mysql limit (18,446,744,073,709,500,000)? i'll take php as my max limit.
Step 4: perform some calculations...
Step 5: validate my result does not exceed my max limit.
Step 6: store the result in mysql.
It's hardware limit of the CPU. You can still use bc math function to work with larger numbers. In mysql it is about aligning bytes, so it knows at which offset is which column.
The multiply result is converted to float.
PHP_INT_MAX is the constant of the largest integer you can use on this build of PHP (32 bit in your case). If your OS and CPU support 64 bit, you can use a 64 bit build of PHP to support a much larger number in an integer. (This causes problems where developers have designed their code around the limits of a 64 bit build and is then used on 32 bit builds, assuming the type matters.)
When you multiply the number by a larger one, PHP recognises this new value is not going to fit in an integer and converts the type to a long or float. In the latter case, you would lose some precision, so it's important you're careful when considering how your code affects variable types. In some languages, you would receive an error for trying to set a value larger than was allowed by that type, because the language would refuse to change the type automatically for you. In this way, PHP is a more basic programming language to use.
<?php
$my_number = PHP_INT_MAX;
var_dump(gettype($my_number), $my_number);
$my_number = $my_number * 1000000000000000000000;
var_dump(gettype($my_number), $my_number);
Output:
string(7) "integer"
int(2147483647)
string(6) "double"
float(2.147483647E+30)
In the world of computing, there are many limits based upon the mathematical model of the computer hardware we use.
For instance, if we decided to represent an integer number in 1 bit of memory, then we would be able to represent the numbers 0 and 1.
If we were to increase that to the more common values of 8, 16, 32 or 64 bits then we can represent the following number of distinct values:
2^8 - 256,
2^16 - 65,536,
2^32 - 4,294,967,296,
or 2^64 - 18,446,744,073,709,551,616.
Of course, if we wish to be able to represent negative numbers then we can sign the integer (use one bit to indicate negative or positive). In the case of a 32 bit signed integer this would allow us to represent the numbers: −(2^31) to 2^31 − 1 or -2,147,483,648 to +2,147,483,647 (the same upper limit as the number in your question).
Integers are arguably the simplest form of number representation in a computer system (beyond straight binary), but because of the inherent limits of the system, we will often need to use other systems for larger numbers that we cannot represent with an integer. PHP will switch from using an integer to using floating point numbers when the limit is exceeded.
The limit you are seeing in PHP is compiled in, and will depend upon the architecture of the compiler. It looks as if your PHP was compiled as 32 bits.
You can read up far more on computer number systems on Wikipedia.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
php short hash
I need to generate a short hash. The shortest possible from urls say under 6 characters.
I need them to be unique just for the same domain, so a hash from
www.example.com/category/sth/blablabla must be different than one from
www.example.com/category2/sth/blabla but not from:
www.example2.com/category/sth/blablabla
Would using md5($url) and then picking some 5 characters out of that result (for example the first, last, middle and 2 other characters) give and unique id?
Would this abbreviated hash be unique as well?
A hash is not unique by definition. It's mathematically impossible to get a unique hash for something longer than the hash, unless it does not vary fully, which is the case for URLs but you cannot exploit it generally. Alternatively, you could use a simple incrementing ID, but that won't allow you to recognize matching URLs.
Either use a really long hash (at least 10 characters, ideally using upper and lower case letters), or accept collisions and handle them appropriately. Which is how actual hash tables work.
For low probability of collisions you can use universal hashing techniques. For example, choose a prime number P. Then for each character of the URL choose a random in the interval [0, P). Compute the hash of the URL as SUM(a[i]*c[i]) mod P, where c[i] is a character in the original URL. Then take the string containing the digits of the obtained integer as the hash.
Read more in this paper: http://www.cs.cmu.edu/~avrim/451/lectures/lect0929.pdf.
Yes, a small change in a URL will change pretty much every character in a good hash. MD5 or SHA1 is probably fine for this. Hence, take the first X characters - and you won't get any improvement by choosing the last X characters, or the first/last/middle. They're all good!
Obviously the more characters you put in your partial hash, the less likely you are to get collisions.
I would try using crc32($url); it will give an integer usually 10-11 digits-long, could be a negative value, but still it will be shorter than 32 chars for md5.
The only problem is that crc32 is not 100% unique, but it's very unlikely that two different URLs will end up with the same checksum (but still there is a possibility).
My PHP program is working with an array of values ranging from 0 to 7. I'm trying to find the most effective way to store those values in PHP. By most effective I mean using the less number of bits.
It's clear that each value only need 3 bits of storage space (b000=0 to b111=7). But what is the most efficient way to store those 3bits values in a binary string ?
I don't know in advance how many 3 bits values I'll need to store or restore, but it might be a lot, so 64bits is clearly not enough.
I was looking into pack() and unpack(): I could store two values in each byte and use a pack('C', $twoValues), but I'm still loosing 2 bits.
Will it work ? Is there a more effective way of storing those values ?
Thanks
You didn't ask if it was a good idea - as many suggested, your benefit of that kind of space compression, is easily lost in the extra processing - but that's another topic :)
You're also not mentioning where you're storing the data after. Whatever that storage location/engine is maybe have further conditions and specialized types (eg a database has a binary column format, might have a byte column format, may even support bit storage etc).
But sticking with the topic, I guess best 3 bit storage is as a nibble (waisting one bit) and I suppose I'd combine two nibbles into a byte (loosing two bits overall). Yes you're loosing two bits (if that's key), but it's simple to combine the two values so you're processing overhead is relatively small:
$byte=$val1*7+$val2;
$val2=$byte%7;$val1=($byte-$val2)/7;
If a byte isn't available, you can combine these up to make 16 (4 stored), 32 (8), 64 (16) bit integers. You can also form an array of these values for larger storage.
I'd consider the above more human readable, but you could also use bit-logic to combine and separate the values:
$combinedbyte=$val1<<3|$val2;
$val2=$combinedbyte&7;$val1=($combinedbyte&56)>>3);
(This is effectively what the PACK/UNPACK commands do)
Alternatively you could encode into characters, since in ASCII the first few are protected, you might as well start at A (A-Z+6 punc+a-z gives you 58 when you only need 49 to store your two values).
$char=chr(($val1*7+$val2)+65); //ord('A')=65
$val2=(ord($char)-65)%7;$val1=(ord($char)-65-$val2)/7;
A series of these encoded characters could be stored as an array or in a null terminated string.
NOTE:
In the case of -say- 64 bit integers above, we're storing 3 bits in 4 so get 64/4=16 storage locations. This means we're waisting 16 further bits (1 per location) so you might be tempted to add another 5 values, for a total of 21 (21*3=63 bits, only 1 wasted). That's certainly possible (with integer math - although most PHP instances don't work # 64 bits, or bit-logic solutions) but it complicates things in the long run - probably more trouble than it's worth.
The best way is to store them as integers and not get involved with packing things bit by bit. Unless you have an actual engineering reason you need these to be stored as 3-bit values (for example, interfacing with hardware), you're just asking for headaches. Keep in mind, esp for odd bit sizes, they become pretty difficult to have direct access to if you do this. And if you are sticking these values in a database, you wouldnt be able to search or index on values packed like this. Store them as integers, or if in a db, perhaps a short integer or byte.
That kind of technique is only necessary if you will have at least half a billion of these. Think about it, the CPU will have to have data in one register, the mask in another and AND them just to get your value out. Now imagine iterating over a list of these that is long enough to justify that kind of space saving technique. A 50% reduction in space and an order of magnitude slower.
Looking at http://php.net/manual/en/language.types.php, you should store them as integers. However, the question is whether to let one integer value represent many 3-bit values or not. The former is more complex but requires less memory, whereas the first is the opposite. If you don't have an extreme need to reduce the amount of memory you use, then I would suggest the latter (use one integer for one 3-bit value).
The main problem with storing many 3-bit values in one integer is figuring out how many 3-bit values there are. You could use an array of integers, and then have an extra integer which states the total number of 3-bit values. However, as also stated in the manual, the number of bits used for an integer value is platform-dependent. So you would have to know whether an integer is 32 bits or 64 bits, or else you may try to store too many values and lose data, or you risk using more memory than needed (which would be a bad thing as you're aiming to use as little memory in the first place).
I would convert each integer to binary, concatenate all of them, and then split the resulting string into bytes. Each byte will be 0-255 so it can be stored as an individual character.
I'm not sure what this is called, which is why I'm having trouble searching for it.
What I'm looking to do is to take numbers and convert them to some alphanumeric base so that the number, say 5000, wouldn't read as '5000' but as 'G4u', or something like that. The idea is to save space and also not make it obvious how many records there are in a given system. I'm using php, so if there is something like this built into php even better, but even a name for this method would be helpful at this point.
Again, sorry for not being able to be more clear, I'm just not sure what this is called.
You want to change the base of the number to something other than base 10 (I think you want base 36 as it uses the entire alphabet and numbers 0 - 9).
The inbuilt base_convert function may help, although it does have the limitation it can only convert between bases 2 and 36
$number = '5000';
echo base_convert($number, 10, 36); //3uw
Funnily enough, I asked the exact opposite question yesterday.
The first thing that comes to mind is converting your decimal number into hexadecimal. 5000 would turn into 1388, 10000 into 2710. Will save a few bytes here and there.
You could also use a higher base that utilizes the full alphabet (0-Z instead of 0-F) or even the full 256 ASCII characters. As #Yacoby points out, you can use base_convert() for that.
As I said in the comment, keep in mind that this is not an efficient way to mask IDs. If you have a security problem when people can guess the next or previous ID to a record, this is very poor protection.
dechex will convert a number to hex for you. It won't obfuscate how many records are in a given system, however. I don't think it will make it any more efficient to store or save space, either.
You'd probably want to use a 2 way crypt function if obfuscation is needed. That won't save space, either.
Please state your goals more clearly and give more background, because this seems a bit pointless as it is.
This might confuse more people than simply converting the base of the numbers ...
Try using signed digits to represent your numbers. For example, instead of using digits 0..9 for decimal numbers, use digits -5..5. This Wikipedia article gives an example for the binary representation of numbers, but the approach can be used for any numeric base.
Using this together with, say, base-36 arithmetic might satisfy you.
EDIT: This answer is not really a solution to the question, so ignore it unless you are trying to hash a number.
My first thought we be to hash it using eg. md5 or sha1. (You'd probably not save any space though...)
To prevent people from using rainbow-tables or brute force to guess which number you hashed, you can always add a salt. It can be as simple as a string prepended to your number before hashing it.
md5 would return an alphanumeric string of exactly 32 chars and sha1 would return one of exaclty 40 chars.