I'm planning to generate and store salt in database and pass it back to UI. I'm using openssl_random_pseudo_bytes function. Looking at the salt result I see following in PHP debug window
and the following in database record
I'm planning to recreate hash on client-side using this salt.
I'm curious if this random string of is okay to work with when passing it back and forth from server to UI? Or should this be encoded before passing it back and forth?
Following is database column definition for SALT column
You should change the column type to VARBINARY (or just BINARY if the length is fixed) instead of VARCHAR. Since the contents of the column are not meant to represent text, binary data types will be more efficient.
In any case, as long as there are no misconfiguration problem you should have no correctness issues in your application even with the column having its current data type.
Related
I asked a question here and I manage to partially implement the advice. Data is now stored encrypted in binary field (varbinary(500)), after I remove the aes-256 encryption and I leave aes-128 (default) codeigniter encryption.
However, I have some questions, and I can't find answers, since I can not find many articles on this subject, so If anyone can answer my questions, or point me to a book, or any other literature for further reading, I would be very grateful.
Why encrypted data must be stored in binary type field? What is wrong with storing it in longtext, or varchar? Does that make the encryption worthless?
Why I must first encode the variable and then encrypt it when I store the data in the binary type of field, and I don't have to do that when I store the data in varchar field?
base64_encode($clientName);
$encClientName = $this->encryption->encrypt($clientName);
In my previous question (see the link on the top) I have been advised to use nonce. Since I didn't know how to use that with codeigniter library, I didn't implement that part. Does that make my data less secure? Can anyone post any snippet code of how to use nonce with the codeigniter?
Again, any link to reading material on this subject (storing encrypted data in the database with php) will be deeply appreciated.
Why encrypted data must be stored in binary type field? What is wrong with storing it in longtext, or varchar? Does that make the encryption worthless?
Encrypted data is binary. It will frequently contain byte sequences which are invalid in your text encoding, making them impossible to insert into a column which expects a string (like VARCHAR or TEXT).
The data type you probably want is either VARBINARY (which is similar to VARCHAR, but not a string) or BLOB (likewise, but for TEXT -- there's also MEDIUMBLOB, LONGBLOB, etc).
Why I must first encode the variable and then encrypt it when I store the data in the binary type of field, and I don't have to do that when I store the data in varchar field?
You don't. This is backwards.
If you were going to use a string-type column to store encrypted data, you could "fake it" by Base64 encoding the data after encryption. However, you're still better off using a binary-type column, at which point you don't need any additional encoding.
In my previous question (see the link on the top) I have been advised to use nonce. Since I didn't know how to use that with codeigniter library, I didn't implement that part. Does that make my data less secure?
Based on what I'm seeing in the documentation, I think the CodeIgniter Encryption library handles this for you by default. You shouldn't have to do anything additional.
In addition to duskwuffs answer, I covered your questions from a more crypto-related viewpoint. He just managed to post a minute before I did :)
Encrypted data must be stored in a binary type field due to the way that Character Encodings work. I recommend you read, if you haven't already, this excellent article by Joel Spolsky that details this very well.
It is important to remember that encryption algorithms operate on raw binary data. That is, a bit string. Literal 1's and 0's that can be interpreted in many ways. You can represent this data as unsigned byte values (255, 255), Hex (0xFF, 0xFF), whatever, they are really just bit strings underneath. Another property of encryption algorithms (or good ones, at least) is that the result of encryption should be indistinguishable from random data. That is, given an encrypted file and a blob of CSPRNG generated random data that have the same length, you should not be able to determine which is which.
Now lets presume you wanted to store this data in a field that expects UTF8 strings. Because the bit string we store in this field could contain any possible sequence of bytes, as we discussed above, we can't assume that the sequence of bytes that we store will denote actual valid UTF8 characters. The implication of this is that binary data encoded to UTF8 and then decoded back to binary is not guaranteed to give you the original binary data. In fact, it rarely will.
Your second question is also somewhat to do with encodings, but the encoding here is base64. Base64 is a encoding that plays very nicely with (in fact, it was designed for) binary data. Base64 is a way to represent binary data using common characters (a-z, A-Z, 0-9 and +, /) in most implementations. I am willing to bet that the encrypt function you are using either uses base64_decode or one of the functions it calls does. What you should actually be interested in is whether or not the output of the encrypt function is a base64 string or actual binary data, as this will affect the type of data field you use in your database (e.g. binary vs varchar).
I believe in your last question you stated that you were using CTR, so the following applies to the nonce used by CTR only.
CTR works by encrypting a counter value, and then xor-ing this encrypted counter value with your data. This counter value is made up of two things, the nonce, and the actual value of the counter, which normally starts at 0. Technically, your nonce can be any length, but I believe a common value is 12 bytes. Because the we are discussing AES, the total size of the counter value should be 16 bytes. That is, 12 bytes of nonce and 4 bytes of counter.
This is the important part. Every encryption operation should:
Generate a new 12 byte nonce to use for that operation.
Your implementation should add the counter and perform the actual encryption.
Once you have the final ciphertext, prepend the nonce to this ciphertext so that the result is len(ciphertext) + 12) bytes long.
Then store this final result in your database.
Repeating a nonce, using a static nonce, or performing more than 2^32 encryption operations with a single 12 byte nonce will make your ciphertext vulnerable.
I'm currently working with data I'd like to temporarily store in my database as encrypted data. I'm not worried about the database getting hacked into, I just want to ensure the people that had entered the data that it is not reachable by any other than themselves. (and me of course)
The data is not meant to be stored permanently in the database since I'm exporting it to a third party application using their API, but since they have a rate limit I need to store the data in our database until the limit is over and I can upload it. (Assuming the rate limit occurs)
The process:
The request I receive from the form is in an array, so to begin with I serialize() the array to get a long string which I will unserialize() later.
Then I want to use a method that lets me convert the string into numbers and back again without losing information.
The reason I want to turn the data into numbers is because I use the HashIds library, which only encodes numbers. To my knowledge it's an extra layer of security I'm happy to add.
Read more on HashIds here: http://hashids.org/
What I have tried:
I tried converting the string into hex numbers, and then the hex numbers into decimals. Unfortunately the number was too large, and i haven't had any luck using biginteger with it.
base64_encode() which is not going to turn the data into numbers, but then base_converting them is. But I couldn't figure out the base converting in php since apparently it's rather odd.
Conclusion:
How can I convert the data I'm receiving from a form request into a short encoded string which can be converted back into the data without too much hassle? I don't quite know all the options PHP offers yet.
UPDATE:
To conclude this thread, I ended up using OpenSSL to encrypt my serialized array. Only problem I ran into was if the request contained a file I wouldn't be able to serialize it and save the object to the database. I do still need a way around this, since the third party application expects the file to be a multipart/formdata object i can't just save the filepath to the database and upload that. But I guess I will have to figure out that one later.
That link http://hashids.org/ provides a pretty clear example. Lets assume that your integer is 15.
$hashids = new Hashids\Hashids('some random string for a salt. Make sure you use the same salt if you want to be able to decode');
$encoded = $hashids->encode(15);
print_r(['hashedId' => $encoded]);
$decoded = $hashids->decode($hashed);
print_r(['decoded' => $decoded]);
So the value of $decoded should equal 15
Update
Sorry - the hashids bit of your question threw me and as such, I misunderstood what you were asking. I will update my answer:
You should really be using https://secure.php.net/openssl_encrypt and https://secure.php.net/manual/en/function.openssl-decrypt.php
I need to store:
array(1,2,3,4,5);
into a mysql blob.
How can i convert this array into binary data?
It depends mostly on how you are using those informations. IDs are usually used to identify a resource, and thus must be unique, not null and indexable.
By those standarts do not use as blob.
Mostly because search by content is slower than as native variable. Also, SQL databases sort the content of a table to ensure faster queries.
If what you need is just storing information and then using another ID to identify this resource (and they can be easily parsed to strings/numbers then do not use blob). A binary file will usually use 8 bytes per char. A number could contain the same information using less total memory. Example, 1902334123 (random keyboard smash) uses 10*8 = 80 bytes in Hard disk, while an 32-bit signed integer could hold it.
Finally, if what you need is just storing several data units, what is your problem with a sequential varchar to be read as string, as it could solve your problem
you can convert to JSON and store to db:
json_encode($array);
and when you return from db:
json_decode($array);
I'm trying to use mcrypt_create_iv to generate random salts. When I test to see if the salt is generated by echo'ing it out, it checks out but it isn't the required length which I pass as a parameter to it (32), instead its less than that.
When I store it in my database table however, it shows up as something like this K??5P?M???4?o???"?0??
I'm sure it's something to do with the database, but I tried to change the collation of it to correspond with the config settings of CI, which is utf8_general_ci, but it doesn't solve the problem, instead it generates a much smaller salt.
Does anyone know of what may be wrong? Thanks for any feedback/help
The function mcrypt_create_iv() will return a binary string, containing \0 and other unreadable characters. Depending on how you want to use the salts, you first have to encode those byte strings, to an accepted alphabet. It is also possible to store binary strings in the database, but of course you will have a problem to display them.
Since salts are normally used for password storing, i would recommend to have a look at PHP's function password_hash(), it will generate a salt automatically and includes it in the resulting hash-value, so you don't need a separate database field for the salt.
I have a MySQL database I am working on in PHP where It will perform address verification from a daily data feed. We would do address correction on our end, because we don't have control over the source of the feed.
I am trying to come up with a method to see if the address has been changed at the source. If it changes then an address verification would be performed in PHP on our MySQL database.
Without storing a copy of the old feed I was thinking it might be better to do a checksum of the fields from the feeds and store this with each record. Then each feed after that it would see if the checksum has changed. Is this the best method to do this? Might there been a PHP function to do all this already? What about something in MySQL? Thanks!
crc32 is probably what you want.
In php: crc32()
In Mysql CRC32()
crc32 is probably a better fit that SHA1 or MD5 for simple comparisons/data integrity:
see here
PHP and MySQL both support the crc32 function which is inexpensive to run; at least less so than a hash algorithm like MD or SHA.
There are various hash methods you can use, either the md5 or sha ones will be ok, you will need to store in your database the hash string to compare to,
Idealy you'd want to do something like
if (sha1(strtoupper($list_of_values) )=== $stored_hashstring){
//skip
}else{
//update
}
Depending of the data you might need to add additional parsing on the strings ie: removing spaces, etc