I am storing serialized data in a mysql and am unsure which field type to choose?
One example of the serialized data output is below,
string(393) "a:3:{s:4:"name";s:22:"PACMAN-Appstap.net.rar";s:8:"trackers";a:6:{i:0;s:30:"http://tracker.ccc.de/announce";i:1;s:42:"http://tracker.openbittorrent.com/announce";i:2;s:36:"http://tracker.publicbt.com/announce";i:3;s:23:"udp://tracker.ccc.se:80";i:4;s:35:"udp://tracker.openbittorrent.com:80";i:5;s:29:"udp://tracker.publicbt.com:80";}s:5:"files";a:1:{s:22:"PACMAN-Appstap.net.rar";i:4147632;}}"
The string lengths of the data can vary greatly upto around 20,000 characters.
I understand that I do not want to use TEXT data type as this could corrupt data because of character sets that it would have to use.
I am stuck as when it comes to use either VARBINARY, BLOB, MEDIUMBLOB etc.
Let us say if I use VARBINARY(20000) does this mean that I can insert a string of 20000 in length safely and if it is over then discard the insert?
I agree with PLB in that you should use BLOB. The length attribute specifies how many bytes can be saved in this column. The main difference between BLOB and VARBINARY is that VARBINARY fills up unused space with padding, wheras with BLOB only the actual length of the data is reserved for one field.
But as PLB said, only use this if you absolutely must, because it slows down the whole DB in most cases. A better solution would be to store the files in your server's filesystem and save the file's path in the DB.
Related
I need to store:
array(1,2,3,4,5);
into a mysql blob.
How can i convert this array into binary data?
It depends mostly on how you are using those informations. IDs are usually used to identify a resource, and thus must be unique, not null and indexable.
By those standarts do not use as blob.
Mostly because search by content is slower than as native variable. Also, SQL databases sort the content of a table to ensure faster queries.
If what you need is just storing information and then using another ID to identify this resource (and they can be easily parsed to strings/numbers then do not use blob). A binary file will usually use 8 bytes per char. A number could contain the same information using less total memory. Example, 1902334123 (random keyboard smash) uses 10*8 = 80 bytes in Hard disk, while an 32-bit signed integer could hold it.
Finally, if what you need is just storing several data units, what is your problem with a sequential varchar to be read as string, as it could solve your problem
you can convert to JSON and store to db:
json_encode($array);
and when you return from db:
json_decode($array);
I am serializing alot of arrays in php that are to be stored in a database using mysql.
The length of the final string can vary greatly from anything inbetween 2000 to 100,000+, I was wondering what would the best column type for this to be?
I currently have it set as LONGTEXT but I feel this is overkill! The database is already active and has around 3million rows this is a new column which will added soon.
Thanks
Always use any BLOB data-type for serializing data so that it does not get cut off and break the serialization in a binary safe manner. If there is not a maximum to the length of the final string then you will need LONGBLOB. If you know that the data won't fill 2^24 characters you could use a MEDIUMBLOB. MEDIUMBLOB is about 16MB while LONGBLOB is about 4GB so I would say you're pretty safe with MEDIUMBLOB.
Why a binary data type? Text data types in MySQL have an encoding. Character encoding will have an effect on how the serialized data is transposed between the different encodings. E.g. when stored as Latin-1 but then read out as UTF-8 (for example because of the database driver connection encoding setting), the serialized data can be broken because binary offsets did shift however the serialized data was not encoded for such shifts. PHP's serialized strings are binary data, not with any specific encoding.
You should choose BLOB (as Marc B noted) per the PHP manual for serialize():
"Note that this [outputs] a binary string which may include null bytes, and needs to be stored and handled as such. For example, serialize() output should generally be stored in a BLOB field in a database, rather than a CHAR or TEXT field."
Source: http://php.net/serialize
Of course J.Money's input regarding sizes must be borne in mind as well - even BLOB has its limits, and if you are going to exceed them then you would need MEDIUMBLOB or LONGBLOB.
I'm referencing this Store GZIP:ed text in mysql?.
I want to store serialized sessions in the database (they are actually stored in a memcached pool but i have this as a failsafe). I am gziping/uncompressing from php.
I want to ask the following:
1) Is this a good move? I am doing this to avoid using mediumtext as the data may be bigger than text. I think/hope i will have a lot of sessions stored there. Is it, in this case, worth to gzip? Table is MyISAM.
2) Do i need to set the encoding of the table field to binary? Or only do that if i have a complete gziped file?
3) Is serializing a bad move, should i use json_encode instead (because of the smaller size i guess)?
Thanks,
You should use a MEDIUMBLOB field instead of MEDIUMTEXT. BLOBs have no encoding, as they are raw byte streams.
I have an Oracle SQL database with a CLOB column that holds a great deal of data. I'm running into the issue that a regular PHP string variable will not hold all of the data that I have in the CLOB column. It will only read in 4618 bits, but my file is much larger. The CLOB column has a series of IP addresses in it. What I need to do is parse the CLOB column so I can extract those IP addresses; however, the string variable won't hold enough data to even get to the portion of the document where the IP addresses are stored. Any thoughts?
Based on your comments:
If you right click in your browser and say "view source" you'll see that you indeed have the entire file.
Most likely you just need to set the appropriate ContentType when emitting the data back to the browser.
First, try increasing the PHP memory limit to see if that has any effect on the error. If the CLOB object being retrieved is truly huge, like 2 GiB, then there is no way (currently) to process it in PHP since the whole object must be held in memory.
If the column has fixed length data, it should be fairly easy to compose an appropriate query which extracts a portion of it. Assuming the data is encoded as fixed-length text, then something like this will work:
// assumes subfield is xxx.xxx.xxx.xxx: 19 fixed characters
SELECT substr (TO_CHAR(gnarly_ipadddress_fieldname), 1, 19) as ipaddr1,
substr (TO_CHAR(gnarly_ipadddress_fieldname), 20, 19) as ipaddr2,
substr (TO_CHAR(gnarly_ipadddress_fieldname), 39, 19) as ipaddr3
FROM table
WHERE (whatever);
If the subfields are variable length, then maybe grabbing something like 16K chunks and parsing it in PHP is the way to go.
So I have been browsing the internet, and came across the MySQL built-in function AES_ENCRYPT. It doesn't seem too hard to use, but some sources tell me to store the encrypted data as a VARCHAR, and some say to store it as a BLOB. What should I store the encrypted data as?
Many encryption and compression functions return strings for which the result might contain arbitrary byte values. If you want to store these results, use a column with a VARBINARY or BLOB binary string data type. This will avoid potential problems with trailing space removal or character set conversion that would change data values, such as may occur if you use a nonbinary string data type (CHAR, VARCHAR, TEXT).
Source: http://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html
If you need to use VARCHAR, rather than BLOB, then convert the encrypted binary to Base64 which only uses printable characters and can be safely stored as VARCHAR. Of course you will need to convert it back from Base64 to binary before decrypting.
I have always used blobs to stored encrypted data in MySQL.
You can use Binary. BINARY in STRING. It have to work. I am using it. Write me answer if it doesn't working.