I'm doing an API call which is being outputted in JSON,
The product field is "ProductID":3468490060026049912
I convert to PHP, json_decode()
Then I output the "Product ID" = float(3.4684900600261E+18)
It gets changed to a float which is rounded, I input this figure into MYSQL and it stays as the rounded figure.
How do I convert from JSON to PHP without it rounding, I need it correct to 19 digits?
You can use the JSON_BIGINT_AS_STRING flag int he $options parameter of json_decode. You will have a string, though.
You probably don't need to store these IDs as integers; it's not like you're going to do any maths on them.
Store them as strings, and you won't have any issues with precision, no matter how many digits they are.
The only reason you'd need to store these as integers is if you're using them as your primary ID field in the database and its doing auto-increment for new records.
This is also the correct way to handle storage for phone numbers, zip codes, and other data that is formatted as a number but is actually just an arbitrary sequence.
The value is too big to be stored as an integer on 32 bit machines (the maximum is about 2*109), that's why PHP converts it to float.
On 64 bit machines PHP can be stored as integer but its value is really close to the limit (which is about 9*1018). Another ID that starts with 9 or is one digit longer doesn't fit and is converted by PHP to float.
But floating point numbers loose the least significant digits.
As a quick workaround you can pass JSON_BIGINT_AS_STRING as the fourth parameter to json_decode() but it doesn't fixes the source of the problem. You can encounter the same problem on the Javascript or the database code.
You better make "ProductID" a string everywhere. It is an identifier, anyway. You don't need to do math operations with it, just to store it and search for it.
Fixed with setting ini_set('precision',19); at the top.
Related
I'm writing a PHP script that looks through a DB table of float values, that tend to be somewhat small, such as:
0.00052
0.00134
0.00103
0.00149
0.00085
0.00068
0.00077
0.00088
0.00169
0.00063
For reasons unknown to me, some values appear in the DB in scientific notation, such as:
1.12305e-06
The table is set to float, and I've tried all manner of functions in PHP to force the numbers to display as decimal, to no avail. Try as I might, I'm unable to get this table of numbers to be consistently decimal in all cases.
Any suggestions on how to resolve this? Have tried typcasting to (float) and using number_format() and several other options, but no change every time.
There seems to be a six digit limit on what is shown out of the CLI (and probably elsewhere). The example you have is 1.12305e-06 which is 0.00000112305 which would be shown as 0.00000 - though clearly it isn't zero.
If you are insisting on using floats or doubles, you will have to force them out using something like round(columnName,5) to force the display in a decimal value. Otherwise, maybe switch to a decimnal data type.
From http://dev.mysql.com/doc/refman/5.0/en/floating-point-types.html
Because floating-point values are approximate and not stored as exact values, attempts to treat them as exact in comparisons may lead to problems. They are also subject to platform or implementation dependencies. For more information, see Section C.5.5.8, “Problems with Floating-Point Values”
Also see this thread on the mysql forums about this exact issue.
The solution in my case turned out to be changing from float to decimal type in the database, so thanks to Romain for the comment that led me to look into that solution!
As commonly discussed, (for example here Storing 0.00001 in MySQL ) the DECIMAL data-type should be used for fields where precision / correctness is required, such as an account balance.
I was wondering however, how PHP handles these values and, if they are internally handled as floats, if there is still a problem when reading these values from the database, doing some calculations and writing them back again. If so, how can we force PHP to keep precision in tact?
The variable is probably a string initially in PHP (when read from the MySQL result object). In general, PHP's floating-point datatype cannot be relied upon to keep the precise decimal value required. You should use an arbitrary-precision mathematics library like GMP. (When you fetch a row of the result object, pass the DECIMAL column value in to the appropriate constructor, and then operate on it using the functions provided by the library you are using.)
To go more into depth: Suppose you have an amount stored in the database, in a DECIMAL(6, 4) column. You want to fetch that into PHP, to do something with it. You issue a query to fetch that column. You fetch the first row of the query into an associative array. Suppose that the value from that row is 2.5674. Your array is now something like array('MyDecimal' => '2.5674') (the number appears as a string). You use (as far as I can tell) gmp_init() to convert that string to a GMP resource. Now you can do mathematics with that number using the other GMP functions. If you want to store a GMP number, you could convert it back to string using gmp_strval() (perhaps you do not have to do this if you are using a database abstraction layer that can handle GMP resources).
You could try using the arbitrary precision features:
http://php.net/manual/en/book.bc.php
Your other option is to store the values as INTs and then convert them when they need to be displayed (i.e. divide by one hundred).
Is there a way to explicitly store my numbers in php as tinyint (1 byte instead of 4).
Or could i only enforce this by storing them 4 by 4 in an int? (using a few binary operations)
I generate these values by breaking a string using str_split and interpretting these bytes as ints via unpack( 'C' , .. ).
Currently i store these values in an array as invdividual integers but it could save alot of space if i could store them somehow as tinyints.
PHP has two data types that you may want to use here: integer and string.
PHP doesn't have any other types you could choose from (float wouldn't be a good choice for integers, the other types are not appropriate).
An int is usually 32 or 64 bits, a string is 1 byte per character.* I propose that unless you have a lot of numbers, you won't ever see any problem with 32 bit ints. If you absolutely positively want to safe space memory** and your numbers have a maximum of 3 digits, you could handle your numbers as strings. There's even the BCMath extension that'll let you operate on string numbers directly without needing to cast them back and forth. It's quite a lot of hassle for possibly very limited gain though.
Seeing that a MySQL TINYINT is usually used for boolean values though, please be aware PHP does have a boolean type...!
* One byte per one-byte character, that is.
** Since PHP scripts are usually only very temporary, you should only have problems with peak memory usage, not storage space. Getting more RAM may be the more efficient solution than playing with types.
My PHP program is working with an array of values ranging from 0 to 7. I'm trying to find the most effective way to store those values in PHP. By most effective I mean using the less number of bits.
It's clear that each value only need 3 bits of storage space (b000=0 to b111=7). But what is the most efficient way to store those 3bits values in a binary string ?
I don't know in advance how many 3 bits values I'll need to store or restore, but it might be a lot, so 64bits is clearly not enough.
I was looking into pack() and unpack(): I could store two values in each byte and use a pack('C', $twoValues), but I'm still loosing 2 bits.
Will it work ? Is there a more effective way of storing those values ?
Thanks
You didn't ask if it was a good idea - as many suggested, your benefit of that kind of space compression, is easily lost in the extra processing - but that's another topic :)
You're also not mentioning where you're storing the data after. Whatever that storage location/engine is maybe have further conditions and specialized types (eg a database has a binary column format, might have a byte column format, may even support bit storage etc).
But sticking with the topic, I guess best 3 bit storage is as a nibble (waisting one bit) and I suppose I'd combine two nibbles into a byte (loosing two bits overall). Yes you're loosing two bits (if that's key), but it's simple to combine the two values so you're processing overhead is relatively small:
$byte=$val1*7+$val2;
$val2=$byte%7;$val1=($byte-$val2)/7;
If a byte isn't available, you can combine these up to make 16 (4 stored), 32 (8), 64 (16) bit integers. You can also form an array of these values for larger storage.
I'd consider the above more human readable, but you could also use bit-logic to combine and separate the values:
$combinedbyte=$val1<<3|$val2;
$val2=$combinedbyte&7;$val1=($combinedbyte&56)>>3);
(This is effectively what the PACK/UNPACK commands do)
Alternatively you could encode into characters, since in ASCII the first few are protected, you might as well start at A (A-Z+6 punc+a-z gives you 58 when you only need 49 to store your two values).
$char=chr(($val1*7+$val2)+65); //ord('A')=65
$val2=(ord($char)-65)%7;$val1=(ord($char)-65-$val2)/7;
A series of these encoded characters could be stored as an array or in a null terminated string.
NOTE:
In the case of -say- 64 bit integers above, we're storing 3 bits in 4 so get 64/4=16 storage locations. This means we're waisting 16 further bits (1 per location) so you might be tempted to add another 5 values, for a total of 21 (21*3=63 bits, only 1 wasted). That's certainly possible (with integer math - although most PHP instances don't work # 64 bits, or bit-logic solutions) but it complicates things in the long run - probably more trouble than it's worth.
The best way is to store them as integers and not get involved with packing things bit by bit. Unless you have an actual engineering reason you need these to be stored as 3-bit values (for example, interfacing with hardware), you're just asking for headaches. Keep in mind, esp for odd bit sizes, they become pretty difficult to have direct access to if you do this. And if you are sticking these values in a database, you wouldnt be able to search or index on values packed like this. Store them as integers, or if in a db, perhaps a short integer or byte.
That kind of technique is only necessary if you will have at least half a billion of these. Think about it, the CPU will have to have data in one register, the mask in another and AND them just to get your value out. Now imagine iterating over a list of these that is long enough to justify that kind of space saving technique. A 50% reduction in space and an order of magnitude slower.
Looking at http://php.net/manual/en/language.types.php, you should store them as integers. However, the question is whether to let one integer value represent many 3-bit values or not. The former is more complex but requires less memory, whereas the first is the opposite. If you don't have an extreme need to reduce the amount of memory you use, then I would suggest the latter (use one integer for one 3-bit value).
The main problem with storing many 3-bit values in one integer is figuring out how many 3-bit values there are. You could use an array of integers, and then have an extra integer which states the total number of 3-bit values. However, as also stated in the manual, the number of bits used for an integer value is platform-dependent. So you would have to know whether an integer is 32 bits or 64 bits, or else you may try to store too many values and lose data, or you risk using more memory than needed (which would be a bad thing as you're aiming to use as little memory in the first place).
I would convert each integer to binary, concatenate all of them, and then split the resulting string into bytes. Each byte will be 0-255 so it can be stored as an individual character.
I'm not sure what this is called, which is why I'm having trouble searching for it.
What I'm looking to do is to take numbers and convert them to some alphanumeric base so that the number, say 5000, wouldn't read as '5000' but as 'G4u', or something like that. The idea is to save space and also not make it obvious how many records there are in a given system. I'm using php, so if there is something like this built into php even better, but even a name for this method would be helpful at this point.
Again, sorry for not being able to be more clear, I'm just not sure what this is called.
You want to change the base of the number to something other than base 10 (I think you want base 36 as it uses the entire alphabet and numbers 0 - 9).
The inbuilt base_convert function may help, although it does have the limitation it can only convert between bases 2 and 36
$number = '5000';
echo base_convert($number, 10, 36); //3uw
Funnily enough, I asked the exact opposite question yesterday.
The first thing that comes to mind is converting your decimal number into hexadecimal. 5000 would turn into 1388, 10000 into 2710. Will save a few bytes here and there.
You could also use a higher base that utilizes the full alphabet (0-Z instead of 0-F) or even the full 256 ASCII characters. As #Yacoby points out, you can use base_convert() for that.
As I said in the comment, keep in mind that this is not an efficient way to mask IDs. If you have a security problem when people can guess the next or previous ID to a record, this is very poor protection.
dechex will convert a number to hex for you. It won't obfuscate how many records are in a given system, however. I don't think it will make it any more efficient to store or save space, either.
You'd probably want to use a 2 way crypt function if obfuscation is needed. That won't save space, either.
Please state your goals more clearly and give more background, because this seems a bit pointless as it is.
This might confuse more people than simply converting the base of the numbers ...
Try using signed digits to represent your numbers. For example, instead of using digits 0..9 for decimal numbers, use digits -5..5. This Wikipedia article gives an example for the binary representation of numbers, but the approach can be used for any numeric base.
Using this together with, say, base-36 arithmetic might satisfy you.
EDIT: This answer is not really a solution to the question, so ignore it unless you are trying to hash a number.
My first thought we be to hash it using eg. md5 or sha1. (You'd probably not save any space though...)
To prevent people from using rainbow-tables or brute force to guess which number you hashed, you can always add a salt. It can be as simple as a string prepended to your number before hashing it.
md5 would return an alphanumeric string of exactly 32 chars and sha1 would return one of exaclty 40 chars.