After switching to Eloquent, when querying our database the values are automatically cast to numeric data types. This results in strange values like 8.0000000000000002E-2 instead of 0.08. This causes all sorts of issues with our reporting system. Is there a flag that can be set to retrieve the values as strings?
We've tried using casts in the Eloquent Model but that seems to happen after the fact. Adding rounding on the web side would be a very significant effort.
Example Query via DB::select($query);
SELECT (CAST(0.08 as float)) as ex1,
(CAST(0.04 as float)) as item2
The casts are to imply the data is stored as float in the database. These are custom queries and therefore are not using querybuilder on them.
Results in:
8.0000000000000002E-2 4.0000000000000001E-2
If you were to run this query directly in MSSQL for example it would show 0.08 and 0.04 which is what our webserver also used to report. I get that floating point numbers have finite precision but we are looking for a solution to mirror the previous behavior.
Try casting to decimal instead of float:
SELECT (CAST(0.08 as decimal)) as ex1,
(CAST(0.04 as decimal)) as item2
You may want to change your column types to be decimal too, especially if you're dealing with currency values.
More about the DECIMAL type in MySQL
Related
I have copied this table from Wikipedia into a PostgreSQL database. The column Cultivated land (km2) became a column of type real. Then I use the PHP command
echo rtrim(rtrim(sprintf('%.10F',$v),'0'),'.');
to display the numbers ($v) in a table (both integers and float), but some values lose precision. For instance, the value from United States, 1669302, becomes 1669300, what is strange, since I expected 10 decimal digits of precision. I thought I have lost the precision when saving into a real column, but converting the column to double precision makes the difference (02) appear again, so it was there somewhere.
I don't think I need double precision, so how can I display the real value correctly? Keep in mind that some columns have decimal places, while some others are bigint, and they also should be displayed correctly.
The problem seems to originate from the way PHP returns results. The values are not returned as the corresponding data type, but rather formatted as a string using PostgreSQL default formatting. This formatting, is different for real and double precision types hence you are seeing different results when you convert the column types of your table. The reason you are seeing this specific result is that PostgreSQL guarantees 6 decimal places for real types and 15 decimal places for double precision.
Setting extra_float_digits
The manual states
Note: The extra_float_digits setting controls the number of extra significant digits included when a floating point value is converted to text for output. With the default value of 0, the output is the same on every platform supported by PostgreSQL. Increasing it will produce output that more accurately represents the stored value, but may be unportable.
Therefore, a simple solution to your problem is to increase extra_float_digits before issuing your SELECT-query:
pg_query($connection, "set extra_float_digits = 3");
Alternatively, you can also specify this change when connecting to your database by adding options to your connection string as follows:
$connection = pg_connect("host=localhost port=5432 dbname=test user=php password=pass connect_timeout=5 options='-c extra_float_digits=3'");
Another option would be to set this flag in the postgresql.conf configuration file of the PostgreSQL server if you have access to the server and want to change the option globally.
Casting the values
A different solution would be to have PostgreSQL return a different string to the PHP backend. This can be achieved by casting your columns to types with different default formatting which avoids cutting off some of the digits. In your case you could either cast to integer or double precision, i.e. instead of using
select cultivated_land from table
you could use
select cultivated_land::integer from table
or
select cultivated_land::double precision from table
Changing data types
Looking at the data you specified, I noticed that all numerical values except those columns specifying percentages contain integers, hence the usage of the integer data type is more suitable in this case. It can fit all the integer values of this table (the maximum being 149,000,000, therefore bigint is not required), requires the same storage size as real (4 bytes) and implies the default formatting of integers that you are looking for.
Update: Background on PostgreSQL-PHP interface and floating point representation
As mentioned above the way the PostgreSQL-PHP interface works is that all values sent from PostgreSQL to PHP are formatted as a string in some type-dependent way. Neither any of the pg_fetch_* functions nor pg_copy_to will provide raw values and all of these functions convert the values to strings in the same manner. As far as I am aware the current PHP interface will not provide you with anything different from a string (which, in my opinion, is not the best interface design).
The reason 18.22 is returned as 18.2199993 can be found in how PostgreSQL converts float4 to strings. You can check the code of how PostgreSQL is internally using float4out and find this relevant line that does the string-conversion:
snprintf(ascii, MAXFLOATWIDTH + 1, "%.*g", ndig, num);
num is the float4-number to be printed as a string. Note however that C will promote the float-variable to a double-variable when calling snprintf. This conversion to double precision results in the value 18.219999313354492 which is why you end up seeing 18.2199993 (you can check this here and will also find some details on floating point number representation on this site).
The takeaway message is that all your float4 values will be converted using this function and the only parameter you can influence is ndig by varying extra_float_digits, however no single value for this variable will suffice all your needs in representing the values as you want them. So as long as you keep using float4 as your data type and use the current PHP-interface to obtain the data you will run into these problems.
I therefore still recommend choosing different data types for your columns. If you think you have a requirement for decimal numbers you might want to investigate decimal data types where you can specify precision and scale as required for your application. If you would like to stick with floating point numbers I suggest rounding the values in PHP before displaying them to the user.
I have single record on a table. So on MySQL when
select myamount from table 1 -- returns amount 420.67
But when i do MySQL as
select sum(myamount) from table 1 -- returns amount 420.8699951171875
should n't it return same amount 420.67 since I have only one record? and how to get amount 420.67 if SUM used.
Any help is appreciated and yes myamount datatype is float.
Float variables are stored in "scientific notation" (the 2,4E+04 format, which is the same as 2,4*10^4). But to make it even worse, it is also stored in binary. When calculating things with numbers stored as float, you may get a bit strange results because of this.
This video by Computerphile describes the problem very nicely.
I am guessing this has came up before, but I couldn't find the answer to my question. Here is a little code snippet:
$stmt = $this -> db -> query("
SELECT
`Field`
FROM
`Table`
WHERE
(`ID` = 33608)");
var_dump($stmt -> fetch());
And this is the result I get:
array(1) { ["Field"]=> float(1.7999999523163) }
However, the data in the MySQL database is 1.8. The type of the field is float(7,4). $this->db is a PDO object.
I have recently migrated to PDO (from AdoDB), and this code was working fine before. I am not sure what went wrong here. Could you point me in the right direction?
Thanks!
As documented under Floating-Point Types (Approximate Value) - FLOAT, DOUBLE:
MySQL performs rounding when storing values, so if you insert 999.00009 into a FLOAT(7,4) column, the approximate result is 999.0001.
Because floating-point values are approximate and not stored as exact values, attempts to treat them as exact in comparisons may lead to problems. They are also subject to platform or implementation dependencies. For more information, see Section C.5.5.8, “Problems with Floating-Point Values”
For maximum portability, code requiring storage of approximate numeric data values should use FLOAT or DOUBLE PRECISION with no specification of precision or number of digits.
Therefore, upon inserting 1.8 into your database, MySQL rounded the literal to 001.8000 and encoded the closest approximation to that number in binary32 format: i.e. 0x3FE66666, whose bits signify:
Sign : 0b0
Biased exponent: 0b01111111
= 127 (representation includes bias of +127, therefore exp = 0)
Significand : 0b[1.]11001100110011001100110
^ hidden bit, not stored in binary representation
= [1.]7999999523162841796875
This equates to:
(-1)^0 * 1.7999999523162841796875 * 2^0
= 1.7999999523162841796875
This is the value that is returned by MySQL to the client. It would appear that AdoDB then inspected the column's datatype and rounded the result accordingly, whereas PDO does not.
If you want exact values, you should use a fixed point datatype, such as DECIMAL.
You have to use DECIMAL field type instead of FLOAT if you want accurate values.
And PDO has nothing to do with it. It's rather related to how computers works in general.
I have one PHP script inserting rows in a MySQL database. Each row has a field 'created_at' which is filled with the value of the PHP function microtime(true), and inserted as a double. (microtime because I need something more precise than to the second)
I have another PHP script that selects rows based on that created_at field.
When I go ahead and select like this:
SELECT * FROM `ms_voltage` WHERE created_at > 1302775523.51878
I receive a resultset with, as the first row, the row with exactly that value for created_at.
This occurs from within my PHP script and from within PhpMyAdmin when manually doing the query. But not always, not for every value. Just once and a while really.
How is this possible? I didn't ask for greater than/equals, I want strictly greater than.
Am I overlooking something type-related perhaps?
Yeah, floating point arithmetic can do that sometimes. To understand why, it's helpful to realize that just as not all numbers can be accurately represented in base 10, not all numbers can be accurately represented in base 2 either.
For example, "1/3" may be written in base 10 as 0.33333 or 0.33334. Neither is really "correct"; they're just the best we can do. A "DOUBLE" in base 10 might be 0.3333333333 or 0.3333333334, which is double the digits, yet still not "correct".
The best options are to either use a DECIMAL value, or use an INT value (and multiply your actual values by, say, 10000 or 100000 in order to get the decimal digits you care about into that int).
The DOUBLE type represent only approximate numeric data values. Try to use the DECIMAL type.
Is your column floating point? Calling microtime with true gives you a float, and that looks like a float, which will have digits after the .51878 that you don't see, so those digits make the stored value greater than the value you have in your query.
Unless you really need the float I'd convert the string result to an int, or even two columns for seconds and useconds. Then you can use > or < on known values without worrying about the imprecision of the floating point value.
I try to store the PHP floating point value 63.59072952118762 into a double precision column in postgres. Postgres stores the value as 63.59073. Does anyone know why? 8 byte should be more than enough for that value. I've tried with the data type numeric, which works when specifying the precision, but that shouldn't really be necessary.
Update: The same problem is present when trying to store 63.5907295, so the suggestion that something happens with the double before it's getting stored seems realistic.
Update II (partly solved): The line where I assign the double parameter looks like this:
$stmt->bindParam(4, $this->latitude);
The thing I didn't know is that PDO defaults its param type to string. I changed it to PDO::PARAM INT in lack of a better alternative (PARAM DOUBLE was not an option), and got 10 digits precision in the double stored in postgres (some progress, at least). I've checked that the numeric type works well, so it seems that numeric is the way to go when using PDO and doubles that has to have a precision of more than 10 decimals.
Anyways, as someone has mentioned, I don't know if it's a must for me to have this kind of precision, but I think the problem in itself deserved to be investigated.
How do you determine what PostgreSQL is storing?
How do you send the data to PostgreSQL?
How do you get the data back again?
How do you display it?
What type is the column in the database?
There are many, many places on the path between PHP and PostgreSQL where there could be confusion about how to represent the data.
It is important to explain how data is inserted into the DBMS. Using a literal value in the INSERT statement leads to a different set of problems from using bound parameters. If you wrote the value out in the SQL:
INSERT INTO SomeTable(SomeColumn) VALUES(63.xxxxxxxxx);
and the data was truncated, you'd have a problem down in PostgreSQL. If you bind the variable, you have to be sure to understand what PHP and the PDO PostgresSQL modules do with the value - is it sent as a double, or as a string, and which code deals with the conversion, and so on.
You run into analogous issues with Perl + DBI + DBD::YourDBMS (DBD::Pg in your case).
Consider using the DECIMAL/NUMERIC type if you need that much precision
PostgreSQL accepts float(1) to float(24) as selecting the real type, while float(25) to float(53) select double precision.
On most platforms PG, the real type has a range of at least 1E-37 to 1E+37 with a precision of at least 6 decimal digits. The double precision type typically has a range of around 1E-307 to 1E+308 with a precision of at least 15 digits (REF)
Which one do you use?