as usual, did my duty to look everywhere for the sol but to no avail.
mysqli_fetch_assoc is (apparently) storing my numbers as strings.
Normally, I could care less, but my site is nearly 100% ajax, and it moves around a lot of data, so all of those json "'s start to add up.
If I'm just grabbing one column value, I can intval, but I want to grab whole rows with associated column names.
Is there a way to get mysqli_fetch_assoc (and PHP PDO's fetch assoc) to store numbers as numbers?
Many thanks in advance!
MySQL can store 64-bit numbers and unsigned numbers, neither of which are supported by default in PHP's int type. If you try to fetch numeric data from the database that overflows the int variables in PHP, you'll lose information. That's why PHP database clients fetch numbers as strings.
Re your comment:
Yes, if you're concerned about the size of JSON, convert the data in your PHP app for more compact representation.
Though since you said you're transferring column names as part of your JSON, it seems like the difference between string-formatted ints and binary ints is not your bottleneck anyway.
I mean, converting a string-int to a binary int reduces the bits by about 50%. But if you have column names as strings anyway, you're only shrinking a small portion of the JSON. The columns are still strings.
Related
I have copied this table from Wikipedia into a PostgreSQL database. The column Cultivated land (km2) became a column of type real. Then I use the PHP command
echo rtrim(rtrim(sprintf('%.10F',$v),'0'),'.');
to display the numbers ($v) in a table (both integers and float), but some values lose precision. For instance, the value from United States, 1669302, becomes 1669300, what is strange, since I expected 10 decimal digits of precision. I thought I have lost the precision when saving into a real column, but converting the column to double precision makes the difference (02) appear again, so it was there somewhere.
I don't think I need double precision, so how can I display the real value correctly? Keep in mind that some columns have decimal places, while some others are bigint, and they also should be displayed correctly.
The problem seems to originate from the way PHP returns results. The values are not returned as the corresponding data type, but rather formatted as a string using PostgreSQL default formatting. This formatting, is different for real and double precision types hence you are seeing different results when you convert the column types of your table. The reason you are seeing this specific result is that PostgreSQL guarantees 6 decimal places for real types and 15 decimal places for double precision.
Setting extra_float_digits
The manual states
Note: The extra_float_digits setting controls the number of extra significant digits included when a floating point value is converted to text for output. With the default value of 0, the output is the same on every platform supported by PostgreSQL. Increasing it will produce output that more accurately represents the stored value, but may be unportable.
Therefore, a simple solution to your problem is to increase extra_float_digits before issuing your SELECT-query:
pg_query($connection, "set extra_float_digits = 3");
Alternatively, you can also specify this change when connecting to your database by adding options to your connection string as follows:
$connection = pg_connect("host=localhost port=5432 dbname=test user=php password=pass connect_timeout=5 options='-c extra_float_digits=3'");
Another option would be to set this flag in the postgresql.conf configuration file of the PostgreSQL server if you have access to the server and want to change the option globally.
Casting the values
A different solution would be to have PostgreSQL return a different string to the PHP backend. This can be achieved by casting your columns to types with different default formatting which avoids cutting off some of the digits. In your case you could either cast to integer or double precision, i.e. instead of using
select cultivated_land from table
you could use
select cultivated_land::integer from table
or
select cultivated_land::double precision from table
Changing data types
Looking at the data you specified, I noticed that all numerical values except those columns specifying percentages contain integers, hence the usage of the integer data type is more suitable in this case. It can fit all the integer values of this table (the maximum being 149,000,000, therefore bigint is not required), requires the same storage size as real (4 bytes) and implies the default formatting of integers that you are looking for.
Update: Background on PostgreSQL-PHP interface and floating point representation
As mentioned above the way the PostgreSQL-PHP interface works is that all values sent from PostgreSQL to PHP are formatted as a string in some type-dependent way. Neither any of the pg_fetch_* functions nor pg_copy_to will provide raw values and all of these functions convert the values to strings in the same manner. As far as I am aware the current PHP interface will not provide you with anything different from a string (which, in my opinion, is not the best interface design).
The reason 18.22 is returned as 18.2199993 can be found in how PostgreSQL converts float4 to strings. You can check the code of how PostgreSQL is internally using float4out and find this relevant line that does the string-conversion:
snprintf(ascii, MAXFLOATWIDTH + 1, "%.*g", ndig, num);
num is the float4-number to be printed as a string. Note however that C will promote the float-variable to a double-variable when calling snprintf. This conversion to double precision results in the value 18.219999313354492 which is why you end up seeing 18.2199993 (you can check this here and will also find some details on floating point number representation on this site).
The takeaway message is that all your float4 values will be converted using this function and the only parameter you can influence is ndig by varying extra_float_digits, however no single value for this variable will suffice all your needs in representing the values as you want them. So as long as you keep using float4 as your data type and use the current PHP-interface to obtain the data you will run into these problems.
I therefore still recommend choosing different data types for your columns. If you think you have a requirement for decimal numbers you might want to investigate decimal data types where you can specify precision and scale as required for your application. If you would like to stick with floating point numbers I suggest rounding the values in PHP before displaying them to the user.
I have two choices of storing date and time in my database.
Generate the time & date from time function in php and then storing in database into int datatype which is of 4 bytes.
Generate the time & date during insertion in database into datetime datatype which is of 8 bytes.
My question is which type will make my SQL queries faster if I use date&time column for sorting.
I always hate it building queries on a DB that contains human unreadable date and time values in int format.
Maybe the query will be a nano second faster if you use int but is it really worth it? I say no!
Use a TIMESTAMP datatype. It's stored as a number, but returned formatted. So it's faster for sorting, and more human-readable.
You are better off using the native format of the database to store date times.
I can see almost no occasion where you would want to use another binary format for this purposes. That would make that component of the database essentially inoperable for other access methods.
As for human readability, you can solve that issue by having views access the tables (in other databases you can define a table with computed columns) that provide the dates in human readable format. Also, tools that know the database should produce the output in human readable format.
Unless you have a very large database or are in a very tightly constrained environment, then don't worry about storage. You are probably not thinking about the extra bits that are stored when you allow NULLs for a column, or the extra padding that might go between fields when they are no aligned on hardware word boundaries, or the empty space on data pages because the records don't align on page boundaries.
If space is such an important consideration, then you might want to develop your own date/time format to see if you can get it down to 2 or 3 bytes.
I am in need of storing a score in a mysql field but this score could be a time,integer or float. My first way of doing this was to identify the score type and then enter it into one of three fields but if the need arises to add a new score type I dont want to have to continually add a field to the database. I remember somewhere down the line someone told me that if you store somethign as a varchar then is wont be able to be read as an integer or float or even date. My question is, can I store all three of those as one specific type but read it any way I need when taking it from the database and using it in my php code?
In my opinion you could model the field as FLOAT except if you absolutely need to know about the type of variable stored. Time can be converted to an integer value by converting to timestamp. Integers are a subset of the real (floating point) numbers set actually so I guess that way you have everything covered. Floating point arithmetic can cause some issues with precision and equality testing though so be careful!
You can use CAST and CONVERT functions to convert the string datatype into another MySQL datatype such as INT, FLOAT, DECIMAL, DATE, DATETIME etc.
There are a few issues. How do you know what datatype is stored in a row? MySQL does have RegExp support but I do not recommend using it in your WHERE clauses. Store the datatype in another column.
Also, using cast functions in the where clause of your query will make them run slow. If you need to search upon/sort by the data you should use proper datatypes. Perhaps you should add one column for each datatype; and for each row, populate only the corresponding column.
mysql will happily convert text to numbers of the appropriate type if you perform a mathematical operation on it. However, it will also convert non-numeric text to zero and perform the same operation on it, so you need to check that you're only pulling fields of the appropriate type beforehand.
This forum post shows how to add a regular expression condition to your query to ensure that you're only pulling fields with numeric data in them. However, I think it's probably wiser to use a separate column to indicate what type of score each record is, and use that to retrieve the appropriate ones.
I don't know how to convert text to a date (putting it through date() doesn't work). However, note that the mysql date format (2012-05-08 11:20:23) has the date elements in order of descending significance. If you just want to find the highest / lowest date, or sort by date, treating them as strings will work fine.
Can we declare a variable with fixed length in PHP?
I'm not asking about trimming or by putting condition do substring.
Can we declare variable just like database char(10).
The reason I'm asking am doing an export process, PHP export data to DB.
In DB I have a field with size 100, and I'm passing a field with length 25, using PHP.
When I look in DB, it's showing some extra space for that field.
Maybe it's your database that is the problem.
The CHAR datatype will always fill up the remaining unused characters when storing data. If you have CHAR(3) and pass 'hi', it will store it as 'hi '. This is true for a lot of relational database engines (MySQL, Postgres, SQLite, etc.).
This is why some database engines also have the VARCHAR datatype (which is variable, like the name says). This one doesn't pad the content with spaces if the data stored in isn't long enough.
In most cases, you are looking for the VARCHAR datatype. CHAR is mostly useful when you store codes, etc. that always have the same length (e.g.: a CHAR(3) field for storing codes like ADD, DEL, CHG, FIX, etc.).
No, a string in PHP is always variable length. You could trim the string to see if extra space is still passed to your DB.
Nope. PHP has no provision to limit string size.
You could simulate something in an object using setter and getter variables, though, throwing an error (or cutting off the data) if the incoming value is larger than allowed.
No, but I really don't think you're having a problem with php. I think you should check your DB2 configuration, perhaps it automatically completes strings with spaces... How much spaces are added? Are they added before? After?
As others have said: No.
I don't understand how it would help anyway. I'm not familiar with DB2 but it sounds like if you have extra spaces, they are either coming in the variable (and thus it should be trimmed) or DB2 does space padding to make the value have 100 characters. If your input is only 25 characters long then if it is doing space padding, it seems it would do it anyway.
If you want to store variable length strings in DB2 then go with VARCHAR, if you always want the same length for each string in the column, define the exact length using CHAR (for postal codes, for instance).
Details on character strings is available here: http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/topic/com.ibm.db2.luw.sql.ref.doc/doc/r0008470.html with a good summary:
Fixed-length character string (CHAR)
All values in a fixed-length string column have the same length, which is determined by the length attribute of the column. The length attribute must be between 1 and 254, inclusive.
Varying-length character strings
There are two types of varying-length character strings:
A VARCHAR value can be up to 32,672 bytes long.
A CLOB (character large object) value can be up to 2 gigabytes minus 1 byte (2,147,483,647 bytes) long.
Of course it then gets more detailed, depending on what sort of encoding you're using, etc... ( like UTF-16 or UTF-32 )
I try to store the PHP floating point value 63.59072952118762 into a double precision column in postgres. Postgres stores the value as 63.59073. Does anyone know why? 8 byte should be more than enough for that value. I've tried with the data type numeric, which works when specifying the precision, but that shouldn't really be necessary.
Update: The same problem is present when trying to store 63.5907295, so the suggestion that something happens with the double before it's getting stored seems realistic.
Update II (partly solved): The line where I assign the double parameter looks like this:
$stmt->bindParam(4, $this->latitude);
The thing I didn't know is that PDO defaults its param type to string. I changed it to PDO::PARAM INT in lack of a better alternative (PARAM DOUBLE was not an option), and got 10 digits precision in the double stored in postgres (some progress, at least). I've checked that the numeric type works well, so it seems that numeric is the way to go when using PDO and doubles that has to have a precision of more than 10 decimals.
Anyways, as someone has mentioned, I don't know if it's a must for me to have this kind of precision, but I think the problem in itself deserved to be investigated.
How do you determine what PostgreSQL is storing?
How do you send the data to PostgreSQL?
How do you get the data back again?
How do you display it?
What type is the column in the database?
There are many, many places on the path between PHP and PostgreSQL where there could be confusion about how to represent the data.
It is important to explain how data is inserted into the DBMS. Using a literal value in the INSERT statement leads to a different set of problems from using bound parameters. If you wrote the value out in the SQL:
INSERT INTO SomeTable(SomeColumn) VALUES(63.xxxxxxxxx);
and the data was truncated, you'd have a problem down in PostgreSQL. If you bind the variable, you have to be sure to understand what PHP and the PDO PostgresSQL modules do with the value - is it sent as a double, or as a string, and which code deals with the conversion, and so on.
You run into analogous issues with Perl + DBI + DBD::YourDBMS (DBD::Pg in your case).
Consider using the DECIMAL/NUMERIC type if you need that much precision
PostgreSQL accepts float(1) to float(24) as selecting the real type, while float(25) to float(53) select double precision.
On most platforms PG, the real type has a range of at least 1E-37 to 1E+37 with a precision of at least 6 decimal digits. The double precision type typically has a range of around 1E-307 to 1E+308 with a precision of at least 15 digits (REF)
Which one do you use?