Related
So I have a PHP form to add a row to a database. If any of the unrequired fields are empty, the default value for that field should be used.
I tried to implement this with MySQLi like this:
$required = $_POST["required"];
$unrequired = isset($_POST["unrequired"])?$_POST["unrequired"]:"DEFAULT(`Unrequired`)";
$sql = $mysqli->prepare("INSERT INTO `table` (`Required`,`Unrequired`) VALUES (?,?)");
$sql->bind_param("is",$required,$unrequired);
$sql->execute();
But when I try to get the value of the unrequired field using SELECT unrequired FROM table WHERE required = 33, I get DEFAULT(`Unrequired`) instead of the default value of the column for varchar columns, and 0 for int and double columns.
Is this problem caused by PHP, or MySQL?
NOTE: Some of the unrequired fields are nullable and some are not. Those which aren't nullable have a set default value. int and double fields' set default value is 1, the rest are nullable.
Can't you just set default value in your database structure. Below you can see an example;
CREATE TABLE example_table
(
P_Id int NOT NULL,
LastName varchar(255) NOT NULL,
FirstName varchar(255) NOT NULL,
Address varchar(255), DEFAULT 'unrequired'
City varchar(255) DEFAULT 'Unrequired'
)
When you create your table like that you are good to go. You can also alter your table
ALTER TABLE example_table
ALTER City SET DEFAULT 'Unrequired'
Alright, I think I figured out the problem.
Since I am using prepared statements, every parameter that I put in the reslting SQL query statement will be converted into safe strings, ints or doubles, so I suppose expressions like DEFAULT(column) will be re-interpreted as "DEFAULT(column)", and will be considered as strings.
Although not a very safe solution, I've considered using not prepared statements, that is concat every parameter and using DEFAULT like in here. To make the queries safer, I'll use real_escape_string() for strings.
For a bit of background, we use Zend Framework 2 and Doctrine at work. Doctrine will always insert NULL for values we do not populate ourselves. Usually this is okay as if the field has a default value, then it SHOULD populate the field with this default value.
For one of our servers running MySQL 5.6.16 a query such as the one below runs and executes fine. Although NULL is being inserted into a field which is not nullable, MySQL populates the field with its default value on insert.
On another of our servers running MySQL 5.6.20, we run the query below and it falls over because it complains that 'field_with_default_value' CANNOT be null.
INSERT INTO table_name(id, field, field_with_default_value)
VALUES(id_value, field_value, NULL);
Doctrine itself does not support passing through "DEFAULT" into the queries it builds so that is not an option. I figure this must be a MySQL server thing of some kind seeing as though it works okay in one version but not another, but unfortunately I have no idea what this could be. Our SQL Mode is also identical on both servers ('NO_AUTO_VALUE_ON_ZERO,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION').
I should probably mention, if I actually run the above SQL in Workbench it still does not work in the same way. So it's not really a Doctrine issue but definitely a MySQL issue of some sort.
Any help on this would be greatly appreciated.
I came across the same problem after a MySQL upgrade. Turns out there is a setting to allow NULL inserts against NOT NULL timestamp fields and get the default value.
explicit_defaults_for_timestamp=0
This is documented at https://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_explicit_defaults_for_timestamp
Based on my research, I would say it could both be a "you" thing and a "MySQL" thing. Check your table definitions with SHOW CREATE TABLE table_name;. Take note of any fields defined with NOT NULL.
The MySQL 5.6 Reference Manual: 13.2.5 INSERT syntax states:
Inserting NULL into a column that has been declared NOT NULL. For
multiple-row INSERT statements or INSERT INTO ... SELECT statements,
the column is set to the implicit default value for the column data
type. This is 0 for numeric types, the empty string ('') for string
types, and the “zero” value for date and time types. INSERT INTO ...
SELECT statements are handled the same way as multiple-row inserts
because the server does not examine the result set from the SELECT to
see whether it returns a single row. (For a single-row INSERT, no
warning occurs when NULL is inserted into a NOT NULL column. Instead,
the statement fails with an error.)
This would imply that it does not matter which SQL mode you are using. If you are doing a single row INSERT (as per your sample code) and inserting a NULL value into a column defined with NOT NULL, it is not supposed to work.
In the same breath, ironically, if you were to simply omit the value from the values list, the MySQL manual says the following, and the SQL mode does matter in this case:
If you are not running in strict SQL mode, any column not explicitly
given a value is set to its default (explicit or implicit) value. For
example, if you specify a column list that does not name all the
columns in the table, unnamed columns are set to their default values.
Default value assignment is described in Section 11.6, “Data Type
Default Values”. See also Section 1.7.3.3, “Constraints on Invalid
Data”.
Thus, you can't win! ;-) Kidding. The thing to do is to accept that NOT NULL on a MySQL table field really means I will not accept a NULL value for a field while performing a single row INSERT, regardless of SQL mode.'
All that being said, the following from the manual is also true:
For data entry into a NOT NULL column that has no explicit DEFAULT
clause, if an INSERT or REPLACE statement includes no value for the
column, or an UPDATE statement sets the column to NULL, MySQL handles
the column according to the SQL mode in effect at the time:
If strict SQL mode is enabled, an error occurs for transactional
tables and the statement is rolled back. For nontransactional tables,
an error occurs, but if this happens for the second or subsequent row
of a multiple-row statement, the preceding rows will have been
inserted.
If strict mode is not enabled, MySQL sets the column to the implicit
default value for the column data type.
So, take heart. Set your defaults in the business logic (objects), and let the data layer take direction from that. Database defaults seem like a good idea, but if they did not exist, would you miss them? If a tree falls in the forest...
According to the documentation, everything works as expected.
Test case:
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.6.16 |
+-----------+
1 row in set (0.00 sec)
mysql> SELECT ##GLOBAL.sql_mode 'sql_mode::GLOBAL',
##SESSION.sql_mode 'sql_mode::SESSION';
+------------------------+------------------------+
| sql_mode::GLOBAL | sql_mode::SESSION |
+------------------------+------------------------+
| NO_ENGINE_SUBSTITUTION | NO_ENGINE_SUBSTITUTION |
+------------------------+------------------------+
1 row in set (0.00 sec)
mysql> SET SESSION sql_mode := 'NO_AUTO_VALUE_ON_ZERO,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT ##GLOBAL.sql_mode 'sql_mode::GLOBAL',
##SESSION.sql_mode 'sql_mode::SESSION';
+------------------------+-----------------------------------------------------------------------------------------------------------------+
| sql_mode::GLOBAL | sql_mode::SESSION |
+------------------------+-----------------------------------------------------------------------------------------------------------------+
| NO_ENGINE_SUBSTITUTION | NO_AUTO_VALUE_ON_ZERO,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+------------------------+-----------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> SHOW CREATE TABLE `table_name`;
+------------+----------------------------------------------------------------------------+
| Table | Create Table |
+------------+----------------------------------------------------------------------------+
| table_name | CREATE TABLE `table_name` ( |
| | `id` INT(11) UNSIGNED NOT NULL, |
| | `field` VARCHAR(20) DEFAULT NULL, |
| | `field_with_default_value` VARCHAR(20) NOT NULL DEFAULT 'myDefault' |
| | ) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+------------+----------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> INSERT INTO `table_name`(`id`, `field`, `field_with_default_value`)
VALUES
(1, 'Value', NULL);
ERROR 1048 (23000): Column 'field_with_default_value' cannot be null
Is it possible to post the relevant part of the structure of your table to see how we can help?
UPDATE
MySQL 5.7, using triggers, can provide a possible solution to the problem:
Changes in MySQL 5.7.1 (2013-04-23, Milestone 11)
...
If a column is declared as NOT NULL, it is not permitted to insert
NULL into the column or update it to NULL. However, this constraint
was enforced even if there was a BEFORE INSERT (or BEFORE UPDATE
trigger) that set the column to a non-NULL value. Now the constraint
is checked at the end of the statement, per the SQL standard. (Bug
#6295, Bug#11744964).
...
Possible solution:
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.7.4-m14 |
+-----------+
1 row in set (0.00 sec)
mysql> DELIMITER $$
mysql> CREATE TRIGGER `trg_bi_set_default_value` BEFORE INSERT ON `table_name`
FOR EACH ROW
BEGIN
IF (NEW.`field_with_default_value` IS NULL) THEN
SET NEW.`field_with_default_value` :=
(SELECT `COLUMN_DEFAULT`
FROM `information_schema`.`COLUMNS`
WHERE `TABLE_SCHEMA` = DATABASE() AND
`TABLE_NAME` = 'table_name' AND
`COLUMN_NAME` = 'field_with_default_value');
END IF;
END$$
mysql> DELIMITER ;
mysql> INSERT INTO `table_name`(`id`, `field`, `field_with_default_value`)
VALUES
(1, 'Value', NULL);
Query OK, 1 row affected (0.00 sec)
mysql> SELECT `id`, `field`, `field_with_default_value` FROM `table_name`;
+----+-------+--------------------------+
| id | field | field_with_default_value |
+----+-------+--------------------------+
| 1 | Value | myDefault |
+----+-------+--------------------------+
1 row in set (0.00 sec)
MySQL actually works as intended, and that behavior seems to be there to stay. MariaDB also works the same way now.
Removing "strict mode" (STRICT_TRANS_TABLES & STRICT_ALL_TABLES) is supposed to revert to the previous behavior, but I personally haven't had any luck with it (maybe I'm doing something wrong, but both my ##GLOBAL.sql_mode & ##SESSION.sql_mode do not contain strict mode).
I think the best solution to this problem is to rely on default values at the PHP level, instead of relying on the Database to provide them. There is an existing answer that explains it pretty well. The comments are also helpful.
That way, you also gain the added benefit that your models/entities will have the default value upon instantiation instead of upon insert in the database. Also, if you want to surface those values to the user after insertion, you can do so without having to do an extra SELECT query after your INSERT.
Another alternative to surface the default values would be to use a RETURNING clause, as is available in PostgreSQL, but not in MySQL (yet). It might be added at some point in the future, but for now MariaDB only has it for DELETE statements. However, I believe that having the default values at the PHP level is still superior; even if you never insert the record, it'll still contain the default values. I've never turned back and used a database default value since putting this into practice.
If you leave out the column (both name and value) from the statement, then the default value will be used.
Some related advice:
Don't have any "non-empty" defaults in your tables, and don't have non-null defaults for nullable columns. Let all values be set from the application.
Don't put business logic on the database-side.
Only define a default if really needed, and only for non-nullable columns. And remove the default when no longer needed. (they come in handy with alter table runs, to set the value of a new column, but then immediately run a new (cheap!) alter to remove the default)
The "empty" mentioned above, is related to the type:
- 0 for numerical columns,
- '' for varchar/varbinary columns,
- '1970-01-01 12:34:56' for timestamps,
- etc.
That saves the application many round trips to the database. If a created row is fully predictable, then the application doesn't need to read it after creating, to find out what it has become. (this assumes: no triggers, no cascading)
With MySQL we make only a few specific exceptions to those strict rules:
Columns called mysql_row_foo, are only set by the database. Examples:
mysql_row_created_at timestamp(6) not null default '1970-01-01 12:34:56.000000',
mysql_row_updated_at timestamp(6) null default null on update current_timestamp,
Unique indexes on not-null columns are welcome, to prevent duplicate data. For example on lookup.brand.name in a table lookup.brand that looks like (id++, name).
The mysql_row_foo columns are like column attributes. They are used by data sync tools, for example. General applications don't read them, and they store their application-side timestamps as epoch values. Examples:
valid_until_epoch int unsigned not null default 0,
last_seen_epoch_ms bigint not null default 0,
What I'm asking is whether when inserting a row in a MySQL database the columns that are null need to be included. For instance, when I use phpMyAdmin to insert a row, it will do something like this:
INSERT INTO table
(
`col1`,
`col2`,
`col3`,
`col4`
)
VALUES
(
'value1',
NULL,
NULL,
'value2'
)
Assuming the fields are null by default, can I remove columns and NULL values from the insert statement?
by default, when you create column it is nullable (unless you have set NOT NULL on it). so when you want to insert record for specific columns only, you can omit that column and the values for it are automatically null.
INSERT INTO table (`col1` ,`col4`)
VALUES ('value1','value2')
you can remove them.
if there is a default value - you will get that instead.
If the column name is also omitted, yes: the default value1 will be assumed.
1 NULL is usually - but not always! - the default value for nullable columns. The schema must be consulted to verify this assumption.
I'm using PHP and I need to insert a new record in a MySQL DB or if it exists simply update it. I read several answers where they propose to use REPLACE or INSERT...ON DUPLICATE KEY UPDATE, however if I well understood, these two option imply the use of unique fields or primary keys (that are unique). In my case the MySQL table has values similar to this:
Timestamp | Query1 | Query2 | Result
--------------------------------------------------------
2012-10-13 08:15:27 | American | Men | here result
2012-10-13 08:15:23 | American | Men | result2
2012-10-13 08:15:27 | American | Women | other result
2012-10-13 08:15:27 | German | Men | here result
Therefore I cannot have primary keys or univoque fields since "query1", "query2" and "Result" can have the same values (e.g. I can have several records with more fields containing "American" (query1) and "Men" (query2) and different results.
At the moment in PHP I'm using:
INSERT INTO results (Timestamp, Query1, Query2, Result) VALUES ('$current_timestamp', '$nationality', '$gender', '')
Which just append all the records and create a huge DB. But what I want to achieve is adding a new record just if the entire combination query1&query2&result doesn't exist. Otherwise I just want to update the "timestamp" field.
For instance if my PHP script produces the following data:
Timestamp | Query1 | Query2 | Result
2012-10-13 08:15:23 | American | Men | result2
and the database already contains this exact combination of "American"&"Men"&"result2" then just the field "timestamp" is updated. Otherwise, if the combination is different (e.g. "American" "Men" "result3) a new record is added to the table.
Thank you in advance for your help
Even absent a single PRIMARY KEY column like a typical AUTO_INCREMENT integer, if those three columns are meant to be a unique combination you ought to define them as a composite key . This will not only preserve uniqueness but also enforce indexing on them as a unit.
CREATE TABLE results (
`Timestamp` DATETIME NOT NULL,
`Query1` VARCHAR() NOT NULL,
`Query2` VARCHAR() NOT NULL,
`Result` VARCHAR() NOT NULL,
/* Composite key across three columns */
PRIMARY KEY (`Query1`,`Query2`,`Result`)
);
You may then use ON DUPLICATE KEY UPDATE when inserting to abort insertion on key violations and update the timestamp instead.
INSERT INTO results (
Timestamp,
Query1,
Query2,
Result
) VALUES (
'$current_timestamp', /* see note below about NOW() */
'$nationality',
'$gender',
''
) ON DUPLICATE KEY UPDATE `Timestamp` = '$current_timestamp';
Note: We assume the PHP variables have been properly escaped already.
Note 2: If the value of $current_timestamp is indeed the current timestamp and not some stored value, I recommend using MySQL's NOW() function rather than pass in a PHP variable.
ON DUPLICATE KEY UPDATE `Timestamp` = NOW()
Same goes for the $current_timestamp in the VALUES() list...
You can use a 'WHERE NOT EXISTS' clause, like that:
INSERT INTO results (Timestamp, Query1, Query2, Result)
SELECT '$current_timestamp', '$nationality', '$gender', '$result'
FROM Dual
WHERE NOT EXISTS (SELECT *
FROM results
WHERE Query1 = '$nationality'
AND Query2 = '$gender'
AND Result = '$result'
)
Here Dual is a stub/fake table that you can use in mysql only to make the query syntaxically correct.
you can use conditional insert like this one:
INSERT INTO results (Timestamp, Query1, Query2, Result)
SELECT '$current_timestamp', '$nationality', '$gender', '$result'
FROM DUAL
WHERE NOT EXISTS (SELECT * FROM results
WHERE Query1 = '$nationality'
AND Query2 = '$gender'
AND Result = '$result')
dual is a special one-column table originally introduced in Oracle
source: http://en.wikipedia.org/wiki/DUAL_table
I have a "bill_date" field that I want to be blank (NULL) until it's been billed, at which point the date will be entered.
I see that MySQL does not like NULL values in datetime fields. Do any of you have a simple way to handle this, or am I forced to use the min date as a "NULL equivalent" and then check for that date?
Thanks.
EDITED TO ADD:
Ok I do see that MySQL will accept the NULL value, but it won't accept it as a database update if I'm updating the record using PHP.
The variable name is $bill_date but it won't leave the variable as NULL if I update a record without sending a value to $bill_date -- I get this error:
Database query failed: Incorrect datetime value: '' for column 'bill_date' at row 1
I assume I need to actually send the word NULL, or leave it out of the update query altogether, to avoid this error? Am I right? Thanks!!!
MySQL does allow NULL values for datetime fields. I just tested it:
mysql> create table datetimetest (testcolumn datetime null default null);
Query OK, 0 rows affected (0.10 sec)
mysql> insert into datetimetest (testcolumn) values (null);
Query OK, 1 row affected (0.00 sec)
mysql> select * from datetimetest;
+------------+
| testcolumn |
+------------+
| NULL |
+------------+
1 row in set (0.00 sec)
I'm using this version:
mysql> select version();
+-----------+
| version() |
+-----------+
| 5.0.45 |
+-----------+
1 row in set (0.03 sec)
Speaking of inserting NULL values through PHP code, given you are using prepared statements (as you definitely should), there is no problem with inserting NULL values. Just bind your variable the usual way, and all PHP variables that contain null values will be stored as NULL in MySQL.
Just make sure that your PHP variable indeed contains NULL and not an empty string.
This is a a sensible point.
A null date is not a zero date. They may look the same, but they ain't. In mysql, a null date value is null. A zero date value is an empty string ('') and '0000-00-00 00:00:00'
On a null date "... where mydate = ''" will fail.
On an empty/zero date "... where mydate is null" will fail.
But now let's get funky. In mysql dates, empty/zero date are strictly the same.
by example
select if(myDate is null, 'null', myDate) as mydate from myTable where myDate = '';
select if(myDate is null, 'null', myDate) as mydate from myTable where myDate = '0000-00-00 00:00:00'
will BOTH output: '0000-00-00 00:00:00'. if you update myDate with '' or '0000-00-00 00:00:00', both selects will still work the same.
In php, the mysql null dates type will be respected with the standard mysql connector, and be real nulls ($var === null, is_null($var)). Empty dates will always be represented as '0000-00-00 00:00:00'.
I strongly advise to use only null dates, OR only empty dates if you can. (some systems will use "virual" zero dates which are valid Gregorian dates, like 1970-01-01 (linux) or 0001-01-01 (oracle).
empty dates are easier in php/mysql. You don't have the "where field is null" to handle. However, you have to "manually" transform the '0000-00-00 00:00:00' date in '' to display empty fields. (to store or search you don't have special case to handle for zero dates, which is nice).
Null dates need better care. you have to be careful when you insert or update to NOT add quotes around null, else a zero date will be inserted instead of null, which causes your standard data havoc. In search forms, you will need to handle cases like "and mydate is not null", and so on.
Null dates are usually more work. but they much MUCH MUCH faster than zero dates for queries.
I had this problem on windows.
This is the solution:
To pass '' for NULL you should disable STRICT_MODE (which is enabled by default on Windows installations)
BTW It's funny to pass '' for NULL. I don't know why they let this kind of behavior.
It depends on how you declare your table. NULL would not be allowed in:
create table MyTable (col1 datetime NOT NULL);
But it would be allowed in:
create table MyTable (col1 datetime NULL);
The second is the default, so someone must've actively decided that the column should not be nullable.
If your datetime column is not accepting null values it might be not nullable, so you can update the column using this command:
ALTER TABLE your_table modify your_coloumn datetime null
And when you want to save a null field, you can leave it alone (not assigning anything to that field), or you can set it to Null (instead of '')
For what it is worth: I was experiencing a similar issue trying to update a MySQL table via Perl. The update would fail when an empty string value (translated from a null value from a read from another platform) was passed to the date column ('dtcol' in the code sample below). I was finally successful getting the data updated by using an IF statement embedded in my update statement:
...
my $stmnt='update tbl set colA=?,dtcol=if(?="",null,?) where colC=?';
my $status=$dbh->do($stmt,undef,$iref[1],$iref[2],$iref[2],$ref[0]);
...
I just discovered that MySQL will take null provided the default for the field is null and I write specific if statements and leave off the quotes. This works for update as well.
if(empty($odate) AND empty($ddate))
{
$query2="UPDATE items SET odate=null, ddate=null, istatus='$istatus' WHERE id='$id' ";
};
I just tested in MySQL v5.0.6 and the datetime column accepted null without issue.
Specifically relating to the error you're getting, you can't do something like this in PHP for a nullable field in MySQL:
$sql = 'INSERT INTO table (col1, col2) VALUES(' . $col1 . ', ' . null . ')';
Because null in PHP will equate to an empty string which is not the same as a NULL value in MysQL. Instead you want to do this:
$sql = 'INSERT INTO table (col1, col2) VALUES(' . $col1 . ', ' . (is_null($col2) ? 'NULL' : $col2). ')';
Of course you don't have to use is_null but I figure that it demonstrates the point a little better. Probably safer to use empty() or something like that. And if $col2 happens to be a string which you would enclose in double quotes in the query, don't forget not to include those around the 'NULL' string, otherwise it wont work.
Hope that helps!