I have a mysql table that has a column with date type.
I'm going to store non-Gregorian date in this column (i.e. Jalali date).
I tested this by using phpMyadmin and storing a Jalali date and no error happend.
Is it a good idea to store non-Gregorian dates in date type?
If not; which is better? storing it as varchar or as timestamp or timestamp as Int or something else?
Is it a good idea to store non-Gregorian dates in date type?
No. Aside that some valid date in one calendar system doesn't exist in another calendar, functions working on DATE typed columns may not work properly. The matter is not just storing data, you need to process this data and for example compare them with CURDATE().
storing it as varchar or as timestamp or timestamp as Int or something else?
If you choose a proper formatting, use two digits for month and day and static number of digits for year, a character string type, CHAR or VARCHAR is fine. Comparing theme against each other is just a lexical comparison and you still can write your functions o procedures to extend functionality.
Choosing TIMESTAMP or DATE changes the question as former represents a specific time but latter represents a specific entry in calendar. If you want put time beside date they still differ in meaning. You should think about issues like daylight-saving time changes which cause some people prefer to put calendar entry (DATE) and some prefer seconds passed from 1 Jan 1970 (TIMESTAMP). e.g. there is two timestamps for 1393-06-30 23:30:00 in Hijri Shamsi calendar based on current Iran government laws.
You can store non-gregorian dates in an integer field in the database.
Examples:
year:1396, month:11, day:17 => 13961117
year:1393, month:4, day:9 => 13930409
by using this, you can query rows to find a specific date and dates that are <=> than a specific date, but unfortunately, you can't compare them against each other.
Internal storage and binary interface of almost all database systems have nothing to do with calendars. you just store date (and possibly time) into database, providing no calendaring information. It's only a simple number of days, seconds or milliseconds past from a specific point in time (usually midnight 1970-01-01).
So you just need to provide an API abstraction of dates in your application layer. Everything else will work. You convert all your dates to Gregorian or Unix timestamp, and send queries to MySQL as usual.
Non Gregorian calendar still operate by Year, Month, Day and Time so it can be stored in 4 separate columns like:
CREATE TABLE `Calendar` (
`Year` int,
`Month` tinyint,
`Day` tinyint,
`Time` time
);
I make possible to store dates without conversion and permit group by Year and Month
Would you recommend using a datetime or a timestamp field, and why (using MySQL)?
I'm working with PHP on the server side.
Timestamps in MySQL are generally used to track changes to records, and are often updated every time the record is changed. If you want to store a specific value you should use a datetime field.
If you meant that you want to decide between using a UNIX timestamp or a native MySQL datetime field, go with the native DATETIME format. You can do calculations within MySQL that way
("SELECT DATE_ADD(my_datetime, INTERVAL 1 DAY)") and it is simple to change the format of the value to a UNIX timestamp ("SELECT UNIX_TIMESTAMP(my_datetime)") when you query the record if you want to operate on it with PHP.
In MySQL 5 and above, TIMESTAMP values are converted from the current time zone to UTC for storage, and converted back from UTC to the current time zone for retrieval. (This occurs only for the TIMESTAMP data type, and not for other types such as DATETIME.)
By default, the current time zone for each connection is the server's time. The time zone can be set on a per-connection basis, as described in MySQL Server Time Zone Support.
I always use DATETIME fields for anything other than row metadata (date created or modified).
As mentioned in the MySQL documentation:
The DATETIME type is used when you need values that contain both date and time information. MySQL retrieves and displays DATETIME values in 'YYYY-MM-DD HH:MM:SS' format. The supported range is '1000-01-01 00:00:00' to '9999-12-31 23:59:59'.
...
The TIMESTAMP data type has a range of '1970-01-01 00:00:01' UTC to '2038-01-09 03:14:07' UTC. It has varying properties, depending on the MySQL version and the SQL mode the server is running in.
You're quite likely to hit the lower limit on TIMESTAMPs in general use -- e.g. storing birthdate.
The below examples show how the TIMESTAMP date type changed the values after changing the time-zone to 'america/new_york' where DATETIMEis unchanged.
mysql> show variables like '%time_zone%';
+------------------+---------------------+
| Variable_name | Value |
+------------------+---------------------+
| system_time_zone | India Standard Time |
| time_zone | Asia/Calcutta |
+------------------+---------------------+
mysql> create table datedemo(
-> mydatetime datetime,
-> mytimestamp timestamp
-> );
mysql> insert into datedemo values ((now()),(now()));
mysql> select * from datedemo;
+---------------------+---------------------+
| mydatetime | mytimestamp |
+---------------------+---------------------+
| 2011-08-21 14:11:09 | 2011-08-21 14:11:09 |
+---------------------+---------------------+
mysql> set time_zone="america/new_york";
mysql> select * from datedemo;
+---------------------+---------------------+
| mydatetime | mytimestamp |
+---------------------+---------------------+
| 2011-08-21 14:11:09 | 2011-08-21 04:41:09 |
+---------------------+---------------------+
I've converted my answer into article so more people can find this useful, MySQL: Datetime Versus Timestamp Data Types.
The main difference is that DATETIME is constant while TIMESTAMP is affected by the time_zone setting.
So it only matters when you have — or may in the future have — synchronized clusters across time zones.
In simpler words: If I have a database in Australia, and take a dump of that database to synchronize/populate a database in America, then the TIMESTAMP would update to reflect the real time of the event in the new time zone, while DATETIME would still reflect the time of the event in the au time zone.
A great example of DATETIME being used where TIMESTAMP should have been used is in Facebook, where their servers are never quite sure what time stuff happened across time zones. Once I was having a conversation in which the time said I was replying to messages before the message was actually sent. (This, of course, could also have been caused by bad time zone translation in the messaging software if the times were being posted rather than synchronized.)
I make this decision on a semantic base.
I use a timestamp when I need to record a (more or less) fixed point in time. For example when a record was inserted into the database or when some user action took place.
I use a datetime field when the date/time can be set and changed arbitrarily. For example when a user can save later change appointments.
I recommend using neither a DATETIME or a TIMESTAMP field. If you want to represent a specific day as a whole (like a birthday), then use a DATE type, but if you're being more specific than that, you're probably interested in recording an actual moment as opposed to a unit of time (day,week,month,year). Instead of using a DATETIME or TIMESTAMP, use a BIGINT, and simply store the number of milliseconds since the epoch (System.currentTimeMillis() if you're using Java). This has several advantages:
You avoid vendor lock-in. Pretty much every database supports integers in the relatively similar fashion. Suppose you want to move to another database. Do you want to worry about the differences between MySQL's DATETIME values and how Oracle defines them? Even among different versions of MySQL, TIMESTAMPS have a different level of precision. It was only just recently that MySQL supported milliseconds in the timestamps.
No timezone issues. There's been some insightful comments on here on what happens with timezones with the different data types. But is this common knowledge, and will your co-workers all take the time to learn it? On the other hand, it's pretty hard to mess up changing a BigINT into a java.util.Date. Using a BIGINT causes a lot of issues with timezones to fall by the wayside.
No worries about ranges or precision. You don't have to worry about what being cut short by future date ranges (TIMESTAMP only goes to 2038).
Third-party tool integration. By using an integer, it's trivial for 3rd party tools (e.g. EclipseLink) to interface with the database. Not every third-party tool is going to have the same understanding of a "datetime" as MySQL does. Want to try and figure out in Hibernate whether you should use a java.sql.TimeStamp or java.util.Date object if you're using these custom data types? Using your base data types make's use with 3rd-party tools trivial.
This issue is closely related how you should store a money value (i.e. $1.99) in a database. Should you use a Decimal, or the database's Money type, or worst of all a Double? All 3 of these options are terrible, for many of the same reasons listed above. The solution is to store the value of money in cents using BIGINT, and then convert cents to dollars when you display the value to the user. The database's job is to store data, and NOT to intrepret that data. All these fancy data-types you see in databases(especially Oracle) add little, and start you down the road to vendor lock-in.
TIMESTAMP is four bytes vs eight bytes for DATETIME.
Timestamps are also lighter on the database and indexed faster.
The DATETIME type is used when you need values that contain both date and time information. MySQL retrieves and displays DATETIME values in YYYY-MM-DD HH:MM:SS format. The supported range is 1000-01-01 00:00:00 to 9999-12-31 23:59:59. The TIMESTAMP data type has a range of 1970-01-01 00:00:01 UTC to 2038-01-09 03:14:07 UTC. It has varying properties, depending on the MySQL version and the SQL mode the server is running in.
DATETIME is constant while TIMESTAMP is affected by the time_zone setting.
TIMESTAMP is 4 bytes Vs 8 bytes for DATETIME.
http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html
But like scronide said it does have a lower limit of the year 1970. It's great for anything that might happen in the future though ;)
Neither. The DATETIME and TIMESTAMP types are fundamentally broken for generic use cases. MySQL will change them in the future. You should use BIGINT and UNIX timestamps unless you have a specific reason to use something else.
Special cases
Here are some specific situations where your choice is easier and you don't need the analysis and general recommendation in this answer.
Date only — if you only care about the date (like the date of the next Lunar New Year, 2022-02-01) AND you have a clear understanding of what timezone that date applies (or don't care, as in the case of Lunar New Year) then use the DATE column type.
Record insert times — if you are logging the insert dates/times for rows in your database AND you don't care that your application will break in the next 17 years, then go ahead and use TIMESTAMP with a default value of CURRENT_TIMESTAMP().
Why is TIMESTAMP broken?
The TIMESTAMP type is stored on disk in UTC timezone. This means that if you physically move your server, it does not break. That's good ✅. But timestamps as currently defined will stop working entirely in the year 2038 ❌.
Every time you INSERT INTO or SELECT FROM a TIMESTAMP column, the physical location (i.e. timezone configuration) of your client/application server is taken into account. If you move your application server then your dates break ❌.
(Update 2022-04-29 MySQL fixed this in 8.0.28 but if your production environment is on CentOS 7 or many other flavors your migration path will be a long time until you get this support.)
Why is VARCHAR broken?
The VARCHAR type allows to unambiguously store a non-local date/time/both in ISO 8601 format and it works for dates past 2037. It is common to use Zulu time, but ISO 8601 allows to encode any offset. This is less useful because while MySQL date and time functions do support string as input anywhere date/time/both are expected, the result is incorrect if the input uses timezone offsets.
Also VARCHAR uses extra bytes of storage.
Why is DATETIME broken?
A DATETIME stores a DATE and a TIME in the same column. Neither of these things have any meaning unless the timezone is understood, and the timezone is not stored anywhere ❌. You should put the intended timezone as a comment in the column because the timezone is inextricably linked to the data. So few people use column comments, therefore this is mistake waiting to happen. I inherited a server from Arizona, so I always need to convert all timestamps FROM Arizona time and then TO another time.
(Update 2021-12-08 I restarted the server after years of uptime and the database client (with upgrades) reset to UTC. That means my application needs to handle dates before and after the reset differently. Hardcode!)
The only situation a DATETIME is correct is to complete this sentence:
Your year 2020 solar new year starts at exactly DATETIME("2020-01-01 00:00:00").
There is no other good use for DATETIMEs. Perhaps you will imagine a web server for a city government in Delaware. Surely the timezone for this server and all the people accessing this server can be implied to be in Delaware, with Eastern Time Zone, right? Wrong! In this millennium, we all think of servers as existing in "the cloud". So it is always wrong to think of your server in any specific timezone, because your server will be moved some day.
Note: MySQL now supports time zone offsets in DATETIME literals (thanks #Marko). This may make inserting DATETIMEs more convenient for you but does not address the incomplete and therefore useless meaning of the data, this fatal issue identifies ("❌") above.
How to use BIGINT?
Define:
CREATE TEMPORARY TABLE good_times (
a_time BIGINT
)
Insert a specific value:
INSERT INTO good_times VALUES (
UNIX_TIMESTAMP(CONVERT_TZ("2014-12-03 12:24:54", '+00:00', ##global.time_zone))
);
Insert a default value (thx Brad):
ALTER TABLE good_times MODIFY a_time BIGINT DEFAULT (UNIX_TIMESTAMP());
Or of course this is much better from your application, like:
$statement = $myDB->prepare('INSERT INTO good_times VALUES (?)');
$statement->execute([$someTime->getTimestamp()]);
Select:
SELECT a_time FROM good_times;
There are techniques for filtering relative times (select posts within the past 30 days, find users that bought within 10 minutes of registering) beyond the scope here.
Depends on application, really.
Consider setting a timestamp by a user to a server in New York, for an appointment in Sanghai. Now when the user connects in Sanghai, he accesses the same appointment timestamp from a mirrored server in Tokyo. He will see the appointment in Tokyo time, offset from the original New York time.
So for values that represent user time like an appointment or a schedule, datetime is better. It allows the user to control the exact date and time desired, regardless of the server settings. The set time is the set time, not affected by the server's time zone, the user's time zone, or by changes in the way daylight savings time is calculated (yes it does change).
On the other hand, for values that represent system time like payment transactions, table modifications or logging, always use timestamps. The system will not be affected by moving the server to another time zone, or when comparing between servers in different timezones.
Timestamps are also lighter on the database and indexed faster.
2016 +: what I advise is to set your Mysql timezone to UTC and use DATETIME:
Any recent front-end framework (Angular 1/2, react, Vue,...) can easily and automatically convert your UTC datetime to local time.
Additionally:
DATETIME can now be automatically set to the current time value How do you set a default value for a MySQL Datetime column?
Contrary to what one might think, DATETIME is FASTER THAN TIMESTAMP,
http://gpshumano.blogs.dri.pt/2009/07/06/mysql-datetime-vs-timestamp-vs-int-performance-and-benchmarking-with-myisam/
TIMESTAMP is still limited to 1970-2038
(Unless you are likely to change the timezone of your servers)
Example with AngularJs
// back-end: format for angular within the sql query
SELECT DATE_FORMAT(my_datetime, "%Y-%m-%dT%TZ")...
// font-end Output the localised time
{{item.my_datetime | date :'medium' }}
All localised time format available here:
https://docs.angularjs.org/api/ng/filter/date
TIMESTAMP is always in UTC (that is, elapsed seconds since 1970-01-01, in UTC), and your MySQL server auto-converts it to the date/time for the connection timezone. In the long-term, TIMESTAMP is the way to go because you know your temporal data will always be in UTC. For example, you won't screw your dates up if you migrate to a different server or if you change the timezone settings on your server.
Note: default connection timezone is the server timezone, but this can (should) be changed per session (see SET time_zone = ...).
A timestamp field is a special case of the datetime field. You can create timestamp columns to have special properties; it can be set to update itself on either create and/or update.
In "bigger" database terms, timestamp has a couple of special-case triggers on it.
What the right one is depends entirely on what you want to do.
Comparison between DATETIME, TIMESTAMP and DATE
What is that [.fraction]?
A DATETIME or TIMESTAMP value can include a trailing fractional
seconds part in up to microseconds (6 digits) precision. In
particular, any fractional part in a value inserted into a DATETIME
or TIMESTAMP column is stored rather than discarded. This is of course optional.
Sources:
MySQL Date/Time data types reference
MySQL Storage Requirements reference
It is worth noting in MySQL you can use something along the lines of the below when creating your table columns:
on update CURRENT_TIMESTAMP
This will update the time at each instance you modify a row and is sometimes very helpful for stored last edit information. This only works with timestamp, not datetime however.
I would always use a Unix timestamp when working with MySQL and PHP. The main reason for this being the default date method in PHP uses a timestamp as the parameter, so there would be no parsing needed.
To get the current Unix timestamp in PHP, just do time();
and in MySQL do SELECT UNIX_TIMESTAMP();.
From my experiences, if you want a date field in which insertion happens only once and you don't want to have any update or any other action on that particular field, go with date time.
For example, consider a user table with a REGISTRATION DATE field. In that user table, if you want to know the last logged in time of a particular user, go with a field of timestamp type so that the field gets updated.
If you are creating the table from phpMyAdmin the default setting will update the timestamp field when a row update happens. If your timestamp filed is not updating with row update, you can use the following query to make a timestamp field get auto updated.
ALTER TABLE your_table
MODIFY COLUMN ts_activity TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP;
The timestamp data type stores date and time, but in UTC format, not in the current timezone format as datetime does. And when you fetch data, timestamp again converts that into the current timezone time.
So suppose you are in USA and getting data from a server which has a time zone of USA. Then you will get the date and time according to the USA time zone. The timestamp data type column always get updated automatically when its row gets updated. So it can be useful to track when a particular row was updated last time.
For more details you can read the blog post Timestamp Vs Datetime .
I always use a Unix timestamp, simply to maintain sanity when dealing with a lot of datetime information, especially when performing adjustments for timezones, adding/subtracting dates, and the like. When comparing timestamps, this excludes the complicating factors of timezone and allows you to spare resources in your server side processing (Whether it be application code or database queries) in that you make use of light weight arithmetic rather then heavier date-time add/subtract functions.
Another thing worth considering:
If you're building an application, you never know how your data might have to be used down the line. If you wind up having to, say, compare a bunch of records in your data set, with, say, a bunch of items from a third-party API, and say, put them in chronological order, you'll be happy to have Unix timestamps for your rows. Even if you decide to use MySQL timestamps, store a Unix timestamp as insurance.
Reference taken from this Article:
The main differences:
TIMESTAMP used to track changes to records, and update every time when the record is changed.
DATETIME used to store specific and static value which is not affected by any changes in records.
TIMESTAMP also affected by different TIME ZONE related setting.
DATETIME is constant.
TIMESTAMP internally converted current time zone to UTC for storage, and during retrieval converted back to the current time zone.
DATETIME can not do this.
TIMESTAMP supported range:
‘1970-01-01 00:00:01′ UTC to ‘2038-01-19 03:14:07′ UTC
DATETIME supported range:
‘1000-01-01 00:00:00′ to ‘9999-12-31 23:59:59′
In my case, I set UTC as a time zone for everything: the system, the database server, etc. every time that I can. If my customer requires another time zone, then I configure it on the app.
I almost always prefer timestamps rather than datetime fields, because timestamps include the timezone implicitly. So, since the moment that the app will be accessed from users from different time zones and you want them to see dates and times in their local timezone, this field type makes it pretty easy to do it than if the data were saved in datetime fields.
As a plus, in the case of a migration of the database to a system with another timezone, I would feel more confident using timestamps. Not to say possible issues when calculating differences between two moments with a sumer time change in between and needing a precision of 1 hour or less.
So, to summarize, I value this advantages of timestamp:
ready to use on international (multi time zone) apps
easy migrations between time zones
pretty easy to calculate diferences (just subtract both timestamps)
no worry about dates in/out a summer time period
For all this reasons, I choose UTC & timestamp fields where posible. And I avoid headaches ;)
Beware of timestamp changing when you do a UPDATE statement on a table. If you have a table with columns 'Name' (varchar), 'Age' (int), and 'Date_Added' (timestamp) and you run the following DML statement
UPDATE table
SET age = 30
then every single value in your 'Date_Added' column would be changed to the current timestamp.
+---------------------------------------------------------------------------------------+--------------------------------------------------------------------------+
| TIMESTAMP | DATETIME |
+---------------------------------------------------------------------------------------+--------------------------------------------------------------------------+
| TIMESTAMP requires 4 bytes. | DATETIME requires 8 bytes. |
| Timestamp is the number of seconds that have elapsed since January 1, 1970 00:00 UTC. | DATETIME is a text displays 'YYYY-MM-DD HH:MM:SS' format. |
| TIMESTAMP supported range: ‘1970-01-01 00:00:01′ UTC to ‘2038-01-19 03:14:07′ UTC. | DATETIME supported range: ‘1000-01-01 00:00:00′ to ‘9999-12-31 23:59:59′ |
| TIMESTAMP during retrieval converted back to the current time zone. | DATETIME can not do this. |
| TIMESTAMP is used mostly for metadata i.e. row created/modified and audit purpose. | DATETIME is used mostly for user-data. |
+---------------------------------------------------------------------------------------+--------------------------------------------------------------------------+
I found unsurpassed usefulness in TIMESTAMP's ability to auto update itself based on the current time without the use of unnecessary triggers. That's just me though, although TIMESTAMP is UTC like it was said.
It can keep track across different timezones, so if you need to display a relative time for instance, UTC time is what you would want.
DATETIME vs TIMESTAMP:
TIMESTAMP used to track changes of records, and update every time when the record is changed.
DATETIME used to store specific and static value which is not affected by any changes in records.
TIMESTAMP also affected by different TIME ZONE related setting.
DATETIME is constant.
TIMESTAMP internally converted a current time zone to UTC for storage, and during retrieval convert the back to the current time zone.
DATETIME can not do this.
TIMESTAMP is 4 bytes and DATETIME is 8 bytes.
TIMESTAMP supported range:
‘1970-01-01 00:00:01′ UTC to ‘2038-01-19 03:14:07′ UTC
DATETIME supported range:
‘1000-01-01 00:00:00′ to ‘9999-12-31 23:59:59′
The major difference is
a INDEX's on Timestamp - works
a INDEX's on Datetime - Does not work
look at this post to see problems with Datetime indexing
I stopped using datetime in my applications after facing many problems and bugs related to time zones. IMHO using timestamp is better than datetime in most of the cases.
When you ask what is the time ? and the answer comes as something like '2019-02-05 21:18:30', that is not completed, not defined answer because it lacks another part, in which timezone ? Washington ? Moscow ? Beijing ?
Using datetimes without the timezone means that your application is dealing with only 1 timezone, however timestamps give you the benefits of datetime plus the flexibility of showing the same exact point of time in different timezones.
Here are some cases that will make you regret using datetime and wish that you stored your data in timestamps.
For your clients comfort you want to show them the times based on their preferred time zones without making them doing the math and convert the time to their meaningful timezone. all you need is to change the timezone and all your application code will be the same.(Actually you should always define the timezone at the start of the application, or request processing in case of PHP applications)
SET time_zone = '+2:00';
you changed the country you stay in, and continue your work of maintaining the data while seeing it in a different timezone (without changing the actual data).
you accept data from different clients around the world, each of them inserts the time in his timezone.
In short
datetime = application supports 1 timezone (for both inserting and selecting)
timestamp = application supports any timezone (for both inserting and selecting)
This answer is only for putting some highlight on the flexibility and ease of timestamps when it comes to time zones , it is not covering any other differences like the column size or range or fraction.
Another difference between Timestamp and Datetime is in Timestamp you can't default value to NULL.
I prefer using timestamp so to keep everything in one common raw format and format the data in PHP code or in your SQL query. There are instances where it comes in handy in your code to keep everything in plain seconds.
This question already has answers here:
Should I use the datetime or timestamp data type in MySQL?
(40 answers)
Closed 9 years ago.
Probably many coders want to ask this question. it is What's the adventages of each one of those MySQL time formats. and which one you will prefer to use it in your apps.
For me i use Unix timestamp because maybe i find it easy to convert & order records with it, and also because i never tried the DATETIME thing. but anyways i'm ready to change my mind if anyone tells me i'm wrong.
Thanks
Timestamp (both PHP ones and MySQL's ones) are stored using 32 bits (i.e. 4 bytes) integers ; which means they are limited to a date range that goes from 1970 to 2038.
DATETIME don't have that limitation -- but are stored using more bytes (8 bytes, if I'm not mistaken)
After, between storing timestamps as seen by PHP, or timestamps as seen by MySQL :
using PHP timestamps means manipulations are easier from PHP -- see Date/Time Functions
using MySQL's timestamps means manipulations are easier from MySQL -- see 11.6. Date and Time Functions
And, for more informations between MySQL's TIMESTAMP and DATETIME datatypes, see 10.3.1. The DATETIME, DATE, and TIMESTAMP Types
As others have said, timestamps can represent a smaller range of datetimes (from 1970 to 2038). However, timestamps measure the number of seconds since the Unix Epoch (1970-01-01 00:00:00 UTC), thereby making them independent of time zone, whereas DATETIME stores a date and time without a time zone. In other words, timestamps unambiguously reference a particular point in time, whereas the exact point in time a DATETIME refers to requires a time zone (which is not stored in a DATETIME field). To see why this can matter, consider what happens if we change our time zone.
Let's say we want to store the datetime 2010-03-27 12:00 UTC. If we store this and retrieve it using a timestamp or DATETIME, then there usually appears to be no difference. However, if the server now changes so that the local time zone is UTC+01, then we get two different results if we pull out the datetime.
If we'd set the field to a DATETIME, it would report the datetime as 2010-03-27 12:00, despite the change in time zone. If we'd set the field to a timestamp, the date would be reported as 2010-03-27 11:00. This isn't a problem with either datatype -- it's just a result of the fact that they store slightly different information.
That really depends. I'll give you 2 examples where one overcome the other:
Timestamp is better than DATETIME when you want to store users session in the database and the session creation time (in Timestamp format) is used for fast row retrieval (with index).
E.g. table may look like this:
[session_create_time AS Timestamp][IP_address AS 32bit Int][etc...]
Having an index on the first two columns can really speed up your queries. If you had a DATETIME value type for the session_create_time field, then it could be taken much more time. Take into account that session queries are executed each time a user request a page, so efficiency is crucial.
DATETIME is better than Timestamp when you want to store a user's date of birth or some historic events that require flexible time range.
Unless digitizing records prior to January 1, 1970, I like the UNIX epoch. Its just a matter of preference, whole unsigned numbers are simpler to deal with when using multiple languages.
Just keep in mind, the epoch starts at January 1, 1970. A lot of companies had been in business for decades, if not longer, prior to that.
I'm using the America/New York timezone. In the Fall we "fall back" an hour -- effectively "gaining" one hour at 2am. At the transition point the following happens:
it's 01:59:00 -04:00
then 1 minute later it becomes:
01:00:00 -05:00
So if you simply say "1:30am" it's ambiguous as to whether or not you're referring to the first time 1:30 rolls around or the second. I'm trying to save scheduling data to a MySQL database and can't determine how to save the times properly.
Here's the problem:
"2009-11-01 00:30:00" is stored internally as 2009-11-01 00:30:00 -04:00
"2009-11-01 01:30:00" is stored internally as 2009-11-01 01:30:00 -05:00
This is fine and fairly expected. But how do I save anything to 01:30:00 -04:00? The documentation does not show any support for specifying the offset and, accordingly, when I've tried specifying the offset it's been duly ignored.
The only solutions I've thought of involve setting the server to a timezone that doesn't use daylight savings time and doing the necessary transformations in my scripts (I'm using PHP for this). But that doesn't seem like it should be necessary.
Many thanks for any suggestions.
I've got it figured out for my purposes. I'll summarize what I learned (sorry, these notes are verbose; they're as much for my future referral as anything else).
Contrary to what I said in one of my previous comments, DATETIME and TIMESTAMP fields do behave differently. TIMESTAMP fields (as the docs indicate) take whatever you send them in "YYYY-MM-DD hh:mm:ss" format and convert it from your current timezone to UTC time. The reverse happens transparently whenever you retrieve the data. DATETIME fields do not make this conversion. They take whatever you send them and just store it directly.
Neither the DATETIME nor the TIMESTAMP field types can accurately store data in a timezone that observes DST. If you store "2009-11-01 01:30:00" the fields have no way to distinguish which version of 1:30am you wanted -- the -04:00 or -05:00 version.
Ok, so we must store our data in a non DST timezone (such as UTC). TIMESTAMP fields are unable to handle this data accurately for reasons I'll explain: if your system is set to a DST timezone then what you put into TIMESTAMP may not be what you get back out. Even if you send it data that you've already converted to UTC, it will still assume the data's in your local timezone and do yet another conversion to UTC. This TIMESTAMP-enforced local-to-UTC-back-to-local roundtrip is lossy when your local timezone observes DST (since "2009-11-01 01:30:00" maps to 2 different possible times).
With DATETIME you can store your data in any timezone you want and be confident that you'll get back whatever you send it (you don't get forced into the lossy roundtrip conversions that TIMESTAMP fields foist on you). So the solution is to use a DATETIME field and before saving to the field convert from your system time zone into whatever non-DST zone you want to save it in (I think UTC is probably the best option). This allows you to build the conversion logic into your scripting language so that you can explicitly save the UTC equivalent of "2009-11-01 01:30:00 -04:00" or ""2009-11-01 01:30:00 -05:00".
Another important thing to note is that MySQL's date/time math functions don't work properly around DST boundaries if you store your dates in a DST TZ. So all the more reason to save in UTC.
In a nutshell I now do this:
When retrieving the data from the database:
Explicitly interpret the data from the database as UTC outside of MySQL in order to get an accurate Unix timestamp. I use PHP's strtotime() function or its DateTime class for this. It can not be reliably done inside of MySQL using MySQL's CONVERT_TZ() or UNIX_TIMESTAMP() functions because CONVERT_TZ will only output a 'YYYY-MM-DD hh:mm:ss' value which suffers from ambiguity problems, and UNIX_TIMESTAMP() assumes its input is in the system timezone, not the timezone the data was ACTUALLY stored in (UTC).
When storing the data to the database:
Convert your date to the precise UTC time that you desire outside of MySQL. For example: with PHP's DateTime class you can specify "2009-11-01 1:30:00 EST" distinctly from "2009-11-01 1:30:00 EDT", then convert it to UTC and save the correct UTC time to your DATETIME field.
Phew. Thanks so much for everyone's input and help. Hopefully this saves someone else some headaches down the road.
BTW, I am seeing this on MySQL 5.0.22 and 5.0.27
MySQL's date types are, frankly, broken and cannot store all times correctly unless your system is set to a constant offset timezone, like UTC or GMT-5. (I'm using MySQL 5.0.45)
This is because you can't store any time during the hour before Daylight Saving Time ends. No matter how you input dates, every date function will treat these times as if they are during the hour after the switch.
My system's timezone is America/New_York. Let's try storing 1257051600 (Sun, 01 Nov 2009 06:00:00 +0100).
Here's using the proprietary INTERVAL syntax:
SELECT UNIX_TIMESTAMP('2009-11-01 00:00:00' + INTERVAL 3599 SECOND); # 1257051599
SELECT UNIX_TIMESTAMP('2009-11-01 00:00:00' + INTERVAL 3600 SECOND); # 1257055200
SELECT UNIX_TIMESTAMP('2009-11-01 01:00:00' - INTERVAL 1 SECOND); # 1257051599
SELECT UNIX_TIMESTAMP('2009-11-01 01:00:00' - INTERVAL 0 SECOND); # 1257055200
Even FROM_UNIXTIME() won't return the accurate time.
SELECT UNIX_TIMESTAMP(FROM_UNIXTIME(1257051599)); # 1257051599
SELECT UNIX_TIMESTAMP(FROM_UNIXTIME(1257051600)); # 1257055200
Oddly enough, DATETIME will still store and return (in string form only!) times within the "lost" hour when DST starts (e.g. 2009-03-08 02:59:59). But using these dates in any MySQL function is risky:
SELECT UNIX_TIMESTAMP('2009-03-08 01:59:59'); # 1236495599
SELECT UNIX_TIMESTAMP('2009-03-08 02:00:00'); # 1236495600
# ...
SELECT UNIX_TIMESTAMP('2009-03-08 02:59:59'); # 1236495600
SELECT UNIX_TIMESTAMP('2009-03-08 03:00:00'); # 1236495600
The takeaway: If you need to store and retrieve every time in the year, you have a few undesirable options:
Set system timezone to GMT + some constant offset. E.g. UTC
Store dates as INTs (as Aaron discovered, TIMESTAMP isn't even reliable)
Pretend the DATETIME type has some constant offset timezone. E.g. If you're in America/New_York, convert your date to GMT-5 outside of MySQL, then store as a DATETIME (this turns out to be essential: see Aaron's answer). Then you must take great care using MySQL's date/time functions, because some assume your values are of the system timezone, others (esp. time arithmetic functions) are "timezone agnostic" (they may behave as if the times are UTC).
Aaron and I suspect that auto-generating TIMESTAMP columns are also broken. Both 2009-11-01 01:30 -0400 and 2009-11-01 01:30 -0500 will be stored as the ambiguous 2009-11-01 01:30.
I think micahwittman's link has the best practical solution to these MySQL limitations: Set the session timezone to UTC when you connect:
SET SESSION time_zone = '+0:00'
Then you just send it Unix timestamps and everything should be fine.
But how do I save anything to 01:30:00
-04:00?
You can convert to UTC like:
SELECT CONVERT_TZ('2009-11-29 01:30:00','-04:00','+00:00');
Even better, save the dates as a TIMESTAMP field. That's always stored in UTC, and UTC doesn't know about summer/winter time.
You can convert from UTC to localtime using CONVERT_TZ:
SELECT CONVERT_TZ(UTC_TIMESTAMP(),'+00:00','SYSTEM');
Where '+00:00' is UTC, the from timezone , and 'SYSTEM' is the local timezone of the OS where MySQL runs.
Mysql inherently solves this problem using time_zone_name table from mysql db.
Use CONVERT_TZ while CRUD to update the datetime without worrying about daylight savings time.
SELECT
CONVERT_TZ('2019-04-01 00:00:00','Europe/London','UTC') AS time1,
CONVERT_TZ('2019-03-01 00:00:00','Europe/London','UTC') AS time2;
This thread made me freak since we use TIMESTAMP columns with On UPDATE CURRENT_TIMESTAMP (ie: recordTimestamp timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP) to track changed records and ETL to a datawarehouse.
In case someone wonder, in this case, TIMESTAMP behave correctly and you can differentiate between the two similar dates by converting the TIMESTAMP to unix timestamp:
select TestFact.*, UNIX_TIMESTAMP(recordTimestamp) from TestFact;
id recordTimestamp UNIX_TIMESTAMP(recordTimestamp)
1 2012-11-04 01:00:10.0 1352005210
2 2012-11-04 01:00:10.0 1352008810
I was working on logging counts of visits of pages and displaying the counts in graph (using Flot jQuery plugin). I filled the table with test data and everything looked fine, but I noticed that at the end of the graph the points were one day off according to labels on x-axis. After examination I noticed that the view count for day 2015-10-25 was retrieved twice from the database and passed to Flot, so every day after this date was moved by one day to right.
After looking for a bug in my code for a while I realized that this date is when the DST takes place. Then I came to this SO page...
...but the suggested solutions was an overkill for what I needed or they had other disadvantages. I am not very worried about not being able to distinguish between ambiguous timestamps. I just need to count and display records per days.
First, I retrieve the date range:
SELECT
DATE(MIN(created_timestamp)) AS min_date,
DATE(MAX(created_timestamp)) AS max_date
FROM page_display_log
WHERE item_id = :item_id
Then, in a for loop, starting with min_date, ending with max_date, by step of one day (60*60*24), I'm retrieving the counts:
for( $day = $min_date_timestamp; $day <= $max_date_timestamp; $day += 60 * 60 * 24 ) {
$query = "
SELECT COUNT(*) AS count_per_day
FROM page_display_log
WHERE
item_id = :item_id AND
(
created_timestamp BETWEEN
'" . date( "Y-m-d 00:00:00", $day ) . "' AND
'" . date( "Y-m-d 23:59:59", $day ) . "'
)
";
//execute query and do stuff with the result
}
My final and quick solution to my problem was this:
$min_date_timestamp += 60 * 60 * 2; // To avoid DST problems
for( $day = $min_date_timestamp; $day <= $max_da.....
So I am not staring the loop in the beginning of the day, but two hours later. The day is still the same, and I am still retrieving correct counts, since I explicitly ask the database for records between 00:00:00 and 23:59:59 of the day, regardless of the actual time of the timestamp. And when the time jumps by one hour, I am still in the correct day.
Note: I know this is 5 year old thread, and I know this is not an answer to OPs question, but it might help people like me who encountered this page looking for solution to the problem I described.