I just found out, running a calendar script, that timestamps in PHP has a limit to 2038. What does it really mean? Why is it 2038 instead of 2050 or 2039? Why a limit if timestamps just count seconds from a given date (1970)?
The limit is imposed by the 4 byte signed integers that most C libraries use for representing that count. Quick math (assumes 365 day years, not exactly correct):
2147483648 seconds ~ 68.1 years
This also implies a lower limit of ~1900. Some libraries have started to introduce 64 bit epoch counts, but they are few and far between for the moment.
The maximum value of a signed 32-bit integer is 2,147,483,647. If you add +1 to that, you get -2,147,483,647. 2,147,483,647 seconds from 01-01-1970 00:00:00 is January 19, 2038. If you add one more second, you get a date somewhere in 1902.
due to the limit of INT datatype on 32 bit machine
http://php.net/manual/en/function.mktime.php
From php.net :
"The maximum possible date accepted by mktime() and gmmktime() is dependent on the current location time zone.
For example, the 32-bit timestamp overflow occurs at 2038-01-19T03:14:08+0000Z. But if you're in a UTC -0500 time zone (such as EST in North America), the maximum accepted time before overflow (for older PHP versions on Windows) is 2038-01-18T22:14:07-0500Z, regardless of whether you're passing it to mktime() or gmmktime()."
my guess is that it is stored in a fixed number of bits, which means a limit on how big the timestamp can get. We could do some math to figure it out exactly.
Related
I was thinking of using TIMESTAMP to store the date+time, but I read that there is a limitation of year 2038 on it. Instead of asking my question in bulk, I preferred to break it up into small parts so that it is easy for novice users to understand as well. So my question(s):
What exactly is the Year 2038 problem?
Why does it occur and what happens when it occurs?
How do we solve it?
Are there any possible alternatives to using it, which do not pose a similar problem?
What can we do to the existing applications that use TIMESTAMP, to avoid the so-called problem, when it really occurs?
I have marked this as a community wiki so feel free to edit at your leisure.
What exactly is the Year 2038 problem?
"The year 2038 problem (also known as Unix Millennium Bug, Y2K38 by analogy to the Y2K problem) may cause some computer software to fail before or in the year 2038. The problem affects all software and systems that store system time as a signed 32-bit integer, and interpret this number as the number of seconds since 00:00:00 UTC on January 1, 1970."
Why does it occur and what happens when it occurs?
Times beyond 03:14:07 UTC on Tuesday, 19 January 2038 will 'wrap around' and be stored internally as a negative number, which these systems will interpret as a time in December 13, 1901 rather than in 2038. This is due to the fact that the number of seconds since the UNIX epoch (January 1 1970 00:00:00 GMT) will have exceeded a computer's maximum value for a 32-bit signed integer.
How do we solve it?
Use long data types (64 bits is sufficient)
For MySQL (or MariaDB), if you don't need the time information consider using the DATE column type. If you need higher accuracy, use DATETIME rather than TIMESTAMP. Beware that DATETIME columns do not store information about the timezone, so your application will have to know which timezone was used.
Other Possible solutions described on Wikipedia
Upgrade your Mysql to 8.0.28 or higher
Are there any possible alternatives to using it, which do not pose a similar problem?
Try wherever possible to use large types for storing dates in databases: 64-bits is sufficient - a long long type in GNU C and POSIX/SuS, or sprintf('%u'...) in PHP or the BCmath extension.
What are some potentially breaking use cases even though we're not yet in 2038?
So a MySQL DATETIME has a range of 1000-9999, but TIMESTAMP only has a range of 1970-2038. If your system stores birthdates, future forward dates (e.g. 30 year mortgages), or similar, you're already going to run into this bug. Again, don't use TIMESTAMP if this is going to be a problem.
What can we do to the existing applications that use TIMESTAMP, to avoid the so-called problem, when it really occurs?
Few PHP applications will still be around in 2038, though it's hard to foresee as the web hardly a legacy platform yet.
Here is a process for altering a database table column to convert TIMESTAMP to DATETIME. It starts with creating a temporary column:
# rename the old TIMESTAMP field
ALTER TABLE `myTable` CHANGE `myTimestamp` `temp_myTimestamp` int(11) NOT NULL;
# create a new DATETIME column of the same name as your old column
ALTER TABLE `myTable` ADD `myTimestamp` DATETIME NOT NULL;
# update all rows by populating your new DATETIME field
UPDATE `myTable` SET `myTimestamp` = FROM_UNIXTIME(temp_myTimestamp);
# remove the temporary column
ALTER TABLE `myTable` DROP `temp_myTimestamp`
Resources
Year 2038 Problem (Wikipedia)
The Internet Will End in 30 Years
When using UNIX Timestamps to store dates, you are actually using a 32 bits integers, that keeps count of the number of seconds since 1970-01-01 ; see Unix Time
That 32 bits number will overflow in 2038. That's the 2038 problem.
To solve that problem, you must not use a 32 bits UNIX timestamp to store your dates -- which means, when using MySQL, you should not use TIMESTAMP, but DATETIME (see 10.3.1. The DATETIME, DATE, and TIMESTAMP Types) :
The DATETIME type is used when you
need values that contain both date and
time information. The supported range
is '1000-01-01 00:00:00' to
'9999-12-31 23:59:59'.
The TIMESTAMP data type has a range
of '1970-01-01 00:00:01' UTC to
'2038-01-19 03:14:07' UTC.
The (probably) best thing you can do to your application to avoid/fix that problem is to not use TIMESTAMP, but DATETIME for the columns that have to contain dates that are not between 1970 and 2038.
One small note, though : there is a very high probably (statistically speaking) that your application will have been re-written quite a couple of times before 2038 ^^ So maybe, if you don't have to deal with dates in the future, you won't have to take care of that problem with the current version of your application...
A quick search on Google will do the trick: Year 2038 problem
The year 2038 problem (also known as Unix Millennium Bug, Y2K38 by analogy to the Y2K problem) may cause some computer software to fail before or in the year 2038
The problem affects all software and systems that store system time as a signed 32-bit integer, and interpret this number as the number of seconds since 00:00:00 UTC on January 1, 1970. The latest time that can be represented this way is 03:14:07 UTC on Tuesday, 19 January 2038. Times beyond this moment will "wrap around" and be stored internally as a negative number, which these systems will interpret as a date in 1901 rather than 2038
There is no easy fix for this problem for existing CPU/OS combinations, existing file systems, or existing binary data formats
http://en.wikipedia.org/wiki/Year_2038_problem has most of the details
In summary:
1) + 2) The problem is that many systems store date info as a 32-bit signed int equal to the number of seconds since 1/1/1970. The latest date that can be stored like this is 03:14:07 UTC on Tuesday, 19 January 2038. When this happens the int will "wrap around" and be stored as a negative number which will be interpreted as a date in 1901. What exactly will happen then, varies from system to system but suffice to say it probably won't be good for any of them!
For systems that only store dates in the past, then I guess you don't need to worry for a while! The main problem is with systems that work with dates in the future. If your system needs to work with dates 28 years in the future then you should start worrying now!
3) Use one of the alternative date formats available or move to a 64-bit system and use 64-bit ints. Or for databases use an alternative time stamp format (eg for MySQL use DATETIME)
4) See 3!
5) See 4!!! ;)
Bros, if you need to use PHP to display timestamps, this is the BEST PHP solution without changing from UNIX_TIMESTAMP format.
Use a custom_date() function. Inside it, use the DateTime. Here's the DateTime solution.
As long as you have UNSIGNED BIGINT(8) as your timestamps in database.
As long as you have PHP 5.2.0 ++
I was asking myself thses questions recently and want to share the solution I landed on for new projects.
Bigint
After reading variouse responses to questions like this one I found that storing a Unix timestamp in a Bigint column is a much better solution moving forward.
Bigints range will cover you from before the begining of time till the year 292277026596 wich may as well be called forever.
Pluss:
It uses the same 8bytes of storage that DATETIME uses.
Its timezone agnostic.
You can still use auto generated timestamps via DEFAULT (unix_timestamp())
range is so massive your server will turn into dust before wraparounds can happen, even if storing time in miliseconds.
DECIMAL
This is the solution I landed on as you get a bit more controll.
You can store an overkill date range like bigint or reduce it to something realistic and use less storage.
In my case I also want to store frations of a second as actual fractions, and I still wanted the timestamps gerated on insert.
Heres the column definition from my create table schema:
`created` decimal(18,2) NOT NULL DEFAULT (unix_timestamp(now(2))) COMMENT 'unix timestamp'
Using decimal(18,2) provides an absolutly overkill time range with the same storage as bigint/datetime while showing fractions of a second down to 2 digits.
Storage is based on the number of digits, 9 digits = 4bytes signed or unsigned it doesnt matter.
You could limit the range to something more realistic and use significantly less than datetime or increase the precision to nanoseconds. You decide whats important.
Another advantage is, if in the distant future you hit the limit of your range, Mysql will just tell you.
No wrap around issue will happen, instead you will get an error preventing the insertion and can easily alter the table again to add another digit.
This makes for a perfect sense to me I highly recommend starting new databases with this appraoch.
As I did't want to upgrade anything, I asked my backend (MSSQL) to do this job instead of PHP!
$qry = "select DATEADD(month, 1, :date) next_date ";
$rs_tmp = $pdo->prepare($qry);
$rs_tmp->bindValue(":date", '2038/01/15');
$rs_tmp->execute();
$row_tmp = $rs_tmp->fetch(PDO::FETCH_ASSOC);
echo $row_tmp['next_date'];
May not be an efficient way, but it works.
I have a problem with processing a SQL Server database dump in PHP.
I have one column called datatime with values like :
0x0000a0af00d7f2eb
I need to extract, in PHP, the date and time values of this column. I don't have SQL Server available so I can't use the obvious solution of CAST(0x0000a0af00d7f2eb AS datetime).
Someone has told me that this hex: 0000a0af00d7f2eb is created by 4 bytes of date and 4 bytes of time.
So I know that:
When I will change 0000a0af (first 4 bytes) to decimal I will get number of days from 1900. That works fine.
But when I'm trying to change last the 4 bytes (so there should be time) : 00d7f2eb to decimal I'm getting something which I can't understand. It should be a time from midnight in milliseconds and sometimes this value is ~3 times lower.
Could anyone help in converting 0000a0af00d7f2eb to date? I know that time is between 5 AM and 11 PM, and the day is in last week.
According to the linked article in the other question linked to by Rene, SQL Server stores 3.33 millisecond intervals in the second set of 4 bytes, not milliseconds. So if you're calculating with milliseconds, you will indeed be getting a time about 1/3 of what it should be. Using your example, let's start by converting to decimal
00d7f2eb -> 14152427 3.3ms intervals
Now multiply out by 3.3 to convert to milliseconds, and divide by 1000 to get seconds
14152427 * 3.3 / 1000 ~ 47127.58
So this represents about 47 thousand seconds after midnight. Dividing by 3600 seconds in an hour
47127.58 / 3600 ~ 13.091
So this represents a time of about 13.1 hours after midnight, which agrees with the result of the cast done in SQL Server.
select CAST(0x0000a0af00d7f2eb AS datetime) as t
is working fine for me. and it returns 'August, 16 2012 13:06:14-0700'.
I just found out, running a calendar script, that timestamps in PHP has a limit to 2038. What does it really mean? Why is it 2038 instead of 2050 or 2039? Why a limit if timestamps just count seconds from a given date (1970)?
The limit is imposed by the 4 byte signed integers that most C libraries use for representing that count. Quick math (assumes 365 day years, not exactly correct):
2147483648 seconds ~ 68.1 years
This also implies a lower limit of ~1900. Some libraries have started to introduce 64 bit epoch counts, but they are few and far between for the moment.
The maximum value of a signed 32-bit integer is 2,147,483,647. If you add +1 to that, you get -2,147,483,647. 2,147,483,647 seconds from 01-01-1970 00:00:00 is January 19, 2038. If you add one more second, you get a date somewhere in 1902.
due to the limit of INT datatype on 32 bit machine
http://php.net/manual/en/function.mktime.php
From php.net :
"The maximum possible date accepted by mktime() and gmmktime() is dependent on the current location time zone.
For example, the 32-bit timestamp overflow occurs at 2038-01-19T03:14:08+0000Z. But if you're in a UTC -0500 time zone (such as EST in North America), the maximum accepted time before overflow (for older PHP versions on Windows) is 2038-01-18T22:14:07-0500Z, regardless of whether you're passing it to mktime() or gmmktime()."
my guess is that it is stored in a fixed number of bits, which means a limit on how big the timestamp can get. We could do some math to figure it out exactly.
What is the last Unix time in decimal? I'm going to set it as the expiration date of cookies (using PHP). Is it possible to cause problem?
Sunday, December 4, 292,277,026,596 AD (on 64-bit systems)
For a 32-bit time_t value, you will overflow in mid to late January 2038. Specifically about 3:14 AM UTC on January 19 (actually somewhere between 3:14 and 3:15 but that should be close enough for you).
The maximum value is 231-1 or 2,147,483,647.
2147483647
I was thinking of using TIMESTAMP to store the date+time, but I read that there is a limitation of year 2038 on it. Instead of asking my question in bulk, I preferred to break it up into small parts so that it is easy for novice users to understand as well. So my question(s):
What exactly is the Year 2038 problem?
Why does it occur and what happens when it occurs?
How do we solve it?
Are there any possible alternatives to using it, which do not pose a similar problem?
What can we do to the existing applications that use TIMESTAMP, to avoid the so-called problem, when it really occurs?
I have marked this as a community wiki so feel free to edit at your leisure.
What exactly is the Year 2038 problem?
"The year 2038 problem (also known as Unix Millennium Bug, Y2K38 by analogy to the Y2K problem) may cause some computer software to fail before or in the year 2038. The problem affects all software and systems that store system time as a signed 32-bit integer, and interpret this number as the number of seconds since 00:00:00 UTC on January 1, 1970."
Why does it occur and what happens when it occurs?
Times beyond 03:14:07 UTC on Tuesday, 19 January 2038 will 'wrap around' and be stored internally as a negative number, which these systems will interpret as a time in December 13, 1901 rather than in 2038. This is due to the fact that the number of seconds since the UNIX epoch (January 1 1970 00:00:00 GMT) will have exceeded a computer's maximum value for a 32-bit signed integer.
How do we solve it?
Use long data types (64 bits is sufficient)
For MySQL (or MariaDB), if you don't need the time information consider using the DATE column type. If you need higher accuracy, use DATETIME rather than TIMESTAMP. Beware that DATETIME columns do not store information about the timezone, so your application will have to know which timezone was used.
Other Possible solutions described on Wikipedia
Upgrade your Mysql to 8.0.28 or higher
Are there any possible alternatives to using it, which do not pose a similar problem?
Try wherever possible to use large types for storing dates in databases: 64-bits is sufficient - a long long type in GNU C and POSIX/SuS, or sprintf('%u'...) in PHP or the BCmath extension.
What are some potentially breaking use cases even though we're not yet in 2038?
So a MySQL DATETIME has a range of 1000-9999, but TIMESTAMP only has a range of 1970-2038. If your system stores birthdates, future forward dates (e.g. 30 year mortgages), or similar, you're already going to run into this bug. Again, don't use TIMESTAMP if this is going to be a problem.
What can we do to the existing applications that use TIMESTAMP, to avoid the so-called problem, when it really occurs?
Few PHP applications will still be around in 2038, though it's hard to foresee as the web hardly a legacy platform yet.
Here is a process for altering a database table column to convert TIMESTAMP to DATETIME. It starts with creating a temporary column:
# rename the old TIMESTAMP field
ALTER TABLE `myTable` CHANGE `myTimestamp` `temp_myTimestamp` int(11) NOT NULL;
# create a new DATETIME column of the same name as your old column
ALTER TABLE `myTable` ADD `myTimestamp` DATETIME NOT NULL;
# update all rows by populating your new DATETIME field
UPDATE `myTable` SET `myTimestamp` = FROM_UNIXTIME(temp_myTimestamp);
# remove the temporary column
ALTER TABLE `myTable` DROP `temp_myTimestamp`
Resources
Year 2038 Problem (Wikipedia)
The Internet Will End in 30 Years
When using UNIX Timestamps to store dates, you are actually using a 32 bits integers, that keeps count of the number of seconds since 1970-01-01 ; see Unix Time
That 32 bits number will overflow in 2038. That's the 2038 problem.
To solve that problem, you must not use a 32 bits UNIX timestamp to store your dates -- which means, when using MySQL, you should not use TIMESTAMP, but DATETIME (see 10.3.1. The DATETIME, DATE, and TIMESTAMP Types) :
The DATETIME type is used when you
need values that contain both date and
time information. The supported range
is '1000-01-01 00:00:00' to
'9999-12-31 23:59:59'.
The TIMESTAMP data type has a range
of '1970-01-01 00:00:01' UTC to
'2038-01-19 03:14:07' UTC.
The (probably) best thing you can do to your application to avoid/fix that problem is to not use TIMESTAMP, but DATETIME for the columns that have to contain dates that are not between 1970 and 2038.
One small note, though : there is a very high probably (statistically speaking) that your application will have been re-written quite a couple of times before 2038 ^^ So maybe, if you don't have to deal with dates in the future, you won't have to take care of that problem with the current version of your application...
A quick search on Google will do the trick: Year 2038 problem
The year 2038 problem (also known as Unix Millennium Bug, Y2K38 by analogy to the Y2K problem) may cause some computer software to fail before or in the year 2038
The problem affects all software and systems that store system time as a signed 32-bit integer, and interpret this number as the number of seconds since 00:00:00 UTC on January 1, 1970. The latest time that can be represented this way is 03:14:07 UTC on Tuesday, 19 January 2038. Times beyond this moment will "wrap around" and be stored internally as a negative number, which these systems will interpret as a date in 1901 rather than 2038
There is no easy fix for this problem for existing CPU/OS combinations, existing file systems, or existing binary data formats
http://en.wikipedia.org/wiki/Year_2038_problem has most of the details
In summary:
1) + 2) The problem is that many systems store date info as a 32-bit signed int equal to the number of seconds since 1/1/1970. The latest date that can be stored like this is 03:14:07 UTC on Tuesday, 19 January 2038. When this happens the int will "wrap around" and be stored as a negative number which will be interpreted as a date in 1901. What exactly will happen then, varies from system to system but suffice to say it probably won't be good for any of them!
For systems that only store dates in the past, then I guess you don't need to worry for a while! The main problem is with systems that work with dates in the future. If your system needs to work with dates 28 years in the future then you should start worrying now!
3) Use one of the alternative date formats available or move to a 64-bit system and use 64-bit ints. Or for databases use an alternative time stamp format (eg for MySQL use DATETIME)
4) See 3!
5) See 4!!! ;)
Bros, if you need to use PHP to display timestamps, this is the BEST PHP solution without changing from UNIX_TIMESTAMP format.
Use a custom_date() function. Inside it, use the DateTime. Here's the DateTime solution.
As long as you have UNSIGNED BIGINT(8) as your timestamps in database.
As long as you have PHP 5.2.0 ++
I was asking myself thses questions recently and want to share the solution I landed on for new projects.
Bigint
After reading variouse responses to questions like this one I found that storing a Unix timestamp in a Bigint column is a much better solution moving forward.
Bigints range will cover you from before the begining of time till the year 292277026596 wich may as well be called forever.
Pluss:
It uses the same 8bytes of storage that DATETIME uses.
Its timezone agnostic.
You can still use auto generated timestamps via DEFAULT (unix_timestamp())
range is so massive your server will turn into dust before wraparounds can happen, even if storing time in miliseconds.
DECIMAL
This is the solution I landed on as you get a bit more controll.
You can store an overkill date range like bigint or reduce it to something realistic and use less storage.
In my case I also want to store frations of a second as actual fractions, and I still wanted the timestamps gerated on insert.
Heres the column definition from my create table schema:
`created` decimal(18,2) NOT NULL DEFAULT (unix_timestamp(now(2))) COMMENT 'unix timestamp'
Using decimal(18,2) provides an absolutly overkill time range with the same storage as bigint/datetime while showing fractions of a second down to 2 digits.
Storage is based on the number of digits, 9 digits = 4bytes signed or unsigned it doesnt matter.
You could limit the range to something more realistic and use significantly less than datetime or increase the precision to nanoseconds. You decide whats important.
Another advantage is, if in the distant future you hit the limit of your range, Mysql will just tell you.
No wrap around issue will happen, instead you will get an error preventing the insertion and can easily alter the table again to add another digit.
This makes for a perfect sense to me I highly recommend starting new databases with this appraoch.
As I did't want to upgrade anything, I asked my backend (MSSQL) to do this job instead of PHP!
$qry = "select DATEADD(month, 1, :date) next_date ";
$rs_tmp = $pdo->prepare($qry);
$rs_tmp->bindValue(":date", '2038/01/15');
$rs_tmp->execute();
$row_tmp = $rs_tmp->fetch(PDO::FETCH_ASSOC);
echo $row_tmp['next_date'];
May not be an efficient way, but it works.