I have a service where users can create events and participate in said events. On event creation, users can assign the start, end dates and times and then fill in the timezone that the event is going to take place in. Upon completion they can invite other users to participate in the event.
When members visit the event page a piece of backend logic checks current time against the event's start time. If the current time (of the user/user's timezone) is within the start and end time of the event, the user will then be able to attend the event (giving them access to a hallway page where they can video conference).
My issue is with the comparison of dates with different timezones. I'm saving all the dates with the default UTC timezone. Example of a start date being
date: 2019-07-27 11:00:00.0 UTC (+00:00)
And then comparing $now with it. $now being the below for example
date: 2019-07-27 18:11:18.747797 Asia/Beirut (+03:00)
$now is being assigned as such
$now = Carbon::now()->tz($userProfile->timezone->key);
But when I do a comparison such as
$now->lte($event->ends_at)
I'm getting a wrong evaluation where even if $now is later than $event->ends_at it is still evaluating as true. Is it a matter of evaluating dates with different timezones?
lte()rely on<=` PHP operator as PHP do it for DateTime objects and it compares naively only numbers. So you should compare only dates in the same timezone (ideally on UTC dates).
$now->copy()->tz('UTC')->lte($event->ends_at->copy()->tz('UTC'))
The ->copy() calls are not needed if you're working with CarbonImmutable (recommended).
I'm creating a calendar that should supoprt both all-day events, and start/end times. The website also has to deal with the user's timezone.
My questions is:
- Events with a start/end time should be localized to the user's timezone, the day of the event may change, and that's the expected functionality.
- All day events should ALWAYS be the same date for all users.
I'm storing the dates in the database as UNIX timestamps, and I'm having trouble with dealing with both of these requirements gracefully.
Thanks!
You can store a single UTC datetime value and adjust it according to (1) user's timezone (See CONVERT_TZ) and (2) an indicator on the event indicating if it is all-day or not. If not all-day, convert the datetime to a date (without timezone conversion), otherwise apply the timezone conversion mentioned above. If you want to save on database fields, use a second datetime for the end, leaving it null for all-day events.
In my "tool box" i'm using this function:
function dataAttuale() {
$now = new DateTime();
$dataAttuale = $now->format(DateTime::ISO8601);
$offset = $now->getOffset();
date_default_timezone_set('UTC');
$nowUTC = new DateTime();
$dataUTC = $nowUTC->format(DateTime::ISO8601);
$orario = array();
$orario['dataAttuale'] = $dataAttuale;
$orario['dataUTC'] = $dataUTC;
$orario['offset'] = $offset;
return $orario;
}
I get this array
Array
(
[dataAttuale] => 2013-10-18T11:03:52+0200
[dataUTC] => 2013-10-18T09:03:52+0000
[offset] => 7200
)
So i could save in a datetime MySql field a datetime referred to UTC.
Now, i've some trouble about this.
1) I would save also offset (in seconds). What's best Mysql field? I think max seconds can be +14hour * 60 * 60 = 50400 and -12hours*60*60 = -43200
2) Do you think is notable save also offset? I.e., for example, several API services return a date in UTC + offset...
Thank you very much!
UPDATE:
Thank you to both people. Now i'm saving in MySQL datetime in UTC format and varchar timezone. With a couple of code I'm getting what I want:
$orario = new DateTime($value['creazione'], new DateTimeZone($value['timezone']));
$orario = $orario->format(DateTime::ISO8601);
The output is (for Europe/Rome)
2013-10-19T09:27:54+0200
And for America/Montreal
2013-10-19T09:29:16-0400
And for Australia/Melbourne
2013-10-19T09:30:31+1100
(difference of minutes//seconds it the time to change in my PHP scripts the default Timezone).
Now I think that:
1) I can laugh about Y2038 bug, abandoning (sigh :( ) timestamp :(
2) I can safely travel around the world and use my own Calendar (naaaa... i'll use forever Google Calendar, of course)
It doesn't make a lot of sense to save the offset. There are two possible values you can be interested in with a timestamp:
the general global timestamp, e.g. "the point in time in this world at which it was 12:52am on Sept. 6 2013 UTC"
the specific local time of some point in time, e.g. "17:34 on Dec. 19th 2012 in Manila, Philippines"
Notice that both of these are actually the same thing, they express a point in time in the notation of wall clock time and date at a specific location or timezone. The only difference is that UTC is a specified standard "location" relative to which other timezone offsets are expressed; but there's no reason Manila in the Philippines couldn't be used for the same purpose.
So when you want to store an absolute timestamp, you either:
decide that all your times are stored in a specific timezone like UTC and simply store that timestamp
decide that you are interested in a specific local time and store the timestamp and its timezone
Either way you need the timestamp and you need to know which timezone it's in. In 1. you decide in advance that all timestamps are in the same defined timezone and don't need to store it, in 2. you explicitly save that timezone information.
An offset is not a good thing to store, because it varies throughout the year. The offset in summer may be +6 hours to UTC, but in winter may be +7. If you need to do date calculations on a localized time later on, an offset is misleading and doesn't help you much. If you know the timezone you're talking about, you can get the offset for any time of the year later on.
MySQL doesn't support a DATETIME + TIMEZONE field (Postgres for example does), so you need to store the timezone (e.g. "Europe/Berlin") in a separate text field. If you don't need to associate a timestamp with a specific location at all, then there's no need for a timezone and you just need to store the normalized timestamp, e.g. normalized to UTC.
MySQL is award of timezones (it does not store the timezone with the date, but it converts it to a normalized format), so most of the time you do not need to have an additional field with the offset.
You just need to make sure that you set the correct time_zone for your connection.
So if you have a date and you want to store it in your database you have different possibilities:
You can use SET time_zone = timezone; for your connection. Way you tell MySQL that the date you send or receive from MySQL should be in the give timezone. MySQL will internally convert it to a normalized format.
If you want to insert dates that have different timezones then set for the time_zone then you could use CONVERT_TZ(dt,from_tz,to_tz). from_tz is the timezone of your date, to_tz the one that is set for your connection.
There are for sure situations where the timezone could matter. If that is true for your case is not exactly clear out of your question.
I am trying to integrate a timezone system in my app, i've really tried hard on avoiding making timezone-aware apps upto now - but its a mandatory requirement now so got no choice. TimeZones it just goes over my head. I've read several topics on PHP.net and also other sites including but not limited to SO. But i never could get the hang of it.
So i was wondering if some one can help me out here :( What i'm looking to make is a preference option in my app to allow users to choose their own timezones from a select menu but the app should also be able to SET/Choose the DST accordingly itself for each user.
Please i'm sure this will help others who are still striving to get the hang of the timezones, so please provide as much detailed explanation as possible, even if you have to consider me a complete dumbo/noob.
Edit for bounty:
I am adding a bounty to this question because I really thing we need a good canonical question about time zones when writing PHP/MySQL apps (thus I'm also adding the MySQL tag). I have found things from many places, but it would be good to have it all together. Charles' answer is great, but I still feel it's lacking somewhat. Here are some things I thought of:
How to store the times in the database from a PHP DateTime object
Should they be stored in DATETIME or TIMESTAMP? What are the benefits or caveats for each?
Do we ever need to worry about time zones with MySQL DATE?
How to insert values using NOW(). Do these need to be converted somehow either before or after the insert?
Is it necessary to set the time zone used by MySQL? If so, how? Should it be done persistently or upon every HTTP request? Does it have to be set to UTC or can it be anything else? Or is the server's time sufficient?
How to retrieve values from MySQL and convert them to a DateTime object. Will putting it straight into DateTime::__construct() suffice or do we need to use DateTime::createFromFormat()?
When to convert to local time and why. Is there ever a time that we would want to convert it before it is echoed back to the user (e.g. to compare to another DateTime object or a static value)?
Is there ever a time we need to worry about Daylight Savings Time (DST)? Why or why not?
What should someone do if they have previously inserted data (e.g. using NOW()) without worrying about the time zone to make sure everything stays consistent?
Anything else you think of that someone should look out for
If possible, try to separate it into logical sections to make it easier for future users to find the information. Be sure to provide code examples where necessary.
This answer has been updated to accomodate the bounty. The original, unedited answer is below the line.
Almost all of the question points added by the bounty owner are in relation to how MySQL and PHP datetimes should interact, in the context of timezones.
MySQL still has pathetic timezone support, which means that the intelligence has to be PHP-side.
Set your MySQL connection timezone to UTC as documented in the link above. This will cause all datetimes handled by MySQL, including NOW(), to be handled sanely.
Always use DATETIME, never use TIMESTAMP unless you very expressly require the special behavior in a TIMESTAMP. This is less painful than it used to be.
It's ok to store the Unix epoch time as an integer if you have to, such as for legacy purposes. The epoch is UTC.
MySQL's preferred datetime format is created using the PHP date format string Y-m-d H:i:s
Convert all PHP datetimes to UTC when storing them in MySQL, which is a trivial thing as outlined below
Datetimes returned from MySQL can be handed safely to the PHP DateTime constructor. Be sure to pass in a UTC timezone as well!
Convert the PHP DateTime to the user's local timezone on echo, no sooner. Thankfully DateTime comparison and math against other DateTimes will take into account the timezone that each is in.
You're still up to the whims of the DST database provided with PHP. Keep your PHP and OS patches up to date! Keep MySQL in the blissful state of UTC to remove one potential DST annoyance.
That addresses most of the points.
The last thing is a doozy:
What should someone do if they have previously inserted data (e.g. using NOW()) without worrying about the time zone to make sure everything stays consistent?
This is a real annoyance. One of the other answers pointed out MySQL's CONVERT_TZ, though I'd personally have done it by hopping between server-native and UTC timezones during selects and updates, 'cause I'm hardcore like that.
the app should also be able to SET/Choose the DST accordingly itself for each user.
You don't need to and should not do this in the modern era.
Modern versions of PHP have the DateTimeZone class, which includes the ability to list named timezones. Named timezones allow the user to select their actual location, and have the system automatically determine their DST rules based on that location.
You can combine DateTimeZone with DateTime for some simple but powerful functionality. You can simply store and use all of your timestamps in UTC by default, and convert them to the user's timezone on display.
// UTC default
date_default_timezone_set('UTC');
// Note the lack of time zone specified with this timestamp.
$nowish = new DateTime('2011-04-23 21:44:00');
echo $nowish->format('Y-m-d H:i:s'); // 2011-04-23 21:44:00
// Let's pretend we're on the US west coast.
// This will be PDT right now, UTC-7
$la = new DateTimeZone('America/Los_Angeles');
// Update the DateTime's timezone...
$nowish->setTimeZone($la);
// and show the result
echo $nowish->format('Y-m-d H:i:s'); // 2011-04-23 14:44:00
By using this technique, the system will automatically select the correct DST settings for the user, without asking the user whether or not they're currently in DST.
You can use a similar method to render the select menu. You can continually reassign the time zone for the single DateTime object. For example, this code will list the zones and their current times, at this moment:
$dt = new DateTime('now', new DateTimeZone('UTC'));
foreach(DateTimeZone::listIdentifiers() as $tz) {
$dt->setTimeZone(new DateTimeZone($tz));
echo $tz, ': ', $dt->format('Y-m-d H:i:s'), "\n";
}
You can greatly simplify the selection process by using some client-side magic. Javascript has a spotty but functional Date class, with a standard method to get the UTC offset in minutes. You can use this to help narrow down the list of likely timezones, under the blind assumption that the user's clock is right.
Let's compare this method to doing it yourself. You'd need to actually perform date math every single time you manipulate a datetime, in addition to pushing a choice off on the user that they aren't going to really care about. This isn't just sub-optimal, it's bat-guano insane. Forcing users to signify when they want DST support is asking for trouble and confusion.
Further, if you wanted to use the modern PHP DateTime and DateTimeZone framework for this, you'd need to use deprecated Etc/GMT... timezone strings instead of named timezones. These zone names may be removed from future PHP versions, so it'd be unwise to do that. I say all of this from experience.
tl;dr: Use the modern toolset, spare yourself the horrors of date math. Present the user with a list of named time zones. Store your dates in UTC, which won't be impacted by DST in any way. Convert datetimes to the user's selected named time zone on display, not earlier.
As requested, here's a loop over the available time zones displaying their GMT offset in minutes. I selected minutes here to demonstrate an unfortunate fact: not all offsets are in whole hours! Some actually switch half an hour ahead during DST instead of a whole hour. The resulting offset in minutes should match that of Javascript's Date.getTimezoneOffset.
$utc = new DateTimeZone('UTC');
$dt = new DateTime('now', $utc);
foreach(DateTimeZone::listIdentifiers() as $tz) {
$local = new DateTimeZone($tz);
$dt->setTimeZone($local);
$offset = $local->getOffset($dt); // Yeah, really.
echo $tz, ': ',
$dt->format('Y-m-d H:i:s'),
', offset = ',
($offset / 60),
" minutes\n";
}
How to store the times in the database from a PHP DateTime object
The SQL-92 standard specified that temporal literals should be passed in SQL using a suitable data-type keyword (e.g. TIMESTAMP for date/time values) followed by a string representation of the value (containing an optional timezone offset if non-default).
Sadly, MySQL is not compliant with this part of the SQL standard. As documented under Date and Time Literals:
Standard SQL permits temporal literals to be specified using a type keyword and a string.
[ deletia ]
MySQL recognizes those constructions and also the corresponding ODBC syntax:
[ deletia ]
However, MySQL ignores the type keyword and each of the preceding constructions produces the string value 'str', with a type of VARCHAR.
The documentation goes on to describe the literal formats which MySQL supports and, notably, explicit timezone offsets are absent. There is a feature request to fix this that is now over seven years old and which does not look likely to be introduced any time soon.
Instead, one must set the session's time_zone variable prior to exchanging date/time values between server and client. Therefore, using PDO:
Connect to MySQL:
$dbh = new PDO("mysql:dbname=$dbname", $username, $password);
$dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, FALSE);
Set the session time_zone to that of the DateTime object:
$qry = $dbh->prepare('SET SESSION time_zone = ?');
$qry->execute([$datetime->format('P')]);
Produce a suitable literal from the DateTime object and pass to MySQL as normal (i.e. as a parameter to a prepared statement).
As described in the documentation, there are a number of possible literal formats that one can use. However, I'd suggest using a string in 'YYYY-MM-DD hh:mm:ss.ffffff' format (note that fractional seconds will be ignored in versions of MySQL prior to 5.6), as it is the closest to the SQL standard; indeed one could prefix the literal with the TIMESTAMP keyword to ensure that one's SQL is portable:
$qry = $dbh->prepare('
UPDATE my_table
SET the_time = TIMESTAMP ?
WHERE ...
');
$qry->execute([$datetime->format('Y-m-d H:i:s.u')]);
Should they be stored in DATETIME or TIMESTAMP? What are the benefits or caveats for each?
PHP DateTime objects should always be stored in TIMESTAMP type columns.
The most fundamental difference is that TIMESTAMP stores timezone information (by storing the value in UTC and converting to/from as required by the time_zone variable above), whereas DATETIME does not. Thus TIMESTAMP is useful for representing a specific moment in time (analogous to PHP DateTime objects), whereas DATETIME is useful for representing the time that is seen on a calendar/clock (as in a photo).
As documented under The DATE, DATETIME, and TIMESTAMP Types:
The DATETIME type is used for values that contain both date and time parts. MySQL retrieves and displays DATETIME values in 'YYYY-MM-DD HH:MM:SS' format. The supported range is '1000-01-01 00:00:00' to '9999-12-31 23:59:59'.
The TIMESTAMP data type is used for values that contain both date and time parts. TIMESTAMP has a range of '1970-01-01 00:00:01' UTC to '2038-01-19 03:14:07' UTC.
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.) By default, the current time zone for each connection is the server's time. The time zone can be set on a per-connection basis. As long as the time zone setting remains constant, you get back the same value you store. If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored. This occurs because the same time zone was not used for conversion in both directions. The current time zone is available as the value of the time_zone system variable. For more information, see Section 10.6, “MySQL Server Time Zone Support”.
The TIMESTAMP data type offers automatic initialization and updating to the current date and time. For more information, see Section 11.3.5, “Automatic Initialization and Updating for TIMESTAMP”.
Note the final paragraph, which often catches out newcomers to MySQL.
It may also be worth adding that, as documented under Data Type Storage Requirements, DATETIME values require 8 bytes for storage whereas TIMESTAMP values only require 4 bytes (the underlying data storage format can be found in Date and Time Data Type Representation).
Do we ever need to worry about time zones with MySQL DATE?
It is only meaningful for a time to be sensitive to timezone. By definition, a date alone is universally the same irrespective of one's timezone and therefore there is no need to "worry about time zones" when using MySQL's DATE data type.
The corollary to this is that, if one has a value that is sensitive to timezone, one must also store its time e.g. in a TIMESTAMP column: using a DATE column causes irreversible loss of significant information.
How to insert values using NOW(). Do these need to be converted somehow either before or after the insert?
As documented under NOW():
Returns the current date and time as a value in 'YYYY-MM-DD HH:MM:SS' or YYYYMMDDHHMMSS.uuuuuu format, depending on whether the function is used in a string or numeric context. The value is expressed in the current time zone.
Since "the value is expressed in the current time zone" and that same "current time zone" will be used in evaluating date/time values, one does not have to worry about time zone when using MySQL's NOW() function (or any of its aliases). Therefore, to insert a record:
INSERT INTO my_table (the_time) VALUES (NOW());
Note that, as mentioned above, MySQL's automatic initialisation of TIMESTAMP columns makes redundant most attempts to use NOW() during record insertion/update.
Is it necessary to set the time zone used by MySQL? If so, how? Should it be done persistently or upon every HTTP request? Does it have to be set to UTC or can it be anything else? Or is the server's time sufficient?
This is already addressed above. One can set MySQL's time_zone variable globally, if so desired and thus avoid having to set it upon every connection. See MySQL Server Time Zone Support for more information.
How to retrieve values from MySQL and convert them to a DateTime object. Will putting it straight into DateTime::__construct() suffice or do we need to use DateTime::createFromFormat()?
As documented under Compound Formats, one of the date/time formats recognised by the parser that PHP uses in DateTime::__construct() is MySQL's output format.
However, since the MySQL output format does not include the timezone, one must be sure to furnish the DateTime constructor with that information through its optional second argument:
$qry = $dbh->prepare('SET SESSION time_zone = ?');
$qry->execute([$timezone->getName()]);
$qry = $dbh->query('SELECT the_time FROM my_table');
$datetime = new DateTime($qry->fetchColumn(), $timezone);
Alternatively, one can have MySQL convert the time to a UNIX timestamp and construct the DateTime object from that:
$qry = $dbh->query('SELECT UNIX_TIMESTAMP(the_time) FROM my_table');
$datetime = new DateTime($qry->fetchColumn());
When to convert to local time and why. Is there ever a time that we would want to convert it before it is echoed back to the user (e.g. to compare to another DateTime object or a static value)?
I'm not sure what you mean by "local time" (local to whom? the RDBMS? the webserver? the webclient?), but comparisons between DateTime objects will handle timezone conversions as necessary (PHP stores the values internally in UTC and only converts for output).
Is there ever a time we need to worry about Daylight Savings Time (DST)? Why or why not?
Generally speaking, if you follow the methodology given above, the only concern for DST is when ensuring that values are rendered to the user in the timezone that they expect.
What should someone do if they have previously inserted data (e.g. using NOW()) without worrying about the time zone to make sure everything stays consistent?
As mentioned above, use of NOW() should never cause problems.
If literal values have been inserted into a TIMESTAMP column whilst the session's time_zone variable was set to an incorrect value, one will need to update those values accordingly. MySQL's CONVERT_TZ() function may prove helpful:
UPDATE my_table SET the_time = CONVERT_TZ(the_time, '+00:00', '+10:00');
How to store the times in the database from a PHP DateTime object
Should they be stored in DATETIME or TIMESTAMP? What are the benefits
or caveats for each?
* UPDATE, clarify my first paragraph*
You can also store a timestamp as an INT. The advantage is that you know in which timezone you have stored your value since timestamp is the current time measured in the number of seconds since the Unix Epoch (January 1 1970 00:00:00 GMT).
see php doc: http://php.net/manual/en/function.time.php
Using a 64 bits operating system, you should not have to worries about the year 2038 issue:
http://en.wikipedia.org/wiki/Year_2038_problem
Timestamp are a lot easier to use to compare date times and more fun to use in objet and array. You could easily use them as keys for your arrays for example.
Do we ever need to worry about time zones with MySQL DATE?
In MySQL, the CURRENT_TIMESTAMP(), CURRENT_TIME(), CURRENT_DATE(), and FROM_UNIXTIME() functions return values in the connection's current time zone, which is available as the value of the time_zone system variable. In addition, UNIX_TIMESTAMP() assumes that its argument is a datetime value in the current time zone.
http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html
How to insert values using NOW(). Do these need to be converted
somehow either before or after the insert?
If you use timestamp, you can rely on PHP function, it is just an integer.
If you use date time, the function curdate allows you to have the current date.
http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html#function_curdate
Is it necessary to set the time zone used by MySQL? If so, how? Should
it be done persistently or upon every HTTP request? Does it have to be
set to UTC or can it be anything else? Or is the server's time
sufficient?
see http://dev.mysql.com/doc/refman/5.0/en/time-zone-support.html
How to retrieve values from MySQL and convert them to a DateTime
object. Will putting it straight into DateTime::__construct() suffice
or do we need to use DateTime::createFromFormat()?
Again, if you use timestamp, it is easier. You know the timestamp timezone, you know the
When to convert to local time and why. DST is easy to manage with timestamp, see functions around timestamp :
http://php.net/manual/en/function.mktime.php
Is there ever a time that we would want to convert it before it is
echoed back to the user (e.g. to compare to another DateTime object or
a static value)?
I think again, timestamp let you works with your date to compare them, extract whatever you need and print what you want.
Is there ever a time we need to worry about Daylight Savings Time
(DST)? Why or why not? What should someone do if they have previously
inserted data (e.g. using NOW()) without worrying about the time zone
to make sure everything stays consistent?
Yes, you should worrie about if you have to create appointment or meeting in an application. I developed two applications, one for clinical appointment and one for workshops appointment that support more than 70000 accounts and huge amounts of record. I stick with timestamp, it is super eady to index, manipulate, compare. The print part comes only on the view.
There are advantages to use datetime in your database. If you have to analyse data from the table in sql direct, it is a lot easier to read, it is more 'human readable'.
I am not sure there will be a fixed answer for this post, since it depends on your needs. Timestamp are very easy to manipulate for the operations (a pragmatical approach). The way you store it depends on your preference, since you can store a date and still convert it to timestamp later. But the timezone is part of the timestamps definition from what I understand.
I've been studying the differences in usage between MySQL's DATETIME and TIMESTAMP. It seems that it's pretty straight forward with DATETIME. I would use the following procedure:
Choose the default timezone for all dates, such as UTC.
Let user select a date from drop-down.
Create new PHP DateTime object with the chosen date, using the user's timezone settings, such as EST.
Convert the object to UTC, and insert into database.
On another page, retrieve datetime value and make a new DateTime object with it, using UTC timezone.
Convert object to user's local time (EST), and display to him.
Now, it seems that mysql's TIMESTAMP column type can help eliminate all of these conversions.
What would the above steps look like with the TIMESTAMP column type?
Would I need to do "SET time_zone = timezone;" in the beginning of each pageload to set the timezone to the location of the user?
Would it be easier to ONLY use one type of date column type per database? If not, it may require two different sets of functions to produce the right date.
Should TIMESTAMP only be used in columns not intended to be shown to the public (so as not to deal with formatting)? Like when a row was created, last edited, etc.
I have not tested any of this approach, but it seems pretty straightforward =)
You shouldn't need to convert dates, just set the time zone when you
read/write from dB to get everything right.
Yes, you will have to set right time zone after connection to dB is made.
You mean to only use datetime or timestamp? It really depends on how you intend to
use the columns. But there isn't a clear have to do.
Same as above, it isn't wrong formatting your data from the dB, with a timestamp you can return date style strings from the dB so no worries
Traditionally timestamp is associated like you mention, and datetime for other dates.
more on locale/time zone:
MYSQL set timezone in PHP code