Recently it came to our attention that our DB was using multiple timezones to record data, and or pass data around, and because of that data started going out of sync. So recently we made some changes to the data (accepting that some of the time data will be misleading or corrupt in the sense that its not 100% accurate. (well the old stuff at least)) That said, we have started making everything we can use GMT as times in/out the problem is now trying to use offsets according to a user defined offset from a predefined list.
So I can query accordingly and display it back to the user accordingly. Lets say I have a user from western america in the pacific time zone, that is an offset of -8, where as if I have someone in the eastern time zone thats a offset of -5
With that dates in and out are currently based on the user preference, where as everything is stored in GMT on the DB so how is it I can pass -N accordingly to something like gmdate() so I can query properly and display properly? I hope this makes sense to someone.
You can't just expect time zones to be a fixed number of hours apart. First off, daylight savings will break that notion, but asides from that, there are a few time zones that aren't hour aligned.
Take a look at this post:
How do you handle timezone difference calculation in PHP?
(There are many similar questions on stackexchange.)
You may want to take a quick look at falsehoods programmers believe about time and a followup with more falsehoods regarding time zones.
Related
I'm curious to know if what I'm considering is bad practice, or if since this is a specific and deliberate choice it is actually a decent idea.
I want to store information for dates for events that occur in specific cities. I want to store that data as UTC timestamps.
Wouldn't it be a good idea to simply store the timestamp and the city id/country id (which is associated with a specific timezone), rather than storing the timezone for each event?
I ask because timezones can change, but city IDs would never change in the DB. Once the server is synced with the latest timezone in the (unlikely) event of a timezone change, the event would be independent and unaffected by that change. However, say a timezone changes its boundaries, then events that occurred in that timezone previously could be outside of it.
Does it seem unwise to do this? I'm just wondering, and I've been scouring for best practices but in this case this actually seems like an OK idea. This works particularly because the application design model would never change- events will ALWAYS be associated with a specific city.
The basic flow would be:
Event data with date/location comes into the system in a standard format like ISO-8601 YYYY-MM-DD string.
System converts date to UTC timestamp and stores the date with the event using that timestamp and the city ID for the event.
When a user requests to view that event, the system pulls the timestamp and city information associated with that event, and uses the city's timezone to format the date accordingly on display.
Is this a terrible idea? Is there a benefit to this, and is the concept of storing the TZ Offset the same idea to eliminate this issue?
Scheduling future time is inherently hard, because you don't know what changes are coming in the future. It is a completely different beast than recording past time.
For past time events, all you really need is the local date and time, and the offset from UTC (many platforms call this a "DateTimeOffset").
But for future events, you don't necessarily know what the offset will be. You can guess what it is based upon your current knowledge of the time zone information, but that information is subject to change. In reality, it changes many times per year as governments of the world change their mind about daylight saving time and other situations.
Because you can't determine the offset reliably, you also cannot determine the exact UTC timestamp. So it's important that you hold on to the original local time. If you're going to calculate a UTC timestamp, you should also recalculate that any time you update your time zone data.
I've already written about this multiple times, (here, here and here). I suggest you read those posts.
Now you brought up one point that I hadn't touched on before, otherwise I'd have marked your question as a duplicate. That is - what if the location of the event moves into a new time zone entirely because the time zone boundaries have changed?
I agree with Deceze, that you need to think about how likely this scenario is and how bad a failure would be. In my opinion, it's probably not worth investing a lot of time to. If you have an event scheduled in the future and that location breaks off into a new time zone, you could always go back and edit the event. You need to ask yourself how much detail your app is expected to know about the time zone changes. Most scheduling systems I've worked with don't handle this aspect.
If it is indeed something that you want to handle, then you will need more than just the city. You should store latitude and longitude coordinates for the location. Then you can use one of these methods to resolve the time zone from those coordinates. But also note that you would want to make sure the source of the time zone boundaries was as up to date as possible.
Also note that the IANA Time Zone database that is the original source for the time zone data, does not keep boundary data at all! Most of the boundary data comes from independent sources, such as Eric Muller's shapefiles, which as of today is aligned with the 2013b data of the IANA database (which is at 2013i), so there have been at least 7 updates to the time zone data that either didn't change any boundaries, or the changes were not tracked.
Whichever way you do it, it will fail in different ways depending on what is changing.
If you store timestamps in the according timezone as 2013-12-29 12:34:56 America/New_York, this will fail if, say, the Bronx suddenly starts their own timezone America/New_York_Bronx with a different offset and your event happened to be in the Bronx.
Decide how likely this is and how bad a failure would be.
If you store timestamps in UTC and the timezone in which the event is happening is redefining their offset (e.g. shifting DST dates around, or entirely shifting to a different offset), the event time may differ from the actual wall clock time at that location. If you store 2013-12-29 12:34:56 UTC for an event at 13:34:56 in Berlin, Germany, and Berlin shifts their DST around, 2013-12-29 12:34:56 UTC may now correspond to 14:34:56 Berlin local time, while the event is still actually happening at 13:34 local time.
Decide how likely this is and how bad a failure would be.
If you store the UTC timestamp and link it to a physical location which you then link to a timezone, you can counteract both problems. But for this you'll have to store the precise physical location, not just "New York", otherwise you just have case 1. with one more intermediate step. If you do store the precise physical location and have a precise way to resolve this location to a timezone and you keep your timezone database up to date, you can handle pretty much all change scenarios.
Decide how practical this is and how much this extra precision is worth to you.
When we were migrating our project from UK hosting to US machines, some of the data that was stored as DATETIME was problematic when we had to display data to clients. We had to chage server configuration for time being but it turned keeping dates as strings or date timestamp in db having them dependent from server configuration is not good idea. since then we do use unix timestamps for any dates in the system and we get rid of problem with timezones once for all. Clients decide to use timezone whatever they want to see. It's much easier to calculate time differences, filter data and so on.
The only problem is to make sure entered data into system it's correctly converted to unix timestamp. That means, user who enters the data should have (in your case) event's timezone when date is converted to timezone, otherwise user will need to calculate timezone on his own.
I'm running into what seems to be a common problem of using a javascript datepicker to display and allow users to select dates from a calendar of available schedule dates for room reservations.
The browser's timezone conversion means that these dates are always off by some margin, so that often when displaying a reserved date to the user, the date will fall on "tomorrow" to the viewer, when the server (the timezone local to the asset or room as stored in the DB) shows them as "today".
I would like the user's browser to ignore the javascript date conversion and just use the actual datetime that is passed in by the database.
However, this even happens on a really simple example like this:
var date = '2013-02-05';
var newdate = new Date(date);
console.log(newdate); // Mon Feb 04 2013 16:00:00 GMT-0800 (PST)
It appears that the date variable is assumed by the browser to be GMT, and when I create a javascript date object from it, it converts that GMT time to my local time.
Is the best practice in this case to use GMT dates in the database, and set the site's local time offset as a variable in the javascript, which can then be used to offset the dates displayed to the end user, and again offset the dates received from the end user for insertion into the database?
This is confusing since there's so many potential pitfalls-- the PHP locale, mysql locale, or browser's locale could all factor into it and mess up the final date. Any advice on ensuring a consistent date value appreciated!
Good question, handling timezones is a mess. Fortunately Javascript has UTC variants of most of its methods. See http://en.wikipedia.org/wiki/Coordinated_Universal_Time
I think the best is to use UTC dates everywhere (except UI maybe). Make sure that server uses and stores UTC dates, and use the javascript UTC methods everywhere. That's the first and most important step, so you know the dates are consistent.
How you present dates in UI is lot less straightforward, and kind of depends on the target audience, application nature etc. I think that's more a subject of discussion, and kind of unanswerable in Stack Overflow (there are other forums for subjective pondering).
I'd say don't trust browser or any kind of geolocation to autoconvert timezones; it should be user configurable, or maybe group/project/install specific setting. Some software are mainly used inside one timezone, and trying to automatically convert timezones might be confusing and annoying for users, if they are used to communicate in some "standard" time. Should user see times in different timezone if he or she is travelling? Sometimes it makes sense, sometimes not, but at least make sure the user knows what logic you are following, both when reading and inputting times.
I develop project management software where it's important these things are handled unambiguously bot by the application, and by users (!). My approach is to always force UTC both for display and input. It's also clearly visible in each date that it is UTC. Handling a variety of timezones would get hairy quick, and I decided it's better to just not do that. I have some helpers in parts of the UI that show the same info in users local timezone etc in fine print. There is a project-wide setting to "hide all timezone-related stuff, and force timezone x", that can be used in smaller projects that are known to never cross timezone boundaries, there is an agreement to use certain zone, or it's just a given and its better to not bother users with that kind of complexity.
EDIT: I should add that, as an example how hairy this can get, sometimes a time is only a time. In some contexts, an event at 15:00 could mean it happens at 15:00 at different places, at their respective local timezones. Uhh....
I plan to capture the start and end of user initiated activities on my website using time() in php. I'm not sure if this is the best way to capture start/end times. Anyway, the data will be stored in MySQL, but again I'm not sure what datatype I should use.
Based on the answers I've read on stackoverflow, the datatype used depends on the purpose of the application.
Purpose of the application
At it's simplest, I want to record start, stop (and duration) of an
activity. Probably using time().
At it's most complicated I'd like to plot statistics based on when
the user did a certain activity, how much time they spent doing the
activity (in total), and when they were the most successful/least
successful etc, etc. (all based on the start/end times) Something to
keep in mind. The users will be from all over the world.
MORE INFO
If an activity is repeated a new record will be made for it. Records will not be updated.
At first, I had planned on storing unix timestamps in MySQL (as an integer datatype?), but from what I understand this is a bad idea, because I will lose a lot of MySQLs ability to process the information. If I store the information as DATETIME, but then move the server, all the times will change based on the local time of the server. Something I found confusing was that TIMESTAMP in MySQL is not the same as a unix timestamp- which is what I would be getting if I used time().
I'm aware that the unix timestamp can only hold dates up to 2038 for some systems, but that isn't a concern (at the moment).
Question: Should I use time() to capture start and end times for user initiated activities? Based on the purpose of the application, what datatype should I use to store the start and stop of user initiated activities?
THANKS
Thanks for the answers everyone. TBH I'm not convinced either way yet, so I'm still doing some research. I chose the TIMESTAMPS option because I really would like to store my information using UTC (GMT). It's a pity though that I will lose out on some of MySQLs inbuilt time functions. Anyway thanks again for your answers.
If you're going worldwide, MySQL's TIMESTAMP is almost universally a good choice over DATETIME, since it stores the time as UTC instead of local time so DST changes won't cause you problems if analyzing in multiple time zones.
Having a non DST changing time zone as a base can be a life saver, converting between multiple time zones with different DST changeover dates can really cause problems, consider for example having a timestamp during the hour that happens twice in a change from summer- to winter time.
Use DATETIME to store the time and use date('Y-m-d H:i:s') to get the current time to store it. When you fetch this value, you will get the time in this format.
To convert it to a timestamp, use $timestamp=strtotime($fetchedValue) To display this in another format use date('H:i',$timestamp). Read about formats from date manual of php
TIMESTAMP can only store values after Jan 1 1970, since it stores timezone data.
So if you are trying to store a date before Jan 1 1970, its better to use DATETIME.
Frankly, TIMESTAMP is useful only if you are actively syncing raw data between two machines with different timezone
I need some guidance with one of my project requirements, I am developing an application which has to deal with various time zones.
Scenario:
User 1 is from India, so his time zone would be GMT+05:30
User 2 is from UK, so his time zone would be GMT+01:00
If the User 1 sends a message to User 2, I want to show the Message Sent/Received Time as per the user’s time zone. For example User 1 sends a message at 6:30 Indian time, when User 2 would view the message it would show as 2:00 UK time.
Here goes my question, whenever I save the message should I convert it to GMT+00, so all my base times stamps are the same and then later when I display the message, I convert it back to User specific time zone. Would this be complex? Is this the best way of doing this?
I like to get views for both saving and displaying, also when I should do the time conversion from optimization point of view. I would need do deal with any/all timezones.
I am developing this application with PHP and MySQL and I am aware of timezone conversion method come with both PHP and MySQL.
I am just trying to figure out the best way of doing this. Look forward to have all valuable suggestions.
Note : As of now I am not much worried with day/light savings.
Thanks
Ravi
When storing the messages, convert, use and store them in a DST neutral timezone - UTC or GMT, for example.
With them store the original timezone offset and the DST offset - this will help with business logic.
When displaying the message, use this data and convert to the local time.
See this question and answers for best practices regarding working with different time zones.
Here goes my question, whenever I save the message should I convert it to GMT+00, so all my base times stamps are the same and then later when I display the message, I convert it back to User specific time zone. Would this be complex? Is this the best way of doing this?
It is. The only caveat is to allow each user to specify their desired timezone to view times in (either as a preference or from their client system's settings or from user database).
I like to get views for both saving and displaying, also when I should do the time conversion from optimization point of view. I would need do deal with any/all timezones.
The best time to convert from client to GMT is in your "business" logic - basically, whatever logic you have which processes web form data. NOT on the database side.
The best time to convert from GMT to client for viewing is right before/in the presentation layer, e.g. when you're printing your HTML.
The reason is that this way, as little of the code as possible needs to worry about timezones.
NOTE This timezone related logic becomes VERY VERY complicated if/when you need to do any date-specific logic (e.g. aggregate based on date as opposed to merely print timestamp to the user).
When saving data to your database, use something like:
INSERT INTO TableName SET TheDateFile=UTC_TIMESTAMP()
And then, presuming that you allow each user to specify their own timezone, you will need to convert the date in your business logic (as mentioned by DVK)
Something like:
DATE_FORMAT(DATE_ADD(TheDateField, INTERVAL 2 HOUR) // or whatever value
I have not done this myself, but I guess it would be simple enough to store the hour interval as per the users timezone.
In any case if you deal with users which are on different timezones, you must separate the timezone from the date. I heard long time ago that wordpress did it the same way:
First step, determine timezone of the date
Second step, set the timezone of the date to GMT (+0)
Third step, store the timezone and the GMT date in two separate columns.
Anyway, if you need more informations, there's a post here
The task:
Time zone chooser widget that allows site visitors to choose their time zone should be generated and populated with reasonable data. It should offer choices like this:
GMT-11 (HH:MM actual time in that zone)
GMT-10 (...)
GMT-1 (...)
... now for the tricky part:
GMT (...)
GMT+1 (...)
... and then all the way until +12.
When user chooses different time zone, server should be able to figure out what time zone to set. This second part isn't much of a problem, once list is populated with correct data.
The problem:
Tricky part is getting actual timezones which are currently in use world-wide. I know how to get entire list of timezones, and how to get their offsets and current DST settings. But, each time zone has two variants: regular time and DST time.
In above list I have marked certain "tricky" part. This is where time zones like Europe/London are. During regular time, Europe/London = GMT, but during DST, Europe/London = GMT+1... Can you see a problem?
What troubles me also is that not all time zones change from DST to regular time and vice-versa, so I can't just populate list with generic data like -11 = GMT-11, -10 = GMT-10, ... and then just apply some logic to locate time zone with appropriate time difference and then set that time zone as a default from that point on.
I'm interested to hear how other sites, that utilize this time zone switching feature, come up with a list of time zones and a logic to choose right time zone, depending on user's choices? Something else you'd like to add or ask?
The most commonly used source for time zone data is the tz/zoneinfo database - it's publically available for free.
As you have noticed yourself, a timezone is an political/administrative entity consisting of a base offset, DST switching dates, and even historical data about changes in these, which you need to correctly deal with historical dates (such as six and a half minutes that officially don't exist).
The tz database (and most apps that use it) therefore represent a timezone not through a base offset, but through the name of the continent and a large City that uses it, often the capital of the country that uses that particular time zone. Examples: "Europe/Berlin" or "Australia/Darwin".
Have a look at what Google Calendar does. It displays the standard time offset as well as the city name, e.g.
(GMT+01:00) Tunis
That seems a pretty reasonable way of representing them, too me. On the other hand, I'm aware that some UIs list the current offset from UTC along with the city. Either way you'll confuse people, to be honest.