I'm overwhelmed trying to get this right: We've got servers located across a dozen time zones, with Apache and MySQL running on all or some of them, as well as MySQL hosted on Amazon RDS.
I want to know "Best Practices", or how to otherwise configure each MySQL and PHP installation so that when a row is added to the database from PHP I'm certain that the value there is actually the UTC time when the event happened, regardless of where the server is located when it happened. Presenting it to the user in any given timezone is not an issue - I just want to know that the datetime columns are actually storing the actual moment in time when something occurred.
As it is now, the Web Servers are set to whatever the local timezone is due to scheduled events, etc., and I'm not sure which parts of the puzzle use which settings from where to come up with whatever is written to the database.
I apologize if the question seems unclear, at this point I don't know what I don't know, so getting a precise question is even challenging. Also, all our dates are in the database as DateTime fields, so storing timestamps isn't possible.
If you restrict yourself to MySQLs DATE and DATETIME types, you can largely ignore time zone issues in MySQL itself. You want to avoid MySQL's TIMESTAMP type because:
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.) By default, the current time zone for each connection is the server's time.
https://dev.mysql.com/doc/refman/5.7/en/datetime.html
As far as reading/writing DATETIME values, you'll get back exactly what you put in, which is good.
That leaves you with the problem of always ensuring you are writing UTC values to the database.
The best way to ensure that PHP is using UTC is to explicitly set it in your application using date_default_timezone_set(). That will ensure that calls like date('Y-m-d H:i:s') will give you the UTC value. It will also ensure that something like (new \DateTime('now'))->getTimezone() will return a UTC \DateTimeZone instance.
You should note, of course, that things get significantly more difficult when you're storing date/time values that you take from users. In those cases, you'll need to somehow determine what timezone the user is in, and handle conversion to UTC before persisting the values. Assuming your users have some per-user timezone setting, you basically do something like:
/** #var \DateTimeZone $userTZ */
$userTz = getUserTimezone();
$dateTime = new \DateTime($user_submitted_date_string, $userTz);
$dateTime->setTimezone(new \DateTime('UTC'));
$dateTimeStr = $dateTime->format('Y-m-d H:i:s');
Quite simple, always store unix time in the database (or if you want microsecond accuracy with microtime ). Then regardless of the timezone of each of your webservers, if two of them recieve a request at the same time, it would be the same integer value that is saved in the database (the database field shoudl obviously be an int (or big int for micro time))
And how to display? Easy with javascript.
new Date(unix_timestamp);
This produces a date and time in the user's timezone.
Related
I am working on codeigniter application which have users from different time zones. Each user has some notifications based on some dates calculation.
So what should be best way to store date into Mysql? Either timestamp or datetime?
First off, you need to be aware of the locale setting for your mysql server. You want the server to be set to use UTC as a neutral setting. Any data that is stored in the mysql server will be relative to the server's timezone setting.
Unless you are dealing with future data (beyond the year 2032) or conversely really old data, the most efficient mysql datatype is the timestamp datatype. Just be careful to turn off the automatic mysql timestamp functionality when you don't want or need it.
Internally to PHP you want to utilize the DateTime class which includes ways to convert from any one timezone to another.
The missing ingredient that has not been mentioned, is that in order for this to work for end users, you need to store their timezone, or utilize functions in the browser to read from the operating system the current timezone of their workstation.
Timezone strings can be stored, and then used after you fetch data from the server, to then convert it and show it to them relative to their timezone.
But again to be clear, all data should be stored as UTC, and this requires that the server be configured to utilize UTC. Well it's a bit more complicated than that, but you can save yourself a lot of trouble when you insure there isn't a mismatch. By the same token your web/application servers (and in fact all servers) ought to be set to UTC, and of course to sync their time using NTP. Most cloud based servers are going to do this by default.
Format doesn't matter as long as you keep your date as DataTime Object. Using PHP you can easily manipulate dates. My personal choice is using UTC offset because is easier for debug purpose. You can easier figure out if your time difference calculation is correct looking on the offset then on time zone name.
More information you will find under DataTime class.
Implementation is basically the same regardless format as long as you are using DataTime Object. This is the best way to calculate time differences in different time zones.
With datetime you get a rich set of tools for interacting with the data (including converting between timezones) and the opportunity to handle dates prior to 1st Jan 1970 and after 19th Jan 2038.
Although the same tools are also applicable to TIMESTAMP, the automatic timezone conversion can get messy.
A further consideration is that TIMESTAMP also acquires some subtle timezone conversions which get rather messed up if you move to maxdb mode or back. Indeed, datetime data will be more portable across different systems.
It does not matter as long as you are using UTC value. However, timestamps are better way to store data from different timezones as it always represents and stored as the standard time(UTC) irrespective of the timezone of client/server.
But, as you are taking the data from user's input it does not make a difference.
For the accurate standard time(UTC), you just need to convert it right according to the user's timezone.
You can retrieve user's timezone either from browser headers(which is a less accurate method) or you can ask the user himself(using a input).
I'm working on a PHP(CodeIgniter)/MySQL application that allows users select when their blog post will be published. How I have designed it to work so far is as below:
User sets the timezone in his/her profile.
User sets the publish date of the blog post. This date/time is assumed to be based on user's timezone.
Based on user's timezone, the publish date is converted to UTC (00:00).
The converted date is stored as DATETIME in mysql.
Server frequently converts the stored DATETIME values to server timezone and publishes the content when their time has come.
I was wondering if there is a better way to handle this situation, considering that my DB server and web server are on different machines on the cloud and might change location as well.
Since you are talking about scheduling of future time, storing only UTC isn't necessarily the best approach.
Keep in mind that time zone rules can (and do) change. If you apply an update to your time zone data (in PHP, it's done with PECL's timezonedb package), then any data you already converted to UTC might be invalid.
The better solution is to store multiple values:
The original local date and time
The original time zone (ex. "America/New_York")
The UTC date and time, converted from the local values
When you want to see if it's time to run the task, you'd compare the current UTC date and time against the stored UTC value.
When you apply time zone updates, you throw out the previous converted UTC values for all future entries, and recalculate new ones based on the updated data.
If you don't do this, then you're losing the intent that the user originally provided. And if you don't keep your time zone data updated, then your system won't be aware of various changes happening around the world, such as the changes made in Egypt earlier this year, or the changes coming in October for Russia.
Last point - If implemented properly, the time zone of your server should not matter. Compare UTC to UTC.
I've been looking around for a few hours now about what's the best way to use timezones in a PHP/MySQL web application, finding a definitive answer is hard. From what I've learnt so far it is best to store everyones stuff in the database in UTC (correct me if I am wrong).
When a user registers I will ask them for there timezone and then store that against there user. This would be in this format as a dropdown menu:
<option value="Europe/London">(GMT) Greenwich Mean Time : London</option>
The app I am building will allow users to set arrangements in the future with people (meetings), much like a calendar. Obviously over the course of a year different timezones have different daylight savings periods, any idea how I would cater for this?
Say a user from the UK sets a meeting for 3:00PM on January 24th 2013 and invites someone who lives in California to this meeting, how do I get it so that the American sees that meeting in his/her timezone and the UK user sees it in his/her timezone? (Note that both users are signed up and have set their timezone).
Does anyone have a clear explanation and maybe some examples for this? Or can point me to where I can find that?
Thanks
I dealt with this situation extensively in a PHP/MySQL application I wrote for a private jet operator a little over a year ago. There are different strategies to handle timezones in these two platforms, but I will explain how I did it. I set the MySQL server to UTC and run each PHP script in the timezone that the user specifies during the signup process for the user profile.
MySQL and PHP (PHP 5.2 and above) both have native datetime datatypes. MySQL's datetime is a primitive data type, while PHP 5.2 and above offers the built-in DateTime class. The MySQL datetime datatype does not include metadata for the timezone, but a PHP DateTime object always includes a timezone. If the PHP datetime constructor does not specify the optional timezone in the second argument, then the PHP datetime constructor uses the php environment variable.
Both MySQL and PHP have default timezone set in the configuration files. MySQL uses the datetime set in the config file for each db connection unless the user specifies a different timezone after connection is started with the command SET time_zone = [timezone];. PHP also sets a timezone environment variable for each script using the timezone set in the server config file, and this environment variable can be overriden using the PHP function date_default_timezone_set() after the script starts.
The PHP DateTime class has a property called timezone, which is a PHP DateTimeZone object. The DateTimeZone object is specified using a string for the exact time zone. The list of timezones is comprehensive, having hundreds of individual time zones across the world. The PHP time zones will account for daylight savings time automatically.
When the user generates a datetime in the web app, construct a PHP datetime object in the timezone of the user's profile. Then use the setTimezone method to modify the DateTime object to the UTC timezone. Now you have the user's datetime in UTC, and you can store the value in the database. Use the DateTime format method to express the data as a string in the format accepted by MySQL.
So the user generates a datetime, and you make a PHP datetime object in the user's specified timezone:
// set using an include file for user profile
$user_timezone = new DateTimeZone('America/New_York');
// 1st arg in format accepted by PHP strtotime
$date_object1 = new DateTime('8/9/2012 5:19 PM', $user_timezone);
$date_object1->setTimezone(new DateTimeZone('UTC'));
$formated_string = $date_object1->format('Y-m-d H:i:s');
$query_string = "INSERT INTO `t_table1` (`datetime1`) VALUES('$formated_string')";
When you retrieve the value from the database, construct in UTC and then convert to the user's time zone.
$query_string = "SELECT `datetime1` FROM `t_table1`";
$date_object1 = new DateTime($datetime_string_from_mysql, new DateTimeZone('UTC'));
$date_object1->setTimezone($user_timezone);
$string_for_display_in_application = $date_object1->format('m/d/Y g:i a');
Using this method, your datetime values are always stored in UTC inside the db, and the user always experiences the values in his/her profile's time zone. PHP will correct for Daylight Savings Time if necessary for each time zone.
One gotcha: This explanation does not cover the MySQL timestamp datatype. I recommend using the MySQL datetime datetype to store datetime values, not the timestamp datatype. The timestamp datatype is covered in the manual here.
Edit:
You can produce an array containing every PHP timezone string using listIdentifiers, which is a static method of the DateTimeZone class.
In MySQL, what you need to do is:
Store each user's chosen timezone someplace you can retrieve it when you're doing database queries on behalf of that user. You can store it as a string.
Right before you do work on behalf of a particular user (for example, storing or retrieving appointment times and dates) do SET time_zone = (stored time zone setting)
This will cause time zones to be converted appropriately to each person's local time.
Edit:
This works because
MySQL tries to use UTC (universal time, formerly known as Greenwich Mean Time) to store DATETIME and TIMESTAMP data items in tables.
It can only do this correctly if it knows the correct local time zone for each data item it is given by applications.
In applications that don't care about different time zones, it does this in a MySQL-server-wide way. See https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html . Most people who run multi-national and multi-time-zone applications set their server time zones to UTC, not to local time, because it makes it much easier to keep things sorted out.
It makes very little sense to try to convert a time from UTC to local time unless you also know the date and the timezone, because local time switches on and off at various times of year. Just try to get this right for all three of Israel, Arizona, and New York, I dare you! Israel switches between daylight and standard time on Passover and Rosh Hashanah; Arizona doesn't switch, and New York switches at the whim of the US federal legislature.
There's a session-scope time_zone setting (SET time_zone = something). If you don't set it, it uses the timezone representation in item 3 above. If you set it, the server will use this as the timezone to convert its internal representation to the representation it sends back in queries.
You can get a list of the names of the available time zones in your MySQL server by issuing SELECT Name from mysql.time_zone_name. This can populate the pulldown menu from which your user can select her time zone. If this query returns no items, take a look at bottom of https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html .
So, that means you can set your session time_zone setting to a particular user's time zone, and then get back all times in that user's time zone. Also, any DATETIME or TIMESTAMP items you INSERT or UPDATE will be converted from that user's time zone to MySQL's internal representation as they are placed in your tables.
Be careful: in web applications with persistent MySQL connections, the work for a new user request will inherit the connection's time_zone setting. Don't forget to reset it for the new user.
If you're running a query returning local time data for more than one user, and those users happen to be in different time zones, you can't take advantage of this MySQL per-session feature set. You can work around this by running different queries for different users and changing the time_zone setting between them.
Or, you can use the MySQL function
CONVERT_TZ(datetime,'UTC','user_time_zone')
or similar on each item.
Alternatively, Java and DotNET have their own high-quality time zone manipulation systems. So, you can make the choice of running your MySQL server with the time_zone setting of UTC, and do all your timezone conversions in your application.
I have gone through many timezone/PHP posts, and most suggest storing your datetime fields in UTC, then using the application users timezone offset when storing and displaying datetime information.
The problem i have I've inherited an application that wasn't timezone aware, and now I need to cater for this.
The server is already set to "EST +11:00 Australia/Melbourne", and there are already applications running from that server. So i can't change this.
Fortunately, I do know a users timezone offset, ie -05:00, etc,.
The application takes Javascript Dates and parses them using PHP's strtotime() function and stores in a MySQL database, like this:
$event_starts = date('Y-m-d H:i:s',
strtotime('Thu Dec 02 2010 11:15:00 GMT+1100 (AUS Eastern Daylight Time)');
So does anyone have any suggestions for the best way on how I can make this application timezone aware considering the server isn't set to UTC?
Many thanks, J.
This is not going to be very easy.
First of all, consider that existing stored dates are in local time of your server, which observes daylight saving time. Any code that has to do anything with these dates except just printing them, now or in the future, will need to convert them to UTC first. If the daylight saving rules are not exactly the same at the point in time where the date was stored and the current time (when the conversion is taking place), your server will use the "current" rules and therefore produce a wrong result. Granted, this scenario may be far-fetched in your specific case (or then again it might not), but it's a very strong warning against storing anything other than UTC.
Assuming that the DST rules remain constant, and that you have PHP >= 5.3.0, you can do this:
Read "original" database date with DateTime::createFromFormat, explicitly specifying the timezone (server's TZ)
Convert to user local time with DateTime::setTimezone (specifying user's TZ)
Display to the user
When receiving user input, you will need to do the reverse:
Create user local time date with with DateTime::createFromFormat, explicitly specifying the timezone (user's TZ)
Convert to server local time with DateTime::setTimezone (specifying server's TZ)
Store in database
Apart from the above, I would suggest taking your application offline at some point and convert all dates in the database to UTC. You would then be rid of the problem discussed earlier (at least in the future, as the past cannot be undone). The "server's TZ" I mention above would then be UTC (regardless of the fact that the actual server may be set to AUS EDT or not, your "working" timezone will be UTC).
You could make use of
1) date_default_timezone_set - Sets the default timezone used by all date/time functions in a script
2) Instead of using this function to set the default timezone in your script, you can also use the INI setting date.timezone to set the default timezone.
The important thing to keep in mind is not UTC, but that all times stored must be standardized to one timezone. So, if your PHP server and your database server both use the same timezone, the only issue that arises is when you need to display a location-aware time to the user or when you allow a user to enter a datetime from another timezone.
PHP has a nice, though somewhat scantly documented class, called DateTime. And some ancillary classes like DateTimeZone, DateInterval, etc. These make converting from db time to user time pretty simple.
So does anyone have any suggestions for the best way on how I can make this application timezone aware considering the server isn't set to UTC?
If you manage to come with any scheme for remapping the timezones its going to be horribly complicated and even more impossible to ever fix properly. Do yourself a favour and get the server timezone to UTC and fix your existing data.
First of allyou have to convert the date time selected by user to timestamp.
You have to use Server time zone offset and save the time to server in GMT.
This is the best way because while displaying the date just add the offset of the user
and convert and show.
I have implemented this for my client as it was an auction site and user may add item from AUS in his time and bidder will be from US. Time zone issues was there and we implemented after a lots of rerence.
You know one thing best and easy way is , do like ebay . just save the user time zone and show time with the time zone. No conversion nothing. Simple and better . 10:35 EST :)
If you wannabe perfect in time zone conversion, think about daylight saving time also. start date and end date on each year will change slightly. If you want to be accurate you have to save the daylight starting and ending date in db and add that difference too .:)
For working with datetime in different timezones and formats you can try to use PHP library Dater (https://github.com/barbushin/dater). Cheers!
I have just realised if I add a particular record to my MySQL database - it will have a date/time of the server and not the particular user and where they are located which means my search function by date is useless! As they will not be able to search by when they have added it in their timezone rather when it was added in the servers timezone.
Is there a way in Codeigniter to globally set time and date specific to a users location (maybe using their IP) and every time I call date() or time() that users timezone is used.
What I am actually asking for is
probably how to make my application
dependent on each users timezone?
Maybe its better to store each users timezone in their profile and have a standard time (servers time) and then convert the time to for each user?
Thanks all
It sounds like what you need to do is store all of the date and times in your system as UTC time (used to be called GMT). This is the base time that everything in the world is calculated off of with adjustments. (eg: Central Time is -6 hours off of UTC)
In MySQL you can use UTC_TIMESTAMP() to get the current UTC time as long as your server and DB are configured with the correct times and timezone settings.
In PHP run this to set the timestamp of PHP to UTC (you will run this in your code so put it on every page or in a centralized index file):
date_default_timezone_set('UTC');
Or you can go directly into PHP.INI and tell it to use UTC time globally. (this may not work if you have multiple websites on a single installation of PHP.
And then anywhere in the system you need to get the current UTC time you can just call:
time();
Then for each user in the system you will need to ask them what timezone they live in and then when you display times make the adjustment for that user. So if it is 5:00PM UTC and I live in Easter US (-5) the time would be 5:00 - 5 hours = 12:00PM.
This can be a long process to get right but once you do your users will find it very useful especially internationally.
I think the easiest way is define a timezone for internal data storage (your server's time zone or UTC) and convert the time according to the user's time zone when outputting it.
I don't know CodeIgniter so I can't point you to the right functions. One prominent library that is timezone aware is Zend_Date: Working with Timezones I have not worked with these functions yet but they look promising.
However, once you know the user's time zone, it's not difficult to put together an own function that adds/substracts the offset when outputting dates and/or times.
Possibly related question:
MySQL: keep server timezone or user timezone?
Take an example of an existing web application such as WordPress and phpBB. Each user have their own timezone setting.
When receiving a content from the user, use local_to_gmt() function in the Date Helper then save the content into database using the gmt date. When fetching the data you will get the time in gmt. Get the user's timezone setting, then display the data in that timezone.
This way, you can save yourself from calculating between two timezone. Just make sure that your server's time is in correct setting, so all your data is in the correct gmt time.
UPDATE:
Recently I review the last project I worked on that have timezone issue. After thinking various scenario, here is the solution for the timezone issue:
All data stored right now already
using server's time. Changing this
will takes times and prone to error,
so I leave it like that.
For the new data from user that set the content date to a certain
date and time, I stored it into 2
column. First column is to store the
data as is, and used to displaying
it as is. Second column will be a
recalculation of the date based on
the user's timezone into server's
timezone. This column is used in the
WHERE statement (filter based on
server date) and for the ORDER
(because this column's value all in
same timezone, which is the server's
timezone).
This way, I only do 1 timezone calculation, which is to convert user date into server date. For displaying , I display the date according to server's datetime. Since all data stored in the same timezone, data can be ordered by the column that hold the server date value.
For the user that have set their timezone, the date from database can be easily recalculated to get the datetime in the user's timezone. Btw, in my application, I display the date using timeago jquery plugins. This plugins need time in the ISO8601 format (UTC time). local_to_gmt() function in CodeIgniter can be used to do this.
Obviously the leap to British Summer Time (Daylight Savings Time) is a big confusion in the world of programming, and I am indeed caught up in that confusion.
The best possible solution I can find (which I will try to coherently explain) when using a timezone sensitive system is this:
The Web Server and Database should both be running off the same machine timezone. I suggest UTC as it is the building blocks of timezone conversions. This will ensure that all of the dates stored in your database are constant, and don't skip any times such as the 1hour jump between Daylight Savings.
At the top of all of your PHP scripts use date_default_timezone_set('Europe/London'); with the specific timezone of the user.
When producing dates from user submitted forms, use gmmktime(); to ensure that the timestamp created is UTC and not altered by the timezone that you have set.
date(); can be used when displaying dates, as this will convert the timestamp to the correct time taking into account the timezone that you have set.
If you do need to show a date in the UTC format then use gmdate(); with the $gm_timestamp that you have taken from the database or created with gmmktime();.
I have written this bit of PHP to help understand the situation.
date_default_timezone_set('UTC');
$gmtime = gmmktime(2,0,0,03,29,2009);
$time = mktime(2,0,0,03,29,2009);
echo $gmtime.'<br />'.date('r',$gmtime).'<br />'.gmdate('r',$gmtime).'<br />';
echo $time.'<br />'.date('r',$time).'<br />'.gmdate('r',$time).'<br />';
date_default_timezone_set('Europe/London');
$gmtime = gmmktime(2,0,0,03,29,2009);
$time = mktime(2,0,0,03,29,2009);
echo $gmtime.'<br />'.date('r',$gmtime).'<br />'.gmdate('r',$gmtime).'<br />';
echo $time.'<br />'.date('r',$time).'<br />'.gmdate('r',$time).'<br />';
Hopefully I've done a good job, but I doubt it because I'm still trying to battle the problem in my head.
UPDATE:
Glad I did this, because I am now having doubts about the user inputted dates.
To get the User inputted date (taking into account their timezone) to match up with the UTC corresponding date in the database, you should put it through mktime(). And then use gmdate('U', $timestamp); to get the true UTC timestamp. (I think)
Example
Looking at it from a reporting side, the user is using the 'Europe/London' timezone. At the start of our PHP script we call date_default_timezone_set('Europe/London');, whilst the Database (and all the records within) is still in UTC.
The user then sends through that they want to select a list of books added to the database between 25/03/2010 10:00 to 30/03/2010 14:00. The PHP script then runs the date variables through mktime($hour, $minute, $second, $month, $day, $year) to generate a correct UTC timestamp. The start date will not change, but PHP knows that the end date is within the BST timezone, so changes the timestamp to UTC accordingly.
When the results are returned back to the user, date('r', $date_added) can be used to show the user the date the book was added to the database according to their set timezone.
This link may help with understanding when it changes. http://www.daylightsavingtime.co.uk/
I think recalculating to user's time is better option, since it gives you normalized time on server, i.e. if you'll need to look up something, that happened (from your point of view) hour ago, you won't have a mess with american, asian and e.g. australian time.
Just ask them for their timezone (usually select with major cities in that timezone) and then recalculate :)
Or, alternatively, you can store two timedates - one for your comparison and one to show, so you won't have so much calculations on serverside.
Also, if recalculating, you can use date helper:
http://ellislab.com/codeigniter/user-guide/helpers/date_helper.html
I've used the MySQL built-in timezone conversion. In the database, all datetimes are stored as UTC. In the select query, I used CONVERT_TZ to convert to the user's timezone. You can specify timezone codes or hour invervals like:
SELECT CONVERT_TZ('2004-01-01 12:00:00','GMT','MET');
SELECT CONVERT_TZ('2004-01-01 12:00:00','+00:00','+10:00');
But, the problem is this does not accommodate for daylight savings times. This is particularly frustrating since many parts of the world either don't honor daylight savings or honor it on different dates. So, if you install the timezone description tables, you can use descriptive names that will account for daylight savings automatically like:
SELECT CONVERT_TZ('2004-01-01 12:00:00', 'UTC', 'US/Eastern');
Codeigniter contains a helper which deals with all manner of date functions.
Codeigniter Date helper
the command gmt_to_local() should help you... the third parameter is for 'daylight_saving'.
Takes a Unix timestamp (referenced to GMT) as input, and converts it to a localized timestamp based on the timezone and Daylight Saving time submitted. Example:
$timestamp = '1140153693';
$timezone = 'UM8';
$daylight_saving = TRUE;
echo gmt_to_local($timestamp, $timezone, $daylight_saving);
Add this line to autoload.php in the application folder/config folder:
$autoload['time_zone'] = date_default_timezone_set('Asia/Kolkata');