I have a question regarding Date and Time for MYSQL. Basically I started building a comments section for a web project I have going on. I was building out the DB table and wanted to know how the DB stores the date and time.
For instance if I am in LA and the time is 7:00 PM will it store the comment at 7PM and say that I am from New York obviously and I see the comment its 10 PM so what time will I see the 7PM that wouldnt really make sense. I am wondering how you solve this problem and differentiate between different time-zones.
I am imagining that the server has its own time zone associated with it so all time is delineated from that which would make things a whole lot easier. But is it right that a person sees their time from a post as 10PM when it was made at 7PM. This is all very confusing to me. Any information on how to regularly deal with this issue or if I have one at all will be greatly appreciated. I am building my site using PHP and Codeigniter.
I personally just store the times in the database under an INT ( 11 ) and use the native PHP time() function to get the time, and then you can convert this into a nicer format of a date using some of the PHP functions http://php.net/manual/en/function.date.php you can get the year, month, day, hour, minute, etc.
You then can easily just add or subtract the # of hours (3600 seconds in an hour) depending on the time zone. May be an easier way to do this, but this is how I am currently doing it :)
Ask the user their timezone and adjust times accordingly.
Show the timezone with the date ("9pm ET")
Do what StackOverflow does, and say "2 mins ago".
Related
I am having an issue getting my head around storing and retrieving information based on a date (no time) and dealing with multiple timezones.
By default my application uses UTC, so create/update times are all in UTC and PHP functions such as date etc all use UTC.
My problem is this:
I allow users to create a news item, the news item has a date (no time, just date).
The application uses UTC, my user is in Sydney, so what I do is when the form loads I pre-populate the date field based of jS M Y for the Australia/Sydney timezone, this way it shows as today for him. All good so far.
When I save the record into the database (into a DATE field) I use the date they entered so if they said 3rd March 2016, then it goes in as 2016-03-03.
When it gets pulled out again it displays fine for them, but if I do a report saying "How many news items published today?", in my report (cron job) it might be looking for a different date, because if this person entered it in early morning, then 3rd March 2016 for Sydney is still going to be 2nd March 2016 in UTC so there will be 0 for today.
The solution it appears would be to convert the date back to UTC when you store it, but how will that work?.. Since it's just a date with no time.
2nd March 2016 00:00:00 UTC would be 2nd March 2016 10:00:00 Sydney
So.. I am not sure how to solve this without them having to also enter a time, which seems redundant.
Always remember that savings time rules vary between countries, such as the USA and EU. Not just when the clocks change but timings of changes across time zones.
In the past I forced a local hour of 12:00 (noon). With a time value the standard conversion routines be used. Choosing noon kept everything well away from any midnight and savings time issues as an hour slip either way would still be in the same day.
The simplest way is to ALWAYS work with full UTC timestamps under the hood.
Convert incoming dates to these straight away, and only convert back to readable, localized formats when rendering.
This way you have a single, solid, point of reference. Everything in your database has logged exactly when it was created (relative to the 1st of January 1970 - UTC).
Then it becomes trivial to order articles in order of creation (regardless of where their authors posted from). It's also much easier to count how many articles were added in the last 24 hours, or on a specific day (for example; seven days ago) -- notice how these examples don't require a date, just ranges of seconds.
This also makes it easy to show "user A" when "user B" added an article, relative to user A's timezone -- you don't need to look up user B's timezone to do the conversion.
Also, give your user the option to add a time to their articles, but don't make it mandatory. Have a sensible default (ideally a dynamic "now"). This is more natural, and should smooth out the midnight and DST anomalies highlighted by Gilbert, when building reports.
It actually sounds like you are doing things correctly already.
The concepts of "today" and a "date" are calendering concepts, covered by the domain of civil time. You can fly around the world with a printed calendar, and you don't need to adjust it for UTC or time zones. A date itself can't be "in UTC".
However, you can say that in the domain of absolute time, that a date in time zone A does not cover the same instantaneous moments of time that are covered by time zone B. At any given time, there could be two different "todays" in effect on the Earth.
By the perspective of your Sydney user, March 3rd was the publishing date. You might have another user in Hawaii publishing simultaneously at March 2nd.
Inherently then, your posting date is the civil date that the item was posted. When you query for all items posted "today", that is based on the civil date of wherever you're at when you're asking.
If this isn't the desired behavior, then change the question. Instead of asking "what was posted today", ask "what was posted in the last 24 hours?" For that, you would indeed need to store the full UTC-based date and time (not just date) of the post. Then you would query it by UTC time from now less 24 hours.
A thought exercise that will help: What should you do if a user posts at 23:59 UTC, and you run a query one minute later? Should it include the result - or not? Does your answer change if it's 23:59 local time and you run the query one minute later in the same time zone? Thinking this through will help you decide what you should actually be storing and querying.
I'll also say that in most cases you should indeed store a UTC based timestamp, but you may want to also store the local date (or datetime) in a separate field. This gives you the ability to do both types of queries.
Well I know that this question may be asked many times but certainly I've few doubts In my mind, and by the way don't comment that what I've tried, I've tried many ways but am just asking what's the correct and easier way as there are many posts lingering out here with different suggestions for accomplishing these tasks, so I'll explode() my question into smaller questions...
So can you people just guide me where am going right or wrong as am sure many people are confused when it comes to date/time
1) Why/how to save time as UTC in MySQL using PHP?
Personally for this I post this using php $year/$month/$day and check the date using checkdate() in mysql date field. so is it ok or should I use timestamp and than explode the retrieved string on front end using php?
2) If I run server from India, should I record default time using date_default_timezone_set("Asia/Calcutta"); and than subtract and add time using php function or while posting only I should check users time zone selected from his user accound and accordingly set a condition kinda:
if(timezoneselected == +5.30) {
echo date_default_timezone_set("Asia/Calcutta");
} elseif(timezoneselected == +anytime) {
echo another country timezone
}
3) last question is how websites like gmail facebook etc manages time? I mean if they are saving datetime according to their server than how they show perfect posted time for each user, even gmail, if I send a mail to another user, my sent time and the person living in another country gets email at his printed local time I mean how we can do this, sorry am not able to explain you perfectly say this example,
facebook:
user from India posts, facebook shows posted 8 mins ago, 9mins ago, fine after sometime they show a real date, and that date is perfect according to the time I posted, however if a person from USA updates, say 8 mins ago 9mins ago on his profile but his original posted time is shown correctly to him, and even correctly to me?
sorry for this question but really this will help me understanding this date/time concept and will also be helpful to future users. Thank you!
Bottom line, you should store everything UTC
When you display times for a particular user, use a timezone of their choosing. Store the timezone of the user, like "Asia/Calcutta" and simply convert the time when displaying it using the date_default_timezone_set method.
I will attempt to answer your questions from the comment here.
You store everything UTC always. It is the baseline. When you display the times associated with anything you convert based on the user. If you want to display Posted 8 mins ago then you are taking the delta between the current UTC time and the UTC time associated with a post. If you send a message from user A (in India) to user B (in Los Angeles, USA) then you would store the message time in UTC. If user A is viewing it, the time would be converted to "Asia/Calcutta" and if user B is viewing it, the time would be converted to "America/Los_Angeles". Using UTC will make your life a lot easier. Trust me.
As described in MySQL Server Time Zone Support:
The current session time zone setting affects display and storage of time values that are zone-sensitive. This includes the values displayed by functions such as NOW() or CURTIME(), and values stored in and retrieved from TIMESTAMP columns. Values for TIMESTAMP columns are converted from the current time zone to UTC for storage, and from UTC to the current time zone for retrieval.
Therefore, if you use TIMESTAMP type columns, MySQL will handle timezone conversion for you automatically: just set the appropriate timezone for the session in its time_zone variable.
I have a social network site I have been working on for a couple years in PHP/MySQL, I am now re-buidling the whole site from scratch again though. This time around I would really like to add in the ability for users to set a timezone for times on my site.
So with that said, I am asking on a guide from start to finish of the steps I need to include to do this please. My old site used mysql datetime for all dates and times and it worked great but I read that it is best to use like a text filed and store all dates and times with UTC, can someone explain how I could do this? Would I still be able to use the now() function in php to save a time to the mysql?
Also I have seen the list that php can generate of all the timezones, the one where it shows like a million (not really) but I am wondering, would it be possible to show some sort of map with images or something and link to the main timezones?
Please any tips for setting a users timezone, I can do that part, but once I have a user's timezone saved and ready to use, how can I make sure the users see's the correct time and how do I save times in the correct time.
Sorry if this was confusing, any help would be great though, thanks.
When working with php, from my experience, it is the best to store timestamps, generated with time(), as an int in the database. While it may not be that easy to read them when looking at the database (as you cannot guess the actual date just from the number), but the timestamp is the most-native thing when working with dates in php.
To save each user's timezone, you can simply save the offset, in either hours, or better (as some timezones are half an hour off) in minutes. So for example my timezone offset would be +60 (UTC+1), while in America most will have something like -480 (UTC-8) or similar.
Then when displaying the times for a user, you just pick the timestamp (which is in UTC time) and the timezone offset and generate a readable date from it using the standard date() function for formatting.
For example:
<?php
$time = time(); // from database or time() for "now"
$offset = -480; // from database
// add the offset to the time you want to display and you have the user's time
echo date( 'd.m.Y H:i', $time + $offset * 60 );
?>
In addition you could then also store the formatting string (d.m.Y H:i) in the database, so that every user can pick his favorite format.
This topic gets complicated especially when you factor in daylight savings and the fact that your server will be in a different time zone from some of your members.
I suggest you take a look at some libraries out there that have been built to handle these issues. The PEAR package Date might be a good place to start. http://pear.php.net/package/Date
Basically, you would want your members to select a timezone for their profile. Then convert all times to UTC (to remove the offset of the server location) and store them in your database as a timestamp. When displaying the times, apply the timezone offset the member chose for themself (maybe with UTC as the default) as well as a date display format.
The company I work for creates applications for the Blackberry platform.
We've been working on a proprietary "analytics system" that allows us to embed code within our applications and have the applications report back some stats to our central servers every time they're run. Currently, the system works ok; however it's only in beta with 100-200 hits per hour. The "hits" are sent to the servers without a problem. We've built a very solid API to handle the acceptance and storage of the hits (in a MySQL DB). We've tested the load and we should be able to accommodate hundreds of thousands of hits per hour without a problem. That's not really a problem.
The problem is showing the stats. We've built a display panel similar to Mint's (haveamint.com), it shows the hits over each hour, the past days, months, weeks, years...etc. The fist version ran straight queries pulling data from the hits table and interpreting it on the fly. That didn't work for very long. Our current solution is that the hits are "queued" for processing and we have a cron come through every 5 minutes taking the hits and sorting them into "caches" for each hour, day, week, month, year...etc. This works amazing and it's incredibly scalable; however, it only works for 1 timezone. Since the entire company has access to this, we're dealing with a few hundred users in various timezones. What I define as "Today" in San Jose is MUCH different than what my colleague in London defines as Today. Since the current solution is only cached to 1 timezone, it's a nightmare for anyone who's checking the data outside of our timezone.
Our current plan to fix this is to create caches for every timezone (40 in total); however, that would mean that we're multiplying the amount of data by 40...that's terrible to me and given that the caches can be very large, multiplying it just sounds like a bad idea; plus, when we go to process the queue, it's going to take a lot more CPU time to put them in 40 different caches.
Any one else have a better idea of how to solve this problem?
(Sorry for such a long question..it's not exactly easy to explain. Thanks all!)
The solution you are proposing has too much redundancy. I would suggest you store the data in at least 30-minute buckets instead of hourly and the time zone be normalized to UTC.
With 30-minute buckets, if a user requests hourly data for 1 - 2PM from -4.5 UTC you can fetch data for 5:30 - 6:30PM from your system and show that. If you store data in one-hour increments you can't service requests to users in time zones with N + 0.5 hour differences.
For daily numbers you would need to aggregate 48 half-hour slots. The slots to pick would be determined by the user's time zone.
It gets interesting when you get to annual data because you end up having to aggregate 17,520 half-hour buckets. To ease that computation I would suggest you get the pre-aggregated annual data per UTC time and the subtract aggregate data for the first for 4.5 hours of the year and add aggregate data for the first 4.5 hours of the next year. This will essentially shift the whole year by 4.5 hours and the work is not that much. Working from here, you can tweak the system further.
EDIT: Turns out Kathmandu is +5.45 GMT so you would need to store the data in 15-minute buckets instead of 30-minute buckets.
EDIT 2: Another easy improvement is around aggregating annual so you don't have to add 17,520 buckets each time and without requiring one aggregate per country. Aggregate the annual data from Jan 02 - Dec 30. Since the maximum time-zone difference between any two countries is 23 hours, this means that you can take the annual data (Jan 02 - Dec 30) and add a few buckets before and after as appropriate. For example for a -5 UTC timezone you would add all buckets on Jan 01 after 0500, all buckets on Dec 31, and on Jan 01 the following year up to 0500 hours.
When designing software that touches multiple timezones, I'd say to always store your date/times in UTC with another field for the original timezone and have a function that takes the time and converts it to and from UTC/timezone. You'll save yourself a lot of trouble to handle the different cases of day switch, daylight savings, people looking at stats from a country from the other side of the earth and so on....
In your case, having the caches in UTC and just adjusting the requests to be converted in UTC should help. Don't store a stat as being "today", store it for hours 00:00:00UTC to 23:59:59UTC and when someone asks for the stats for today in New York, do the conversion.
As far as I can see, you are looking for the storage part of a data warehouse system here (your reports would be the front-end).
Actually, the way commercial systems are doing it, is the cache you described: Preaggregate your tables and create caches of them. The only way to accelerate your queries is to make the database system do less for them. This means less data, which in turn means less time spent in iterating the data or less data in the indices.
That said, I would either propose the "40 cache solution" (are there really more than 24 timezones). You should be able to trivially parallelize the sorting queue by creating copies of the data.
Another way to do this, would be to cache at hour granularity and then aggregate the hours into days (or 30 minutes if your timezones require this). This means you cache at a finer granularity than your daily cache but at a coarser granularity than the original data.
this kind of data is usually stored using round-robin or circular databases. check this http://www.shinguz.ch/MySQL/mysql_20070223.html and this http://techblog.tilllate.com/2008/06/22/round-robin-data-storage-in-mysql/ to know how they work and how to implement it under MySQL
this might be stupid question, but ill ask anyway.
do i really need to implement gmt on site to get proper timestamp on a post? well im building a site like twitter. my feeds are working fine and showing me the time i need, like it shows "posted 11 minutes, posted yesterday, posted 5 days ago.. " just like this site, so i was thinking it should work for everyone properly, what you think? or do you think i need to implement gmt on it and do you recommand or have any article on it for implementing it proper way?
sorry if this sound stupid :O
In the long term, it is best to store time stamps always in GMT/UTC, no matter how the user interface is rendered. This has two advantages:
when changing the UI at some point, you won't need to touch the data
it gives a clear design guideline (in terms of a clear right/wrong decision): any specific function dealing with time can be independently reviewed wrt. processing timestamps in UTC
As for specific programming guidelines: these depend very much on the functionality you want to execute. Most of the time, you need either from-utc or to-utc operations.
It's a good point, but consider the case when 1 day ago suggests "yesterday", and it's currently 1:05am for some user. I suppose if you're using days = hours % 24 rather than days = datediff(then, now).days, then it would work fine.
I think the issue is: how accurate does your site need to be? If it's for medical readings, so someone can know when their medication is due, then yes, you want to account for timezones properly, and give accurate times rather than just "x ago". If it's just for "joe said hi two days ago", though, then it's not a big deal.
When you're talking about time periods like hours, minutes and seconds, then that's not even anything to do with timezones. Look at the timestamp of when it was created, and look at the timestamp now. Do some subtraction and voila!
The only time it's slightly strange is when you are in the range of a couple of days. Does yesterday mean "some time in the 24 hours prior to the last midnight", or does it mean "over 24 hours ago"? In any case, if the only level of granularity you are providing is "days", then once it's gone past about 2 days, then it doesn't really matter.
One nice way to avoid confusion is the same method used here on SO: put the readable, friendly date on the screen ("yesterday"), but put the exact time (in GMT or in the user's TZ) as a tooltip.