Introduction to my website
My website is for visitors in Korea(AKA Republic of Korea).
And the server for My website is in the United States of America.
And PHPMyAdmin displays EDT when it runs a query SELECT ## system_time_zone.
Structure of my website
When I first uploaded my website to this server in October this year, I checked the DB time.
And it seemed that there was a time difference of 13 hours with Korea. So I added 3600 * 13 seconds to DB time(without setting timezone) as follows.
const Offset = 3600 * 13;
$SelectNow = $PDO->prepare('SELECT DATE_ADD(NOW(), INTERVAL '.Offset.' SECOND)');
$SelectNow->execute() or exit;
$DbNow = $SelectNow->fetchColumn();
My website takes $DbNow as above and uses it in various situations.
For example, in the posting situation, enter $DbNow in the datetime field of the INSERT INTO query as follows:
$WriteNote = $PDO->prepare('INSERT INTO table_note(my_datetime, my_contents) VALUES("'.$DbNow.'", :my_contents)');
$WriteNote->bindValue(':my_contents', $my_contents, PDO::PARAM_STR);
$WriteNote->execute();
The problem situation
One day in November of this year, when I wrote a post and checked the date field(my_datetime) of the post, I got an additional time difference of one hour with Korea.
Apparently, at the end of October, I corrected the time difference of 3600 * 13. And then I confirmed that it matches the Korean time. However, in November, There is a time difference of one hour!
Guess the cause
It seems that US summer time is being applied to the DB server of my website. Did I guess right?
My question
1) How can I solve this time difference fundamentally?
Is it correct to convert DB time to KST?
Or is it the correct way to convert to UTC and then added 3600 * x to UTC?
2) Even though the problem is resolved, some of the existing data in my DB has a time difference of one hour with Korean time.
What query do I use if I want to select the data with a time difference?
And how much more or subtract it from the data to get rid of the 1 hour time difference?
Use UTC to store time in Database.
change your queries to insert with UTC datetimes.
Use external libraries to convert UTC to respective timezones.
(below are the my personal recommendation.)
There may be best of it.
PHP : Carbon
Javascript : Moment, moment timezone.
No, it takes timezone of Database server resides in.
little manual verification, or create a job to change all dates in UTC.
Edit:
http://carbon.nesbot.com/docs/
I mean you can create a script and run with cron job.
Related
What to accomplish:
I want to export the user's activity (from the previous week) every monday morning between 4 and 5 am respecting the user's local timezone.
Issue:
Timezones..
Situation:
I'm running a PHP web service with the timezone set to UTC for the script and MySQL database. So all timestamps in the system are set to UTC and the user's timezone is stored with their account information.
I have a CRON scheduled to run and call the script every hour at the 30 minute mark.
Now the step I'm "stuck" at is on the part on how to determine if there are users in the system with a timezone set where it is currently between 04 and 05 am on a monday (when it's 19:30 UTC time for the server for example).
Current solution:
Thanks to #deceze in the comments for this suggestion!
(which seems to be an efficient way or at least the most logical way, which I completely skipped for some reason)
Do the timezone conversion on database level to filter the users.
WHERE
DATE_FORMAT(CONVERT_TZ(NOW(), '+00:00', users.raw_offset), "%w%H") = '104'
Previous solution:
Fetch all the active users, iterate them, create a new \DateTime with the timezone that is specified by the user, do an "if monday between 4-5 am" check, if so, execute the script for that user and continue with the next user. But this seems really inefficient if we would talk about thousands of users or something?
$users = (new Models\User) -> getActiveUsers();
foreach ($users as $user) {
$localTimeUser = new \DateTime("now", new \DateTimeZone($user -> getTimezone()));
if ($localTimeUser -> format('NH') === '104') {
// execute
}
}
I could also "reverse" the determination by doing something like "in which timezone is it currently monday morning and get me the users with that timezone" but I haven't figured out the best way to check for that yet.
This is the first situation where I'm having trouble with timezones and I'm having a really hard time on figuring out the most efficient way to do something like this.
Is there a really straightforward solution that I'm currently missing or is checking thousands of user's timezones the only doable way?
Subnote:
The value 104 I compare with is just the easiest way I thought of, to have one value that indicates the day and the hour (1 = monday, 04 = 04:00 - 04:59 am) instead of checking between values etc.
I want to use the following code to SELECT data older then 1 minute.
SELECT * FROM `auth_temp` WHERE date > (NOW() - INTERVAL 1 MINUTE)
But it didn't work. Then I checked some other topics and one person talked about the server time, I just asked my host and he said the server time is: 15:30
When at my place and the logs in MySQL it is 21:30, aka 6 hours later.
Anyone how I should asjust my code to that?
Thank you all!
You are hitting a timezone issue. Most servers run on UTC. If you have a TIMESTAMP as the field type, MySQL will convert the time from server time to UTC and back. You can adjust what MySQL considers server time using SET time_zone = timezone; (Docs). If you actually care about timezones it is advisable to just use UTC and convert in your application.
Your current SQL statement will only select data newer than 1 minute. Change it to:
SELECT * FROM auth_temp WHERE date < (NOW() - INTERVAL 1 MINUTE)
This will select data that is older than one minute. If you are using NOW() for setting the date column when you are inserting the row then that small fix should do it even if the time zones are different between your application and database layer. If you are setting the date column from your application layer you will have syncing issues if the time zone is set differently than the database layer.
It sounds like the MySQL server is either running in a different time zone or running on Universal Time (UTC) which is common. Running MySQL on UTC time is a good way to deal with users in multiple time zones. In your code, you should be able to synchronize the time zones in use on the database and application layers if it's set to UTC time easily. If it's set to a different time zone, it should be possible as well but not recommended.
I have a MySQL database containing details of shops in different time zones. The timezone of each store is stored in iana format and the MySQL datetimes are stored in UTC.
I wish to execute some php code at the end of the day for each shop.
This is how I am thinking to approach this but is there a better way?
Set a cron to run hourly at xx:59:59.
Get the current date at the top of the php script. Is the script guaranteed to get the correct date at 23:59:59?
Use SQL query to return all shops that are at the end of the current day. I'm not sure how to check this in the query?
Perform the end of the day processing on those stores.
You can run an hourly cron job any time you wish.
Why do you need the date here? You wrote that you're interested in the servers, which are at 23:59:59 localtime, so time is relevant, I think, not the date. Also, if your details server is very busy, your script might run too late and get the next day's date.
If IANA time zone format means offset to UTC, you could simply look for shops having a timezone like
24 - current time(UTC) +/- DST
Negative timezone offsets work similar, e.g. 24 + offset. So timezone offset -01:00 would become +23:00
Nothing to say here.
I'm developing a website where I have to deal with different possible timezones from the users. This becomes a great problem since the website hosts time-delicate events like auctions.
All dates/times on the server are in UTC. Database stores everything in UTC timestamps. PHP default timezone is set to UTC too (date_default_timezone_set('UTC');).
Now, my problem is how I should interact with the users, whether I'm only showing a date or, more important, I'm reading a date/time from user input.
A concrete example:
An auction has a deadline, which I store in database as UTC.
When I view the auction on the website, a javascript timer uses a Date object to calculate the remaining time. It automatically converts the timezone to GMT+0100 (my local timezone). So if the deadline is '2013-08-08 10:46:08' (UTC), the javascript date object will return Aug 08 2013 11:26:15 GMT+0100 (GMT Standard Time).
If the current time is greater than 11:46:08 then the timer says that the remaining time is 00:00 (which is correct).
But if I try to insert a bid, the server accepts since the condition on the MySQL INSERT evaluates to true:
INSERT INTO Bids ... WHERE ... AND auction_deadline > NOW() ...
( because auction_deadline = '2013-08-08 10:46:08' and NOW() = '2013-08-08 10:26:50')
All this mumbo jumbo of timezone melts my brains. What am I missing here? I'm almost certain that storing all dates/times in UTC inside the database is the best. I just can't think crystal clear how do deal with it between the user and the database.
Your problem doesn't involve timezones at all, just the fact that clients can turn their clocks or have their clock skewed considerably. For that the fix is to poll the server every once in a while for an offset fix to use in calculations.
In fact, you don't even need date objects. There is a certain universal instant in time when the auction ends. Let's say it is 1375960662823. Right now, the universal instant in time is 1375960669199, so from that we see that the auction ends in 6 seconds (1375960662823 - 1375960669199 ~ 6000 ). It will end in 6 seconds regardless if I am in Morocco or Japan. Do you understand it yet?
To generate these numbers, on the client side you can call var now = Date.now() + skewFix where skewFix is the correction that needs to applied in case client has time skew or manually set their computer to wrong time.
In PHP, you can generate it with $now = time() * 1000;
This is rather a typical subject yet very complex for most to understand. First thing, you never mention the DAYLIGHT SAVING. yeah I am increasing your tension :).
Now let us see how we can do this. You did a good job by saving the Time in UTC. Now, I hope you have registered members and that each member has ability to set their preferred timezone, otherwise you will show Server' timezone based time to them.
When you through "start time" to user you must send them after converting UTC time to their time, similarly when you accept TIME from browser be it user action or javascript you need to convert that time to UTC considering the fact that user is that time zone that he select for his profile.
Hope that clear the idea on where you are going wrong? Please read through day light saving as that will play an important role too when you move ahead with other logic on same.
EDIT:
You can use javascript's Timezone offset, for auto submission and user input based on his settings.
Date in JavaScript uses local timezone. You should get UTC time for the user and send it to the server
new Date
Thu Aug 08 2013 17:00:14 GMT+0530 (India Standard Time)
(new Date("Thu Aug 08 2013 17:00:14")).toUTCString();
"Thu, 08 Aug 2013 11:30:14 GMT"
This will resolve the timezone issue between the server and client.
You said
( because auction_deadline = '2013-08-08 10:46:08' and NOW() = '2013-08-08 10:26:50')
In MySQL - NOW returns the current time in the server's local time zone (docs).
You probably want something like UTC_TIMESTAMP which returns the current time in UTC (docs).
Also - you probably shouldn't accept any input time from the client JavaScript at all. Only trust your own clock. When a bid is placed, use the time on your server in MySQL or in PHP. Don't accept it as input.
You can do the following
once page is loaded, send an ajax request to server with timezone offset of user. You can get timezone offset using the following code.
var curdate = new Date()
var offset = curdate.getTimezoneOffset()
offset is timezone offset in minute.
I think it will help.
everytime when you get the date from the clientside, you can use the getUTC to convert to UTC date ie:
var todayDate = new Date();
var todayDateInUTC = new Date(todayDate.getUTCFullYear(), todayDate.getUTCMonth(), todayDate.getUTCDate(), todayDate.getUTCHours(), todayDate.getUTCMinutes(), todayDate.getUTCSeconds());
so right before you insert the bid date to database, use the getUTC functions to convert it into UTC format.
I'm using the America/New York timezone. In the Fall we "fall back" an hour -- effectively "gaining" one hour at 2am. At the transition point the following happens:
it's 01:59:00 -04:00
then 1 minute later it becomes:
01:00:00 -05:00
So if you simply say "1:30am" it's ambiguous as to whether or not you're referring to the first time 1:30 rolls around or the second. I'm trying to save scheduling data to a MySQL database and can't determine how to save the times properly.
Here's the problem:
"2009-11-01 00:30:00" is stored internally as 2009-11-01 00:30:00 -04:00
"2009-11-01 01:30:00" is stored internally as 2009-11-01 01:30:00 -05:00
This is fine and fairly expected. But how do I save anything to 01:30:00 -04:00? The documentation does not show any support for specifying the offset and, accordingly, when I've tried specifying the offset it's been duly ignored.
The only solutions I've thought of involve setting the server to a timezone that doesn't use daylight savings time and doing the necessary transformations in my scripts (I'm using PHP for this). But that doesn't seem like it should be necessary.
Many thanks for any suggestions.
I've got it figured out for my purposes. I'll summarize what I learned (sorry, these notes are verbose; they're as much for my future referral as anything else).
Contrary to what I said in one of my previous comments, DATETIME and TIMESTAMP fields do behave differently. TIMESTAMP fields (as the docs indicate) take whatever you send them in "YYYY-MM-DD hh:mm:ss" format and convert it from your current timezone to UTC time. The reverse happens transparently whenever you retrieve the data. DATETIME fields do not make this conversion. They take whatever you send them and just store it directly.
Neither the DATETIME nor the TIMESTAMP field types can accurately store data in a timezone that observes DST. If you store "2009-11-01 01:30:00" the fields have no way to distinguish which version of 1:30am you wanted -- the -04:00 or -05:00 version.
Ok, so we must store our data in a non DST timezone (such as UTC). TIMESTAMP fields are unable to handle this data accurately for reasons I'll explain: if your system is set to a DST timezone then what you put into TIMESTAMP may not be what you get back out. Even if you send it data that you've already converted to UTC, it will still assume the data's in your local timezone and do yet another conversion to UTC. This TIMESTAMP-enforced local-to-UTC-back-to-local roundtrip is lossy when your local timezone observes DST (since "2009-11-01 01:30:00" maps to 2 different possible times).
With DATETIME you can store your data in any timezone you want and be confident that you'll get back whatever you send it (you don't get forced into the lossy roundtrip conversions that TIMESTAMP fields foist on you). So the solution is to use a DATETIME field and before saving to the field convert from your system time zone into whatever non-DST zone you want to save it in (I think UTC is probably the best option). This allows you to build the conversion logic into your scripting language so that you can explicitly save the UTC equivalent of "2009-11-01 01:30:00 -04:00" or ""2009-11-01 01:30:00 -05:00".
Another important thing to note is that MySQL's date/time math functions don't work properly around DST boundaries if you store your dates in a DST TZ. So all the more reason to save in UTC.
In a nutshell I now do this:
When retrieving the data from the database:
Explicitly interpret the data from the database as UTC outside of MySQL in order to get an accurate Unix timestamp. I use PHP's strtotime() function or its DateTime class for this. It can not be reliably done inside of MySQL using MySQL's CONVERT_TZ() or UNIX_TIMESTAMP() functions because CONVERT_TZ will only output a 'YYYY-MM-DD hh:mm:ss' value which suffers from ambiguity problems, and UNIX_TIMESTAMP() assumes its input is in the system timezone, not the timezone the data was ACTUALLY stored in (UTC).
When storing the data to the database:
Convert your date to the precise UTC time that you desire outside of MySQL. For example: with PHP's DateTime class you can specify "2009-11-01 1:30:00 EST" distinctly from "2009-11-01 1:30:00 EDT", then convert it to UTC and save the correct UTC time to your DATETIME field.
Phew. Thanks so much for everyone's input and help. Hopefully this saves someone else some headaches down the road.
BTW, I am seeing this on MySQL 5.0.22 and 5.0.27
MySQL's date types are, frankly, broken and cannot store all times correctly unless your system is set to a constant offset timezone, like UTC or GMT-5. (I'm using MySQL 5.0.45)
This is because you can't store any time during the hour before Daylight Saving Time ends. No matter how you input dates, every date function will treat these times as if they are during the hour after the switch.
My system's timezone is America/New_York. Let's try storing 1257051600 (Sun, 01 Nov 2009 06:00:00 +0100).
Here's using the proprietary INTERVAL syntax:
SELECT UNIX_TIMESTAMP('2009-11-01 00:00:00' + INTERVAL 3599 SECOND); # 1257051599
SELECT UNIX_TIMESTAMP('2009-11-01 00:00:00' + INTERVAL 3600 SECOND); # 1257055200
SELECT UNIX_TIMESTAMP('2009-11-01 01:00:00' - INTERVAL 1 SECOND); # 1257051599
SELECT UNIX_TIMESTAMP('2009-11-01 01:00:00' - INTERVAL 0 SECOND); # 1257055200
Even FROM_UNIXTIME() won't return the accurate time.
SELECT UNIX_TIMESTAMP(FROM_UNIXTIME(1257051599)); # 1257051599
SELECT UNIX_TIMESTAMP(FROM_UNIXTIME(1257051600)); # 1257055200
Oddly enough, DATETIME will still store and return (in string form only!) times within the "lost" hour when DST starts (e.g. 2009-03-08 02:59:59). But using these dates in any MySQL function is risky:
SELECT UNIX_TIMESTAMP('2009-03-08 01:59:59'); # 1236495599
SELECT UNIX_TIMESTAMP('2009-03-08 02:00:00'); # 1236495600
# ...
SELECT UNIX_TIMESTAMP('2009-03-08 02:59:59'); # 1236495600
SELECT UNIX_TIMESTAMP('2009-03-08 03:00:00'); # 1236495600
The takeaway: If you need to store and retrieve every time in the year, you have a few undesirable options:
Set system timezone to GMT + some constant offset. E.g. UTC
Store dates as INTs (as Aaron discovered, TIMESTAMP isn't even reliable)
Pretend the DATETIME type has some constant offset timezone. E.g. If you're in America/New_York, convert your date to GMT-5 outside of MySQL, then store as a DATETIME (this turns out to be essential: see Aaron's answer). Then you must take great care using MySQL's date/time functions, because some assume your values are of the system timezone, others (esp. time arithmetic functions) are "timezone agnostic" (they may behave as if the times are UTC).
Aaron and I suspect that auto-generating TIMESTAMP columns are also broken. Both 2009-11-01 01:30 -0400 and 2009-11-01 01:30 -0500 will be stored as the ambiguous 2009-11-01 01:30.
I think micahwittman's link has the best practical solution to these MySQL limitations: Set the session timezone to UTC when you connect:
SET SESSION time_zone = '+0:00'
Then you just send it Unix timestamps and everything should be fine.
But how do I save anything to 01:30:00
-04:00?
You can convert to UTC like:
SELECT CONVERT_TZ('2009-11-29 01:30:00','-04:00','+00:00');
Even better, save the dates as a TIMESTAMP field. That's always stored in UTC, and UTC doesn't know about summer/winter time.
You can convert from UTC to localtime using CONVERT_TZ:
SELECT CONVERT_TZ(UTC_TIMESTAMP(),'+00:00','SYSTEM');
Where '+00:00' is UTC, the from timezone , and 'SYSTEM' is the local timezone of the OS where MySQL runs.
Mysql inherently solves this problem using time_zone_name table from mysql db.
Use CONVERT_TZ while CRUD to update the datetime without worrying about daylight savings time.
SELECT
CONVERT_TZ('2019-04-01 00:00:00','Europe/London','UTC') AS time1,
CONVERT_TZ('2019-03-01 00:00:00','Europe/London','UTC') AS time2;
This thread made me freak since we use TIMESTAMP columns with On UPDATE CURRENT_TIMESTAMP (ie: recordTimestamp timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP) to track changed records and ETL to a datawarehouse.
In case someone wonder, in this case, TIMESTAMP behave correctly and you can differentiate between the two similar dates by converting the TIMESTAMP to unix timestamp:
select TestFact.*, UNIX_TIMESTAMP(recordTimestamp) from TestFact;
id recordTimestamp UNIX_TIMESTAMP(recordTimestamp)
1 2012-11-04 01:00:10.0 1352005210
2 2012-11-04 01:00:10.0 1352008810
I was working on logging counts of visits of pages and displaying the counts in graph (using Flot jQuery plugin). I filled the table with test data and everything looked fine, but I noticed that at the end of the graph the points were one day off according to labels on x-axis. After examination I noticed that the view count for day 2015-10-25 was retrieved twice from the database and passed to Flot, so every day after this date was moved by one day to right.
After looking for a bug in my code for a while I realized that this date is when the DST takes place. Then I came to this SO page...
...but the suggested solutions was an overkill for what I needed or they had other disadvantages. I am not very worried about not being able to distinguish between ambiguous timestamps. I just need to count and display records per days.
First, I retrieve the date range:
SELECT
DATE(MIN(created_timestamp)) AS min_date,
DATE(MAX(created_timestamp)) AS max_date
FROM page_display_log
WHERE item_id = :item_id
Then, in a for loop, starting with min_date, ending with max_date, by step of one day (60*60*24), I'm retrieving the counts:
for( $day = $min_date_timestamp; $day <= $max_date_timestamp; $day += 60 * 60 * 24 ) {
$query = "
SELECT COUNT(*) AS count_per_day
FROM page_display_log
WHERE
item_id = :item_id AND
(
created_timestamp BETWEEN
'" . date( "Y-m-d 00:00:00", $day ) . "' AND
'" . date( "Y-m-d 23:59:59", $day ) . "'
)
";
//execute query and do stuff with the result
}
My final and quick solution to my problem was this:
$min_date_timestamp += 60 * 60 * 2; // To avoid DST problems
for( $day = $min_date_timestamp; $day <= $max_da.....
So I am not staring the loop in the beginning of the day, but two hours later. The day is still the same, and I am still retrieving correct counts, since I explicitly ask the database for records between 00:00:00 and 23:59:59 of the day, regardless of the actual time of the timestamp. And when the time jumps by one hour, I am still in the correct day.
Note: I know this is 5 year old thread, and I know this is not an answer to OPs question, but it might help people like me who encountered this page looking for solution to the problem I described.