I've been asked to migrate data from one table to another. The old data includes a series of dates entered in a mishmash of whatever format the user felt like using at the time, the new format requires separate FROM date and TO date fields in MM/YYYY.
So some dates are a single date, such as DD.MM.YYYY or DD/MM/YY or DD YYYY or YYYY or again whatever. Some dates are both dates, such as DD.MM.YYYY-DD/YYYY or DD/MM/YY--DD/MM/YYYY. So just a mess. There are only a couple hundred rows but I don't feel like going through and changing them manually if I can avoid it.
Most of the Google results are for converting one format to another format, how can I convert from a mess of formats to one?
Using a single expression to encompass all combinations is hard so just do the patterns one by one:
UPDATE olddata,newdata
SET newdata.date=STR_TO_DATE(olddate.date, "%d.%m.%Y")
WHERE olddata.id=newdata.id
AND olddata.date REGEXP '[:digit:]{1,2}\.[:digit:]{1,2}.[:digit:]{4}'
AND newdata.date IS NULL
Do this a multiple times for each date format, and use the right expression in STR_TO_DATE. Experiment selecting with the right regulate expression before doing the update.
Eventually you'll have enough record to edit manually
Related
I am trying to optimise how I use dates across my SQL/PHP site. I'm familiar with date() and strtotime() functions, and have created projects which have stored data on the server side in both unix and datetime. I know that changing date doesn't use massive amounts of resource, but I'm trying to understand how to code in the most efficient way.
What I'm trying to work out now is what the quickest/most efficient practise is for storing dates on a server, i.e: What is the most effective combination of SQL FROM_UNIXTIME, CONVERT, UNIX_TIMESTAMP and PHP strtotime and date functions, for a typical table involving frequent CRUD of the date fields. Take the following example:
I have an SQL table which contains 3 date columns. All 3 of these can be updated quite regularly by multiple users of the site, and entries are also entered in to it as follows:
mysql_query("INSERT INTO `regular` (
`regularid`,
`propertyid`,
`userid`,
`billdate`,
`billfrequency`,
`enddate`,
`amount`,
`description`,
`payee`,
`payer`,
`lastpayment`
)
VALUES (
'', '{$_POST['propertyid']}', '$varuserid', FROM_UNIXTIME($date), '{$_POST['billfrequency']}', 'FROM_UNIXTIME($edate)', '$amount', '{$_POST['description']}', '{$_POST['payee']}', '{$_POST['payer']}', 'FROM_UNIXTIME($ldate)'
)") or die(mysql_error);
Print "A regular payment for £".$_POST['amount']." has been created<br/>\n";
Entering in to the database I parse the date fields from a user entry form to a variable in mm / dd / yyyy format, validate it with the strtotime() or mktime() functions, and then submit it using from_unixtime.
However, then displaying information from the database I use:
date("d-m-Y", strtotime($val['billdate']))
That means I take a dd/mm/yyyy date format from the user and another one from the database. I then convert these to unixtime using strtotime, and then convert it back to British date format using date - This is surely doubling? if not tripling? the server query.
My question is in 2 parts.
Firstly, is it quickest a) storing all date formats in unixtime in sql and converting to date format through the server side query or b) storing all date formats in date format and converting to unixtime only when accessing them
(I thought the latter would be best, but my example above shows that even doing that I seem to be using more server time than I ought to)
Secondly, could you point me in the right direction for how I should carry out my own time benchmarks of scripts - I'm only really beginning to understand testing as I'm still relatively new to Php, but I'm keen to learn.
Thanks so much!
Use MySQL's "unix_timestamp()", it's short and can be converted to readable full date within a second by using php, bash etc.
I have a DB table with relation 1: N where N represents multiple dates for one event. The column with the dates is DateTime type, and I would like to keep the Time option for later use, but it won't be so bad if I have to change it to Date type.
The problem comes when I have to show those multiple dates in some GUI. I get the dates with the GROUP_CONCAT function which means that in JavaScript I operate with a string with comma-separated values representing the different dates, which by now is in the default SQL DateTime format - YYYY-mm-dd hh:mm:ss.
I use the split(',') function to get each date-time value and what I can do is to change the type of the SQL column to Date so when I split the string in JavaScript to end up with YYYY-mm-dd values. Which should be reversed to dd-mm-YYYY for the GUI.
I'm not sure how to proceed here. I have in mind two main options:
First: Maybe there's a way to use dd-mm-YYYY format in SQL which will solve all the problems.
Second: some kind of (complex?!?) String manipulation in JavaScript to split the string of dates into an array with multiple elements and then try to format each element the way I need.
Honestly - I want to avoid the second option, but don't know if the first is possible, and maybe, there's another way that I haven't think of.
Try this..
SELECT GROUP_CONCAT(DATE_FORMAT( date_time_column, '%d-%m-%Y' )) FROM test_table;
First of all I advise to use the native Date/DateTime-format everywhere in your code and only use localized variations like dd-mm-yyyy only where you really want to display it. First reason: it is consistent in your code. Second reason: Sorting.
Example:
2012-03-01 > 01-03-2012
2012-05-12 > 12-05-2012
2012-01-03 > 03-01-2012
Sorted by the native format you'll get...
1: 2012-01-03
2: 2012-03-01
3: 2012-05-12
Sorted by the output format dd-mm-yyyy it will look like this...
1: 01-03-2012
2: 03-01-2012
3: 12-05-2012
...and I doubt this is what most people want.
Adjusting the output via SQL
You can change the output in your SELECT-query , most RDBMS offer functions for this. For example, in MySQL it is DATE_FORMAT, which looks like this:
SELECT DATE_FORMAT(NOW(), '%d-%m-%Y');
Adjusting the output via SQL
You can also change the output using php, which is explained here.
Adjusting the output via JavaScript
It isn't hard to do this in JavaScript, too: Here is a great SO post which explains it in detail.
I have a DATETIME string and I need only the DATE in my script to perform some searches in my database. Currently, I have two scenarios in my mind, but don't know which of them is faster.
The first scenario:
In my MYSQL database, I have two columns: datetime (which is a DATETIME type) and date (which is a DATE type).
Then, in my PHP script, each time I save a record, I will insert my known string to the datetime field, and then convert it to fit the date field (I was thinking of something like: $date = date("Y-m-d", strtotime($datetime))).
This way, all the necessary pieces are stored in my database and I can retrieve them on the fly (both the datetime and the date fields).
The second scenario:
The MYSQL database should consist only of the datetime column.
My PHP script will insert the known string to the datetime field without any other modifications.
And when I retrieve my data, I would do something like: SELECT datetime, DATE(datetime) FROM ...
Conclusion
Which of these scenarios is faster and therefore should be used? Should date formats be made on save or on retrieve? Is MYSQL faster than PHP on formatting dates? Is it better to store everything in the database and retrieve as it is, or store only the minimum and format on retrieve? Which of these scenarios is the best practice?
Thank you!
It depends of your usecases:
If you are only going to need the date for reading, then go with a single datetime column, conversion from datetime to date is cheap enough.
If you are going to select rows at a given date (like WHERE date = '2011-08-01'), then go for a date column, as this will allow mysql to use the indexes on the date column if you have added one.
If you are going to select rows in a date range, then go for a datetime column. You could do things like WHERE datetime >= '2011-08-01' AND datetime < '2011-08-16'.
The second one is the best and fast as you are getting the value based on the requirement. Rather getting some value and working on it later.
imho
datetime, or even unsigned integer (unix timestamp) is better for range filtering
datetime allow date-time function, it could be useful for aggregate function
avoid formatted data from mysql (that's mean raw)
anything related to presentation is PHP duty
Definitely depends on your situation - if you will be reading (a lot) more than writing, you can store both. But I'd go for storing one field (datetime) and convert that, either in PHP or while retrieving it from MySQL (convert datetime to char in the format you like)
I'm trying to store a week schedule (e.g. Monday, 9am-5pm, etc.). I do not have the need to store the dates; I just need to save the following: day, from time, to time.
So, say I have the following time values:
1:20pm
1320
8:00 AM
etc
Assuming that the values are actual valid times, how do I convert these strings into MySQL Time type? And how do I do the reverse? (I'm using PHP.)
Also, how do I query for something like this: find every store that is open on Mondays between 2pm and 3pm? Do I just do something like: WHERE day = 1 AND from_time >= 2pm AND to_time <= 3pm (changing '2pm' and '3pm' to whatever their converted values are, of course)? Or is there some MySQL function better suited for such queries?
MySQL has built in conversion for unix timestamps to a MySQL date:
INSERT INTO table (the_date) VALUES (FROM_UNIXTIME(your_timestamp));
…and the other way around…
SELECT UNIX_TIMESTAMP(the_date) FROM table;
You can use the DAY() and DAYOFWEEK() functions in your WHERE conditionals to convert your MySQL timestamps into the relevant units for you to do your query.
You might need to play around a bit with your schema to determine the best structure to allow you to get the functionality you need here. E.g. it might not make sense to store the day-of-week in a datetime field at all.
MySQL has a TIME data type - it will store values in the hh:mi:ss format, and you can use the TIME_FORMAT function to change the presentation to however you'd like.
Try not to rely on functions applied to the column for comparison - IE:
WHERE TIME_TO_SEC(column) = 123
...because it will render an index, if one exists on the column, useless -- ensuring a table scan (worst performing option).
MySQL understands the ISO 8601 date format, so you have to give time in the form "08:00:00" for 8:00 AM.
You can just use a string with a valid ISO 8601 time in your WHERE clause.
http://en.wikipedia.org/wiki/ISO_8601
http://dev.mysql.com/doc/refman/5.1/en/time.html
http://dev.mysql.com/doc/refman/5.1/en/datetime.html
I have found a proper solution to my "problem" but even after reading mysql pages, I don't understand the logic behind it.
I currently store registration information in my system in a "datetime" formatted field in one of my tables (YYYY-MM-DD hh:mm:ss).
When I want to display the data on one of my php pages, simply posting the exact field data shows the format mentioned above.
I would THINK simply using date("Y-m-d",$row["DATE"]) where $row["DATE"] corresponds to the particular row value would return the desired format.
Instead I have to use:date("Y-m-d", strtotime($row["DATE"])).
Why is this? My $row["DATE"] field is not a string in the first place. Should I be able to simple rearrange the data stored in a datetime field? Wasn't that the purpose of rebuilding my entire tableset to accomodate datetime?
MySQL has a built in function called date_format which you can use to display the date how you want to.
SELECT DATE_FORMAT(date_field, '%Y-%m-%d') as date_field FROM table_name
The manual has the list of formats and the variables needed to display it that way. Using this method there will be no need to have PHP convert it etc. Plus it is less code on PHP side for something MySQL can handle easily.
EDIT
Sorry, just read you were looking for an explanation.
PHP's date function takes in a UNIX timestamp, which MySQL is not using. MySQL uses a real date format IE: YYYY-MM-DD HH:MM:SS, as you know, this is to be compliant for years later. The UNIX timestamp has a limited range from something like 1969 to 2037 that it is valid for, which makes it really useful for "timestamping" of items such as a chat box message or items they are not expected to be around post those dates, where as the MySQL DATETIME should not die out until the year changes to 5 digits or the world ends.
Read the WIKI on UNIX timestamp for more information on it.
MySQL does allow you to select dates in unix timestamp format, which allows them to be used more easily in PHP, exactly as you requested.
The previous answer seemed to ignore this point, or downplay it due to the range restriction on the unix timestamp, but if it's what you're looking for...
SELECT UNIX_TIMESTAMP(datefield) as u_datefield FROM table
will give you the date in timestamp format, which you can use as you suggested in PHP:
<?php
$showdate = date("Y-m-d",$row['u_datefield']);
?>
As the previous answer suggests, unix timestamps do have a limited range, so if you need dates prior to 1970 or after 2038 it may not be suitable, but for everyday use today it's great.
The main advantage of using timestamps over date strings is that timestamps can be added and subtracted, which is much harder with a date in string format.