Creating hour groups for time series data MySQL - php

Question

I have a MySQL database with data recorded every 15 minutes. For simplicity, lets assume there are 2 fields:
DATETIME Created
Double Value
I would like to draw a chart which needs for each hour the opening, min, max, and closing values for an hour. To do this I need to return results from my MySQL query to my PHP to create a JSON. I would like to do this in the MySQL query so that the response is cached.
Here is an example of the problem, given 9 data points trying to get 2 hour groups:
Creation Value
2014-03-25 12:15:00 413.17011
2014-03-25 12:00:00 414
2014-03-25 11:45:00 415
2014-03-25 11:30:00 415
2014-03-25 11:15:00 415.5
2014-03-25 11:00:00 415.5
2014-03-25 10:45:00 416
2014-03-25 10:30:00 416
2014-03-25 10:15:00 415.99
I would need:
Hour 1 (11:15:00 to 12:15:00)
Open: 415.5
Close: 413.17011
High: 415.5
Low: 413.17011
Hour 2 (10:15:00 to 11:15:00)
Open: 415.99
Close: 415.5
High: 416
Low: 415.5
Of course for the full 24 hours this would need repeating, this is just an example.
Any help is really appreciated!
Here is the current MySQL dump for the example (Using MySQL version 2.6.4-pl3):
--
-- Table structure for table `exampleTable`
--
CREATE TABLE `exampleTable` (
`created` datetime NOT NULL,
`value` double NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci;
--
-- Dumping data for table `exampleTable`
--
INSERT INTO `exampleTable` VALUES ('2014-03-25 12:15:00', 413.17011);
INSERT INTO `exampleTable` VALUES ('2014-03-25 12:00:00', 414);
INSERT INTO `exampleTable` VALUES ('2014-03-25 11:45:00', 415);
INSERT INTO `exampleTable` VALUES ('2014-03-25 11:30:00', 415);
INSERT INTO `exampleTable` VALUES ('2014-03-25 11:15:00', 415.5);
INSERT INTO `exampleTable` VALUES ('2014-03-25 11:00:00', 415.5);
INSERT INTO `exampleTable` VALUES ('2014-03-25 10:45:00', 416);
INSERT INTO `exampleTable` VALUES ('2014-03-25 10:30:00', 416);
INSERT INTO `exampleTable` VALUES ('2014-03-25 10:15:00', 415.99);

Answer 1

Get it to work
You might try
SELECT
DATE(created) AS day,
HOUR(created) AS hour,
(
SELECT Value FROM `table` AS b
WHERE DATE(a.created) = DATE(b.created)
AND HOUR(a.created) = HOUR(b.created)
ORDER BY created ASC LIMIT 1
) AS Open,
(
SELECT Value FROM `table` AS b
WHERE DATE(a.created) = DATE(b.created)
AND HOUR(a.created) = HOUR(b.created)
ORDER BY created DESC LIMIT 1
) AS Close,
MIN(value) AS Low,
MAX(value) AS High
FROM `table` AS a
GROUP BY DATE(created), HOUR(created)
this groups all your rows by DATE+HOUR and computes the MIN respectively MAX as Low or High. To find the first and last row for Open and Close, the easiest in SQL syntax is a subselect. It selects all rows which are relevant for the current row, and sorts them ascending or descending. Then selects the first row.
Please consider that this groups only by hour. Instead of
Hour 1 (11:15:00 to 12:15:00)
Hour 2 (10:15:00 to 11:15:00)
this groups like
Hour 1 (11:00:00 to 11:59:00)
Hour 2 (10:00:00 to 10:59:00)
If you want to keep the 15 minutes offset, you may subtract this from your created timestamp (created - INTERVAL 15 MINUTE) at all occurrences of created in the sql query above.
I created a working sqlfiddle for you.
Performance
Just as hint: If you can, you might want to split date and time into two columns (of types date and time). This way you do not need to cast DATE() on created everytime, but can use the new date column instead. You can then add a combined index to this new columns too, which speeds up your query. See this sqlfiddle for an example.

Answer 2

To get your grouping right, you can use
FLOOR(( UNIX_TIMESTAMP(myTable.dateCreated) - 900 ) / 3600)
where 3600 sets the interval at 1 hour and the - 900 sets the offset at 00:15
Since you need the MIN() and MAX for each of your four values, you'll need to JOIN the main table to itself but grouped by the min or max (based on the column).
finally, you have each sub-query (joined table) calculate the grouping hour above so you can use that to join them. Here's what I cam up with (with slightly different column names and
SELECT openDate,Open,Close,High,Low
FROM (SELECT FLOOR(( UNIX_TIMESTAMP(myTable.dateCreated) - 900 ) / 3600)
AS
theHour,
myTable.value AS Open,myTable.dateCreated openDate
FROM myTable
JOIN (SELECT value,MIN(dateCreated) AS dateCreated
FROM myTable
GROUP BY FLOOR(( UNIX_TIMESTAMP(dateCreated) - 900 )
/ 3600)
) AS
aggTable
ON aggTable.dateCreated = myTable.dateCreated) AS
openTable
LEFT JOIN (SELECT FLOOR(( UNIX_TIMESTAMP(myTable.dateCreated) - 900
) /
3600) AS
theHour
,
myTable.value AS Close,myTable.dateCreated closeDate
FROM myTable
JOIN (SELECT value,MAX(dateCreated) AS dateCreated
FROM myTable
GROUP BY FLOOR(( UNIX_TIMESTAMP(dateCreated) - 900 ) / 3600)
) AS
aggTable
ON aggTable.dateCreated = myTable.dateCreated) AS closeTable
ON openTable.theHour = closeTable.theHour
LEFT JOIN (SELECT
FLOOR((
UNIX_TIMESTAMP(myTable.dateCreated) - 900 ) / 3600) AS
theHour,
MAX(
value)
AS High
FROM myTable
GROUP BY theHour) AS highTable
ON closeTable.theHour = highTable.theHour
LEFT JOIN (SELECT
FLOOR((
UNIX_TIMESTAMP(myTable.dateCreated) - 900 ) / 3600) AS
theHour,
MIN(
value)
AS Low
FROM myTable
GROUP BY theHour) AS lowTable
ON highTable.theHour = lowTable.theHour