I am graphing data from a large database (150K+ rows) and some data points have identical timestamps with different price values.
For example:
time => 1502050000
price => 1
time => 1502050000 // identical timestamp
price => 1.1
In the SQL query, I want to ignore duplicate timestamps. I have found that DISTINCT will likely do the job, but I'm stuck with applying this to a datetime field in my database. Here is my working SQL query which pulls in duplicate timestamps.
SELECT time, price FROM price_table WHERE time >= '" . $data_from . "' ORDER BY time ASC
My goal is to get unique timestamps only, and avoid querying duplicates.
You can use aggregation:
SELECT time, MIN(price) as price
FROM price_table
WHERE time >= '" . $data_from . "'
GROUP BY time
ORDER BY time ASC;
Of course, this begs the question of what you want for price when there are duplicates.
Related
I have a column in a MySQL database table that has times (just time of the day ex. "15:45:00") however when I am sorting by time, it doesn't seem to order the rows correctly in order of time. Here is a screenshot from the query:
SELECT * FROM stop_times ORDER BY day ASC, departure_time ASC
Do I need to cast the column in a certain form or convert the time in a way for this to work?
I have an online store with thousands of orders and I'm writing some queries to figure out how much money each supplier (brand) has made on the site. I have the following queries, one for every month of the selected year:
$filterJan = "$filterYear-01";
$queryJan = "SELECT price, quantity FROM order_items WHERE productID='$productID' AND timestamp LIKE '%$filterJan%' LIMIT 10000";
$suppliersQueryFilter = mysql_query($queryJan, $connection) or die(mysql_error());
while($rowF = mysql_fetch_assoc($suppliersQueryFilter)) {
$price = $rowF["price"]*$rowF["quantity"];
$totalJan = $totalJan+$price;
}
** and so on for each month **
It takes ages to load (we're talking over 60 seconds at least) and I know it is not efficient in any shape or form. For each month these queries are searching through thousands of records. Is there a more efficient way or writing this to:
a) Reduce the amount of code to maybe 1 query
b) Make it more efficient to increase loading times
$filterYear contains a year, like 2009.
So what this query does is it selects how much money has been made for each month for a selected year (which is assigned to $filterYear). So the result it generates is a table with Jan, Feb, March... with how much money has been made each month, so £2345, £2101, etc...
You should be storing your timestamp as an actual mysql datetime value, which would make things like
GROUP BY YEAR(timestamp), MONTH(timestamp)
WHERE timestamp BETWEEN $initialtime AND $finaltime
trivially possible. That'd reduce your multiple essentially identical repeated queries to just one single query.
You can use derived values for this, but it'll be less efficient than using a native datetime field:
GROUP BY SUBSTR(timestamp, 0, 4), SUBSTR(timestamp, 6,2)
For best performance, you'd want to submit a query something like this to the database:
SELECT DATE_FORMAT(i.timestamp,'%Y-%m') AS `month`
, SUM(i.price*i.qty) AS `total`
FROM order_items i
WHERE i.productID = 'foo'
AND i.timestamp >= '2013-01-01'
AND i.timestamp < '2013-01-01' + INTERVAL 12 MONTH
GROUP
BY DATE_FORMAT(i.timestamp,'%Y-%m')
(This assumes that the timestamp column is MySQL datatype TIMESTAMP, DATETIME or DATE)
Using the deprecated mysql_ interface, you want to avoid SQL Injection vulnerabilities using the mysql_real_escape_string function. (A better option would be to use the mysqli or PDO interface, and use a prepared statement with bind placeholders.)
We want the predicates on the timestamp to be on the BARE column, so MySQL can make use of an available suitable index for a range scan operation, rather than requiring a full scan of every row in the table.
We also want to use the power of the server to quickly derive a total, and return just the total, rather than retrieving every flipping row, and processing each of those rows individually (RBAR = row by agonizing row)
The GROUP BY clause and the SUM() aggregate function are tailor made to suit this result.
With mysql_ interface, the query would look something like this:
$querytext = "
SELECT DATE_FORMAT(i.timestamp,'%Y-%m') AS `month`
, SUM(i.price*i.qty) AS `total`
FROM order_items i
WHERE i.productID = '" . mysql_real_escape_string($thisProductID) . "'
AND i.timestamp >= '" . mysql_real_escape_string($filterYear) . "-01-01'
AND i.timestamp < '" . mysql_real_escape_string($filterYear) . "-01-01' +
INTERVAL 12 MONTH
GROUP BY DATE_FORMAT(i.timestamp,'%Y-%m')";
# for debugging
#echo $querytext;
I'm using mysql and I'm having trouble thinking of a query to count the number of users/visitors for a certain date range. The way that I'm currently doing it is using php, I select the date range and process the data in a for loop and then just count them there. It's actually pretty easy, but the problem is that this method does not work for bigger data of a few million rows. The alternative is to count the distinct values using mysql only and just return a count and not actual data by utilizing the index on the timestamp column. Also, converting the column to a datetime is not an option. Any ideas how I can achieve this?
Here's a sample result set of what I need:
date | count
5-01-13 14
5-02-13 44
5-03-13 23
5-04-13 13
My problem is that I don't know how to group the timestamp column by day.
That should do the trick:
SELECT DATE('datetimecolumn'), COUNT(*)
FROM Table
GROUP BY DATE('datetimecolumn')
You just have to do the same, but instead add a group by clause:
SELECT myDate, count(distinct myField) as cnt
FROM myTable
WHERE myDate BETWEEN ? and ?
GROUP by myDate;
Where the "?" are the dates you use in your original query.
Trying to get my head around, If I select a record such as WHERE item_id='$item_id' AND date(datetime)='2012-06-25' and if that record does not exist so I want to get the nearest-latest record after that date. How can I achieve that in a query?
All I can think of the only way right now is if num_of_rows is 0 then I add 3 days period ahead to that day and search again and get the DESC datetime LIMIT 1 (in case there are multiple rows). But who knows I can do it with just a query.
The record could have multiple rows in one day. So if a particular date has no record, how to get the next nearest available data given the same $item_id?
SELECT *
FROM table
WHERE field <= '2012-06-25'
ORDER BY field DESC
LIMIT 1
I think this is what you are looking for:
SELECT *
FROM my_table
WHERE datetime BETWEEN '2012-06-25 00:00:00' AND
DATE_ADD('2012-06-25 00:00:00', INTERVAL 3 DAY)
ORDER BY datetime ASC
LIMIT 1;
also create index on field datetime for faster performance.
This will bring back the item closest to the date that you enter into the query. It won't however look for before or after, just find the closest date to what you enter in.
select
min(abs(DATEDIFF(date(datetime),'2012-06-25'))) as minDiff
,yourID
from table1
group by yourID
order by 1 asc;
I have this little script that shows one wisdom each day.
so I have three columns.
Id wisdom timestamp
1 wisdon 1 4/1/2012
2 wisdon 2 4/1/2012
3 wisdon 3 4/2/2012
and I want to fetch array of one wisdom for each day
I looked around your website, but unfortunately I didn't find something similar to what I want.
also I got this code
$sql = mysql_query("SELECT DISTINCT id FROM day_table group by timestamp");
but this also not working.
any ideas?
is it possible to make a counter of 24 hours update wisdom date?
please give me some help.
You can make another table that is called wisdom_of_day
The table would have the following columns, id, wisdom_id, date
Basically each day you can randomly select a wisdom from your wisdom table and insert it into the wisdom day table. You can also add a constraint to your date column so it is distinct. It is important that it is a date column and not a timestamp since you don't care about time.
Then you can retrieve the wisdom of the day by querying based on the date.
It's possible I read your question wrong and you just want to select one wisdom for each day, but you want to show multiple days and you want to get the data from your table.
If so, the reason your query is not working is because you are grouping by a timestamp which includes the date and time. You need to group it by date for it to group like you want.
Here is a query that will group by the day correctly. This will only work if you have a timestamp field and are not storing a unix timstamp on an int column.
select id, wisdom, date(timestamp) date_only from day_table group by date_only order by date_only asc;
Hmm, I noticed that your timestamp values are in some kind of date format, maybe as a string? If so the above query probably won't work.
First compute number of days since 1970
SELECT DATEDIFF(CURDATE(), '1970-01-01')
Then insert this number inside RAND, for example:
SELECT * FROM table ORDER BY RAND(15767) LIMIT 1;
Rand with number as argument is deterministic.
Full query:
SELECT * FROM table ORDER BY RAND((SELECT DATEDIFF(CURDATE(), '1970-01-01'))) LIMIT 1;