I want to do a query with thousands of tuples. I need to save the first ID, last ID and date saved in a historic table by day in a new table. I have data from 2020 to 2022. Every day could 600.000 rows or more. I have thought two solutions:
Doing a query every time with limit 600.000 and save the first id, last id and date, all of this order by dates or ids.
Doing a query day by day and get the first and the last id.
The problems are that these querys could delay so much because i am doing orderings.
I´m doing this with SQL and need execute this in PHP with a cron every day to save the data of the day. First, i´m building the new table with the data of past.
Someone would know one tip or antoher form to do this.
THANKS!
You can do this (result here)
select date, min(id) as min, max(id) as max
from logs
group by date
Related
I'm making a PHP based site designed to display line graphs based on data over time. Where the user selects a time range and gets a graph corresponding to what was selected.
The problem is that to calculate any given point, I need to know the previous record. I have no way of knowing when it was, it may have been an hour or a week before hand, but it could have been a minute.
So is there anyway, from within SQL, to specify a time range and one record before that?
You can do another query that gets the last record before the time range:
SELECT *
FROM yourTable
WHERE time < #start_time
ORDER BY time DESC
LIMIT 1
You can combine this with the original query using UNION.
I have a query that counts the "Xp" difference per day from my database, this all works as it should however it groups from midnight-midnight, what I would like to do is group 3am to 3am.
However another issue I think I may have is that my query may not always have the rows being the exact second at 3am due to the fact that it has to run a huge query and retrieve data from another website per user profile, so it should get all data after 3am, but before maybe 4am or something, so it has enough time to get all of the rows.
my current mysql is:
SELECT FROM_UNIXTIME(date, '%Y%m%d') AS YYYYMMDD, MAX(xp)-MIN(xp) AS xp_gain
FROM skills
WHERE userID = '$checkID'
AND skill = '$skill'
AND date >= '$date'
GROUP BY YYYYMMDD
ORDER BY date ASC
The best way to handle this is to add (if you can) another column that is just a DATE (not a DATETIME) and have this field rollover from one day to the next at 3am, (you can to this by subtracting 3 hours from the current time when doing the INSERT).
This gives you a couple of benefits, especially with a large number of rows:
It is much faster to query or group by a DATE than a range of
DATETIME
It will always query the rows at the exact second of 3am,
regardless of how long the query takes.
I have a database of events that are used to fill a PHP calender. I want to also have a page that lists three upcoming events in date order when ever the page loads. The records have a separate field for day, month and year. The issue i run in to is getting records that accrue after the current date.
Table structure will be good to have but I will guess the query that you need. Use that as a point to start from.
SELECT * FROM events WHERE CONCAT(year,'-',month,'-',date) > DATE_FORMAT(NOW(),'%Y-%m-%d')
ORDER BY year DESC, month DESC, date DESC LIMIT 3;
You are going to have to build a date from your three fields to test against current date, or split current date into year, month, day and test all three against data in your table.
Got to say, not a brill decision on someones part to split date like that.
Need table structure before we can properly help query wise.
I have this little script that shows one wisdom each day.
so I have three columns.
Id wisdom timestamp
1 wisdon 1 4/1/2012
2 wisdon 2 4/1/2012
3 wisdon 3 4/2/2012
and I want to fetch array of one wisdom for each day
I looked around your website, but unfortunately I didn't find something similar to what I want.
also I got this code
$sql = mysql_query("SELECT DISTINCT id FROM day_table group by timestamp");
but this also not working.
any ideas?
is it possible to make a counter of 24 hours update wisdom date?
please give me some help.
You can make another table that is called wisdom_of_day
The table would have the following columns, id, wisdom_id, date
Basically each day you can randomly select a wisdom from your wisdom table and insert it into the wisdom day table. You can also add a constraint to your date column so it is distinct. It is important that it is a date column and not a timestamp since you don't care about time.
Then you can retrieve the wisdom of the day by querying based on the date.
It's possible I read your question wrong and you just want to select one wisdom for each day, but you want to show multiple days and you want to get the data from your table.
If so, the reason your query is not working is because you are grouping by a timestamp which includes the date and time. You need to group it by date for it to group like you want.
Here is a query that will group by the day correctly. This will only work if you have a timestamp field and are not storing a unix timstamp on an int column.
select id, wisdom, date(timestamp) date_only from day_table group by date_only order by date_only asc;
Hmm, I noticed that your timestamp values are in some kind of date format, maybe as a string? If so the above query probably won't work.
First compute number of days since 1970
SELECT DATEDIFF(CURDATE(), '1970-01-01')
Then insert this number inside RAND, for example:
SELECT * FROM table ORDER BY RAND(15767) LIMIT 1;
Rand with number as argument is deterministic.
Full query:
SELECT * FROM table ORDER BY RAND((SELECT DATEDIFF(CURDATE(), '1970-01-01'))) LIMIT 1;
I'm building a system that shows "events for this month", listed by day and hour.
When I create the event, I set a start date, an end date and a hour.
Let's say that one event starts at 07-10-2011 and ends 07-12-2011. The problem is that some days in this date range will not feature the event. As an example, this event may happen all days and at the same hour, except some few days where it will not happen or has a different hour (think about a show with an opening date different than the rest of the days).
I'm using PHP, MySQL and Codeigniter and my doubt is about the right way to save those dates in the database. Another table with all the dates and the event ID, or save them all in a field inside the event row? Or something else?
Thanks
I'd create two tables. The first table is an events table, and the other is an events_dates table. This way you can create a single event and have as many dates linked to it as you want.
The events_dates table can be as detailed or simple as you want. If it were me, I'd probably have a start_time and end_time column, as well as an event_id and any other data you want.
I would store the range date in a table and then create an "exception" table where you can store your exceptions.