I currently have a table with 1,100,000 rows which contains user's data.
Its format is sort of like this:
User_Id Date Action
I was wondering, instead of searching each time on the whole table for the actions that were made by a specific user on a specific date by doing the following:
SELECT Action FROM USERS_TABLE WHERE Date=08092014 AND User_Id=5
SELECT Action FROM USERS_TABLE WHERE Date=09092014 AND User_Id=5
SELECT Date FROM USERS_TABLE WHERE Action="Shopping" AND User_Id=5
SELECT Date FROM USERS_TABLE WHERE Action="Eating" AND User_Id=5
etc.
Maybe I could do something like that:
SELECT * FROM USERS_TABLE WHERE User_Id=5
And on top of this query's results I could run the above queries, which I think will result a faster execution time (correct me if I'm wrong)
Do you guys know how to do that?
You could combine all of those queries into one query using an or.
SELECT *
FROM USERS_TABLE
WHERE (Date = 09092014 OR Date = 08092014)
AND (Action="Shopping" OR Action="Eating")
AND User_Id = 5
I assume you have a table with unique users ids. if you don't, you might consider it? How can a profile be managed if there is no single entry for a single user? anyway that's not my business, but let's just assume you have such a table, with a unique field with the User_Id
it's named USERS here
SELECT Action,Date
FROM USERS
LEFT JOIN USERS_TABLE AS Actions
ON (Actions.User_Id=USERS.User_Id AND Date IN (08092014,09092014))
LEFT JOIN USERS_TABLE AS Dates
ON (Dates.User_Id=USERS.User_Id AND Action IN ("Shopping","Eating"))
WHERE USERS.User_Id=5
be sure to index User_Id, Date And Action since we are searching on them.
I would do a crosstab query, after I indexed the User_Id column -
SELECT `Date`,
SUM(IF(`Action` = 'Eating', 1, 0)) AS `Eating`,
SUM(IF(`Action` = 'Shopping', 1, 0)) AS `Shopping`
FROM `USERS_TABLE`
WHERE `User_Id` = 5
GROUP BY `Date`
You'll get a result like this -
+-------------+---------------+----------+
Date Eating Shopping
+-------------+---------------+----------+
2002-03-01 59 72
2002-03-02 28 0
2002-03-03 22 17
2002-03-04 36 13
2002-03-06 12 0
+-------------+---------------+----------+
For expediency I might store this data in a temp table (with a user id column). This can be modified to accept date ranges and other limitations. That gives me some additional flexibility down the line when I need to aggregate date from multiple users.
I think what you mean is answered by this:
select action, actiondate
from
(select *
from USERS_TABLE
where user_id = 5) as filter
Fiddle here.
The derived table basically acts as the filter you describe.
Whether it would be any faster is hard to predict - I'd run it on your production system, and see what the query plan says.
Related
I have an orders grid holding 1 million records. The page has pagination, sort and search options. So If the sort order is set by customer name with a search key and the page number is 1, it is working fine.
SELECT * FROM orders WHERE customer_name like '%Henry%' ORDER BY
customer_name desc limit 10 offset 0
It becomes a problem when the User clicks on the last page.
SELECT * FROM orders WHERE customer_name like '%Henry%' ORDER BY
customer_name desc limit 10 offset 100000
The above query takes forever to load. Index is set to the order id, customer name, date of order column.
I can use this solution https://explainextended.com/2009/10/23/mysql-order-by-limit-performance-late-row-lookups/ if I don't have a non-primary key sort option, but in my case sorting is user selected. It will change from Order id, customer name, date of order etc.
Any help would be appreciated. Thanks.
Problem 1:
LIKE "%..." -- The leading wildcard requires a full scan of the data, or at least until it finds the 100000+10 rows. Even
... WHERE ... LIKE '%qzx%' ... LIMIT 10
is problematic, since there probably not 10 such names. So, a full scan of your million names.
... WHERE name LIKE 'James%' ...
will at least start in the middle of the table-- if there is an index starting with name. But still, the LIMIT and OFFSET might conspire to require reading the rest of the table.
Problem 2: (before you edited your Question!)
If you leave out the WHERE, do you really expect the user to page through a million names looking for something?
This is a UI problem.
If you have a million rows, and the output is ordered by Customer_name, that makes it easy to see the Aarons and the Zywickis, but not anyone else. How would you get to me (James)? Either you have 100K links and I am somewhere near the middle, or the poor user would have to press [Next] 'forever'.
My point is that the database is not the place to introduce efficiency.
In some other situations, it is meaningful to go to the [Next] (or [Prev]) page. In these situations, "remember where you left off", then use that to efficiently reach into the table. OFFSET is not efficient. More on Pagination
I use a special concept for this. First I have a table called pager. It contains an primary pager_id, and some values to identify a user (user_id,session_id), so that the pager data can't be stolen.
Then I have a second table called pager_filter. I consist of 3 ids:
pager_id int unsigned not NULL # id of table pager
order_id int unsigned not NULL # store the order here
reference_id int unsigned not NULL # reference into the data table
primary key(pager_id,order_id);
As first operation I select all records matching the filter rules from and insert them into pager_filter
DELETE FROM pager_filter WHERE pager_id = $PAGER_ID;
INSERT INTO pager_filter (pager_id,order_id,reference_id)
SELECT $PAGER_ID pager_id, ROW_NUMBER() order_id, data_id reference_id
FROM data_table
WHERE $CONDITIONS
ORDER BY $ORDERING
After filling the filter table you can use an inner join for pagination:
SELECT d.*
FROM pager_filter f
INNER JOIN data_table d ON d.data_id = f.reference id
WHERE f.pager_id = $PAGER_ID && f.order_id between 100000 and 100099
ORDER BY f.order_id
or
SELECT d.*
FROM pager_filter f
INNER JOIN data_table d ON d.data_id = f.reference id
WHERE f.pager_id = $PAGER_ID
ORDER BY f.order_id
LIMIT 100 OFFSET 100000
Hint: All code above is not tested pseudo code
is it possible to sellect a row from a table like so
SELECT * FROM users WHERE username=? FROM date 2015-02-17 08:12:54 TO date 2015-02-17 09:12:54
Because I have a login table, and I want to check to make sure they're not bruteforcing.
Is that possible?
The syntax looks like this:
SELECT u.*
FROM users u
WHERE u.username = ? AND
u.date BETWEEN '2015-02-17 08:12:54' AND '2015-02-17 09:12:54'
If you have a large table, this query can take advantage of an index on users(username, date).
Yes its possible
The smaller date has to be listed first
Example
WHERE BETWEEN '12/19/2012' AND '1/17/2013'
I have several tables in my database such as
comments
status
events
I’m trying to create an SQL query procedure which counts data from these different tables based on the userID entered and then sum up the counts to create a unique valued. This is what i’ve tried so far but i’m having problems with the syntax. Where am I going wrong??
SELECT COUNT(user_id) AS comments FROM comment
WHERE user_id= userID
UNION ALL
SELECT COUNT(creator_id) AS events FROM event
WHERE creator_id=userID;
In a union, the fields are combined based on order. So giving the count field a different name in each part of the union does not make two fields. It becomes the same field in the end. To differentiate which value came from which table, add a hardcoded string literal like so:
SELECT COUNT(user_id) AS rows, 'comment' as tablename FROM comment
WHERE user_id= userID
UNION ALL
SELECT COUNT(creator_id) AS rows, 'event' as tablename FROM event
WHERE creator_id=userID;
I'm trying to create a query that will select all dates between two dates
This is my query:
$query = "SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN '$date1' AND '$date2' AND D1.D1_ID = D2.D2_ID";
The trouble is, it is not returning anything, but not producing an error either
So I tried inputting it directly into phpMyAdmin like this
SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN '2011-01-01' AND '2011-12-12'
AND D1.D1_ID = D2.D2_ID`
then like this
SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN '2011-01-01' AND '2011-12-12'
and like this
SELECT * FROM D1
WHERE DATE_ADDED BETWEEN '2011-01-01' AND '2011-12-12'
and I just get
MySQL returned an empty result set (i.e. zero rows). ( Query took 0.0003 sec )
Yes, my tables exist, and so do the columns :)
In the first cases the lack of results could be because of the inner join. For a result to be in the set it would require a record in both tables, ie. a record from d1 would not appear unless d2 also had that id in the d2_id column. To resolve this, if that is correct for your business logic, use left join.
However, the last of your cases (without the join) suggests the reasons is a lack of matching records in the first (left) table d1.
Without the full dataset we can't really comment further, since all the code you are running is perfectly valid.
If you always want to select an entire year it is easer to select it like this:
SELECT * FROM D1 WHERE YEAR(DATE_ADDED) = 2011;
Please implement below code
SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN DATE_FORMAT('2011-01-01','%Y-%m-%d')
AND DATE_FORMAT('2011-12-12','%Y-%m-%d')
AND D1.D1_ID = D2.D2_ID`
Alright, I'm trying to figure out why I can't understand how to do this well...
I have two tables:
invoices:
id
userID
amount
date
payments:
id
userID
amount
date
So, the goal here is to join both tables, where the userID matches whatever I want it to be - and then return everything ordered by date (most recent at the top). However, because there is a date field in each of the tables, I'm not sure how MySQL will handle things... will is sort by both dates automatically? Here's what I was thinking...
"SELECT DISTINCT *
FROM invoices,payments
WHERE {$userID} = invoice.userID
OR {$userID} = payments.userID
ORDER BY date DESC";
But, it's starting to become clear to me that maybe this isn't even the right use of a join command... maybe I need to just get all data on each table alone, then try to sort it somehow with PHP? If that's the better method, what's a good way to do this type of DATE sort while keeping all row data in tact?
I should add, the TIME inside the unix timestamp (that's how "date" is stored) is NOT negligible - it should sort by the date and time.
Thanks all...
If the columns of both tables are the same, you can use a UNION
SELECT X.*
FROM ( SELECT `id`,
`userID`,
'INVOICE' AS PTYPE
`amount`,
`date`
FROM `invoices`
WHERE {$userID} = userID
UNION
SELECT `id`,
`userID`,
'PAYMENT' AS PTYPE
`amount`,
`date`
FROM `payments`
WHERE {$userID} = userID
) X
ORDER BY X.`date`
EDIT
Read the relevant section of the MySQL manual on UNIONS. There are other ways of phrasing this, but this is my preferred style - it should be clear to anybody reading that the ORDER BY clause applies to the result of both sides of the UNION. A carelessly written UNION - even with an ORDER BY - may still leave the final resultset in indeterminate order.
The purpose of the PTYPE is that this query returns an extra column called PTYPE, that indicates whether each individual row is an INVOICE or a PAYMENT... ie. which of the two tables it comes from. It's not mandatory, but can often be useful within a union
Because you have two identical fields named date, MySQL will not know which one you're trying to order by.
"SELECT DISTINCT *
FROM invoices,payments
WHERE {$userID} = invoice.userID
OR {$userID} = payments.userID
ORDER BY invoices.date, payments.date DESC";
This would sort on the invoice date, then the payment date - if that's what you are trying to find out
If your data tipe is Date, Timestamp, or anything related, the SGBD will order it properly. If that was what you've asked.
But if the datatype is String, even when dates is store, it will not sort the way you want.