Getting random record from database with group by - php

Hello i have a question on picking random entries from a database. I have 4 tables, products, bids and autobids, and users.
Products
-------
id 20,21,22,23,24(prime_key)
price...........
etc...........
users
-------
id(prim_key)
name user1,user2,user3
etc
bids
-------
product_id
user_id
created
autobids
--------
user_id
product_id
Now a multiple users can have an autobid on an product. So for the next bidder I want to select a random user from the autobid table
example of the query in language:
for each product in the autobid table I want a random user, which is not the last bidder.
On product 20 has user1,user2,user3 an autobidding.
On product 21 has user1,user2,user3 an autobidding
Then I want a resultset that looks for example like this
20 – user2
21 – user3
Just a random user. I tried miximg the GOUP BY (product_id) and making it RAND(), but I just can't get the right values from it. Now I am getting a random user, but all the values that go with it don't match.
Can someone please help me construct this query, I am using php and mysql

The first part of the solution is concerned with identifying the latest bid for each product: these eventually wind up in temporary table "latest_bid".
Then, we assign randon rank values to each autobid for each product - excluding the latest bid for each product. We then choose the highest rank value for each product, and then output the user_id and product_id of the autobids with those highest rank values.
create temporary table lastbids (product_id int not null,
created datetime not null,
primary key( product_id, created ) );
insert into lastbids
select product_id, max(created)
from bids
group by product_id;
create temporary table latest_bid ( user_id int not null,
product_id int not null,
primary key( user_id, product_id) );
insert into latest_bid
select product_id, user_id
from bids b
join lastbids lb on lb.product_id = b.product_id and lb.created = b.created;
create temporary table rank ( user_id int not null,
product_id int not null,
rank float not null,
primary key( product_id, rank ));
# "ignore" duplicates - it should not matter
# left join on latest_bid to exclude latest_bid for each product
insert ignore into rank
select user_id, product_id, rand()
from autobids a
left join latest_bid lb on a.user_id = lb.user_id and a.product_id = lb.product_id
where lb.user_id is null;
create temporary table choice
as select product_id,max(rank) choice
from rank group by product_id;
select user_id, res.product_id from rank res
join choice on res.product_id = choice.product_id and res.rank = choice.choice;

You can use the LIMIT statement in conjunction with server-side PREPARE.
Here is an example that selects a random row from the table mysql.help_category:
select #choice:= (rand() * count(*)) from mysql.help_category;
prepare rand_msg from 'select * from mysql.help_category limit ?,1';
execute rand_msg using #choice;
deallocate prepare rand_msg;
This will need refining to prevent #choice becoming zero, but the general idea works.
Alternatively, your application can construct the count itself by running the first select, and constructing the second select with a hard-coded limit value:
select count(*) from mysql.help_category;
# application then calculates limit value and constructs the select statement:
select * from mysql.help_category limit 5,1;

Related

How do i optimise a mysql query that updates rank order in nested loop?

There is a score link table that holds a list of item_ids, the category_ids they are assigned to, and a score.
What I am trying to do is
For each category, grab all the items assigned to it, ordered by highest score first.
This then defines that items RANK ORDER in that category.
I then want to store that items RANK in the same table against the matching item_id and category_id.
This works great with the PHP MySQL code i have, but unfortunately there are 10000 categories and so takes about 30mins before timing out.
Here is an example of what im doing:
SELECT category_id FROM tbl_categories /* - this just grabs a list of categorys to loop through */
Loop through each row returned {
SELECT item_id, score
FROM tbl_scores
WHERE category_id = CATEGORY_ID ORDER BY score DESC
MYCOUNT = 0
Loop through each results, incrementing the count and storing in another tables {
MYCOUNT = MYCOUNT + 1
UPDATE tbl_scores
SET rank = MYCOUNT
WHERE item_id = ITEMID AND category_id = CATEGORYID
}
}
Try this query to get the rank for each product (using session variables, see more info here MySql - How get value in previous row and value in next row?)
SELECT
s.categoryid ,
s.itemid,
s.score,
#rowid := IF (#prev_categroy = s.categoryid, #rowid+1, 0),
#prev_categroy := s.categoryid AS curr
FROM scores s, (SELECT #rowid:=0, #prev_categroy := NULL) AS init
ORDER BY s.categoryid, s.score DESC
I would store the results of this (ranks) in a temporary table and then update in another statement with a simple join by item_id, supposing the size of the table is not huge.

How to query for many to many relationship between products and filters in MySQL?

I have three tables viz. tb_filters, tb_products, and tb_products_to_filters. The structure of these tables along with some dummy data is given by:
tb_filters:
CREATE TABLE IF NOT EXISTS `tb_filters`
(
`filter_id` INT (11) AUTO_INCREMENT PRIMARY KEY,
`filter_name` VARCHAR (255)
);
INSERT INTO `tb_filters`
(`filter_name`)
VALUES ('USB'),
('High Speed'),
('Wireless'),
('Ethernet');
tb_products:
CREATE TABLE IF NOT EXISTS `tb_products`
(
`product_id` INT (11) AUTO_INCREMENT PRIMARY KEY,
`product_name` VARCHAR (255)
);
INSERT INTO `tb_products`
(`product_name`)
VALUES ('Ohm precision shunt resistor'),
('Orchestrator Libraries'),
('5cm scanner connection'),
('Channel isolated digital'),
('Network Interface Module');
tb_products_to_filters:
CREATE TABLE IF NOT EXISTS `tb_products_to_filters`
(
`id` INT (11) AUTO_INCREMENT PRIMARY KEY,
`product_id` INT (11),
`filter_id` INT (11)
);
INSERT INTO `tb_products_to_filters`
(`product_id`, `filter_id`)
VALUES (1, 1),
(2, 2),
(3, 3),
(4, 3),
(1, 3);
By looking into above "tb_products_to_filters" table, my required queries are:
When filter id = 1 and 3 are selected via checkbox on the page, all those products which belong to filter id 1 as well as filter id 3 must be fetched from the database. In this case, the product with id 1 should come.
Second, when only one filter (say id = 3) is checked, then all those products which fall under this id should be fetched. In this condition, the products id 1, 3 and 4 will come.
If filter id 2 is selected, then only one product with id = 2 will come.
If combination of filter (2 and 3) is selected, then no product will come because there is no product which belongs to both of them.
What is the way of writing queries to obtain above goal?
Please note that I want to include columns: product_id, product_name, filter_id and filter_name to display data in table result set.
EDIT:
The output should match below when filter ids 1 and 3 were checked:
EDIT 2:
I'm trying below query to fetch results when filter 1 and 3 were checked:
SELECT `p`.`product_id`, `p`.`product_name`,
GROUP_CONCAT(DISTINCT `f`.`filter_id` ORDER BY `f`.`filter_id` SEPARATOR ', ') AS filter_id, GROUP_CONCAT(DISTINCT `f`.`filter_name` ORDER BY `f`.`filter_name` SEPARATOR ', ') AS filter_name
FROM `tb_products` AS `p` INNER JOIN `tb_products_to_filters` AS `ptf`
ON `p`.`product_id` = `ptf`.`product_id` INNER JOIN `tb_filters` AS `f`
ON `ptf`.`filter_id` = `f`.`filter_id` GROUP BY `p`.`product_id`
HAVING GROUP_CONCAT(DISTINCT `ptf`.`filter_id` SEPARATOR ', ') = ('1,3')
ORDER BY `p`.`product_id`
But unfortunately, it returns an empty set. Why?
You can use the HAVING clause with GROUP_CONCAT :
SELECT t.product_id,tp.product_name,
GROUP_CONCAT(t.filter_id) as filter_id,
GROUP_CONCAT(tb.filter_name) as filter_name
FROM tb_products_to_filters t
INNER JOIN tb_filters tb ON(t.filter_id = tb.filter_id)
INNER JOIN tb_products tp ON(t.product_id = tp.product_id)
WHERE t.filter_id IN(1,3)
GROUP BY t.product_id
HAVING COUNT(distinct t.filter_id) = 2
You can adjust this any way you want. Note that the number of arguments placed inside the IN() should be the same as the COUNT(..) = X
EDIT:
A DISTINCT keyword is required in GROUP_CONCAT while fetching those columns otherwise all the filters would come in the list. I tried it by doing
SELECT t.product_id,tp.product_name,
GROUP_CONCAT(DISTINCT t.filter_id ORDER BY `t`.`filter_id` SEPARATOR ', ') as filter_id,
GROUP_CONCAT(DISTINCT tb.filter_name ORDER BY tb.filter_name SEPARATOR ', ') as filter_name
FROM tb_products_to_filters t
INNER JOIN tb_filters tb ON(t.filter_id = tb.filter_id)
INNER JOIN tb_products tp ON(t.product_id = tp.product_id)
WHERE t.filter_id IN(1,3)
GROUP BY t.product_id
HAVING COUNT(distinct t.filter_id) = 2
But still all the filter names (Ethernet, High Speed, USB, Wireless) are coming in the list. How to list only those filter names whose corresponding filter id (1, 3) are in the string?

Speed up slow 3 table MYSQL query

I'm querying 3 tables in an eCommerce site.
orders Table:
id
order_number
name
etc...
order_lines table:
id
order_number
sku
quantity
etc.
products Table
id
sku
title
ship_by (INT)
etc.
order_number links orders table to order_lines table. SKU links order_lines table to products table.
Notice the ship_by column in the products table, this denotes which supplier will ship the item. The query needs to pull orders for a particular supplier. Orders may contain items sold by different suppliers.
This is the query I managed to cobble together:
SELECT
orders.`order_number` as `orderId`,
orders.`shipping` as `fPostageCost`,
FROM_UNIXTIME(orders.`time`) as `dReceievedDate`,
(
CASE orders.`dob`
WHEN 'COURIER 3 DAY' THEN 'UK_SellersStandardRate'
WHEN 'COURIER 24 HOUR' THEN 'UK_OtherCourier24'
ELSE orders.`dob`
END
)
...(plus a number more)
FROM orders
INNER JOIN order_lines
ON orders.order_number = order_lines.order_number
WHERE
(
(SELECT COUNT(order_lines.sku)
FROM order_lines, products
WHERE order_lines.sku = products.sku
AND products.ship_by = 1
AND order_lines.order_number = orders.order_number) > 0
)
AND ( orders.`printed` = 'N' OR orders.`printed` IS NULL )
AND orders.status = 'Awaiting Despatch'
AND ( orders.payment_status = 'Success' OR orders.payment_status = 'Paypal Paid' OR orders.payment_status = 'Manual Payment' )
GROUP BY orders.`order_number`
ORDER BY orders.order_number ASC
It is taking about 7 seconds to execute the query.
If I remove the line 'AND order_lines.order_number = orders.order_number' from the second SELECT query it is executed almost instantly, but without this line it doesn't work as I need it to.
Aside from adding the 'ship_by' column into the order_lines table (which I'd rather not do since I'd have to change a lot of php code), is there any way of modifying this query to speed it up?
The query is being run from outside of PHP so it has to be a pure mysql query.
Here is the EXPLAIN on the query:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY order_lines ALL NULL NULL NULL NULL 9627 Using temporary; Using filesort
1 PRIMARY orders eq_ref order_idx order_idx 17 .order_lines.order_number 1 Using where
2 DEPENDENT SUBQUERY order_lines ALL NULL NULL NULL NULL 9627 Using where
2 DEPENDENT SUBQUERY products ref sku_2,sku,sku_3 sku_2 63 order_lines.prod_code 11 Using where
Thanks
Your order_lines table needs an index on order_number at the very least. We would need to see the schema to best choose one. Perhaps a composite index would pick up speed in other areas.
But in choosing index changes, it must be carefully weighed against other queries in your system, and the impact on insert and update speeds.
The goal shouldn't be to make 10% of your app fast, at the expense of the 90.
To show very useful information, publish show create table tableName for relevant tablenames. (versus describing it free-hand, like my table has this and that). We need to see the schema for orders, order_lines, products.
The manual page on Create Index
Idk how large the table is on your inline view but if it is large then
it may help to use a union instead of having to query the view for every record.
Something like.
SELECT L1*
FROM
(SELECT
orders.`order_number` as `orderId`,
0 AS SKU_CNT,
orders.`shipping` as `fPostageCost`,
FROM_UNIXTIME(orders.`time`) as `dReceievedDate`,
(
CASE orders.`dob`
WHEN 'COURIER 3 DAY' THEN 'UK_SellersStandardRate'
WHEN 'COURIER 24 HOUR' THEN 'UK_OtherCourier24'
ELSE orders.`dob`
END
)
...(plus a number more)
FROM orders
INNER JOIN order_lines
ON orders.order_number = order_lines.order_number
WHERE ( orders.`printed` = 'N' OR orders.`printed` IS NULL )
AND orders.status = 'Awaiting Despatch'
AND ( orders.payment_status = 'Success' OR orders.payment_status = 'Paypal Paid' OR orders.payment_status = 'Manual Payment' )
UNION ALL
Select
0 AS ORDERID,
COUNT(OL.SKU) AS SKU_CNT,
0 as `fPostageCost`,
NULL as `dReceievedDate`,
...
FROM ORDER_LINES OL,
PRODUCTS PR
WHERE order_lines.sku = products.sku
AND products.ship_by = 1
AND order_lines.order_number = orders.order_number
GROUP BY ORDERID, ORDER_LINES_CNT, fPostageCost`, dReceievedDate`,
)L1
WHERE L1.SKU_CNT > 0
GROUP BY L1.`order_number`
ORDER BY L1.order_number ASC
You can replace the subquery with exists:
WHERE EXISTS (SELECT 1
FROM order_lines ol2 JOIN
products p2
ON ol2.sku = p.sku AND
p2.ship_by = 1
ol.order_number = o.order_number
) AND
. . .
But, your snippet of the query is not using order_lines in the outer query. I suspect you can just get rid of it with the aggregation:
FROM orders o
WHERE EXISTS (SELECT 1
FROM order_lines ol2 JOIN
products p2
ON ol2.sku = p2.sku AND
p2.ship_by = 1
ol.order_number = o.order_number
) AND
. . .
This will at least return all the orders information -- and the lack of aggregation may simplify other parts of the query. If some versions of the query are running fast, then indexes are probably set up reasonable.

condition update cells within the same column

I need to update cells within a specific column based upon ids in another column. The column names are Prod_ID, Lang_ID and Descr:
Prod_ID | Lang_ID | Descr
--------+---------+------
A101 | 1 | TextA
A101 | 2 | TextB
A101 | 3 | TextC
For a group of rows with the same Prod_ID, I need to replace all subsequent descriptions (Descr column) with the description of the first row. The row with the correct description has always Lang_ID = 1. Also, the table may not be sorted by Lang_ID.
Example: TextA (Lang_ID = 1) should replace TextB and TextC because the Prod_IDs of the rows match.
You mentioned in a comment elsewhere that the "master" lang_id is always 1. That simplifies things greatly, and you can do this with a simple self-join (no subqueries :-)
This query selects all lang_1 rows, then joins them with all non-lang_1 rows of the same prod_id and updates those.
If Lang_ID=1 is always the "first"
UPDATE products
LEFT JOIN products as duplicates
ON products.Prod_ID=duplicates.Prod_ID
AND duplicates.Lang_ID != 1
SET duplicates.Descr = products.Descr
WHERE products.Lang_ID = 1
edit: If Lang_ID=1 may not be the "first"
you can join the table to itself via a an intermediate join which finds the lowest Lang_ID for that row. I have called the intermediate-join "lang_finder".
UPDATE products
LEFT JOIN (SELECT Prod_ID, MIN(Lang_ID) as Lang_ID FROM products GROUP BY Prod_ID) as lang_finder
ON products.prod_id=lang_finder.prod_id
LEFT JOIN products as cannonical_lang
ON products.Prod_ID = cannonical_lang.Prod_ID
AND lang_finder.Lang_ID = cannonical_lang.Lang_ID
SET products.Descr = cannonical_lang.Descr
Note that while it does use a subquery, it does not nest them. The subquery essentially just adds a column to the products table (virtually) with the value of the lowest Lang_ID, which then allows a self-join to match on that. So if there were a product with Lang_ID 3, 4, & 5, this would set the Descr on all of them to whatever was set for Lang_ID 3.
How about this?
UPDATE myTable dt1, myTable dt2
SET dt1.Descr = dt2.Descr
WHERE dt1.Prod_ID=dt2.Prod_ID;
Demo at sqlfiddle
Assuming that the correct description is always in the row of a group of rows with the same Prod_ID where Lang_ID has the smallest value, this MySQL query should work:
UPDATE your_table AS t1
JOIN (
SELECT Prod_ID, Descr
FROM (
SELECT *
FROM your_table
ORDER BY Lang_ID
) AS t3
GROUP BY Prod_ID
) AS t2
ON t1.Prod_ID = t2.Prod_ID
SET t1.Descr = t2.Descr;
The above can be used e.g. if Lang_ID is a primary or unique key. It also works if the corresponding Lang_ID has always the same minimum value (e.g. = 1) but in that case much less complex queries like this one are possible.

Calculating a row position in the table

Let's say I have a table like this (this is just a simplified example, the real table I'm talking about is much more complex):
CREATE TABLE media (
id INT NOT NULL AUTO_INCREMENT,
title VARCHAR(255) NOT NULL,
description TEXT NOT NULL,
voted INT NOT NULL DEFAULT 0,
rating FLOAT NOT NULL DEFAULT 0
) ENGINE = INNODB;
The voted column represents a number of votes the item has received and the rating column represents the total rating of the item.
Now, what I want to do is select a single item from the table based on id, something like:
SELECT m.* FROM media AS m WHERE id = 5;
But, in addition, I want to calculate the position of this row based on the rating column and fetch that as an additional column, let's say called a site_rank (so the bigger the rating of the row is the higher its site_rank will be, I hope I explained it well). My guess is this can be achieved with a subselect query but I'm not sure how to do it.
Any help?
SELECT count(*) FROM media WHERE rating > (SELECT rating FROM media WHERE id = 5);
This will output high rank for most voted media.
If you want low rank for the most voted (like, the most voted gets the rank of 1), just reverse the sign in the subquery.
SELECT mo.*,
(
SELECT COUNT(*)
FROM media mi
WHERE (mi.rating, mi.id) <= (mo.rating, mo.id)
) AS rank
FROM media mo
WHERE mo.id = 5
SELECT
m1.*,
(SELECT COUNT(*) FROM media AS m2 WHERE m2.rating > m1.rating) AS site_rank
FROM
media AS m1
WHERE
id = 5;
Note that this does not define a complete ordering because there might be items with equal rating and they will report the same site rank.
Does MySQL support the standard SQL rank() function? It does exactly what you want.
select
*,
rank() over (
order by rating desc
) as site_rank
from media

Categories