SQL query where different - php

I'm trying to run a query to select customer audience, but it should select the customers who didn't get an email before. The email tracking comes from another table. This is the original query:
SELECT
c.customers_firstname,
c.customers_lastname,
o.orders_id,
o.customers_id,
c.customers_email_address
FROM
orders o,
customers c,
order_status s
WHERE
o.customers_id = c.customers_id
AND o.orders_id = s.orders_id
AND o.orders_status = s.orders_status_id
ORDER BY
o.orders_id ASC
Now, I need to check another table called tracking and see if the customer already exists in that table and if so, skip it.
This is what I've tried, but it doesn't seem to work:
SELECT
c.customers_firstname,
c.customers_lastname,
o.orders_id,
o.customers_id,
c.customers_email_address
FROM
orders o,
customers c
INNER JOIN
tracking t
ON
c.customers_id = t.customers_id,
order_status s
WHERE
o.customers_id = c.customers_id
AND o.orders_id = s.orders_id
AND o.orders_status = s.orders_status_id
AND c.customers_id NOT LIKE t.customers_id
ORDER BY
o.orders_id ASC
What am I doing wrong? Or is there any way to do this better?
ADDED: I totally forgot one more important factor - tracking table has "module" column and I need results only from "contact" module. So, in other words, I need to filter out customers who already exist in the tracking table, but only if associated with contact module, not any other module.

This is equivalent to your original query:
SELECT c.customers_firstname
, c.customers_lastname
, o.orders_id
, o.customers_id
, c.customers_email_address
FROM orders o
JOIN customers c
ON c.customers_id = o.customers_id
JOIN order_status s
ON s.orders_id = o.orders_id
AND s.orders_status_id = o.orders_status
ORDER
BY o.orders_id ASC
Add an anti-join
To meet your specification, you can use an "anti-join" pattern. We can add this to the query, before the ORDER BY clause:
LEFT
JOIN tracking t
ON t.customers_id = o.customers_id
WHERE t.customers_id IS NULL
What that's going to do is find all matching rows from the tracking table, based on the customers_id. For any rows that the query doesn't find a matching row(s) in the tracking table, it will generate a dummy row from tracking which consists of all NULL values. (That's one way of describing what an OUTER JOIN does.)
The "trick" now is to throw out all the rows that matched. And we do that by checking for a NULL value of customers_id from the tracking table (in the WHERE clause). For a match, that column won't be NULL. (The equals comparison in the join predicate guarantees us that.) So we know that if we get a NULL value for t.customers_id, that there wasn't a match.
So, this query returns the specified result set:
SELECT c.customers_firstname
, c.customers_lastname
, o.orders_id
, o.customers_id
, c.customers_email_address
FROM orders o
JOIN customers c
ON c.customers_id = o.customers_id
JOIN order_status s
ON s.orders_id = o.orders_id
AND s.orders_status_id = o.orders_status
LEFT
JOIN tracking t
ON t.customers_id = o.customers_id
WHERE t.customers_id IS NULL
ORDER
BY o.orders_id ASC
Other approaches
There are other approaches, but the anti-join is frequently the best performer.
Some other options are a NOT EXISTS predicate and a NOT IN predicate. I can add those, though I expect those solutions will be provided in other answers before I get around to it.
Starting with that first query (equivalent to the query in your question), we could also use a NOT EXISTS predicate. We'd add this before the ORDER BY clause:
WHERE NOT EXISTS
( SELECT 1
FROM tracking t
WHERE t.customers_id = o.customers_id
)
To use a NOT IN predicate, again, add this before the ORDER BY clause:
WHERE o.customers_id NOT IN
( SELECT t.customers_id
FROM tracking t
WHERE t.customers_id IS NOT NULL
)
(You may have some guarantee that tracking.customers_id is not null, but in the more general case, it's important that the subquery NOT return a NULL value, so we include a WHERE clause so that we have that guaranteed.)
With appropriate indexes, the anti-join pattern usually performs better than either the NOT EXISTS or the NOT IN, but not always.

Like spencer7593 suggested you can do the antijoin but instead of
LEFT JOIN tracking t ON t.customers_id = o.customers_id
WHERE t.customers_id IS NULL
you can write easier
JOIN tracking t ON t.customers_id = o.customers_id

Related

How to increase the speed of MySQL query with extra condition?

I'm trying to speed up the following query as it takes quite long to run: now it's 'only' about 1.5 seconds, but it will certainly get slower with more rows (which will 10x over the next period).
Basically, I want to show all the rows from the orders table for the user, and per row show the total order amount (which is the SUM of the orders_products table).
SELECT
orders.order_number,
orders.order_date,
companies.company_name,
COALESCE(SUM(orders_products.product_price * orders_products.product_quantity),0) AS order_value
FROM orders
LEFT JOIN companies ON companies.id = orders.company_id
LEFT JOIN orders_products ON orders_products.order_id = orders.id
LEFT JOIN users ON users.id = orders.user_id
WHERE orders.user_id = '$user_id'
AND companies.user_id = '$user_id'
GROUP BY orders.id ORDER BY orders.order_date DESC, orders.order_number DESC
I've tried adding another condition AND orders_products.user_id = '$user_id'. Speed wise the query was about 12x faster (yeah!) but the problem is that not all orders have products in them. In this case, the orders without products in them are not returned.
How do I change my query so that despite of an order not having products in them, it still is returned (with total order value 0), whilst also speeding up the query?
Thank you in advance for your help!
You might find it faster to use a correlated subquery:
SELECT o.order_number, o.order_date, c.company_name,
(SELECT COALESCE(SUM(op.product_price * op.product_quantity), 0)
FROM orders_products op
WHERE op.order_id = o.id
) AS order_value
FROM orders o LEFT JOIN
companies c
ON c.id = o.company_id AND c.user_id = o.user_id
WHERE o.user_id = '$user_id'
ORDER BY o.order_date DESC, o.order_number DESC
This gets rid of the outer aggregation which is often a performance win.
Then for performance you want the following indexes:
orders(user_id, order_date desc, order_number_desc, company_id)
companies(id, company_id, company_name)
orders_products(order_id, product_price, product_quantity)

MySQL query (28K rows) overloads the server

We are running OpenCart store and now we have to export order data for 6 months (around 17K orders).
Opencart has a builtin solution, when you select the orders you need and export, but it works fine with less than 500 orders.
I've decided to make a standalone script based on the query it uses for its original export.
I'm not publishing the export part, since my dedicated server and the store freezes when I run the following query and consequent loop for mysql_fetch_assoc.
$query = "SELECT o.*, op.name, op.model,op.order_product_id, op.product_id, op.quantity, op.price, op.total AS ptotal, op.tax,
(SELECT ot.value FROM order_total ot WHERE ot.order_id = o.order_id AND ot.code = 'sub_total') AS sub_total,
(SELECT ot.value FROM order_total ot WHERE ot.order_id = o.order_id AND ot.code = 'credit') AS store_credit,
(SELECT `name` FROM order_history oh INNER JOIN `order_status` os on oh.order_status_id=os.order_status_id WHERE oh.order_id = o.order_id and os.language_id='2'
ORDER BY order_history_id DESC LIMIT 1) as order_status
FROM `order` o inner join `order_product` op on o.order_id=op.order_id WHERE o.date_added BETWEEN '2014-01-01 00:00:00' AND '2014-06-30 03:59:59'";
$sql = mysql_query($query);
do {
echo $o['model']."<br/>"; // here we will have a part using PEAR Excel basically
} while ($o=mysql_fetch_assoc($sql));
$endtime = microtime();
echo $endtime-$starttime;
Indexes are set. Additional index was set on oh.order_id. No luck. The same request for 1 particular order id runs in 0.003ms.
My process list shows "Sending data" for the above shown query and "Waiting for table level lock" for anothers.
Could you kindly assist?
I think you could optimize your SQL a bit to prevent locks. The following might help, but recognize that I don't have your schema or data to test this on, so you may need to tweak.
SELECT o.*, op.name, op.model,op.order_product_id, op.product_id,
op.quantity, op.price, op.total AS ptotal, op.tax,
ot.sub_total, ot.store_credit, os.os_name
FROM `order` o inner join `order_product` op on o.order_id=op.order_id
INNER JOIN (SELECT t.order_id,
MAX(case when t.code = 'sub_total' t.value else 0 end case) as sub_total,
MAX(case when t.code = 'credit' t.value else 0 end case) as store_credit
FROM order_total t GROUP BY t.order_id) ot ON ot.order_id = o.order_id
INNER JOIN (SELECT * FROM (SELECT oh.order_id, `name` AS os_name
FROM order_history oh
INNER JOIN `order_status` s on oh.order_status_id=s.order_status_id
WHERE s.language_id='2' ORDER BY order_history_id DESC) t1
GROUP BY t1.order_id) as os ON os.order_id = o.order_id
WHERE o.date_added BETWEEN '2014-01-01 00:00:00' AND '2014-06-30 03:59:59'
I would also probably do a create temp table temp_order_history from select ... and then select * from temp_order_history Also, you may need to do outer join instead of inner join if either of those sub selects have missing data. In those cases, you'd just get NULLs for those columns.

nested queries and calculations all in the same query - is it possible?

I am running a MySQL query to get all "users" with current orders.
(It is possible for a user to have more than 1 associated orders in the db/query).
However i also want to get the total order value for each user and total order count for each user that is being returned (within the below query).
I could do these calculations in PHP, but feel it is possible and would be neater all done within the same SQL query (if possible).
This is the basic query with no attempt to make the above calculations
SELECT u.UserID, u.UserName,
o.OrdersID, o.OrderProductName, o.OrderProductQT, o.OrderTotalPrice, o.tUsers_UserID, o.tOrderStatus_StatusID, o.OrderDate, o.OrderDateModified, o.OrderVoid, o.tProducts_ProductID,
os.OrderStatusName,
ud.UserDetailsName, ud.UserDetailsPostCode,
p.ProductName, p.ProductImage1
FROM tusers u
INNER JOIN torders o ON o.tUsers_UserID = u.UserID
INNER JOIN torderstatus os ON os.OrderStatus_StatusID = o.tOrderStatus_StatusID
INNER JOIN tuserdetails ud ON ud.tUsers_UserID = u.UserID
LEFT JOIN tproducts p ON p.ProductID = o.tProducts_ProductID
WHERE o.tOrderStatus_StatusID = ?
GROUP BY u.UserID
ORDER BY OrdersID DESC
I have tried various nested select queries, but none of them work (right)
Is what i want to do possible in SQL or should i just do it all in PHP once i have the returned query results?
Any advice is much appreciated
You can embed the slightly modified queries into another query. For instance:
SELECT userid, SUM(orderid) FROM orders GROUP BY userid
and
SELECT userid, SUM(distinct productid)
FROM
orders o
INNER JOIN orderlines ol on ol.orderid = o.orderid
GROUP BY
userid
can be combined to:
SELECT
u.userid
u.fullname,
(SELECT SUM(orderid)
FROM orders o
WHERE o.userid = u.userid) as ORDERCOUNT,
(SELECT SUM(distinct productid)
FROM
orders o
INNER JOIN orderlines ol on ol.orderid = o.orderid
WHERE
o.userid = u.userid) as UNIQUEPRODUCTS
FROM
users u
Note that the latter query will return all users and will return NULL for ORDERCOUNT or UNIQUEPRODUCTS when the subquery doesn't return anything (when a user doesn't have orders). Also, the query will fail when a subquery returns more than 1 row, which should never happen in the example I posted.

mysql query become slow when data reached upto 100000

I build a query a month ago on a website. It was working fine. But after a month I was informed that the website become very slow to load the page.
When I search for the problem, I found that my query is executing very slow to fetch the data from mysql database. Then I check for the database and found that the 4 tables which I was using by joins, have around 216850, 167634, 64000, 931 rows respectively.
I have already have indexed that tables. So, where I'm lacking. Please help guys.
[Edit]
Table1: user_alert
Records: 216850
DB Type: InnoDB
Indexes: id(primary)
Table2: orders
Records: 167634
DB Type: InnoDB
Indexes: id(primary), order_id, customer_id
Table3: user_registration
Records: 64000 around
DB Type: InnoDB
Indexes: id(primary), email_address
Table4: cities
Records: 931
DB Type: InnoDB
Indexes: id(primary)
Query:
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address, o.order_id,
CASE WHEN ct.city_name IS NULL THEN uas.city_name ELSE ct.city_name END AS city_name
FROM `user_alert` uas
LEFT JOIN orders o ON o.customer_id = uas.user_id
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
GROUP BY uas.user_id
ORDER BY uas.create_date DESC
GROUP BY is used to aggregate values up. For example if you wanted the count of orders by a user you could use COUNT(o.order_id).....GROUP BY uas.user_id. There are multiple orders for each user, but the aggregate function is just counting them here. However if you just select o.order_id when you have a GROUP BY uas.user_id it doesn't know which of the possibly many order_id values to return for that user id.
In this case it possibly doesn't matter as it looks like the order table is the only one where there is multiple rows per use. If you want the latest one you could just use MAX(o.order_id) (assuming that the order_id is assigned is order). But if you wanted the order value it becomes more difficult.
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address, MAX(o.order_id) AS LatestOrderId,
IFNULL(ct.city_name, uas.city_name) AS city_name
FROM `user_alert` uas
LEFT JOIN orders o ON o.customer_id = uas.user_id
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
GROUP BY uas.user_id
ORDER BY uas.create_date DESC
If you wanted the (say) value of the latest order then it becomes more difficult.
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address, Sub1.MaxOrderId AS LatestOrderId, o.order_value
IFNULL(ct.city_name, uas.city_name) AS city_name
FROM `user_alert` uas
LEFT JOIN (SELECT customer_id, MAX(order_id) AS MaxOrderId FROM orders GROUP BY customer_id) Sub1 ON Sub1.customer_id = uas.user_id
LEFT OUTER JOIN orders o ON o.customer_id = Sub1.user_id AND o.order_id = Sub1.MaxOrderId
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
ORDER BY uas.create_date DESC
Or doing a bit of a fiddle based on GROUP_CONCAT
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address,
SUBSTRING_INDEX(GROUP_CONCAT(o.order_id ORDER BY o.order_id DESC), ',', 1) AS LatestOrderId,
SUBSTRING_INDEX(GROUP_CONCAT(o.order_value ORDER BY o.order_id DESC), ',', 1) AS LatestOrderValue,
IFNULL(ct.city_name, uas.city_name) AS city_name
FROM `user_alert` uas
LEFT OUTER JOIN orders o ON o.customer_id = uas.user_id AND o.order_id = Sub1.MaxOrderId
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
GROUP BY uas.user_id
ORDER BY uas.create_date DESC

Showing COUNT in LEFT JOIN subquery?

I have a query that selects order info between a selected time period. I want to include a where clause that limits the order info to all orders that have only 1 order total(through out all time).
Here is what I have so far:
SELECT o.orders_id, o.customers_id, o.customers_name, o.payment_method, o.date_purchased,o.orders_status, o.shipping_status, ot.value
FROM orders as o
LEFT JOIN orders_total as ot ON o.orders_id = ot.orders_id
WHERE date_purchased between '2011-07-30' AND '2011-08-30 23:59:59'
AND ot.class = 'ot_total'
AND o.customer_service_id = ''
OR o.customer_service_id IS NULL
ORDER BY orders_id DESC
This query gives me all orders in the specified time period. I need to include a subquery(or something similar) that counts all previous(through out all time) orders(order_count) BY customers_id. Then include a 'HAVING order_count < 2' in the where clause.
Is this possible? Does this make sense?
Just add this in you where close:
AND (
SELECT COUNT(o.id)
FROM orders o2
WHERE o2.customers_id = o.customers_id
) < 2
Or if you want to return the orders count, add it in your SELECT clause, and add a HAVING clause:
SELECT o.orders_id, ..., (
SELECT COUNT(o.id)
FROM orders o2
WHERE o2.customers_id = o.customers_id
) as orders_count
...
HAVING orders_count < 2

Categories