Consider a sample table with these rows:
+----+----------+-------+
| id | postcode | value |
+----+----------+-------+
| 1 | A1A3A3 | one |
| 2 | A1A3A4 | two |
| 3 | A1A3B | three |
| 4 | A1A3C | four |
| 5 | A1A3D | five |
| 6 | A1A3 | six |
| 7 | A1A | seven |
| 8 | A1 | eight |
+----+----------+-------+
My goal is to perform a query, whereby it steps down through the postcode column until an exact match is found.
Let's say my starting query parameter is A1A3E9. The expected return value, based on the sample table, would be six. It is important to note, that each step down, I remove one character from the end of the starting query parameter.
So first I would try and find a match for A1A3E9, and then A1A3E, and then A1A3 and so forth.
Currently, I achieve this simply with a series of IF/ELSE blocks, like this:
IF
EXISTS (
SELECT value FROM table
WHERE postcode=:userPost6_1
)
BEGIN
SELECT value FROM table
WHERE postcode=:userPost6_2
END
ELSE IF
EXISTS (
SELECT value FROM table
WHERE postcode=:userPost5_1
)
BEGIN
SELECT value FROM table
WHERE postcode=:userPost5_2
END
ELSE IF
EXISTS (
SELECT value FROM table
WHERE postcode=:userPost4_1
)
BEGIN
SELECT value FROM table
WHERE postcode=:userPost4_2
END
ELSE IF
EXISTS (
SELECT value FROM table
WHERE postcode=:userPost3_1
)
BEGIN
SELECT value FROM table
WHERE postcode=:userPost3_2
END
ELSE IF
EXISTS (
SELECT value FROM table
WHERE postcode=:userPost2_1
)
BEGIN
SELECT value FROM table
WHERE postcode=:userPost2_2
END
Note that I am using parameter binding in PHP, so just for context, my parameter bindings ultimately look like this:
$stmt->bindValue(':userPost6_1', "A1A3E9", PDO::PARAM_STR);
$stmt->bindValue(':userPost6_2', "A1A3E9", PDO::PARAM_STR);
$stmt->bindValue(':userPost5_1', "A1A3E", PDO::PARAM_STR);
$stmt->bindValue(':userPost5_2', "A1A3E", PDO::PARAM_STR);
$stmt->bindValue(':userPost4_1', "A1A3", PDO::PARAM_STR);
$stmt->bindValue(':userPost4_2', "A1A3", PDO::PARAM_STR);
$stmt->bindValue(':userPost3_1', "A1A", PDO::PARAM_STR);
$stmt->bindValue(':userPost3_2', "A1A", PDO::PARAM_STR);
$stmt->bindValue(':userPost2_1', "A1", PDO::PARAM_STR);
$stmt->bindValue(':userPost2_2', "A1", PDO::PARAM_STR);
I do not have have any concerns so far as performance is concerned, as I have an index on the postcode column (which contains 40,000+ rows). My concern is purely that this is visually, an unpleasant query to look at.
My question: Is there a cleaner way to write this query?
Here is one method:
select top (1) t.*
from t
where 'A1A3E9' like t.postcode + '%'
order by t.postcode desc;
The only issue is that your multiple if statements are probably faster. Getting performance is a real challenge with this type of problem. One method uses multiple joins:
select v.pc, coalesce(t0.value, t1.value, t2.value, . . . )
from (values ('A1A3E9')) v(pc) left join
t t0
on t0.postcode = v.pc left join
t t1
on t1.postcode = t0.postcode is null and
(case when len(v.pc) > 1 then left(v.pc, len(v.pc) - 1) end) left join
t t2
on t1.postcode is null and
t2.postcode = (case when len(v.pc) > 2 then left(v.pc, len(v.pc) - 2) end) left join
. . .
I would first spool all the potentially matching rows, eg:
select *
into #matches
from t
where postcode like 'AI%'
This can use an index on postcode and so should be cheap. Then whatever query you run against the matches will just operate over this subset. Even writing a UDF that compares postcode to a literal, eg:
select top 1 *
from #matches
order by dbo.NumberOfMatchingCharacters(postcode,'A1A3E9') desc
Related
I'd like some help combining Multiple SQL queries into one...
I have a search box for orderid or sampleref. An order may have up to 99 sampleref in it so I want the customer to be able to pull up a list of all sampleref associated with their order number regardless of if they search by orderid or one of their sampleref. Essentially what I want to do is,
SELECT `orderid` as OrderNumber FROM `results` WHERE `sampleref` = 'TEST12345';
SELECT * FROM `results` WHERE `orderid` = OrderNumber GROUP BY `sampleref`;
For clarity I'm putting this into a PHP script for a Maria DB mysql server
Here is a sample database
+----+---------+-----------+
| id | orderid | sampleref |
+----+---------+-----------+
| 1 | 101388 | TEST12345 |
| 2 | 101388 | TEST54321 |
| 3 | 333444 | ABC123 |
| 4 | 333444 | ABC321 |
+----+---------+-----------+
Thanks
Henry
Following will give you what you are looking for.
select r2.orderid, r2.sampleref
from result r
join result r2 on r.orderid = r2.orderid
where r.sampleref = 'TEST12345' or r.orderid = <orderid>
You can use or with a correlated subquery:
SELECT r.*
FROM results r
WHERE r.orderid = $orderid OR
EXISTS (SELECT 1
FROM results r2
WHERE r2.orderid = r.orderid AND r2.sampleref = $sampleref
);
Note: This takes two parameters -- either the order id or the sample ref. The first condition returns everything with the same order, if that is given. The second returns everything with the same order as the given sample ref.
I have performance issue in below query:
SELECT t.local_branch_revenue, t.total_payment,
(SELECT SUM(IF(cpo.real_account_type = 'HQ', 0, cpo.payment_amount)) AS cpo_payment_amount
FROM customer_payment_options cpo
WHERE tran_id=t.id
AND cpo.payment_type != 'WALLET' AND cpo.payment_type != 'REWARD_CREDIT'
GROUP BY cpo.tran_id)
as cpo_payment_amount,
b.ben_firstname, b.ben_lastname
FROM transaction t
LEFT JOIN beneficiary b
ON b.id=t.ben_id
WHERE t.local_branch_id='31'
AND DATE(t.date_added) < '2016-04-07'
AND source_country_id='40'
AND t.transaction_status != 'CANCELLED'
EXPLAIN
+----+--------------------+-------+--------+----------------------------------------+----------------------------------------+---------+-----------------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+-------+--------+----------------------------------------+----------------------------------------+---------+-----------------+------+-------------+
| 1 | PRIMARY | t | ref | local_branch_id,source_country_id | local_branch_id | 5 | const | 2 | Using where |
+----+--------------------+-------+--------+----------------------------------------+----------------------------------------+---------+-----------------+------+-------------+
| 1 | PRIMARY | b | eq_ref | PRIMARY | PRIMARY | 8 | mtesdb.t.ben_id | 1 | |
+----+--------------------+-------+--------+----------------------------------------+----------------------------------------+---------+-----------------+------+-------------+
| 2 | DEPENDENT SUBQUERY | cpo | ref | tran_id_payment_type_real_account_type | tran_id_payment_type_real_account_type | 9 | mtesdb.t.id | 1 | Using where |
+----+--------------------+-------+--------+----------------------------------------+----------------------------------------+---------+-----------------+------+-------------+
As you can see, it's using indexes from possible key. But still query takes about 13 sec.
I also have index over transaction table: (ben_id, company_id, source_country_id, date_added, tran_owner). But, it's not even coming in possible keys section.
Let me know if you need table schema.
What am I missing here?
Dependent subqueries don't perform very well in MySQL ... the query planner doesn't transform them to JOINed subqueries efficiently. (They're OK in Oracle and SQL Server, but who has the money for those?) So, a good bet for you is to refactor your query to eliminate the dependent subquery.
Here's your subquery. Let's refactor it as an independent subquery. We'll get rid of the WHERE tran_id=t.id and move it, later, to an ON clause.
SELECT tran_id,
SUM(IF(real_account_type = 'HQ',
0,
payment_amount)) AS cpo_payment_amount
FROM customer_payment_options
WHERE payment_type != 'WALLET'
AND payment_type != 'REWARD_CREDIT'
GROUP BY tran_id
Notice you can simplify this as follows -- your IF() clause excludes rows with real_account_type = 'HQ'. You can do that in the WHERE clause instead.
SELECT tran_id,
SUM(payment_amount) AS cpo_payment_amount
FROM customer_payment_options
WHERE payment_type != 'WALLET'
AND payment_type != 'REWARD_CREDIT'
AND real_account_type != 'HQ'
GROUP BY tran_id
A compound index on (tran_id, payment_type, real_account_type, payment_amount) may help this subquery run faster. But the presence of those three != clauses guarantees a full index scan; there's no way to random access any index for those.
This generates a virtual table containing one row per tran_id with the sum you need.
Next we need to join that into your main query.
SELECT t.local_branch_revenue,
t.total_payment,
IFNULL(cposum.cpo_payment_amount,0) cpo_payment_amount,
b.ben_firstname, b.ben_lastname
FROM transaction t
LEFT JOIN beneficiary b ON b.id=t.ben_id
LEFT JOIN (
SELECT tran_id,
SUM(payment_amount) AS cpo_payment_amount
FROM customer_payment_options
WHERE payment_type != 'WALLET'
AND payment_type != 'REWARD_CREDIT'
AND real_account_type != 'HQ'
GROUP BY tran_id
) cposum ON t.id = cposum.tran_id
WHERE t.local_branch_id='31'
AND DATE(t.date_added) < '2016-04-07'
AND source_country_id='40'
AND t.transaction_status != 'CANCELLED'
Do you see how we've changed the dependent summary subquery into its own virtual table? That lets the query planner run that query just once, rather than once for each row in the main query. That helps a lot.
The IFNULL() gets you a numeric value for cpo_payment_amount, rather than NULL, for transaction rows lacking any corresponding customer_payment_options rows.
A compound index on the transaction table on (local_branch_id, source_country_id, date_added) will help this query; the query engine can random access the local_branch_id and source_country_id values, then range scan the date_added value.
How do you learn to do this yourself? http://use-the-index-luke.com/ is a good start.
WHERE t.local_branch_id='31'
AND DATE(t.date_added) < '2016-04-07'
AND source_country_id='40'
Change that date test to simply t.date_added < '2016-04-07' ! Otherwise the following index suggestions won't work.
What table is source_country_id in?? If it is in t, then you need INDEX(local_branch_id, source_country_id, date_added). If it is not in t, then INDEX(local_branch_id, date_added).
Please provide SHOW CREATE TABLE if you need further discussion.
I am not sure if there is no answer here on Stack Overflow, but I did not find exact thing there.
So, I have a table with data like this:
id | sector | type | level
| | |
1 | TMT | Long | First
2 | Energy | Long | Second
3 | TMT | Short | Third
4 | Other | Long | First
5 | TMT | N/A | Sixth
6 | Other | Short | First
What I need is summary of each different value in each field, like this:
Sector TMT: 3
Sector Energy: 1
Sector Other: 2
Type Long: 3
Type Short: 2
Type N/A: 1
Level First: 3
Level Second: 1
Level Third: 1
Level Sixth: 1
Type and Level have fixed values. Sectors could be dynamic, I mean there is no final list as they are adding with each item.
For now I am selecting all rows and processing it with PHP:
SELECT `sector`, `type`, `level` FROM `table`;
and for example:
<?php
$result['type_long'] = 0;
$result['type_short'] = 0;
$result['type_na'] = 0;
foreach ($rows as $row)
{
if ($row['type'] == 'Long')
{
$result['type_long']++;
}
else if ($row['type'] == 'Short')
{
$result['type_short']++;
}
else if ($row['type'] == 'N/A')
{
$result['type_na']++;
}
}
etc. For 10000 records it's not really fast cause I need to select all rows from database and then do something to them via PHP.
Is there any way I can do this faster with MySQL?
Thanks!
This is a straightforward application of the union of several summary queries. A typical summary query would be
SELECT Sector, COUNT(*) cnt
FROM mytable
GROUP BY Sector
To get your exact result set you could do something like this:
SELECT CONCAT('Sector ', Sector, ':') item, COUNT(*) cnt FROM mytable GROUP BY Sector
UNION ALL
SELECT CONCAT('Type ', Type, ':') item, COUNT(*) cnt FROM mytable GROUP BY Type
UNION ALL
SELECT CONCAT('Level', Level, ':') item, COUNT(*) cnt FROM mytable GROUP BY Level
If you want this to be optimally fast, you might try adding indexes on the three columns.
MySQL is quite good at handling summary queries with high performance. Even with multiple separate SELECT queries in this UNION, it will perform an order of magnitude better than transferring all your raw data rows from the server to your MySQL program.
I have a database with three tables in it:
places:
id | name | latitude | longitude |
------|--------|------------|------------|
1 | place1 | 11.123456 | 76.123456 |
------|--------|------------|------------|
2 | place2 | 23.123456 | 65.123456 |
etc ...
categorized_places:
id | place_id | cat_id |
------|----------|--------|
1 | 1 | 2 |
------|----------|--------|
2 | 2 | 1 |
etc ...
places_visited:
id | user_name | user_email | place_id |
------|-----------|------------|----------|
1 | user_1 | x#mail.com | 2 |
------|-----------|------------|----------|
2 | user_2 | y#mail.com | 2 |
There's also a fourth named categories, but it's not important in this.
I'm trying to filter the places from the places-table to show the user the nearest place, that he/she has not yet visited.
$cur_cat is set on the previous page, where the user selects which kind of place he/she would like to visit.
$cur_user and $cur_user_email are based on $_SESSION variables
$max_lat, $max_lon, $min_lat and $min_lon are based on the users current position
I'm using this code in php (with PDO), but it always returns zero results:
$get_places = $db->prepare("
SELECT
places.id,
places.name,
places.latitude,
places.longitude
FROM
places,
categorized_places,
places_visited
WHERE
places.id = categorized_places.place_id
AND categorized_places.cat_id = '$cur_cat'
AND places.latitude <= '$max_lat'
AND places.latitude >= '$min_lat'
AND places.longitude <= '$max_lon'
AND places.longitude >= '$min_lon'
AND places_visited.user_name = '$cur_user'
AND places_visited.user_email = '$cur_user_email'
AND places.id != places_visited.place_id
");
$get_places->execute();
The code always shows 0 results and throws no error. I've also made sure, that the places are not already in the places_visited table.
I've stared at this for so very long now, and I just can't figure out the error.
Any help would be very appreciated!
Your query is doing inner joins. So, it can only return places that the user has visited. No way that it can return places that a user hasn't visited. Before proceeding further, here is a simple rule: Never use commas in the from clause. ANSI standard explicit JOIN syntax has been around for over two decades, and you should use it. In fact, in this case, you need it, because you need an outer join:
SELECT p.id, p.name, p.latitude, p.longitude
FROM places p INNER JOIN
categorized_places cp
ON p.id = cp.place_id LEFT JOIN
places_visited pv
ON pv.place_id = p.id AND
pv.user_name = '$cur_user' AND
pv.user_email = '$cur_user_email'
WHERE cp.cat_id = '$cur_cat' AND
p.latitude <= '$max_lat' AND
p.latitude >= '$min_lat' AND
p.longitude <= '$max_lon' AND
p.longitude >= '$min_lon' AND
pv.place_id IS NULL;
What this does is it matches the conditions to all the places visited, using an outer join. Then the condition pv.place_id IS NULL chooses the ones that have not been visited. Note that the conditions on the places_visited table go in the ON clause. The conditions on the other two tables remain in the WHERE clause. In general, when using LEFT OUTER JOIN, the filters on the first table stay in the WHERE clause. The filters on the second table go in the ON clause.
I also introduced table aliases. These help make queries easier to write and to read.
I'd like to know how I check whether one or more of the elements (numbers in this case) in a string, eg. '1,2,3,5' are in another ,eg. '3,4,5,6'
3 and 5 are common elements to each string in that example.
In this case it is to create a SQL query based on the string comparisons.
One column value in a db contains one number string, and needs to be compared to another. I need results that match values of each string.
$results = $db->query("SELECT * FROM db
WHERE comparisonString IN (".$idsString.")
")->fetchAll (PDO::FETCH_ASSOC);
But its not quite working... it could be a lateral or syntactic answer.
MORE SPECIFICALLY, I am only getting a result when the FIRST element in the comaprisonString matches the other string elements.
Ideally the solution will look something like this:
$results = $db->query("SELECT * FROM db
WHERE ELEMENTS IN comparisonString IN (".$idsString.")
")->fetchAll (PDO::FETCH_ASSOC);
"ELEMENTS IN" is made up syntax, but that's the sort of thing I'm after
First of all it smells like a bad schema design. Don't store delimited strings of values in your database. Normalize your data by creating a many-to-many table. It will pay off big time enabling you to normally maintain and query your data.
In the meantime if you're using MySQL and assuming that your table looks something like
CREATE TABLE Table1
(
id int not null auto_increment primary key,
column_name varchar(128)
);
and let's say you have sample data
| ID | COLUMN_NAME |
|----|-------------|
| 1 | 3,4,5,6 |
| 2 | 4,6,22 |
| 3 | 7,5,11 |
| 4 | 9,12,1,3 |
| 5 | 8,32,16 |
and you want to select all rows where column_name contains one or more values from a list 1,2,3,5 you can do either
SELECT *
FROM table1
WHERE FIND_IN_SET(1, column_name) > 0
OR FIND_IN_SET(2, column_name) > 0
OR FIND_IN_SET(3, column_name) > 0
OR FIND_IN_SET(5, column_name) > 0
or
SELECT *
FROM table1 t JOIN
(
SELECT id
FROM table1 t JOIN
(
SELECT 1 value UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 5
) s
ON FIND_IN_SET(s.value, t.column_name) > 0
GROUP BY id
) q
ON t.id = q.id
Output (in both cases):
| ID | COLUMN_NAME |
|----|-------------|
| 1 | 3,4,5,6 |
| 3 | 7,5,11 |
| 4 | 9,12,1,3 |
Here is SQLFiddle demo