Related
I am wondering if there is away (possibly a better way) to order by the order of the values in an IN() clause.
The problem is that I have 2 queries, one that gets all of the IDs and the second that retrieves all the information. The first creates the order of the IDs which I want the second to order by. The IDs are put in an IN() clause in the correct order.
So it'd be something like (extremely simplified):
SELECT id FROM table1 WHERE ... ORDER BY display_order, name
SELECT name, description, ... WHERE id IN ([id's from first])
The issue is that the second query does not return the results in the same order that the IDs are put into the IN() clause.
One solution I have found is to put all of the IDs into a temp table with an auto incrementing field which is then joined into the second query.
Is there a better option?
Note: As the first query is run "by the user" and the second is run in a background process, there is no way to combine the 2 into 1 query using sub queries.
I am using MySQL, but I'm thinking it might be useful to have it noted what options there are for other DBs as well.
Use MySQL's FIELD() function:
SELECT name, description, ...
FROM ...
WHERE id IN([ids, any order])
ORDER BY FIELD(id, [ids in order])
FIELD() will return the index of the first parameter that is equal to the first parameter (other than the first parameter itself).
FIELD('a', 'a', 'b', 'c')
will return 1
FIELD('a', 'c', 'b', 'a')
will return 3
This will do exactly what you want if you paste the ids into the IN() clause and the FIELD() function in the same order.
See following how to get sorted data.
SELECT ...
FROM ...
WHERE zip IN (91709,92886,92807,...,91356)
AND user.status=1
ORDER
BY provider.package_id DESC
, FIELD(zip,91709,92886,92807,...,91356)
LIMIT 10
Two solutions that spring to mind:
order by case id when 123 then 1 when 456 then 2 else null end asc
order by instr(','||id||',',',123,456,') asc
(instr() is from Oracle; maybe you have locate() or charindex() or something like that)
If you want to do arbitrary sorting on a query using values inputted by the query in MS SQL Server 2008+, it can be done by creating a table on the fly and doing a join like so (using nomenclature from OP).
SELECT table1.name, table1.description ...
FROM (VALUES (id1,1), (id2,2), (id3,3) ...) AS orderTbl(orderKey, orderIdx)
LEFT JOIN table1 ON orderTbl.orderKey=table1.id
ORDER BY orderTbl.orderIdx
If you replace the VALUES statement with something else that does the same thing, but in ANSI SQL, then this should work on any SQL database.
Note:
The second column in the created table (orderTbl.orderIdx) is necessary when querying record sets larger than 100 or so. I originally didn't have an orderIdx column, but found that with result sets larger than 100 I had to explicitly sort by that column; in SQL Server Express 2014 anyways.
SELECT ORDER_NO, DELIVERY_ADDRESS
from IFSAPP.PURCHASE_ORDER_TAB
where ORDER_NO in ('52000077','52000079','52000167','52000297','52000204','52000409','52000126')
ORDER BY instr('52000077,52000079,52000167,52000297,52000204,52000409,52000126',ORDER_NO)
worked really great
Ans to get sorted data.
SELECT ...
FROM ...
ORDER BY FIELD(user_id,5,3,2,...,50) LIMIT 10
The IN clause describes a set of values, and sets do not have order.
Your solution with a join and then ordering on the display_order column is the most nearly correct solution; anything else is probably a DBMS-specific hack (or is doing some stuff with the OLAP functions in standard SQL). Certainly, the join is the most nearly portable solution (though generating the data with the display_order values may be problematic). Note that you may need to select the ordering columns; that used to be a requirement in standard SQL, though I believe it was relaxed as a rule a while ago (maybe as long ago as SQL-92).
Use MySQL FIND_IN_SET function:
SELECT *
FROM table_name
WHERE id IN (..,..,..,..)
ORDER BY FIND_IN_SET (coloumn_name, .., .., ..);
For Oracle, John's solution using instr() function works. Here's slightly different solution that worked -
SELECT id
FROM table1
WHERE id IN (1, 20, 45, 60)
ORDER BY instr('1, 20, 45, 60', id)
I just tried to do this is MS SQL Server where we do not have FIELD():
SELECT table1.id
...
INNER JOIN
(VALUES (10,1),(3,2),(4,3),(5,4),(7,5),(8,6),(9,7),(2,8),(6,9),(5,10)
) AS X(id,sortorder)
ON X.id = table1.id
ORDER BY X.sortorder
Note that I am allowing duplication too.
Give this a shot:
SELECT name, description, ...
WHERE id IN
(SELECT id FROM table1 WHERE...)
ORDER BY
(SELECT display_order FROM table1 WHERE...),
(SELECT name FROM table1 WHERE...)
The WHEREs will probably take a little tweaking to get the correlated subqueries working properly, but the basic principle should be sound.
My first thought was to write a single query, but you said that was not possible because one is run by the user and the other is run in the background. How are you storing the list of ids to pass from the user to the background process? Why not put them in a temporary table with a column to signify the order.
So how about this:
The user interface bit runs and inserts values into a new table you create. It would insert the id, position and some sort of job number identifier)
The job number is passed to the background process (instead of all the ids)
The background process does a select from the table in step 1 and you join in to get the other information that you require. It uses the job number in the WHERE clause and orders by the position column.
The background process, when finished, deletes from the table based on the job identifier.
I think you should manage to store your data in a way that you will simply do a join and it will be perfect, so no hacks and complicated things going on.
I have for instance a "Recently played" list of track ids, on SQLite i simply do:
SELECT * FROM recently NATURAL JOIN tracks;
I am using group by for like statement as i have database structure like this.
I want to get the count of workingzone groupby.but if i try to group by, then wrong output will appear as output will group by 99 and 99, as in figure.
My sql code is:
select count(organization),working_zone from `projects` where `district` = 12 and (`working_zone` = 99 or `working_zone` LIKE "99," or `working_zone` LIKE ",99") group by `organization`;
my desired result is:
count |working_zone
____6| 99
____3| 100
(99),(,99),(99,) should be grouped by doing sum and result should be 6.
You have an awful data structure -- although I wouldn't be surprised if the data is okay and you are really working off the result of a (reasonable) query. You should not be storing raw data in comma-delimited lists. Instead, use junction tables.
Why is having a separate row for each pair the SQLish way of storing data? Consider these reasons:
SQL has pretty based string functions (compared to other programming environments).
Data should be stored in its native type; don't store numbers as strings.
Foreign key relationships should be explicitly declared, and you can't declare a foreign key relationship using comma-delimited strings.
The primary SQL mechanism for optimizing queries are indexes, and comma-delimited lists preclude the use of indexes.
Sometimes, though, you are stuck with someone else's bad design decisions. If so, one solution uses a table of working zones:
select count(*), wz.working_zone
from projects p join
working_zones wz
on find_in_set(wz.working_zone, p.working_zone) > 0
where p.district; = 12 and
find_in_set(99, p.working_zone) > 0
group by wz.working_zone;
SELECT COUNT(organization),working_zone FROM table WHERE working_zone HAVING '99'
UNION ALL
SELECT COUNT(organization),SUBSTRING(working_zone,4) FROM table WHERE working_zone = '99,100'
I have some tasks that I have to store in my database. And each task has an array of dates in which the tasks were completed. I've learn that it is better to not use a array (serialize) to store dates, but instead make another table. So I did:
taskTable contains columns: taskID, userid, description, name
task_days contains columns: taskID, day
But Im having trouble with php,
usually I can easily send my data to client with:
function getTasks(){
$app = \Slim\Slim::getInstance();
$userid = $app->request->params('userid');
$db = getDB();
$result = $db->prepare("Select * From taskTable where userid = ?");
$result->execute(array($userid));
$result->setFetchmode(PDO::FETCH_ASSOC);
echo json_encode($result->fetchAll());
}
I encode it, then client can easily read it as an array of JSON. But now with two tables, I'm not sure how to do it efficiently. I know I can get the required information with this query:
Select * from taskTable as t, task_days as d where t.taskID = d.taskID
But how do I make it so the days will be in an array associated with the correct task.
Do I first Select * From taskTable where userid = $userid, then for each task, I will do a query on table task_days? that seems extremely inefficient though.
So I want something like the following:
[
{taskid: 123, userid: 1, description: "do task", name: "tony", day:[1998-01-02, 1998-02-03]},
{taskid: 124, userid: 2, description: "do task2", name: "Ann", day:[2016-01-02, 2016-02-03, 2016-01-01]},
...
]
There's a couple of approaches.
1) One approach, as you already outline, is to run a query that returns the the columns from just `taskTable`. And for each row returned, run another query to get the associated rows from task_days. And you are right, that's usually not the most efficient approach. But for a reasonably small number of rows, performance should be reasonable as long as appropriate indexes are available.)
2) Another approach, assuming `taskid` is the primary key of `taskTable` is to perform a join, and use a "GROUP BY" to collapse the rows. The "GROUP_CONCAT" aggregate function can convert the multiple values of `day` from the `task_days` table into a single string. For example:
SELECT t.taskid
, t.userid
, t.description
, t.name
, GROUP_CONCAT(d.day ORDER BY d.day) AS `day`
FROM taskTable t
LEFT
JOIN task_days d
ON d.taskid = t.taskid
GROUP BY t.taskid
ORDER BY t.taskid
This would return the day as a string, not an array. If you need an array, your code would need to do that. (As a convenient way to do that, the PHP explode function might be suitable.)
NOTE: the length of the string returned by GROUP_CONCAT is limited by group_concat_max_len variable, and also by max_allowed_packet.
3) Another way to approach this is to perform a join operation, and pull back the "duplicated" task information, ordered by taskid and day
SELECT t.taskid
, t.userid
, t.description
, t.name
, d.day
FROM taskTable t
LEFT
JOIN task_days d
ON d.taskid = t.taskid
ORDER BY t.taskid, d.day
That would get a result set like this:
taskid userid description name day
------ ------ ----------- ----- ----------
123 1 do task tony 1998-01-02
123 1 do task tony 1998-02-03
124 2 do task2 Ann 2016-01-02
124 2 do task2 Ann 2016-02-03
124 2 do task2 Ann 2016-01-01
Then your code would need to do some rudimentary "control break" processing. Basically, compare the taskid of the current row to the taskid from the previous row. If they match, you are processing just a new `day` value for the same task.
If the taskid of the current row is different than the taskid from the previous row, then you are starting a new task.
Your code would effectively be ignoring the duplicated rows from `taskTable`, basically squinting at the result set and seeing it like this:
taskid userid description name day
------ ------ ----------- ----- ----------
- 123 1 do task tony 1998-01-02
+ 1998-02-03
- 124 2 do task2 Ann 2016-01-02
+ 2016-02-03
+ 2016-01-01
FOLLOWUP
The second option is closest to your original implementation, a comma separated list of values as a string, in a character column.
As far as storing a comma separated list, that's a SQL anti-pattern, and it's usually best avoided it. Multi-valued attributes can be stored in a separate table, like you have done.
The exception would be if you never, ever need the database to see the values in the list as separate values.
If you are storing that "list of dates" as if it were an image, for example like the contents of a jpeg... if you always store the entire value into the column, and always extract the contents of the column as a single value... if never need to search for an individual date, or add a date to an existing list, or remove a date from a list... and if you never need the database to enforce any constraints on the values, or do any validation of the contents...
If all of those conditions are satisfied, only then might it make sense to store a comma separated list as a single column.
My personal preference, if the implementation is targeted only to MySQL, would be the second option... using GROUP_CONCAT. If the length of the string generated by the GROUP_CONCAT exceeds group_concat_max_len, the string will be truncated, with no warning or error. (I believe that's a limitation in bytes, and not characters.)
The safest coding practice would be to do perform a query:
SELECT ##session.group_concat_max_len
save the value returned by that. Then, for the values returned from the GROUP_CONCAT expression, compare the length (in bytes) to the saved value, to see if truncation has occurred. (If the length of the returned string is less than the value of group_concat_max_len, then you can be pretty confident that truncation has not occurred.) It's also possible to override the current value of the variable (before you run the statement containing GROUP_CONCAT, with a separate SET statement. Something like this:
SET SESSION group_concat_max_len = 131072 ;
(Just be careful not to exceed max_allowed_packet.)
So let's say I have a Database called xjet and a table called samples where I have 3 samples with the colums: Id, Frequency and Temperature. It looks like this:
Id=1, Frequency=10000, Temperature=50
Id=2, Frequency=10000, Temperature=30
Id=3, Frequency=10000, Temperature=50
I want to not have columns shown where their value is always the same, so in this case I want the query to show:
Id=1, Temperature=50
Id=2, Temperature=30
Id=3, Temperature=50
Presuming of course that I don't know the values of temperature and frequency ahead. How can I do such a query?
I'm using MySqli and PHP if it's relevant
I am not sitting at a LAMP setup but here is how you would write something like this.
(using ms sql2008) its very similar to mysql command. What makes your query a bit vague is, are you going to pass frequency ? The reason when you don't display the frequency how is something/someone going to know the grouping of the values for frequency?
Create the test table and load values:
create table #testTable
(
id int identity(1,1),
Frequency int default null,
Temperature decimal default null
)
INSERT into #testTable (Frequency, Temperature)
VALUES
(10000, 50),
(10000, 30),
(10000, 50)
;
declare #someValue int = 10000;
select
tt.id,
tt.Temperature
from
#testTable tt
where
tt.Frequency = #someValue
The code above works for only one value.
Now you said you don't know the frequency before hand the code below will produce the results for you given you don't provide a frequency. The problem with code below is you can see via the output how the rows are grouped. (The code below just uses the table from above.)
select
z.id,
z.Temperature
from
(
select distinct frequency from #testTable tt group by tt.Frequency
) as t
inner join #testTable z on t.Frequency = z.Frequency;
I eventually solved it by employing PHP...thank you everyone for the responses
I have a database with this kind of a table, has more than 10 million rows.
ID colA colB Length
1 seq1 seq11 1
2 seq1 seq11 11
3 seq3 seq33 21
4 seq3 seq33 14
I want to loop though colA first, get the relevant colB value, and check if there are any other occurrences of the same value.
For example in colB (seq11) there are 2 occurrences of colA(seq1), this time I have to combine those and output the sum of the length. Similar to this:
ID colA colB Length
1 seq1 seq11 12
2 seq3 seq33 35
I am a bit Java guy, but because my colleague has written everything in PHP and this will be just an adding, I need a PHP solution.
With Java I would have used hashmap, so that I would have the colA data once and just increment the value of "Length Column".
I tried this query in order to group by occurences:
SELECT COUNT(*) SeqName FROM SeqTable GROUP BY SeqName HAVING COUNT(*)>0;
This is something easily achieved within SQL rather than in programming logic:
SELECT colA, colB, SUM(Length) as `length_sum`
FROM SeqTable
GROUP BY colA, colB
Of course you would still need PHP to iterate through the result set and do whatever it is you want to do with the data.
In PHP you can use an array like an hash map
$array = Array();
$array['seq1'] = Array();
$array['seq1']['seq11'] = 0;
$array['seq1']['seq11']++;
Or you can use an SQL query like this one:
select id,colA,colB,sum(Length) as Length from {tableName} group by colA,colB order by colA, colB;