I'm trying to sum columns x through x+n in an SQL table. Essentially, I have multiple tables that contain grades in them and a user_id. I want to sum all the grades to come up with a total grade column without specifying the column names as the names and number of columns changes with each table. For instance, one table might have columns (user_id, calculations, prelab, deductions) while another might have (user_id, accuracy, precision, graphs, prelab, deductions).
I could rename my columns col1, col2, col3, col4, col5, etc., but I can't figure out how to get around the varying number of columns.
As far as I know, there is no way to sum groups of columns without actually specifying the column names directly in SQL. It seems to me like this is a badly designed schema, but that's a separate topic.
In any your case, you're going to need to create a new column in each table that contains the sum of all the grades in that particular table, say called total, and then, do something like this:
select user_id, sum(table1.total, table2.total, table3.total)
from table1, table2, table3
where table1.user_id = table2.user_id
and table2.user_id = table3.user_id
group by user_id
1) You could write some pl/sql to go and hit the data dictionary and get the columns and then construct dynamic sql to do the work of adding them up correctly.
2) Or you could create views on top of the tables that contain the user_id and the sum of the interesting columns (the views themselves could be constructed programmatically - but that only needs to happen once rather than every time you want the totals).
But either of the above is probably over-kill compared to simply fixing your schema.
The following procedure would likely do the trick.
It will look for all column names for the given tableName in the INFORMATION_SCHEMA.COLUMNS table (excluding 'userid' - This may be subject to change if the name you use is different).
The procedure also creates a temporary table (this is also subject to improvement - it would probably be better to do a 'drop if exists before the create) to store the sum up to a point.
The items inside the loop is just building an SQL UPDATE statement with the given tableName argument and the columnName from the cursor and doing the math.
To test this (after creation):
call myProcedure('tableName');
DELIMITER //
DROP PROCEDURE IF EXISTS myProcedure //
CREATE PROCEDURE
myProcedure( tableName varchar(32) )
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE columnName varchar(64);
DECLARE cur1 CURSOR FOR SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = tableName and COLUMN_NAME <> 'userid';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur1;
CREATE TEMPORARY TABLE intermediateresults(userid integer, sumOfScores integer);
SET #st1 = CONCAT('INSERT INTO intermediateresults (SELECT DISTINCT userid, 0 FROM ',tableName,' )' );
PREPARE stmt3 FROM #st1;
EXECUTE stmt3;
looping: LOOP
FETCH cur1 into columnName;
IF done THEN
LEAVE looping;
END IF;
SET #st1 = CONCAT('UPDATE intermediateresults set sumOfScores = sumOfScores + COALESCE( (SELECT ', columnName, ' FROM ',tableName, ' t WHERE t.userid=intermediateresults.userid) , 0)' );
PREPARE stmt3 FROM #st1;
EXECUTE stmt3;
DEALLOCATE PREPARE stmt3;
END LOOP;
CLOSE cur1;
SELECT * FROM intermediateresults;
DROP table intermediateresults;
END
//
DELIMITER ;
What might be of interest when doing this kind of thing:
INFORMATION_SCHEMA also has data on:
DATA_TYPE: which can be used to test if a specific column has the actual type you are expecting - a condition such as DATA_TYPE='int' can be added to the cursor definition to make sure that it is in fact an int (assuming that the columns to be summed are in fact INTs)
ORDINAL_POSITION: which can be used if you know in which order the columns are supposed to arrive (for cases where the last four are housekeeping, for instance)
TABLE_SCHEMA: the procedure above rather assumes that the table is only present in the current default schema. Using this would require an additional parameter in the procedure and a slight change in the constructed SQL statements.
I have created a procedure which returns a single column of officeid
call officetree(15);
I need to get list of employee under officeid's return by officetree procedure
select * from master_employee where officeid in ( here i want put my officeids return from procedure)
Is this possible to achieve this if yes what is the syntax for that.
Inside the procedure
Below ofcid is parameter of procedure
select `ofc_id`
from (select * from master_office
order by `ofc_parent_id`, `ofc_id`) master_office,
(select #pv := ofcid) office
where (find_in_set(`ofc_parent_id`, #pv) > 0
and #pv := concat(#pv, ',', `ofc_id`)) or ofc_id=ofcid
No, AFAIK, you can not use a SP as sub query in MySQL.
Ref: Using a stored procedure as subquery
Ref: https://forums.mysql.com/read.php?10,556522,556538#msg-556538
Is it possible to call stored procs in MySQL 5.5 subqueries.
No.
And for a suggestion, use stored procedure as few as possible (my ten-year experience tells me)
I think there is no way to use a stored procedure result like a subquery.
Your alternatives:
Use the statement from the procedure as subquery.
Fetch the IDs from the SP in PHP and execute a second query with the fetched IDs. select * from master_employee where officeid in ( list of previously fetched IDs )
Use a more powerfull design for the tree structure like "materialized path" or "transitive closure table"
I have a Oracle table that has nearly 150k records and 65 columns. There is a requirement from my customer that They need to search the whole table with a single search value and that too can be wildcard search.
And the worst part is that i am using php language to do that.
So i have constructed a query which has all 65 columns to search for a single string value.
Sample query:
SELECT * FROM <TABLE_NAME>
WHERE (col1 || col2 || col3 ... || col65) LIKE '%<SEARCH_VALUE>%'
The query is giving output for some fields and sometimes it is failing if we pass some address related fields.
On some scenario, particularly with wild card type search and if it is outputting more than 1000 rows.
I have done these test using php language. but at the same when i try to execute the query using SQL Developer. It is giving results all the time. So i got confused.
And more over the Oracle server that i am trying to access is an external server and it is not within our environment.
Please suggest me how to implement this feature or idea and i really appreciate your assistant.
If you are using Oracle 11g, you may add a virtual column to the table
ALTER TABLE <TABLE_NAME> ADD
SEARCH_COLUMN GENERATED ALWAYS AS (col1 || ' ' || col2 || ' ' || col3 ... || col65);
Then create an index on the newly created virtual column. The index will store the concatenated string value of all 65 columns. You will not have to change any queries in your application as Oracle will automatically calculate the value of the virtual column with the values of other 65 columns.
Then you can just use the index virtual column for your searches
SELECT * FROM <TABLE_NAME>
WHERE SEARCH_COLUMN LIKE '%<SEARCH_VALUE>%'
This looks like a pretty horrible design. One problem that comes to mind is shown in the example below
col1 col2
===========
ABC DEF
CDX ZZZ
If your search condition is "CD", both of these rows will match, since you are searching on the concatenation of all columns.
I'm using this trick to search across columns And i know that the solution is slow.
create table test_example as select * from user_objects where rownum <= 1000;
select t2.* from
xmltable('for $i in ora:view("test_example")/ROW
where fn:exists($i/*[contains(.,$var_text_to_serach)])
return $i' passing 'change_here' as "var_text_to_serach" columns id varchar(100) path 'OBJECT_ID') t1
join test_example t2 on t1.id = t2. OBJECT_ID
;
ora:view - allows to query tables within xquery expression. Function creates xml-s from table rows.
xmltable - allows to map xquery-xml result to relational model.
FLWOR expresion - inside xmltable i'm using FLWOR expresion it is acronym from words FOR, LET, WHERE, ORDER, RETURN
fn:exists($i/*[contains(.,$var_text_to_serach)]) - check if any of columns contains pattern.
columns id varchar(100) path 'OBJECT_ID' extract id column from XML. In my example OBJECT_ID is uniquer value in table.
passing 'change_here' as "var_text_to_serach" - "change_here" is text to search
to use this example you have to change test_example to your table, and take care about join condition and .
Consider using a user defined data store to create a text index on all of the columns and then search with the contains operator..
Sonething like this
begin
begin
ctx_ddl.drop_preference('mymcds');
exception
when others then
null;
end;
ctx_ddl.create_preference('mymcds', 'multi_column_datastore');
ctx_ddl.set_attribute('mymcds', 'columns', 'NAME, POI_STREET_NAME,
POI_CITY, POI_POSTCODE, ACTUAL_ADDRESS');
end;
/
create index ADDRESS_SEARCH_IDX
on MAP_POI(NAME)
indextype is ctxsys.context
parameters ('datastore mymcds section group ctxsys.auto_section_group')
/
A search with contains(NAME,'Foo') should search all the columns indexed.
So let's say I have a Database called xjet and a table called samples where I have 3 samples with the colums: Id, Frequency and Temperature. It looks like this:
Id=1, Frequency=10000, Temperature=50
Id=2, Frequency=10000, Temperature=30
Id=3, Frequency=10000, Temperature=50
I want to not have columns shown where their value is always the same, so in this case I want the query to show:
Id=1, Temperature=50
Id=2, Temperature=30
Id=3, Temperature=50
Presuming of course that I don't know the values of temperature and frequency ahead. How can I do such a query?
I'm using MySqli and PHP if it's relevant
I am not sitting at a LAMP setup but here is how you would write something like this.
(using ms sql2008) its very similar to mysql command. What makes your query a bit vague is, are you going to pass frequency ? The reason when you don't display the frequency how is something/someone going to know the grouping of the values for frequency?
Create the test table and load values:
create table #testTable
(
id int identity(1,1),
Frequency int default null,
Temperature decimal default null
)
INSERT into #testTable (Frequency, Temperature)
VALUES
(10000, 50),
(10000, 30),
(10000, 50)
;
declare #someValue int = 10000;
select
tt.id,
tt.Temperature
from
#testTable tt
where
tt.Frequency = #someValue
The code above works for only one value.
Now you said you don't know the frequency before hand the code below will produce the results for you given you don't provide a frequency. The problem with code below is you can see via the output how the rows are grouped. (The code below just uses the table from above.)
select
z.id,
z.Temperature
from
(
select distinct frequency from #testTable tt group by tt.Frequency
) as t
inner join #testTable z on t.Frequency = z.Frequency;
I eventually solved it by employing PHP...thank you everyone for the responses
I have a column in one of my table where I store multiple ids seperated by comma's.
Is there a way in which I can use this column's value in the "IN" clause of a query.
The column(city) has values like 6,7,8,16,21,2
I need to use as
select * from table where e_ID in (Select city from locations where e_Id=?)
I am satisfied with Crozin's answer, but I am open to suggestions, views and options.
Feel free to share your views.
Building on the FIND_IN_SET() example from #Jeremy Smith, you can do it with a join so you don't have to run a subquery.
SELECT * FROM table t
JOIN locations l ON FIND_IN_SET(t.e_ID, l.city) > 0
WHERE l.e_ID = ?
This is known to perform very poorly, since it has to do table-scans, evaluating the FIND_IN_SET() function for every combination of rows in table and locations. It cannot make use of an index, and there's no way to improve it.
I know you said you are trying to make the best of a bad database design, but you must understand just how drastically bad this is.
Explanation: Suppose I were to ask you to look up everyone in a telephone book whose first, middle, or last initial is "J." There's no way the sorted order of the book helps in this case, since you have to scan every single page anyway.
The LIKE solution given by #fthiella has a similar problem with regards to performance. It cannot be indexed.
Also see my answer to Is storing a delimited list in a database column really that bad? for other pitfalls of this way of storing denormalized data.
If you can create a supplementary table to store an index, you can map the locations to each entry in the city list:
CREATE TABLE location2city (
location INT,
city INT,
PRIMARY KEY (location, city)
);
Assuming you have a lookup table for all possible cities (not just those mentioned in the table) you can bear the inefficiency one time to produce the mapping:
INSERT INTO location2city (location, city)
SELECT l.e_ID, c.e_ID FROM cities c JOIN locations l
ON FIND_IN_SET(c.e_ID, l.city) > 0;
Now you can run a much more efficient query to find entries in your table:
SELECT * FROM location2city l
JOIN table t ON t.e_ID = l.city
WHERE l.e_ID = ?;
This can make use of an index. Now you just need to take care that any INSERT/UPDATE/DELETE of rows in locations also inserts the corresponding mapping rows in location2city.
From MySQL's point of view you're not storing multiple ids separated by comma - you're storing a text value, which has the exact same meaing as "Hello World" or "I like cakes!" - i.e. it doesn't have any meaing.
What you have to do is to create a separated table that will link two objects from the database together. Read more about many-to-many or one-to-many (depending on your requirements) relationships in SQL-based databases.
Rather than use IN on your query, use FIND_IN_SET (docs):
SELECT * FROM table
WHERE 0 < FIND_IN_SET(e_ID, (
SELECT city FROM locations WHERE e_ID=?))
The usual caveats about first form normalization apply (the database shouldn't store multiple values in a single column), but if you're stuck with it, then the above statement should help.
This does not use IN clause, but it should do what you need:
Select *
from table
where
CONCAT(',', (Select city from locations where e_Id=?), ',')
LIKE
CONCAT('%,', e_ID, ',%')
but you have to make sure that e_ID does not contain any commas or any jolly character.
e.g.
CONCAT(',', '6,7,8,16,21,2', ',') returns ',6,7,8,16,21,2,'
e_ID=1 --> ',6,7,8,16,21,2,' LIKE '%,1,%' ? FALSE
e_ID=6 --> ',6,7,8,16,21,2,' LIKE '%,6,%' ? TRUE
e_ID=21 --> ',6,7,8,16,21,2,' LIKE '%,21,%' ? TRUE
e_ID=2 --> ',6,7,8,16,21,2,' LIKE '%,2,%' ? TRUE
e_ID=3 --> ',6,7,8,16,21,2,' LIKE '%,3,%' ? FALSE
etc.
Don't know if this is what you want to accomplish. With MySQL there is feature to concatenate values from a group GROUP_CONCAT
You can try something like this:
select * from table where e_ID in (Select GROUP_CONCAT(city SEPARATOR ',') from locations where e_Id=?)
this one in for oracle ..here string concatenation is done by wm_concat
select * from table where e_ID in (Select wm_concat(city) from locations where e_Id=?)
yes i agree with raheel shan .. in order put this "in" clause we need to make that column into row below code one do that job.
select * from table where to_char(e_ID)
in (
select substr(city,instr(city,',',1,rownum)+1,instr(city,',',1,rownum+1)-instr(city,',',1,rownum)-1) from
(
select ','||WM_CONCAT(city)||',' city,length(WM_CONCAT(city))-length(replace(WM_CONCAT(city),','))+1 CNT from locations where e_Id=? ) TST
,ALL_OBJECTS OBJ where TST.CNT>=rownum
) ;
you should use
FIND_IN_SET Returns position of value in string of comma-separated values
mysql> SELECT FIND_IN_SET('b','a,b,c,d');
-> 2
You need to "SPLIT" the city column values. It will be like:
SELECT *
FROM table
WHERE e_ID IN (SELECT TO_NUMBER(
SPLIT_STR(city /*string*/
, ',' /*delimiter*/
, 1 /*start_position*/
)
)
FROM locations);
You can read more about the MySQL split_str function here: http://blog.fedecarg.com/2009/02/22/mysql-split-string-function/
Also, I have used the TO_NUMBER function of Oracle here. Please replace it with a proper MySQL function.
IN takes rows so taking comma seperated column for search will not do what you want but if you provide data like this ('1','2','3') this will work but you can not save data like this in your field whatever you insert in the column it will take the whole thing as a string.
You can create a prepared statement dynamically like this
set #sql = concat('select * from city where city_id in (',
(select cities from location where location_id = 3),
')');
prepare in_stmt from #sql;
execute in_stmt;
deallocate prepare in_stmt;
Ref: Use a comma-separated string in an IN () in MySQL
Recently I faced the same problem and this is how I resolved it.
It worked for me, hope this is what you were looking for.
select * from table_name t where (select (CONCAT(',',(Select city from locations l where l.e_Id=?),',')) as city_string) LIKE CONCAT('%,',t.e_ID,',%');
Example: It will look like this
select * from table_name t where ',6,7,8,16,21,2,' LIKE '%,2,%';