Searching a Oracle table for a value - php

I have a Oracle table that has nearly 150k records and 65 columns. There is a requirement from my customer that They need to search the whole table with a single search value and that too can be wildcard search.
And the worst part is that i am using php language to do that.
So i have constructed a query which has all 65 columns to search for a single string value.
Sample query:
SELECT * FROM <TABLE_NAME>
WHERE (col1 || col2 || col3 ... || col65) LIKE '%<SEARCH_VALUE>%'
The query is giving output for some fields and sometimes it is failing if we pass some address related fields.
On some scenario, particularly with wild card type search and if it is outputting more than 1000 rows.
I have done these test using php language. but at the same when i try to execute the query using SQL Developer. It is giving results all the time. So i got confused.
And more over the Oracle server that i am trying to access is an external server and it is not within our environment.
Please suggest me how to implement this feature or idea and i really appreciate your assistant.

If you are using Oracle 11g, you may add a virtual column to the table
ALTER TABLE <TABLE_NAME> ADD
SEARCH_COLUMN GENERATED ALWAYS AS (col1 || ' ' || col2 || ' ' || col3 ... || col65);
Then create an index on the newly created virtual column. The index will store the concatenated string value of all 65 columns. You will not have to change any queries in your application as Oracle will automatically calculate the value of the virtual column with the values of other 65 columns.
Then you can just use the index virtual column for your searches
SELECT * FROM <TABLE_NAME>
WHERE SEARCH_COLUMN LIKE '%<SEARCH_VALUE>%'

This looks like a pretty horrible design. One problem that comes to mind is shown in the example below
col1 col2
===========
ABC DEF
CDX ZZZ
If your search condition is "CD", both of these rows will match, since you are searching on the concatenation of all columns.

I'm using this trick to search across columns And i know that the solution is slow.
create table test_example as select * from user_objects where rownum <= 1000;
select t2.* from
xmltable('for $i in ora:view("test_example")/ROW
where fn:exists($i/*[contains(.,$var_text_to_serach)])
return $i' passing 'change_here' as "var_text_to_serach" columns id varchar(100) path 'OBJECT_ID') t1
join test_example t2 on t1.id = t2. OBJECT_ID
;
ora:view - allows to query tables within xquery expression. Function creates xml-s from table rows.
xmltable - allows to map xquery-xml result to relational model.
FLWOR expresion - inside xmltable i'm using FLWOR expresion it is acronym from words FOR, LET, WHERE, ORDER, RETURN
fn:exists($i/*[contains(.,$var_text_to_serach)]) - check if any of columns contains pattern.
columns id varchar(100) path 'OBJECT_ID' extract id column from XML. In my example OBJECT_ID is uniquer value in table.
passing 'change_here' as "var_text_to_serach" - "change_here" is text to search
to use this example you have to change test_example to your table, and take care about join condition and .

Consider using a user defined data store to create a text index on all of the columns and then search with the contains operator..
Sonething like this
begin
begin
ctx_ddl.drop_preference('mymcds');
exception
when others then
null;
end;
ctx_ddl.create_preference('mymcds', 'multi_column_datastore');
ctx_ddl.set_attribute('mymcds', 'columns', 'NAME, POI_STREET_NAME,
POI_CITY, POI_POSTCODE, ACTUAL_ADDRESS');
end;
/
create index ADDRESS_SEARCH_IDX
on MAP_POI(NAME)
indextype is ctxsys.context
parameters ('datastore mymcds section group ctxsys.auto_section_group')
/
A search with contains(NAME,'Foo') should search all the columns indexed.

Related

groupby with column field having like column

I am using group by for like statement as i have database structure like this.
I want to get the count of workingzone groupby.but if i try to group by, then wrong output will appear as output will group by 99 and 99, as in figure.
My sql code is:
select count(organization),working_zone from `projects` where `district` = 12 and (`working_zone` = 99 or `working_zone` LIKE "99," or `working_zone` LIKE ",99") group by `organization`;
my desired result is:
count |working_zone
____6| 99
____3| 100
(99),(,99),(99,) should be grouped by doing sum and result should be 6.
You have an awful data structure -- although I wouldn't be surprised if the data is okay and you are really working off the result of a (reasonable) query. You should not be storing raw data in comma-delimited lists. Instead, use junction tables.
Why is having a separate row for each pair the SQLish way of storing data? Consider these reasons:
SQL has pretty based string functions (compared to other programming environments).
Data should be stored in its native type; don't store numbers as strings.
Foreign key relationships should be explicitly declared, and you can't declare a foreign key relationship using comma-delimited strings.
The primary SQL mechanism for optimizing queries are indexes, and comma-delimited lists preclude the use of indexes.
Sometimes, though, you are stuck with someone else's bad design decisions. If so, one solution uses a table of working zones:
select count(*), wz.working_zone
from projects p join
working_zones wz
on find_in_set(wz.working_zone, p.working_zone) > 0
where p.district; = 12 and
find_in_set(99, p.working_zone) > 0
group by wz.working_zone;
SELECT COUNT(organization),working_zone FROM table WHERE working_zone HAVING '99'
UNION ALL
SELECT COUNT(organization),SUBSTRING(working_zone,4) FROM table WHERE working_zone = '99,100'

Advanced search in mysql column with row of words separated by coma

Hello everyone as the topic says I am looking for alternative or advanced using of "LIKE".
I have column which contains a row of words p.e. "keyword1,keyword2,another_keyword" and when I use
$sql = mysql_query("SELECT * FROM table WHERE `column` LIKE '%keyword1%' ");
It hardly find it p.e. this example works but when i try to find shorter strings it has problems and sometimes it does not find anything.
I tried put a whitespace after comas and it helped but if there is a way where I can search for match with this specification of column I would be happy.
You may move keywords into individual table.
Or you can use SET field type, if the list of your keywords don't change.
Storing comma separated list of your words is a bad idea example using like in your scenario is hard to find the exact work in comma separated list instead you can add new table which relates to your current table and store each the word in a new row with the associated identity like
table1
id title
1 test1
2 test2
kewords_table
table1_id word
1 word1
1 word2
1 word3
and query will be
select t.*
from table1 t
join kewords_table k
on(t.id = k.table1_id)
where k.word = 'your_keyword'
If you can't alter your structure you can use find_in_set()
SELECT * FROM table WHERE find_in_set('your_keyword',`column`) > 0
try something like this:
SELECT * FROM tablename
WHERE column LIKE '%keyword1%'
OR column LIKE '%keyword2%';
for more info see here:Using SQL LIKE and IN together
MySQL allows you to perform a full-text search based on very complex queries in the Boolean mode along with Boolean operators. This is why the full-text search in Boolean mode is suitable for experienced users.
First You have to add FULLTEXT index to that perticuler column :
ALTER TABLE table_name ADD search_column TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL, ADD FULLTEXT search_column (search_column);
Run following query for searching :
SELECT * FROM table WHERE MATCH(search_column) AGAINST("keyword1")
for more info see here : https://dev.mysql.com/doc/refman/8.0/en/fulltext-boolean.html

Uniquely identify same column name of two different table after join - mysql

I have 2 tables. suppose a & b
a has id, name, roll. b has id,group,name
This name column data are not same. How can I select and uniquely identify them?
I know about
SELECT a.id,a.name,a.group FROM a,b ............
I know this. But this is an example. I am working with huge amount of data with 20-30 columns in each table. So I don't want to write the column names I need to select rather I want to write the names that I want to exclude.
Like
SELECT * Except b.name............
OR is there any way to uniquely identify after join. Like
.......... a,b WHERE a.name as name1
Please don't ask why those column names are same. I admit it was a mistake. But it's already implemented and heavily used. So finding another way. Is there any simple way to exclude a column while merging them?
Well, you can't write the names you wish to exclude. That is not how SQL works.
However, if writing out 20-30 column names is that much of a burden, you can use information_schema.columns. I write it that way, because 20-30 column names is not particularly large and writing them out is probably less effort than writing the question.
But, back to the solution. It looks something like this:
select concat(c.column_name, ' as ', 'a_', column_name, ', ')
from information_schema.columns c
where table_name = 'a' ;
You might want to include the table schema as well.
As an IDEA, what you can do is, if you want to avoid columns of specific table & your statements have multiple table, you can try following,
Suppose you have 20 columns in table a & 5 columns in table b, you want to avoid col2,col3 & col4 of table b. Standard method is that you should write name of all columns of table a & required columns of table b. But you can avoid to write long list of 20 columns of table by writing a.* & then type required columns of table b. Please see below statement.
Select a.*,b.col1,b.col4,b.col5 from a,b
But if you require to exclude some columns from both table, then I think there is no other way than writing all required column names from both table.
There is no way to exclude a column in SQL SELECT Statement, you can only select a column. You can give alias name to columns while selecting them like below, so that you can identity columns using those alias names.
SELECT a.id as [column1],a.name as [column2],a.group as [column3] FROM a,b ............
There is no way to exclude a specific column but you can avoid to write all columns name and easy your job by below steps-
Step1: Execute below query-
SELECT a.*,b.* FROM a,b ............limit 1;
Step2: Export it into csv format with headings.
Step3: Copyp first (heading) row from csv.
Step4: Delete columns, those are not required and use other columns in your query.
There's only one waY i could see-
first create a temorary table
CREATE TEMPORARY TABLE IF NOT EXISTS mytable
(id int(11) NOT NULL, PRIMARY KEY (id)) ENGINE=MyISAM;
then put your column in temporary table-
SELECT * INTO mytable
FROM YourTable
/* Drop the cloumns that are not needed */
ALTER TABLE mytable
DROP COLUMN ColumnToDrop
/* Get results and drop temp table */
SELECT * FROM #TempTable
DROP TABLE #TempTable

Mysql with regular expression

I have a query regarding regular expression.I have design a table which contain three column one column contain member ids which are separated by commas.I am showing you my table structure.Please follow
send_id member_id
1 1211,23,34
2 1,23
I want to select only send_id 2 data which contain member_id as 1.
this is the query that i am using
SELECT * FROM table WHERE column REGEXP '^[1]+$';
but this query giving me both row.Please help me.
With Regards
Rahul
Never store separate values in one column
Normalize your structure like
send_id member_id
1 1211
1 23
1 34
2 1
2 23
If you still want your regex, then it will be
SELECT * FROM t WHERE column REGEXP '(^|[^0-9])1([^0-9]|$)'
First, you should be normalizing your data so you're not in this horrible mess in the first place. Here's a good resource explaining normalization.
Second, I believe your problem lies with your regular expression. Try this instead:
SELECT * FROM table WHERE column REGEXP '^[1]$';
The regular expression you're using uses the [1]+ group. The + means it has to match [1] 1 or more times, hence why you're getting two rows instead of one. Removing the + means it will match [1] once.
However, that still won't fix your problem, as more than one row contains 1. This is why normalization is so important.
Having multiple values inside a column isn't a good practice for designing a DB.
You should normalize your data, i.e., put just one piece of atomic information inside each element of your table.
You can find more information regarding to this in Wikipedia:
http://en.wikipedia.org/wiki/Database_normalization
Like they have told you, perfect solution would be normalize your data, I think Alma Do Mundo answer explains it quite well.
If you want to use REGEXP anyway you have to take in account four approaches; id is the only one, id is the first, id is in the middle and id is at the end. I have use id=74 for the example:
SELECT * FROM table WHERE member_id REGEXP '(^74$|^74,|,74,|,74$)';
depending on your requirements, you should either normalize your data i.e. make 3 tables, one with the send ID, one with the member id, and one that combines the two, then you can link them up with INNER JOINS.
However, if you are going to do it that way, you can use a "WHERE member_id LIKE %1%" to pull in all the relevant fields. You'll have to use the application to filter the relevant records.
In any case, if you're not going to normalize the data you will have to use the front end to filter out the results.
An example of the inner join syntax would look like this
SELECT * FROM SendTable
JOIN Send_Member ON SendTable.send_id = Send_Member.send_id
JOIN Member ON Member.member_id = Send_Member.member_id
WHERE Member.member_id = 1;
where the schema looks like:
Sendtable:
send_Id (primary key)
...other fields
Send_Member:
send_id (primary key and foreign key to SendTable)
member_id (primary key and foreign key to member)
...any fields you might want that are relevant to the particular send table and member table link
Member:
member_id (primarykey)
...other fields

Comma separated values in MySQL "IN" clause

I have a column in one of my table where I store multiple ids seperated by comma's.
Is there a way in which I can use this column's value in the "IN" clause of a query.
The column(city) has values like 6,7,8,16,21,2
I need to use as
select * from table where e_ID in (Select city from locations where e_Id=?)
I am satisfied with Crozin's answer, but I am open to suggestions, views and options.
Feel free to share your views.
Building on the FIND_IN_SET() example from #Jeremy Smith, you can do it with a join so you don't have to run a subquery.
SELECT * FROM table t
JOIN locations l ON FIND_IN_SET(t.e_ID, l.city) > 0
WHERE l.e_ID = ?
This is known to perform very poorly, since it has to do table-scans, evaluating the FIND_IN_SET() function for every combination of rows in table and locations. It cannot make use of an index, and there's no way to improve it.
I know you said you are trying to make the best of a bad database design, but you must understand just how drastically bad this is.
Explanation: Suppose I were to ask you to look up everyone in a telephone book whose first, middle, or last initial is "J." There's no way the sorted order of the book helps in this case, since you have to scan every single page anyway.
The LIKE solution given by #fthiella has a similar problem with regards to performance. It cannot be indexed.
Also see my answer to Is storing a delimited list in a database column really that bad? for other pitfalls of this way of storing denormalized data.
If you can create a supplementary table to store an index, you can map the locations to each entry in the city list:
CREATE TABLE location2city (
location INT,
city INT,
PRIMARY KEY (location, city)
);
Assuming you have a lookup table for all possible cities (not just those mentioned in the table) you can bear the inefficiency one time to produce the mapping:
INSERT INTO location2city (location, city)
SELECT l.e_ID, c.e_ID FROM cities c JOIN locations l
ON FIND_IN_SET(c.e_ID, l.city) > 0;
Now you can run a much more efficient query to find entries in your table:
SELECT * FROM location2city l
JOIN table t ON t.e_ID = l.city
WHERE l.e_ID = ?;
This can make use of an index. Now you just need to take care that any INSERT/UPDATE/DELETE of rows in locations also inserts the corresponding mapping rows in location2city.
From MySQL's point of view you're not storing multiple ids separated by comma - you're storing a text value, which has the exact same meaing as "Hello World" or "I like cakes!" - i.e. it doesn't have any meaing.
What you have to do is to create a separated table that will link two objects from the database together. Read more about many-to-many or one-to-many (depending on your requirements) relationships in SQL-based databases.
Rather than use IN on your query, use FIND_IN_SET (docs):
SELECT * FROM table
WHERE 0 < FIND_IN_SET(e_ID, (
SELECT city FROM locations WHERE e_ID=?))
The usual caveats about first form normalization apply (the database shouldn't store multiple values in a single column), but if you're stuck with it, then the above statement should help.
This does not use IN clause, but it should do what you need:
Select *
from table
where
CONCAT(',', (Select city from locations where e_Id=?), ',')
LIKE
CONCAT('%,', e_ID, ',%')
but you have to make sure that e_ID does not contain any commas or any jolly character.
e.g.
CONCAT(',', '6,7,8,16,21,2', ',') returns ',6,7,8,16,21,2,'
e_ID=1 --> ',6,7,8,16,21,2,' LIKE '%,1,%' ? FALSE
e_ID=6 --> ',6,7,8,16,21,2,' LIKE '%,6,%' ? TRUE
e_ID=21 --> ',6,7,8,16,21,2,' LIKE '%,21,%' ? TRUE
e_ID=2 --> ',6,7,8,16,21,2,' LIKE '%,2,%' ? TRUE
e_ID=3 --> ',6,7,8,16,21,2,' LIKE '%,3,%' ? FALSE
etc.
Don't know if this is what you want to accomplish. With MySQL there is feature to concatenate values from a group GROUP_CONCAT
You can try something like this:
select * from table where e_ID in (Select GROUP_CONCAT(city SEPARATOR ',') from locations where e_Id=?)
this one in for oracle ..here string concatenation is done by wm_concat
select * from table where e_ID in (Select wm_concat(city) from locations where e_Id=?)
yes i agree with raheel shan .. in order put this "in" clause we need to make that column into row below code one do that job.
select * from table where to_char(e_ID)
in (
select substr(city,instr(city,',',1,rownum)+1,instr(city,',',1,rownum+1)-instr(city,',',1,rownum)-1) from
(
select ','||WM_CONCAT(city)||',' city,length(WM_CONCAT(city))-length(replace(WM_CONCAT(city),','))+1 CNT from locations where e_Id=? ) TST
,ALL_OBJECTS OBJ where TST.CNT>=rownum
) ;
you should use
FIND_IN_SET Returns position of value in string of comma-separated values
mysql> SELECT FIND_IN_SET('b','a,b,c,d');
-> 2
You need to "SPLIT" the city column values. It will be like:
SELECT *
FROM table
WHERE e_ID IN (SELECT TO_NUMBER(
SPLIT_STR(city /*string*/
, ',' /*delimiter*/
, 1 /*start_position*/
)
)
FROM locations);
You can read more about the MySQL split_str function here: http://blog.fedecarg.com/2009/02/22/mysql-split-string-function/
Also, I have used the TO_NUMBER function of Oracle here. Please replace it with a proper MySQL function.
IN takes rows so taking comma seperated column for search will not do what you want but if you provide data like this ('1','2','3') this will work but you can not save data like this in your field whatever you insert in the column it will take the whole thing as a string.
You can create a prepared statement dynamically like this
set #sql = concat('select * from city where city_id in (',
(select cities from location where location_id = 3),
')');
prepare in_stmt from #sql;
execute in_stmt;
deallocate prepare in_stmt;
Ref: Use a comma-separated string in an IN () in MySQL
Recently I faced the same problem and this is how I resolved it.
It worked for me, hope this is what you were looking for.
select * from table_name t where (select (CONCAT(',',(Select city from locations l where l.e_Id=?),',')) as city_string) LIKE CONCAT('%,',t.e_ID,',%');
Example: It will look like this
select * from table_name t where ',6,7,8,16,21,2,' LIKE '%,2,%';

Categories