I've read quite a few similar posts but none solves my case, which could well be because of my lack of sufficient knowledge, so please bear with me.
One of the search options in my terminological dictionary is "whole words only". At first I was using
WHERE ".$source." RLIKE '[[:<:]]".$keyword."[[:>:]]'
However, this failed to match whole words for the first or second $keyword when there is more than one. Then I found
WHERE ".$source." REGEXP '[[:<:]]".$keyword."[[:>:]]'
and
WHERE ".$source." REGEXP '(^| )".$keyword."( |$)'
while searching these forums
I just tested both of the above in my PhpMyAdmin and found out that the former executes in 0.0740 seconds, while the latter takes twice as long, 0.1440 seconds, so I guess I should stick with the former.
What bothers me the most is the huge discrepancy in results, e.g. searching for a single word ("tool"):
Using the [[:<:]] and [[:>:]] word boundary in PhpMyAdmin returns 34 results.
Using (^| ) and ( |$) in PhpMyAdmin returns 26 results.
Running the #1 regexp in my PHP script returns 34 results (this is the correct number).
Here's the whole MySQL block:
foreach($keywords as $keyword) {
$query = $db->query("SELECT * FROM ".DICTIONARY_TABLE." " .
"JOIN ".DICTIONARY_THEMES." ON ".DICTIONARY_TABLE.".theme_id = ".DICTIONARY_THEMES.".theme_id ".
"LEFT JOIN ".DICTIONARY_DEFINITIONS." ON ".DICTIONARY_TABLE.".term_id = ".DICTIONARY_DEFINITIONS.".term_id ".
"WHERE ".DICTIONARY_TABLE.".".$source." REGEXP '(^| )".$keyword."( |$)'".
//"WHERE ".DICTIONARY_TABLE.".".$source." REGEXP '[[:<:]]".$keyword."[[:>:]]'".
" ORDER BY ".DICTIONARY_TABLE.".theme_id, ".DICTIONARY_TABLE.".".$source."");
}
I've commented out the search option I'm not using.
Now, if I try TWO keywords, e.g. "cutting tool", I still get 34 results in the page. I'm unsure if I'm doing this right in PhpMyAdmin:
SELECT * FROM `asphodel_dictionary_terms` WHERE english REGEXP '[[:<:]]cutting[[:>:]]';
SELECT * FROM `asphodel_dictionary_terms` WHERE english REGEXP '[[:<:]]tool[[:>:]]'
This returns 44 results for "cutting" and 34 results for "tool". The query using (^| )... returns 37 + 26 results respectively.
Any feedback that would help me sort things out would be appreciated!
The database contains a total of 109,000 entries in the main table, there are 82 themes in the DICTIONARY_THEMES table and 727 entries in the DICTIONARY_DEFINITIONS table. Not a huge database and it won't grow much bigger.
You are getting different results because the two regexes are not identical.
(^| ) means : either the beginning of the string or a space (( |$) has the same meaning at end of string).
[[:<:]] and [[:>:]] are word boundaries : conceptually this refers to characters that separate words, and usually regex engines interpret it as something like : anything but a digit, a letter or an underscore.
So basically the first pattern is more restrictive than the second (space, beginning and end of string are word boundaries, but there are others).
If you have more than one keyword to search for, you would need to repeat the regex matches, like :
WHERE
".$source." RLIKE '[[:<:]]".$keyword1."[[:>:]]'
OR ".$source." RLIKE '[[:<:]]".$keyword2."[[:>:]]'
Or create a new regex by combining the keywords :
WHERE
".$source." RLIKE '[[:<:]](".$keyword1.")|(".$keyword2.")[[:>:]]'
NB : for search requirement, you should consider using MySQL Full Text Search, which are primarily built for the purpose of searching for full words (there are pre-requisites, though).
Related
I have a databse with keywords coloumn
Need to search the database on the basis of query done by user.
Every keyword has word "outlet" at the end but user will only search "gul ahmad" not "gul ahmad outlet". For this i used following query and things worked fine to get results and found complete result "Gul Ahmad Outlet"
$sql = "SELECT keywords FROM table WHERE keywords REGEXP '([[:blank:][:punct:]]|^)$keyword([[:blank:][:punct:]]|$)'";
Now i have 2 issues
1. If the word "outlet is in between the query words then it does not find the word. e.g if user search "kohistan lahore", database has an outlet named "kohistan outlet lahore" but it does not find the keyword in database and returns empty. How to tell database to include "outlet" in between, at the start or athe end to find and match the result.
if some user search "nabeel's outlet" database has it but due to " ' " this query returns empty without any result.
What you can do is that you can match your column values with just the first word
of your search expression(i.e nabeel's outlet). I believe this way you will be able to cover all your scenarios.
select
*
from `outlets`
where REPLACE(`name`,'\'','') regexp SUBSTRING_INDEX('nabeels outlet', ' ', 1)
Look at this fiddle and test yourself : http://sqlfiddle.com/#!9/b3000/21
Hope it helps.
Much simpler: [[:<:]]$keyword[[:>:]] -- This checks for "word boundary" instead of space or punctuation or start/end of string. And $keyword = "nabeel's" should not be a problem.
Don't you want to always tack on "outlet"?
REGEXP "[[:<:]]$keyword[[:>:]] outlet"
And, yes, you must escape certain things, such as the quotes that will be used to quote the regex string. PHP's addslashes() is one way.
I have this REGEX expression I use to find word in a text through an SQL query
[SQL QUERY] where REGEX concat(^|’| | | |\'|>|-)','Mercure','( | | |.|,|<|-) = 1
That work the way I want (for some reasons, the traditional word boundaries don't do the job). I would like to add to each capturing group respectively an opening parenthesis and a closing one. But when I try :
(^|’| | | |\'|>|-|\()','Mercure','( | | |.|,|<|-|\))
Then I get an error with my PHP script, although regex101 doesn't seem to find this odd. Does anyone have an idea how to solve that ?
WHERE col REGEXP "(^|[’ '>-])Mercure[ .,<-]"
I don't understand having multiple blanks.
It is not easy to look for > matched only by < -- that would probably take a long list of choices.
One way to approach the problem (if it is complex) is to do only this in MySQL:
WHERE col LIKE '%Mercure%'
then use PHP code to filter out the rows that don't match the complex boundary conditions.
I need: a regexp for MySQL (php, PDO) usage. Regexp should find all numbers between brackets [ ], except number 150.
So I would like to get:
[3]
[25464]
[510]
But I would like to exclude:
[150]
What I got:
\[{1}((?!150)[0-9])+\]{1}
and it works fine for newest version of MySQL, but I need something that would work also on an older version (probably 5.1).
Problem: Currently I get an error:
1139 - Got error 'repetition-operator operand invalid' from regexp
I know I can't use ?. How can I replace it?
Additional info (edit):
I'm redesigning the database and that's why I need to write this
II edit - why I need this:
I need to retrieve all rows which in column "content" contains only one specified [150]. One column 'content' can contain zero [nr] or one specific [nr] or many different [nrs].
WHERE content REGEXP '\[{1}((?!150)[0-9])+\]{1}' = 0 AND content LIKE '%[150]%'
Try this one, Hope this works.
$string='[3]
[25464]
[510]
[150]
[100]';
preg_match_all('/\[(?!150)\d+\]/', $string,$matches);
print_r($matches);
This will match all digits(except 150) with brackets.
$matches[0] will contain desired result...
WHERE x REGEXP '\[[[:digit:]]+\]'
AND NOT x REGEXP '\[150\]'
However, that will reject aaa[123]bbb[150]ccc. Should it be rejected? (Please give some sample data that be matched / rejected. Your goal is not crystal clear.)
Since LIKE is faster than REGEXP, this will be a little faster:
WHERE x REGEXP '\[[[:digit:]]+\]'
AND x NOT LIKE '%[150]%'
Thanks for taking time to read my question.
I have created a MySQL table, a HTML form and a program in PHP which connects the form to MySQL table and retrieves sequences for column Annotations which is text data type.
This column has characters and also has one or more of hyphen, comma, parentheses, period or spaces.
Please look at the following code that I used for select query:
$values=mysql_query("SELECT Sequence
FROM oats
WHERE Foldchange = '$Foldchange' AND
RustvsMockPvalue = '$RustvsMockpvalue' AND
Annotations REGEXP '%$Annotation%[-]+'");
Here $Annotation is the form variable which holds the value entered by the user in the form. Annotations is the column name in the MySQL table.
Annotations column has characters A-Z or a-z and one or more of hyphen, comma, space or parentheses like the following.
Sequence is another text column in the MySQL table but does not have ,./().
Example data from Annotations column:
ADP, ATP carrier protein, mitochondrial precursor (ADP/ATP translocase) (Adenine nucleotide translocator) (ANT).
I am not able to retrieve Sequence column data when I search for any Annotations column data with comma, parentheses, period and slash. It works fine for those records which does not have these ,.()/.
I tried to use LIKE instead of REGEX but it didn't work either.
A record from mysql table:(columns that you see below: contigid,source,genelength,rustmeans, mockmeans,foldchange,pvalue,rustmockteststatistic,Annotations and Sequence)
as_rcr_contig_10002 ORME1 2101 506.33 191 -2.18 2.21E-10 -6.35 Tesmin/TSO1-like, CXC domain containing protein. AACAATTCCCCTCAACCAACCTTTTATTTCATCCCATTTTTATCATCTGTCCGGTTACAGATTTTGCTTCCAGTTAGGTGCCACTTCTTCAAACGCTCAACCCTTACCCACTACCACCCCACCAAAACCAACCCCCCAAGATGCAGTTCATCACTCTCGCCGTTGCTTTTGCTTTCTTTGCTGGTGCCANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNCTTTTGCTTTCTTTGCTGGTGCCACCTCGTCGCCGGTTTCCATGGACCCCAAAGCCGAGAAGTCCGGCTCCTCGGGATCCGGTGGCGCCCCTCTGGGCACTGCTAGCCCCTATCCCCAAAGNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNGGTGGCCCTCAGTCGCCAGGCTCTGGCCAACCCGGTAGGATGCCATGGGGTAGCGACCAATCTGCCTACGGTGGTGGTTTCCCTTATGGATCATTCCCCTCGGTTTCGGGGCAATCCCAATCGACGGCCTATGCTCAAGCTCAATCATCCAGTTTCCCCTCAAACGGTGTCCCGACACACTCCTCGGCCTCCGCCCAAGCGCAATCATCCGGTCCTGGACAAGCTCAGGCAGCCGCTTCTGCCCAGGTTCCCGGCGGCCCCCACGGTCAAGGTTCTAACGGATTTGGCGCACAAGGCCAGTTTGGACAGAACGGGCAGAACGGCCTCTATGGTCAAGACGGCAATGGCTTTAGTGCCCAAGGCCAATTTGGACAGAGTGGACAGAATGGCTTCTATGGTCA
Could someone please help me in the correct syntax of the SELECT syntax? Thank you.
You need to familiarise yourself with regex - it's its ownittke language.
Use REGEXP with the right regex:
WHERE ...
AND Annotations REGEXP '[-A-Za-z(). ]+'
AND Annotations NOT_REGEXP '[A-Za-z]+'
If mysql supported regex look aheads, this could be done in one test.
,
First of all, you are not using REGEXP properly.
You should check the differences between LIKE and REGEXP.
REGEXP use Regular expresions, which have very particular syntax.
LIKE use simple text remplacement with key characters like % or _
Here you are using REGEXP with %, that's why it's not working. % is a key character for LIKE only.
But in REGEXP, . and - are special characters that you need to escape to.
If you want to check several characters, REGEXP is the way to go :
Annotations REGEXP '.*$Annotation.*[\-(),\.]+.*'
This match :
.* : 0 to n characters
$Annotation : Your keyword
.* : 0 to n characters
[\-(),\.]+ : At least 1 character from the list : - ( ) , .
.* : 0 to n characters
Tell us if that match your data.
Since we can't craft a Regular Expression that would work in your case without getting into some crazy matching schemes (orders and so forth), In order to find what you're looking for, you'll need to custom construct the SQL statement and luckily you're using PHP.
Here I'm starting with a simple space delimited entry. Remember that you can't wrap something with parenthesis because the parenthesis might not match up in your result set.
$search_input = 'ADP ANT';
//example of array from a search page full of check boxes or fields
$annSearches = explode(' ',$search_input);
/*annSearches is now and array with ADP,ANT*/
$sql = "SELECT Sequence FROM oats WHERE Foldchange = '$Foldchange' AND RustvsMockPvalue = '$RustvsMockpvalue'";
foreach ($annSearches as $Annotation){
$sql .= " AND Annotations LIKE '%$Annotation%'";
}
The output SQL statement would look like this (wrapped for clarity):
SELECT Sequence FROM oats WHERE
Foldchange = '$Foldchange'
AND RustvsMockPvalue = '$RustvsMockpvalue'
AND Annotations LIKE '%ADP%'
AND Annotations LIKE '%ANT%';
If you do a really long query, this will get slower and slower as MySQL has to run through every record in the database over and over for the results.
FULLTEXT SEARCH OPTION
Another way that you could potentially do this is to enable FULLTEXT search functionality on the Annotations field in the table in the database.
ALTER TABLE oats ADD FULLTEXT(Annotations);
This would allow you to do a search something like this:
Sequence FROM oats WHERE
Foldchange = '$Foldchange'
AND RustvsMockPvalue = '$RustvsMockpvalue'
MATCH(Annotations) AGAINST ('ADP ANT')
I think I'm going crazy with this...
I've tried a lot of combinations and I can't get with the good one.
I need to find all the SQL queries in a PHP code after having read it with a file_get_contents().
Of course, all those queries are variable assignations like:
$sql1 = "
SELECT *
FROM users u
WHERE u.name LIKE '%".$name."%' AND ... ;
";
or
$sql2 = "
SELECT *
FROM users u
WHERE u.id = ".$user_id;
or
$sql3 = '
SELECT *
FROM users u
ORDER BY u.surname1 DESC
'; //this query blablabla.......
So you can see that there are many factors to take in account for PHP variables.
First I've tried with an approximation based on getting the variable itself combined with the getting it's content...
I've tried to find specific words from SQL in a regex pattern too...
Whatever...
I don't know how to do it.
Getting all the variable and it's assignation, grouping the assignation and after it, looping through matches searching for special SQL words (that's what I've right now, but it doesn't work cause assignation regex part).
Directly searching for a SQL queries with a good regex?
PHP variables (specifically strings), contains partially concatenations with other variables, double and single quoted strings, comments at the end of ";" or in the middle...
So what can I do?
So far, that's my variable regex part:
$regex_variable = '\\$([a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*)\s*[\+\-\*\/\%\.\&\|\^\<\>]*=\s*';
Which I concatenate with $regex_sql which I've tried different forms:
//$regex_sql = '(["\'])(.*?)\2\s*;';
//$regex_sql = '(["\'])([^;]*?)\2\s*;';
//$regex_sql = '(?<!")\b\w+\b|(?<=")\b[^"]+';
//$regex_sql = '([^;]+)(?<=["\']);(?!["\'])';
//$regex_sql = '(.*?;)[^\\$]*';
None of those correctly works.
Can you help me please? I'm sure the best approximation it's getting all the variable itself, and after it, testing the assignation for containing some special SQL words like SELECT, WHERE, UNION, ORDER, ...
So much thanks in advance!
Mark.
edit:
To add that of course, variables with queries could have any kind of form. Those from above are just simple examples.
We're talking about things such:
$s = 'insert into tabletest(a,b,c) values('asd','r32r32','fdfdf')';
or
$where = 'where a=2';
$sql="select distinct * from test ".$where;
or
$a = '
select *
from users
left outer join ...
inner join ...
left join ...
where ...
group by ...
having ...
order by ...
limit ...
...
';
or
...
Imagine a lot of programmers, creating queries inside the code, anyone doing it at their own way... :\
I've to get ALL of them. At least, maximise the results... ^^'
I suggest you take a look at the PHP Tokenizer - you can use it to tokenize your source (i.e. parse it so it is easier to comprehend) then you can look through the tokens for strings and variables that match your requirements, knowing that each token ; ends a line of code.
Don't know if this is what you are looking for :
preg_match_all('/\$.*?=(.*?)(?<=[\'"]);/s', $subject, $result, PREG_PATTERN_ORDER);
$result = $result[1];
This will have all the assignments(assignations) stored in $result. I tested it with all your samples.
Sorry if you wanted something else.
Explanation :
"
\$ # Match the character “\$” literally
. # Match any single character
*? # Between zero and unlimited times, as few times as possible, expanding as needed (lazy)
= # Match the character “=” literally
( # Match the regular expression below and capture its match into backreference number 1
. # Match any single character
*? # Between zero and unlimited times, as few times as possible, expanding as needed (lazy)
)
(?<= # Assert that the regex below can be matched, with the match ending at this position (positive lookbehind)
['\"] # Match a single character present in the list “'\"”
)
; # Match the character “;” literally
"