Mysql, PHP, searching for multiple words - php

I'm trying to search a table for specific words.
Say I have a list of words: printer,network,wireless,urgent
I only want to return those rows where all of these words are in it.
SELECT * FROM tickets WHERE concat(subject,body) REGEXP "printer|network|wireless|urgent"
will return any row with any one of these words. How can I make it so that it will only return those rows where all of these words are in it.
Thanks,

There are two ways to do this. The first is the rather obvious approach. Let's say you have all the words that need to appear in an array called $necessaryWords:
$sql = 'SELECT ... FROM ...'; // and so on
$sql .= ' WHERE 1';
foreach ($necessaryWords as $word)
$sql .= ' AND concat(subject,body) LIKE "%' . $word . '%"'; //Quotes around string
However, using %foo% is rather slow, as no indexes can be used, so this query might cause performance issues with huge tables and/or a high number of necessary words.
The other approach would be a FULLTEXT index on subject and body. You could the use the fulltext MATCH IN BOOLEAN MODE like this:
$sql = 'SELECT ... FROM ...'; // and so on
$sql .= ' WHERE MATCH(subject,body) AGAINST("';
foreach ($necessaryWords as $word)
$sql .= ' +' . $word;
$sql .= '")';
Note that your table must use MyISAM in order to use FULLTEXT indexes. UPDATE: As of MySQL 5.6, InnoDB supports FULLTEXT indexes as well. I guess this could be the better choice performance wise. Further documentation on the fulltext in boolean mode can be found in the manual.

not the best way, but:
SELECT * FROM tickets WHERE
concat(subject,body) REGEXP "printer" AND
concat(subject,body) REGEXP "network" AND
concat(subject,body) REGEXP "wireless" AND
concat(subject,body) REGEXP "urgent"

SELECT * FROM tickets WHERE
concat(subject,body) LIKE "%printer%" AND
concat(subject,body) LIKE "%network%" AND
concat(subject,body) LIKE "%wireless%" AND
concat(subject,body) LIKE "%urgent%"

Not sure this would work with MySQL regex engine but the regex (using lookarounds) below can achieve what you are looking for. Will find the words of interest irrespective of the order in which they occur:
^(?=.*printer)(?=.*network)(?=.*wireless)(?=.*urgent).*$
Demo: http://www.rubular.com/r/XcVz5xMZcb
Some regex lookaround examples here: http://www.rexegg.com/regex-lookarounds.html

Alternative answer, just because I thought of it when I looked at your question. I do not know whether it would be faster than the other answers (most likely no):
(SELECT * FROM tickets WHERE subject LIKE "%printer%" OR body LIKE "%printer%")
UNION
(SELECT * FROM tickets WHERE subject LIKE "%network%" OR body LIKE "%network%")
UNION
(SELECT * FROM tickets WHERE subject LIKE "%wireless%" OR body LIKE "%wireless%")
UNION
(SELECT * FROM tickets WHERE subject LIKE "%urgent%" OR body LIKE "%urgent%")
UPDATE:
This is wrong

Related

How to select rows by substring? Similar to Google Suggest [duplicate]

I have mysql table that has a column that stores xml as a string. I need to find all tuples where the xml column contains a given string of 6 characters. Nothing else matters--all I need to know is if this 6 character string is there or not.
So it probably doesn't matter that the text is formatted as xml.
Question: how can I search within mysql?
ie
SELECT * FROM items WHERE items.xml [contains the text '123456']
Is there a way I can use the LIKE operator to do this?
You could probably use the LIKE clause to do some simple string matching:
SELECT * FROM items WHERE items.xml LIKE '%123456%'
If you need more advanced functionality, take a look at MySQL's fulltext-search functions here: http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html
Using like might take longer time so use full_text_search:
SELECT * FROM items WHERE MATCH(items.xml) AGAINST ('your_search_word')
SELECT * FROM items WHERE `items.xml` LIKE '%123456%'
The % operator in LIKE means "anything can be here".
Why not use LIKE?
SELECT * FROM items WHERE items.xml LIKE '%123456%'
you mean:
SELECT * FROM items WHERE items.xml LIKE '%123456%'
When you are using the wordpress prepare line, the above solutions do not work. This is the solution I used:
$Table_Name = $wpdb->prefix.'tablename';
$SearchField = '%'. $YourVariable . '%';
$sql_query = $wpdb->prepare("SELECT * FROM $Table_Name WHERE ColumnName LIKE %s", $SearchField) ;
$rows = $wpdb->get_results($sql_query, ARRAY_A);

My code search is really slow

This is my code:
$sql = "SELECT *,MATCH (CUIT,DENOMINACION) AGAINST ('%{$word}%') AS Score FROM cuits WHERE";
$sql_end = '';
foreach($words as $word) {
$sql_end .= " AND MATCH (CUIT,DENOMINACION) AGAINST ('%{$word}%' IN BOOLEAN MODE) AND BAJA=0 ";
$sql_end2 = "order by DENOMINACION limit $inicio, $TAMANO_PAGINA";
}
It is very slow. How can I improve the speed?
It doesn't make sense in FULLTEXT search to use LIKE-style wildcard strings %. So, change AGAINST ('%{$word}%') to AGAINST ('{$word}') and see if it helps.
Also, FULLTEXT can match multiple words at once, so there's no need to repeat the whole MATCH clause for each word. Instead try
SELECT *,MATCH (CUIT,DENOMINACION) AGAINST ('word word word word') ...
You may, if the series of words is a phrase, wish to stop using the boolean FULLTEXT mode and use the natural language mode as shown in your first MATCH clause.
Be aware that FULLTEXT works very strangely indeed on small tables, with less than a few hundred rows. That makes testing on small tables difficult.
Finally, make sure your FULLTEXT index is constructed correctly.

having trouble search through mysql database

I have two questions regarding my script and searching. I have this script:
$searchTerms = explode(' ', $varSearch);
$searchTermBits = array();
foreach($searchTerms as $term){
$term = trim($term);
if(!empty($term)){
$searchTermBits[] = "column1 LIKE '%".$term."%'";
}
}
$sql = mysql_query("SELECT * FROM table WHERE ".implode(' OR ', $searchTermBits)."");
I have a column1 with a data name "rock cheer climbing here"
If I type in "rock climb" this data shows. Thats perfect, but if I just type "Rocks", it doesn't show. Why is that?
Also, How would I add another "column2" for the keyword to search into?
Thank you!
Searching that string for "rocks" doesn't work, because the string "rocks" doesn't exist in the data. Looking at it, it makes sense to you, because you know that the plural of "rock" is "rocks", but the database doesn't know that.
One option you could try is removing the S from search terms, but you run into other issues with that - for example, the plural of "berry" is "berries", and if you remove the S, you'll be searching for "berrie" which doesn't get you any further.
You can add more search terms by adding more lines like
$searchTermBits[] = "column1 LIKE '%".$term."%'";
and replacing ".$term." with what you want to search for. For example,
$searchTermBits[] = "column1 LIKE '%climb%'";
One other thing to note... as written, your code is susceptible to SQL injection. Take this for example... What if the site visitor types in the search term '; DROP TABLE tablename; You've just had your data wiped out.
What you should do is modify your searchTermBits[] line to look like:
$searchTermBits[] = "column1 LIKE '%" . mysql_real_escape_string($term) . "%'";
That will prevent any nastiness from harming your data.
Assuming the data you gave is accurate, it shouldn't match because you're using "Rocks" and the word in the string is "rock". By default mysql doesn't do case sensitive matching, so it's probably not the case.
Also, to avoid sql injection, you absolutely should be using mysql_real_escape_string to escape your content.
Adding a second column would be pretty easy as well. Just add two entries to your array for every search term, one for column1 and one for column2.
Your column1 data rock cheer climbing here your search criteria %Rocks% it doesn't fit at all as rocks is not in your column1 data
you can add column2 as you do for column1 then put it all together by using an AND operator (column1 LIKE "%rock%" OR column1 LIKE "%climb%") AND (column2 LIKE "%rope%" OR column2 LIKE "%powder%")
TIPS:
If your table/schema are using xx_xx_ci collation (then this is mean case insensitive,mysql doesn't care case sensitive) but if other then you need to make sure that the search term must be case sensitive(mysql do case sensitive).

Php/mysql search engine

How I do exact match or anywhere match keywords on the table with php/mysql search engine.
For example: i added a url with keywords: UFC 131 online
And 2nd url with keywords: UFC 132 online
and if i enter UFC 132 on the search box how do i make the 2nd url to come up first?
Because if i use this code
//explode out search term
$search_exploded = explode(" ",$search);
foreach($search_exploded as $search_each)
{
//construct query
$x++;
if ($x==1)
$construct .= "title LIKE '%$search_each%'";
else
$construct .= " or title = '$search'";
and i search for ufc 132 online it gives me ufc 131 online result first.
You could implement something like this:
SELECT *, MATCH (title) AGAINST ('searchstring' IN BOOLEAN MODE) +
CASE
WHEN title = 'searchstring' THEN 10
ELSE 0
END AS relevance FROM `wrestlingsucks`
HAVING relevance > 1.0
ORDER BY relevance DESC LIMIT 100
This is a very basic example, all it will do is add 10 to a sudo relevance column if its an exact match. You could however get as complicated as you want with full text and natural matching.
MySQL has superior search capabilities for full text search than handcrafting your searches. Take a look at fulltext-search
From the documentation MATCH (col1,col2,...) AGAINST (expr [search_modifier])
Use Levenshtein algorithm in MySQL.
Levenshtein matching is a metric for
measuring the amount of difference
between two sequence, here it is
strings. By default MySQL does not
have this function, but you can write
and add one.
Please take a look at the code here and add that code as a system function in MySQL, please see the example below on how to get the similarity of two strings.
Please see: https://github.com/rakesh-sankar/Tools/blob/master/MySQL/Levenshtein.txt
Example: SELECT column1,
LEVENSHTEIN(column1, 'matchme') AS
perfectmatch FROM sometable ORDER BY
perfectmatch DESC

Php/ MySql 'Advanced Search' Page

I'm working on an 'advanced search' page on a site where you would enter a keyword such as 'I like apples' and it can search the database using the following options:
Find : With all the words, With the
exact phrase , With at least one of
the words, Without the words
I can take care of the 'Exact phrase' by:
SELECT * FROM myTable WHERE field='$keyword';
'At least one of the words' by:
SELECT * FROM myTable WHERE field LIKE '%$keyword%';//Let me know if this is the wrong approach
But its the 'With at least one of the words' and 'Without the words' that I'm stuck on.
Any suggestions on how to implement these two?
Edit: Regarding 'At least one word' it wouldn't be a good approach to use explode() to break the keywords into words, and run a loop to add
(field='$keywords') OR ($field='$keywords) (OR)....
Because there are some other AND/OR clauses in the query also and I'm not aware of the maximum number of clauses there can be.
I would suggest the use of MySQL FullText Search using this with the Boolean Full-Text Searches functionality you should be able to get your desired result.
Edit:
Requested example based on your requested conditions ("Its just one field and they can pick either of the 4 options (i.e 1 word, exact words, at least 1 word, without the term).")
I am assuming you are using php based on your initial post
<?php
$choice = $_POST['choice'];
$query = $_POST['query'];
if ($choice == "oneWord") {
//Not 100% sure what you mean by one word but this is the simplest form
//This assumes $query = a single word
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('{$query}' IN BOOLEAN MODE)");
} elseif ($choice == "exactWords") {
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('\"{$query}\"' IN BOOLEAN MODE)");
} elseif ($choice == "atLeastOneWord") {
//The default with no operators if given multiple words will return rows that contains at least one of the words
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('{$query}' IN BOOLEAN MODE)");
} elseif ($choice == "withoutTheTerm") {
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('-{$query}' IN BOOLEAN MODE)");
}
?>
hope this helps for full use of the operators in boolean matches see Boolean Full-Text Searches
You could use
With at least one of the words
SELECT * FROM myTable WHERE field LIKE '%$keyword%'
or field LIKE '%$keyword2%'
or field LIKE '%$keyword3%';
Without the word
SELECT * FROM myTable WHERE field NOT LIKE '%$keyword%';
I'm not sure you could easily do those search options in a naive manner as the other two.
It would be worth your while implementing a better search engine if you need to support those scenarios. A simple one that could probably get you by is something along these lines:
When an item is added to the database, it is split up into the individual words. At this point "common" words (the, a, etc...) are removed (probably based on a common_words table). The remaining words are added to a words table if they are not already present. There is then a link made between the word entry and the item entry.
When searching, it is then a case of getting the word ids from the word table and the appropriate lookup of item ids in the joining table.
Search is notoriously difficult to do well.
You should Consider using a third party search engine using something like Lucene or Sphider.
Giraffe and Re0sless pooseted 2 good answers.
notes:
"SELECT * " sucks... only select the columns that you need.
Re0sless puts a "OR" between keywords.
- you should eliminate common words (" ","i","am","and"..etc)
- mysql has a 8kb i belive limit on the size of the query, so for really long SELECTS you should slipt it into separate queries.
- try to eliminate duplicate keywords (if i search for "you know you like it" the SELECT should basically only search for "you" once and elimnate common words as "it")
Also try to use "LIKE" and "MATCH LIKE" (see mysql man page) it could do wonders for "fuzzy" searches

Categories