This is my code:
$sql = "SELECT *,MATCH (CUIT,DENOMINACION) AGAINST ('%{$word}%') AS Score FROM cuits WHERE";
$sql_end = '';
foreach($words as $word) {
$sql_end .= " AND MATCH (CUIT,DENOMINACION) AGAINST ('%{$word}%' IN BOOLEAN MODE) AND BAJA=0 ";
$sql_end2 = "order by DENOMINACION limit $inicio, $TAMANO_PAGINA";
}
It is very slow. How can I improve the speed?
It doesn't make sense in FULLTEXT search to use LIKE-style wildcard strings %. So, change AGAINST ('%{$word}%') to AGAINST ('{$word}') and see if it helps.
Also, FULLTEXT can match multiple words at once, so there's no need to repeat the whole MATCH clause for each word. Instead try
SELECT *,MATCH (CUIT,DENOMINACION) AGAINST ('word word word word') ...
You may, if the series of words is a phrase, wish to stop using the boolean FULLTEXT mode and use the natural language mode as shown in your first MATCH clause.
Be aware that FULLTEXT works very strangely indeed on small tables, with less than a few hundred rows. That makes testing on small tables difficult.
Finally, make sure your FULLTEXT index is constructed correctly.
Related
I am using MySql FullText indexing to search data from database.
Here is the query
$search_input_text = 'the_string_to_be_search';
$searchArray = explode(" ", $search_input_text);
$query="SELECT * FROM car_details
WHERE MATCH (car_trim) AGAINST ('";
foreach ($searchArray as $word) {
$query .= "+".$word."* ";
}
$query .= "' IN BOOLEAN MODE) LIMIT $start, $limit";
The query is executing fine but it has a bug, if you look at the column name you will find car_trim which is inside the MATCH() function. The column has only 3 different types of values in the database which are 'T5', 'T6' and 'T5 premier'.
When I type 'Premier' in the search bar and hit Enter, it fetches the results whose values contain the word 'Premier'. But when I type T5 or T6 , it returns an empty record. Please be sure that there are lots of records with car_trim='T5', car_trim='T6' or car_trim='T5 Premier'
I am not getting that what can be the problem with the strings T5 and T6.
MySQL has two key parameters when using full text search (and a few other important ones). The key parameters are the minimum word length and the stop words list. In short, MySQL ignores words that are less than 3 or 4 characters (depending on the storage engine) or that are in the stop word list.
Your examples ("T5" and "T6") are too short -- based on the parameter defaults.
Some other configuration parameters might be of interest, such as the maximum word length and the characters that are valid for words.
You can change the parameters for full text indexing and re-build the index.
Here is a good place to start in understanding this.
I am running MySQL version 5.1.57
I have a HTML form where a user can insert a search-string. I create a $_SESSION on this string, then I run it in a MySQLquery. Something like this:
<?php
$sql = mysql_query ("SELECT
s.student_firstname, s.student_lastname
FROM students s
WHERE (
s.student_firstname LIKE '%$searchstring%' OR
s.student_lastname LIKE '%$searchstring%
)
AND s.isActive = '1' ");
?>
The problem is when a user is searching for multiple words. Then my query fails because it is trying to match the string against the values in either column.
I've read something about MySQL FULLTEXT indexing but as far as I understand, it only works on MyISAM tables(?). How can I be able to search for multiple words using the environment that I have?
I think you should split your searched string on space (" ") and insert each segment in your query, or in another query. For example :
$str = "word1 word2";
With that you search first for the whole string "word1 word2" and after you search in you database for "word1" and "word2".
With this solution you should handle a word ignore list, because words like "a, an, the, or, ..." shouldn't be seek ...
I'm not sure there is an other way with an innoDB table ... The best solution is obviously to use the "match against" command, but it's only available with a full text index under MyISAM.
How I do exact match or anywhere match keywords on the table with php/mysql search engine.
For example: i added a url with keywords: UFC 131 online
And 2nd url with keywords: UFC 132 online
and if i enter UFC 132 on the search box how do i make the 2nd url to come up first?
Because if i use this code
//explode out search term
$search_exploded = explode(" ",$search);
foreach($search_exploded as $search_each)
{
//construct query
$x++;
if ($x==1)
$construct .= "title LIKE '%$search_each%'";
else
$construct .= " or title = '$search'";
and i search for ufc 132 online it gives me ufc 131 online result first.
You could implement something like this:
SELECT *, MATCH (title) AGAINST ('searchstring' IN BOOLEAN MODE) +
CASE
WHEN title = 'searchstring' THEN 10
ELSE 0
END AS relevance FROM `wrestlingsucks`
HAVING relevance > 1.0
ORDER BY relevance DESC LIMIT 100
This is a very basic example, all it will do is add 10 to a sudo relevance column if its an exact match. You could however get as complicated as you want with full text and natural matching.
MySQL has superior search capabilities for full text search than handcrafting your searches. Take a look at fulltext-search
From the documentation MATCH (col1,col2,...) AGAINST (expr [search_modifier])
Use Levenshtein algorithm in MySQL.
Levenshtein matching is a metric for
measuring the amount of difference
between two sequence, here it is
strings. By default MySQL does not
have this function, but you can write
and add one.
Please take a look at the code here and add that code as a system function in MySQL, please see the example below on how to get the similarity of two strings.
Please see: https://github.com/rakesh-sankar/Tools/blob/master/MySQL/Levenshtein.txt
Example: SELECT column1,
LEVENSHTEIN(column1, 'matchme') AS
perfectmatch FROM sometable ORDER BY
perfectmatch DESC
I'm trying to create a small search for my site. I've tried using full-text index search, but I could never get it to work. Here is what I've come up with:
if(isset($_GET['search'])) {
$search = str_replace('-', ' ', $_GET['search']);
$result = array();
$titles = mysql_query("SELECT title FROM Entries WHERE title LIKE '%$search%'");
while($row = mysql_fetch_assoc($titles)) {
$result[] = $row['title'];
}
$tags = mysql_query("SELECT title FROM Entries WHERE tags LIKE '%$search%'");
while($row = mysql_fetch_assoc($tags)) {
$result[] = $row['title'];
}
$text = mysql_query("SELECT title FROM Entries WHERE entry LIKE '%$search%'");
while($row = mysql_fetch_assoc($text)) {
$result[] = $row['title'];
}
$result = array_unique($result);
}
So basically, it searches through all the titles, body-text, and tags of all the entries in the DB. This works decently well, but I'm just wondering how efficient would it be? This would only be for a small blog, too. Either way I'm just wondering if this could be made any more efficient.
There's no way to make LIKE '%pattern%' queries efficient. Once you get a nontrivial amount of data, using those wildcard queries performs hundreds or thousands of times slower than using a fulltext indexing solution.
You should look at the presentation I did for MySQL University:
http://www.slideshare.net/billkarwin/practical-full-text-search-with-my-sql
Here's how to get it to work:
First make sure your table uses the MyISAM storage engine. MySQL FULLTEXT indexes support only MyISAM tables. (edit 11/1/2012: MySQL 5.6 is introducing a FULLTEXT index type for InnoDB tables.)
ALTER TABLE Entries ENGINE=MyISAM;
Create a fulltext index.
CREATE FULLTEXT INDEX searchindex ON Entries(title, tags, entry);
Search it!
$search = mysql_real_escape_string($search);
$titles = mysql_query("SELECT title FROM Entries
WHERE MATCH(title, tags, entry) AGAINST('$search')");
while($row = mysql_fetch_assoc($titles)) {
$result[] = $row['title'];
}
Note that the columns you name in the MATCH clause must be the same columns in the same order as those you declared in the fulltext index definition. Otherwise it won't work.
I've tried using full-text index search, but I could never get it to work... I'm just wondering if this could be made any more efficient.
This is exactly like saying, "I couldn't figure out how to use this chainsaw, so I decided to cut down this redwood tree with a pocketknife. How can I make that work as well as the chainsaw?"
Regarding your comment about searching for words that match more than 50% of the rows.
The MySQL manual says this:
Users who need to bypass the 50% limitation can use the boolean search mode; see Section 11.8.2, “Boolean Full-Text Searches”.
And this:
The 50% threshold for natural language
searches is determined by the
particular weighting scheme chosen. To
disable it, look for the following
line in storage/myisam/ftdefs.h:
#define GWS_IN_USE GWS_PROB
Change that line to this:
#define GWS_IN_USE GWS_FREQ
Then recompile MySQL. There is no need
to rebuild the indexes in this case.
Also, you might be searching for stopwords. These are words that are ignored by the fulltext search because they're too common. Words like "the" and so on. See http://dev.mysql.com/doc/refman/5.1/en/fulltext-stopwords.html
Using LIKE is NOT fulltext.
You need to use ... WHERE MATCH(column) AGAINST('the query') in order to access a fulltext search.
MySQL Full-text search works -- I would look into it and debug it rather than trying to do this. Doing 3 separate MySQL queries will not be anywhere near as efficient.
If you want to try to make that much efficient you could separate the LIKE statements in one query with OR between them.
I'm working on an 'advanced search' page on a site where you would enter a keyword such as 'I like apples' and it can search the database using the following options:
Find : With all the words, With the
exact phrase , With at least one of
the words, Without the words
I can take care of the 'Exact phrase' by:
SELECT * FROM myTable WHERE field='$keyword';
'At least one of the words' by:
SELECT * FROM myTable WHERE field LIKE '%$keyword%';//Let me know if this is the wrong approach
But its the 'With at least one of the words' and 'Without the words' that I'm stuck on.
Any suggestions on how to implement these two?
Edit: Regarding 'At least one word' it wouldn't be a good approach to use explode() to break the keywords into words, and run a loop to add
(field='$keywords') OR ($field='$keywords) (OR)....
Because there are some other AND/OR clauses in the query also and I'm not aware of the maximum number of clauses there can be.
I would suggest the use of MySQL FullText Search using this with the Boolean Full-Text Searches functionality you should be able to get your desired result.
Edit:
Requested example based on your requested conditions ("Its just one field and they can pick either of the 4 options (i.e 1 word, exact words, at least 1 word, without the term).")
I am assuming you are using php based on your initial post
<?php
$choice = $_POST['choice'];
$query = $_POST['query'];
if ($choice == "oneWord") {
//Not 100% sure what you mean by one word but this is the simplest form
//This assumes $query = a single word
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('{$query}' IN BOOLEAN MODE)");
} elseif ($choice == "exactWords") {
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('\"{$query}\"' IN BOOLEAN MODE)");
} elseif ($choice == "atLeastOneWord") {
//The default with no operators if given multiple words will return rows that contains at least one of the words
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('{$query}' IN BOOLEAN MODE)");
} elseif ($choice == "withoutTheTerm") {
$result = mysql_query("SELECT * FROM table WHERE MATCH (field) AGAINST ('-{$query}' IN BOOLEAN MODE)");
}
?>
hope this helps for full use of the operators in boolean matches see Boolean Full-Text Searches
You could use
With at least one of the words
SELECT * FROM myTable WHERE field LIKE '%$keyword%'
or field LIKE '%$keyword2%'
or field LIKE '%$keyword3%';
Without the word
SELECT * FROM myTable WHERE field NOT LIKE '%$keyword%';
I'm not sure you could easily do those search options in a naive manner as the other two.
It would be worth your while implementing a better search engine if you need to support those scenarios. A simple one that could probably get you by is something along these lines:
When an item is added to the database, it is split up into the individual words. At this point "common" words (the, a, etc...) are removed (probably based on a common_words table). The remaining words are added to a words table if they are not already present. There is then a link made between the word entry and the item entry.
When searching, it is then a case of getting the word ids from the word table and the appropriate lookup of item ids in the joining table.
Search is notoriously difficult to do well.
You should Consider using a third party search engine using something like Lucene or Sphider.
Giraffe and Re0sless pooseted 2 good answers.
notes:
"SELECT * " sucks... only select the columns that you need.
Re0sless puts a "OR" between keywords.
- you should eliminate common words (" ","i","am","and"..etc)
- mysql has a 8kb i belive limit on the size of the query, so for really long SELECTS you should slipt it into separate queries.
- try to eliminate duplicate keywords (if i search for "you know you like it" the SELECT should basically only search for "you" once and elimnate common words as "it")
Also try to use "LIKE" and "MATCH LIKE" (see mysql man page) it could do wonders for "fuzzy" searches