I would like to ask if anybody can help me with this query
SELECT count(MATCH(product_text) AGAINST('lorem*' in boolean mode)) AS score FROM table_products WHERE MATCH (text) AGAINST ('lorem*' in boolean mode) limit 0,50000
There is fulltext index on column text. A.m. query return sum of rows as score.
What I wont is to count fulltext search results. If the number of results(rows)
is higher than 50000, than count-sum 50000 will be returned, otherwise the exact count of
of results is returned.
Problem is that it is not fast on table width 1,5 million rows if user try to find for example word "lorem" and this word appears in table f.e. more than 500 000 x.
I tried also
SELECT id,name,product_text,price, MATCH(product_text) AGAINST('lorem*' in boolean mode) AS score FROM table_products WHERE MATCH(product_text) AGAINST('lorem*' in boolean mode) and show_product='1' limit 0,50000
... width php mysql_num_rows
Another problem is that in following query mysql use fulltext index only
and sort by another column or by score is than slow
SELECT id,name,product_text,price, MATCH(product_text) AGAINST('lorem*' in boolean mode) AS score FROM table_products WHERE MATCH(product_text) AGAINST('lorem*' in boolean mode) and show_product='1' order by score desc limit 0,50000
resp. (..order by name, order by price)
Is there another better and faster way?
Many thanks for any help.
Related
I've built an internal DB / Search Engine for art creatives. I'm trying to create a search criterion where you can query one column in the database and also search several columns in the database for a phrase search using FullText Search. The example of a search query might be: November {and} Black Friday. November would search for creatives matching the created_for column and the black friday would search headline, subheadline and additional_text columns with a fulltext search. Any ideas of how to accomplish this would be really helpful!
SELECT
(SELECT * FROM headlines WHERE created_for = '$searchString' AND image_slug <> '')
(SELECT *, MATCH(headline) AGAINST('$fullText' IN BOOLEAN MODE) AS MultiScore, MATCH(subheadline, additional_text) AGAINST('$fullText' IN BOOLEAN MODE) AS MultiSecondScore
FROM `headlines`
WHERE MATCH(headline, subheadline, additional_text) AGAINST('$fullText' IN BOOLEAN MODE))
I've tried adding a UNION statement before the second Select statement, but I get an error message saying the columns don't match. Not sure what I've got wrong here, but thanks in advance for your help!
Use AND in the WHERE clause.
SELECT *, MATCH(headline) AGAINST('$fullText' IN BOOLEAN MODE) AS MultiScore, MATCH(subheadline, additional_text) AGAINST('$fullText' IN BOOLEAN MODE) AS MultiSecondScore
FROM headlines
WHERE created_for = '$searchString'
AND image_slug <> ''
AND MATCH(headline, subheadline, additional_text) AGAINST('$fullText' IN BOOLEAN MODE)
UNION would get results that match either of the criteria, not both of them. And when you use UNION, both subqueries have to return the same number of columns -- you would have to add extra columns to the first query to match the MultiScore and MultiSecondScore columns of the first query.
I have a table with 400,000 rows. I am using fulltext search on my website. When the table has 250,000 rows or less, searches take a few seconds to load, however when increasing this to 400,000 it is taking 35 seconds to search for a product.
Here is an example query I use for searching:
SELECT *,
(CASE WHEN CHAR_LENGTH(index.brand_name) > 18
THEN CONCAT(SUBSTRING(index.brand_name, 1, 18), '...')
ELSE index.brand_name
END) AS brand_name
from `index`
where MATCH (`product_name`) AGAINST (:search IN BOOLEAN MODE) and `category_id` = '1' order by MATCH (`product_name`) AGAINST (:search2 IN BOOLEAN MODE) desc
I used the explain statement which showed:
id,select_type,table,type,possible_keys,key,key_len,ref,rows,Extra
1,SIMPLE,index,ALL,NULL,NULL,NULL,NULL,385624,"Using where; Using filesort"
I then added a fulltext index for the field product_name, and ran explain again:
id,select_type,table,type,possible_keys,key,key_len,ref,rows,Extra
1,SIMPLE,index,fulltext,ft_product_name,ft_product_name,0,NULL,1,"Using where; Using filesort"
However this has not made any improvement to the speed, can anyone help?
I'm getting a bit confused when trying to set up a search utilizing fulltext search in boolean mode. Here is the query I'm using:
$query = "SELECT *,
MATCH(title) AGAINST('$q' IN BOOLEAN MODE) AS score
FROM results
WHERE MATCH(title) AGAINST('$q' IN BOOLEAN MODE)
ORDER BY score DESC";
When I run a search for +divorce+refinance, the results returned are:
1) Divorce: Paying Off Spouse = Rate/Term Refinance
2) Divorce - What to Look Out For Regarding Divorced Borrowers
Am I right in thinking that the second result should not be appearing, as it does not have both words? If not, how can I create that functionality?
Maybe I am mistaken, but if you search this string +divorce+refinance you get a weird result. If you want to search both words, your should search for +divorce +refinance (with a space between).
I tested it and it returns only one row:
Divorce: Paying Off Spouse = Rate/Term Refinance
Your problem relates to the create a prioritized boolean query and for this type of query one has to go in depth of Boolean search and to now how the Boolean search is performed. In simple words let me explain you why the second number result of result is shown.
Once should first understand what does Boolean means in programming?
It means either condition is true or false i,e 0 to 1.
Now let me explain for the Boolean search is performed? You have given two words. Let us search the row by row in Boolean mode. Search engine start and searches the row by row now where ever the First word is found, it makes the record true and give score as 1 to the rows in which the first word is found and also prepare the numbers of words found in the row.
Now it moves the next word and do the same process gives the record True and makes a list of records wherever the word is found and also prepare the number of words found in the row.
Now there are two rows of results are available and they are clubbed and with the priority is given to the words with the maximum number of words and row here is the main problem lies.
Example
First >>> total nos. >> Second >> total nos. >>> Final >> row
Word >>> Results >> Word >>>> of words > > > Results >>no >>Answer
1 >>>>>>>> 2 >>>>>>>>1>>>>>>>>>1>>>>>>>>1.33>>>> 1 >>> 1.33
0 >>>>>>>> 0 >>>>>>>>2>>>>>>>>>2>>>>>>>>1.25>>>> 2 >>> 1.25
0 >>>>>>>> 0 >>>>>>>>1>>>>>>>>>0>>>>>>>>1.25>>>> 3 >>> 1
While clubbing two results lists when true added with false then result is true, as if you add 1 + 0 = 1 and the results are should with value more than 1. So, while scoring the relevancy to the words found it is always found that the search engine shows the results where it found any word.
Scoring relevancy queries are done in two types either ignore the scores which are equal to one and only do calculations on the records who's score is greater than 1. Second is to make such a query that it never shows the records equal to one. As in your case you can so the below things also to get the correct results for two words:
SELECT *, ( (1.3 * (MATCH(title) AGAINST ('+term +term2' IN BOOLEAN MODE))) + (0.6 * (MATCH(text) AGAINST ('+term +term2' IN BOOLEAN MODE))) ) AS score FROM results WHERE ( MATCH(title, text) AGAINST ('+term +term2' IN BOOLEAN MODE) ) HAVING relevance > 0 ORDER BY relevance DESC;
I know that using the word HAVING make the query little slow but there is no other solution available. Hope this solves your query.
I've got a car searching website and when I set up the search system it failed to give any dynamic searching. For instance, it's got 500 cars on their and ~5 are 'Toyota Ist'.
So when I search 'Ist' I get no results. (See query)
SELECT *,MATCH(aTitle, aDescribe, aCarModel, aCarWheels, aCarStereo, aCarIntTrim, aCarTrans, aCarDrive, aCarFuel, aCarPlate, aCarColour) AGAINST('toyota ist' IN BOOLEAN MODE) AS score FROM at_auction WHERE status = '1' AND aCarYear >= 1992 AND aCarYear <= 2012 AND startBid >= 0 AND startBid <= 20000 AND MATCH(aTitle, aDescribe, aCarModel, aCarWheels, aCarStereo, aCarIntTrim, aCarTrans, aCarDrive, aCarFuel, aCarPlate, aCarColour) AGAINST('toyota ist' IN BOOLEAN MODE) AND closeTime >= '201201060842' ORDER BY opt_feature DESC, score DESC, score DESC LIMIT 0,10
But if I search 'Toyota Ist', i'll get a whole lot of Toyota Results. And the Ist car isn't neccessarily high in the list.
So the problem underlying is how do I set it up so that if someone searches just one word, say the cars model, it'll return the row... and how it can return the row if they search with multiple words like 'toyota camry', although that actually seems to work.
The MATCH fields are all FULLTEXT and aCarModel etc store the cars model and they're usually just one word like 'Ist' or 'Camry'.
Thanks.
If over 50% of the cars are "Toyota", then "Toyota" will be ignored in the search. Also, the FULLTEXT doesn't index words under 4 letters.
50% limit and other tuning bits about FULLTEXT
http://dev.mysql.com/doc/refman/5.1/en/fulltext-fine-tuning.html
Minimum word length
http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_ft_min_word_len
I would like to tweak the results returned by this Fulltext-query:
$STH = $DBH->prepare('SELECT *,
MATCH (title,title_under,subject) AGAINST (:query) AS score
FROM articles
WHERE MATCH(title,title_under,subject) AGAINST(:query IN BOOLEAN MODE)
order by score desc');
Is there a way to return the score calculated by mysql so that I can run my own conditions for adding/subtracting points before parsing the results?
Yes, the "MATCH() AGAINST() AS score" in your SELECT-statement will already do just that, it'll return the score calculated by MySQL.
I notice you executing the FT search IN BOOLEAN MODE in the WHERE-clause, but not in the SELECT however.