MySQL - Full Text Search - Losing index - php

So here is the scenario:
MySQL
have 1 MYISAM Table
colum named v.value has a full text index
Basic query works fine, uses the index as expected:
SELECT p.online_identifier
FROM (...)
WHERE r.area_id = 3 AND s.state_id= 4
AND (snap.area_has_catalogues_attributes_id = 7028
AND MATCH (v.value) AGAINST('+SomeBrand' IN BOOLEAN MODE))
Now when I add an OR, the full text search index (on v.value) is not used.
I run Explain to verify it.
The query would look something like this:
(...)
WHERE r.area_id = 3 AND s.state_id= 4 AND
(snap.area_has_catalogues_attributes_id = 7028 AND MATCH (v.value) AGAINST('+SomeBrand' IN BOOLEAN MODE))
OR (snap.area_has_catalogues_attributes_id = 7045 AND MATCH (v.value) AGAINST('+OtherBrand' IN BOOLEAN MODE))
I dont understand why.
Any ideas?

Here is something interesting: The query you have is actually causing the FULLTEXT indexing to be ignored. Don't worry, it is not your fault. I wrote about this before : Is there a way to hint to query optimizer to MySQL which constraints should be done first?
After looking over my answer and its subsequent links, now let's look at your original query:
SELECT p.online_identifier
FROM (...)
WHERE r.area_id = 3 AND s.state_id= 4
AND (snap.area_has_catalogues_attributes_id = 7028
AND MATCH (v.value) AGAINST('+SomeBrand' IN BOOLEAN MODE))
You will have to execute the FULLTEXT search alone and retrieve IDs only.
Use those IDs to join to the other table aliases.
Let me take a query from one of my other links to demonstrate what to do to your query
Instead of this query
select * from seeds WHERE MATCH(text) AGAINST
("mount cameroon" IN BOOLEAN MODE) = 4;
you must refactor it into something like this:
SELECT B.* FROM
(
SELECT id,MATCH(text) AGAINST
("mount cameroon" IN BOOLEAN MODE) score
FROM seeds
WHERE MATCH(text) AGAINST
("mount cameroon" IN BOOLEAN MODE)
) A
INNER JOIN seeds B USING (id)
WHERE A.score = 4;
Give it a Try !!!

Related

#variable mysql return null value on php mysql execution

I have this mysql need to run in php:
$sql_subject_summary = "SELECT
c.subject_code_id, c.subject_name, #total_target:= SUM(s.total_target_question) AS total_target_question,
#total_correct := ROUND((RAND() * (#total_target-10))+10) AS total_correct,
(#total_correct / #total_target)*100 AS percent, c.icon_filename
FROM
edu_subject_code c LEFT JOIN wkp_wg_student_subject s ON c.subject_code_id=s.subject_code_id
WHERE s.student_id=$student_id AND s.week_id = $current_week_id
$sql_inject
GROUP BY s.subject_code_id ";
However, the values of #total_correct, #total_target return null on php mysql execution .
When I run in mysql IDE, then the result is ok.
How to solve this problem?
That's right cause for that to work you need to use SELECT INTO... construct like select col1 into #arg1 from tbl1. Moreover since you are running the query from PHP why you need that at all? if you really need that then consider wrapping the query in a stored procedure and have those parameter as OUT parameter.
Well it's return null cause you are not selecting those parameter. After your query executes, you need to select those parameter saying select #total_correct, #total_target;.
Why don't you just run the query as is (like below) and fetch the specific columns value
SELECT
c.subject_code_id, c.subject_name, SUM(s.total_target_question) AS total_target_question,
ROUND((RAND() * (#total_target-10))+10) AS total_correct,
(#total_correct / #total_target)*100 AS percent, c.icon_filename
FROM
edu_subject_code c LEFT JOIN wkp_wg_student_subject s ON c.subject_code_id=s.subject_code_id
WHERE s.student_id=$student_id AND s.week_id = $current_week_id
$sql_inject
GROUP BY s.subject_code_id

Return the string equivalent of a SELECT that only returns one value

I have a SELECT query that only returns one value (not one row, but just one "thing"). It looks like
SELECT IF( EXISTS( SELECT * FROM `Blah` WHERE <whatever> ), 1, 0);
I tried fiddling around with mysqli_result::fetch_assoc, mysqli_result::fetch_object and mysqli_result::fetch_array, and all of them are being really weird. Is there a mysqli_result::fetch_string for special cases like this that just outputs a string on which I can just call intval?
You can do:
SELECT IF( EXISTS( SELECT * FROM `Blah` WHERE <whatever> ), 1, 0) AS exists;
And use any of the functions above to get row number 0's 'exists' column.
Aswell as casraf's answer, another option might be to incorporate a count into this - you might want to find out just how many blah's match your criteria later. Might be easier to read back when you re-visit your code in a years time, too.
SELECT count(*) cnt FROM `Blah` WHERE <whatever>
You'd be able to do a boolean check on the "cnt" variable (i.e. a count of 0 would == false) but also use it to find out how many matches you've got.

Regex Replace in MySql Select query

I have this SQL query:
SELECT * FROM pages WHERE page_title = 'paul_mccartney'
But often in page_title the value is like 'paul_mccartney (musician)'. And in this case I get NULL back. I had the idea to regex replace everything which is between ( and ) including the ():
SELECT * FROM pages WHERE REPLACE(REGEXP('/\([^\*].*\)/U'), '', page_title) = 'paul_mccartney'
But it doesn't work. Is my idea possible or not? And how?
Although your question is about using RegEx but have you looked into MySQL fulltext search functions?
https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
https://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html
Using the following tutorials if you have a row that is like this:
Id title
1 paul_mccartney (musician)
Using the example SQL provided will return a result:
SELECT * FROM articles
WHERE MATCH (title) AGAINST ('paul_mccartney' IN BOOLEAN MODE)
You could try a LIKE and see how that works:
SELECT * FROM pages WHERE page_title LIKE '%paul_mccartney%'

How to return multiple rows in a LEFT JOIN

I have a situation where lets say i'm trying to get the information about some food. Then I need to display all the information plus all the ingredients in that food.
With my query, i'm getting all the information in an array but only the first ingredient...
myFoodsArr =
[0]
foodDescription = "the description text will be here"
ratingAverage = 0
foodId = 4
ingredient = 1
ingAmount = 2
foodName = "Awesome Food name"
typeOfFood = 6
votes = 0
I would like to get something back like this...
myFoodsArr =
[0]
foodDescription = "the description text will be here"
ratingAverage = 0
foodId = 4
ingArr = {ingredient: 1, ingAmount: 4}, {ingredient: 3, ingAmount: 2}, {ingredient: 5, ingAmount: 1}
foodName = "Awesome Food name"
typeOfFood = 6
votes = 0
This is the query im working with right now. How can I adjust this to return the food ID 4 and then also get ALL the ingredients for that food? All while at the same time doing other things like getting the average rating of that food?
Thanks!
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood, c.ingredient, c.ingAmount, AVG(b.foodRating) AS ratingAverage, COUNT(b.foodId) as tvotes
FROM `foods` a
LEFT JOIN `foods_ratings` b
ON a.foodId = b.foodId
LEFT JOIN `foods_ing` c
ON a.foodId=c.foodId
WHERE a.foodId=4
EDIT:
Catcall introduced this concept of "sub queries" I never heard of, so I'm trying to make that work to see if i can do this in 1 query easily. But i just keep getting a return false. This is what I was trying with no luck..
//I changed some of the column names to help them be more distinct in this example
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood, AVG(b.foodRating) AS ratingAverage, COUNT(b.foodId) as tvotes
FROM foods a
LEFT JOIN foods_ratings b ON a.foodId = b.foodId
LEFT JOIN (SELECT fId, ingredientId, ingAmount
FROM foods_ing
WHERE fId = 4
GROUP BY fId) c ON a.foodId = c.fId
WHERE a.foodId = 4";
EDIT 1 more thing related to ROLANDS GROUP_CONCAT/JSON Idea as a solution 4 this
I'm trying to make sure the JSON string im sending back to my Flash project is ready to be properly parsed Invalid JSON parse input. keeps popping up..
so im thinking i need to properly have all the double quotes in the right places.
But in my MySQL query string, im trying to escape the double quotes, but then it makes my mySQL vars not work, for example...
If i do this..
GROUP_CONCAT('{\"ingredient\":', \"c.ingredient\", ',\"ingAmount\":', \"c.ingAmount\", '}')`
I get this...
{"ingredient":c.ingredient,"ingAmount":c.ingAmount},{"ingredient":c.ingredient,"ingAmount":c.ingAmount},{"ingredient":c.ingredient,"ingAmount":c.ingAmount}
How can i use all the double quotes to make the JSON properly formed without breaking the mysql?
This should do the trick:
SELECT food_ingredients.foodId
, food_ingredients.foodName
, food_ingredients.foodDescription
, food_ingredients.typeOfFood
, food_ingredients.ingredients
, AVG(food_ratings.food_rating) food_rating
, COUNT(food_ratings.foodId) number_of_votes
FROM (
SELECT a.foodId
, a.foodName
, a.foodDescription
, a.typeOfFood
, GROUP_CONCAT(
'{ingredient:', c.ingredient,
, ',ingAmount:', c.ingAmount, '}'
) ingredients
FROM foods a
LEFT JOIN foods_ing c
ON a.foodsId = c.foodsId
WHERE a.foodsId=4
GROUP BY a.foodId
) food_ingredients
LEFT JOIN food_ratings
ON food_ingredients.foodId = food_ratings.foodId
GROUP BY food_ingredients.foodId
Note that the type of query you want to do is not trivial in any SQL-based database.
The main problem is that you have one master (food) with two details (ingredients and ratings). Because those details are not related to each other (other than to the master) they form a cartesian product with each other (bound only by their relationship to the master).
The query above solves that by doing it in 2 steps: first, join to the first detail (ingredients) and aggregate the detail (using group_concat to make one single row of all related ingredient rows), then join that result to the second detail (ratings) and aggregate again.
In the example above, the ingredients are returned in a structured string, exactly like it appeared in your example. If you want to access the data inside PHP, you might consider adding a bit more syntax to make it a valid JSON string so you can decode it into an array using the php function json_decode(): http://www.php.net/manual/en/function.json-decode.php
To do that, simply change the line to:
CONCAT(
'['
, GROUP_CONCAT(
'{"ingredient":', c.ingredient
, ',"ingAmount":', c.ingAmount, '}'
)
, ']'
)
(this assumes ingredient and ingAmount are numeric; if they are strings, you should double quote them, and escape any double quotes that appear within the string values)
The concatenation of ingredients with GROUP_CONCAT can lead to problems if you keep a default setting for the group_concat_max_len server variable. A trivial way to mitigate that problem is to set it to the maximum theoretical size of any result:
SET group_concat_max_len = ##max_allowed_packet;
You can either execute this once after you open the connection to mysql, and it will then be in effect for the duration of that session. Alternatively, if you have the super privilege, you can change the value across the board for the entire MySQL instance:
SET GLOBAL group_concat_max_len = ##max_allowed_packet;
You can also add a line to your my.cnf or my.ini to set group_concat_max_lenght to some arbitrary large enough static value. See http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_group_concat_max_len
One obvious solution is to actually perform two queries:
1) get the food
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood
FROM `foods` a
WHERE a.foodsId=4
2) get all of its ingredients
SELECT c.ingredient, c.ingAmount
FROM `foods_ing` c
WHERE c.foodsId=4
This approach has the advantage that you don't duplicate data from the "foods" table into the result. The disadvantage is that you have to perform two queries. Actually you have to perform one extra query for each "food", so if you want to have a listing of foods with all their ingredients, you would have to do a query for each of the food record.
Other solutions usually have many disadvantages, one of them is using GROUP_CONCAT function, but it has a tough limit on the length of the returned string.
When you compare MySQL's aggregate functions and GROUP BY behavior to SQL standards, you have to conclude that they're simply broken. You can do what you want in a single query, but instead of joining directly to the table of ratings, you need to join on a query that returns the results of the aggregate functions. Something along these lines should work.
select a.foodId, a.foodName, a.foodDescription, a.typeOfFood,
c.ingredient, c.ingAmount,
b.numRatings, b.avgRating
from foods a
left join (select foodId, count(foodId) numRatings, avg(foodRating) avgRating
from foods_ratings
group by foodId) b on a.foodId = b.foodId
left join foods_ing c on a.foodId = c.foodId
order by a.foodId

MySQL Full-text search - search for short words

I have a problem. I made a simple search engine which searches by brand and model of car. For reasons of query performance and a lot of data in database, I decided to use full-text search. It's ok, but now I come across the problem:
I would like to find all cars with brand "Audi" and with model "Q7". For now, I have this SQL query, but it doesn't work right, because of word length "Q7":
SELECT `a`.`id`, `a`.`title`, `a`.`askprice`, `a`.`description`, `a`.`picture`
FROM (`mm_ads` as a)
WHERE `a`.`category` = '227'
AND `a`.`askprice` >= '0'
AND `a`.`askprice` <= '144000'
AND (MATCH(a.title) AGAINST ('+audi +q7' IN BOOLEAN MODE ))
GROUP BY `a`.`id`
ORDER BY `a`.`id` ASC
LIMIT 30
I don't have access to modify MySQL config file, to set ft_min_word_len to value 2. For now value is 3. Is there any other way to deal with that?
Here is another problem:
I would like to get all cars brand "BMW" and model "116". For example, I have a car named BMW, 1, 116i. My SQL query is:
`SELECT `a`.`id`, `a`.`title`, `a`.`askprice`, `a`.`description`, `a`.`picture`
FROM (`mm_ads` as a)
WHERE `a`.`category` = '227'
AND `a`.`askprice` >= '0'
AND `a`.`askprice` <= '144000'
AND (MATCH(a.title) AGAINST ('+bmw +116' IN BOOLEAN MODE))
GROUP BY `a`.`id`
ORDER BY `a`.`id` ASC
LIMIT 30`
Search return 0 rows. Why? All input strings ("BMW", "116") are min length 3. What am I doing wrong?
Regards, Mario
I had a similar issue when dealing with match against (regarding text length) and my answer was to strlen the string first and switch between like and match against for shorter words. Not what I would call graceful, but it was all I could do since I too had no access to the config.
As for the second question, are you sure the default isn't 4? I recall I couldn't search on the term "art" in my case. 3 letters. Had to go with like on everything below 4 chars.
Unless you have access to the config file and can change it I fear there is very little to do.
A change to ft_min_word_len requires a server restart and a full rebuild of the full text index.
As found here
Try this:
for this search: "bmw 116i"
(MATCH(a.title) AGAINST ('+bmw +116i "bmw 116i"' IN BOOLEAN MODE ))
not the best solution but might help...

Categories