I'm a developer/designer for a community driven website: http://www.thegamesdb.net
The problem we have is quite straight forward:
Pac-Man is a game on the site. A person should be able to search "pacman" or "pac man" and the "Pac-Man" result should be shown. Currently, this does not happen.
Search code snippet is below but full code can be seen at: http://code.google.com/p/thegamesdb/source/browse/trunk/tab_listseries.php
if ($function == 'Search')
{
$query = "SELECT g.*, p.name FROM games as g, platforms as p WHERE GameTitle LIKE '%$string%' and g.Platform = p.id";
if(!empty($sortBy))
{
$query .= " ORDER BY $sortBy, GameTitle ASC";
}
else
{
$query .= " ORDER BY GameTitle";
}
}
I'm not that familiar with coding search techniques, so any help would be appreciated... I've tried searching around on the net and all I've found is some pre-fabricated site search engines. We do not really want to go down this route... it's a little overkill for our needs.
Looking forward to some discussion,
Alex
MySQL has a string comparison feature called SOUNDS LIKE. This may be a good use case for it.
http://dev.mysql.com/doc/refman/5.1/en/string-functions.html#operator_sounds-like
You would probably modify like so:
SELECT * FROM blah WHERE SOUNDEX(column) LIKE CONCAT('%', SOUNDEX($search_string), '%')
ultimately, you will need to decide how to perform a fuzzy match.
for your example - you might consider removing all whitespace, and any '-' character - then perform the like '%$string%' match. you might also consider UPPER on both sides of that LIKE.
you can do more or less in your own way of determining fuzziness.
Related
I have query I have came up with for a search bar. I know I can use OR to split the statement to check different columns:
$query = mysql_query('SELECT * FROM PRODUCTS WHERE cats LIKE "%'.$s.'%" OR sub_cats LIKE "%'.$s.'%"');
But if I want to check more than 2 or three columns is the re a way doing something like this to speed things up a bit:
$query_string = explode('&&=',$_GET['q']);
$db_str = 'SELECT * FROM products WHERE name, cats, sub_cats, desc, brand ';
for($x = 0; $x < count($query_string); $x++){
$enc = mysql_real_escape_string($query_string[$x]);
if($x == (count($query_string) -1)){
$db_str .= 'LIKE "%'.$enc.'%"';
}else{
$db_str .= 'LIKE "%'.$enc.'%" ';
}
}
$query = mysql_query($db_str);
I normally used POST for search bars but I fancy giving GET a go it looks more user friendly to me.
I don't know if this would speed things up (you would have to test on your data). However, the following is an alternative with only one comparison:
WHERE concat_ws('|', name, cats, sub_cats, desc, brand) LIKE '%$enc%'
Because your like has a wildcard at the beginning, the query does not use regular indexes. So this version should not slow things down much, if at all.
The answer to your performance problem, though, may be a full text index.
Your code will not work like that, even ignoring the deprecated API. You would need to do something like this:
$searchFields = ['name', 'cats', 'sub_cats', 'desc', 'brand'];
$searchTerm = mysql_real_escape_string($query_string[$x]);
$filters = implode(' OR ', array_map(function($field) use ($searchTerm) {
return "$field LIKE '%{$searchTerm}%'";
}, $searchFields));
$query = mysql_query('SELECT * FROM products WHERE '.$filters);
It's far from optimal though, and will quickly become slow as database and userbase grow. It is however exactly how for example phpMyAdmin implements its database-wide searches. If your database is larger than a few records, or is likely to attract more than a handful of searches simultaneously, you should implement a smarter solution like full text search or implement an external search engine like Apache SOLR, Amazon CloudSearch and the like - relational databases like MySQL on their own just aren't good at messy searching nor were they ever meant to be.
As for this remark:
I normally used POST for search bars but I fancy giving GET a go it
looks more user friendly to me.
This isn't really the good argument to choose. As a rule:
GET is for repeatable requests, that the user can F5 on as much as they want.
POST is for non-repeatable requests, that are usually considered harmful if (overly) repeated.
If your name is Google and you have a gazillion servers all over the world, use GET. If you already know your search has bad performance, stick with POST and don't tempt the lions.
I'm trying to search a table for specific words.
Say I have a list of words: printer,network,wireless,urgent
I only want to return those rows where all of these words are in it.
SELECT * FROM tickets WHERE concat(subject,body) REGEXP "printer|network|wireless|urgent"
will return any row with any one of these words. How can I make it so that it will only return those rows where all of these words are in it.
Thanks,
There are two ways to do this. The first is the rather obvious approach. Let's say you have all the words that need to appear in an array called $necessaryWords:
$sql = 'SELECT ... FROM ...'; // and so on
$sql .= ' WHERE 1';
foreach ($necessaryWords as $word)
$sql .= ' AND concat(subject,body) LIKE "%' . $word . '%"'; //Quotes around string
However, using %foo% is rather slow, as no indexes can be used, so this query might cause performance issues with huge tables and/or a high number of necessary words.
The other approach would be a FULLTEXT index on subject and body. You could the use the fulltext MATCH IN BOOLEAN MODE like this:
$sql = 'SELECT ... FROM ...'; // and so on
$sql .= ' WHERE MATCH(subject,body) AGAINST("';
foreach ($necessaryWords as $word)
$sql .= ' +' . $word;
$sql .= '")';
Note that your table must use MyISAM in order to use FULLTEXT indexes. UPDATE: As of MySQL 5.6, InnoDB supports FULLTEXT indexes as well. I guess this could be the better choice performance wise. Further documentation on the fulltext in boolean mode can be found in the manual.
not the best way, but:
SELECT * FROM tickets WHERE
concat(subject,body) REGEXP "printer" AND
concat(subject,body) REGEXP "network" AND
concat(subject,body) REGEXP "wireless" AND
concat(subject,body) REGEXP "urgent"
SELECT * FROM tickets WHERE
concat(subject,body) LIKE "%printer%" AND
concat(subject,body) LIKE "%network%" AND
concat(subject,body) LIKE "%wireless%" AND
concat(subject,body) LIKE "%urgent%"
Not sure this would work with MySQL regex engine but the regex (using lookarounds) below can achieve what you are looking for. Will find the words of interest irrespective of the order in which they occur:
^(?=.*printer)(?=.*network)(?=.*wireless)(?=.*urgent).*$
Demo: http://www.rubular.com/r/XcVz5xMZcb
Some regex lookaround examples here: http://www.rexegg.com/regex-lookarounds.html
Alternative answer, just because I thought of it when I looked at your question. I do not know whether it would be faster than the other answers (most likely no):
(SELECT * FROM tickets WHERE subject LIKE "%printer%" OR body LIKE "%printer%")
UNION
(SELECT * FROM tickets WHERE subject LIKE "%network%" OR body LIKE "%network%")
UNION
(SELECT * FROM tickets WHERE subject LIKE "%wireless%" OR body LIKE "%wireless%")
UNION
(SELECT * FROM tickets WHERE subject LIKE "%urgent%" OR body LIKE "%urgent%")
UPDATE:
This is wrong
I'm building a website that has records that need to be searched through. I just realized that the search function is too precise. It's a recipe website, so for example, if the user types "keylime pie", the recipe named "Key Lime Pie" won't show up in the results. I'm not sure if there's a script that I can get for this, but I'd really appreciate some help.
Here's my current query:
SELECT * FROM `recipes` WHERE
`recipe_title` LIKE '%$search%' // Key Lime Pie
OR `recipe_summary` LIKE '%$search%' // I love key lime pie
OR `recipe_categories` LIKE '%$search%' //desserts, pies
//... etc
Thanks!
Like queries will soon take down your MySQL database.
If it's not too complicated for you, use Sphinx for searching on mysql it will give you nice results based on keyword density and keyword weight etc. And it's really really fast.
Your best bet may be to look into full text searching. MySQL only supports full text search in MyISAM by default, but if you're running MySQL 5.6 or later, you can do it in InnoDB, at well.
Alternatively, you can run dedicated full text search tools such as Lucene or Sphinx. These are more sophisticated tools that include things like relevance ranking, and may even be able to handle spelling differences/errors (depending on the tool).
Not sure about those double quotes, try this:
SELECT * FROM `recipes` WHERE (`recipe_title` LIKE '%$search%'
OR `recipe_summary` LIKE '%$search%'
OR `recipe_categories` LIKE '%$search%')
I have found it useful to add a related table where you add all the keywords pertaining to a specific entry.
So if the name of your recipe is "Key Lime Pie" you could have one of the keywords be "KeyLime Pie". That allows for typos by the user or allows you to add obvious related terms that people might look for.
This very simple problem could also be solved by adding a column in your recipes table where you store the name (or other short fields) without spaces (and maybe without punctuation). So for instance the name "Key Lime Pie" would be also stored as "KeyLimePie".
Now you can find the recipe by searching for the name "keylime pie": first remove the spaces (and maybe also the punctuation) from the search term. "keylime pie" becomes "keylimepie". That way your query will return the "Key Lime Pie" entry.
You can also expand your search by adding partial LIKE specifically for the beginning or the end of the searched term.
Assuming you store the title without spaces in a column called recipe_title_no_space
"SELECT * FROM recipes
WHERE
recipe_title_no_space LIKE '%".$search."'
OR
recipe_title_no_space LIKE '".$search."%'"
That way you will fin a match for "keylime" and also for "pie keylime". Be careful that this does not return too many/poor results in some cases.
There are a few ways of customising search results as stated above, but if you wanted to experiment...
$searchArray = explode(' ', $search);
$idArray = array();
for($i=0; $i<count($searchArray) $i++) {
$query = "SELECT * FROM table WHERE recipe_title LIKE '%$search%'
AND recipe_summary LIKE '%$search%' AND recipe_categories LIKE '%$search%'";
$result = mysqli_query($link, $query);
$row = mysqli_fetch_array($result);
$idArray = array_merge($idArray, $row['id']);
}
$mostMatches = array_count_values($idArray);
foreach($mostMatches as $key => $value) {
if($value > 1) {$popularResultId[] = $key;}
}
for($i=0; $i<count($popularResultId)$i++) {
mysqli_data_seek($result, $popularResultId[$i]);
$row = mysqli_fetch_array($result);
echo "Title: {$row['recipe_title']}\n\r";
echo "Summary: {$row['recipe_summary']}\r\n";
echo "category: {$row['recipe_categories']}\r\n";
}
If my theory is correct you should be able to tune the results by changing the "if($value > 1)" up or down, you could even have it auto adjust depending on how big your database got if you wanted.
EDIT somehoe the acceptabe answer was DELETED :(
The solution to the problem was to use a function like this:
ifnull(trees.primarycolor, '') LIKE %,
Start Question:
Hello Friends of the internet, Family , Colleagues and random strangers. I have a predicament that is sure to excite the webmasters out there.
My query is as follows:
$result = mysql_query("SELECT trees.*
$bottom_query
FROM trees
INNER JOIN price
ON trees.ID=price.treeid
where trees.primarycolor like
'$primary_color'
and trees.secondarycolor like
'$secondary_color'
and trees.fallcolor like
'$fall_color'
and trees.droughtresistant like
'$drought_resistant'
and trees.height like
'$height'
and trees.spread like
'$spread'
and trees.trunkcolor like
'$trunk_color'
and trees.flowering like
'$flowering'
and trees.fruitproducing like
'$fruit_producing'
");
Where the $var_name are either a string such as red or green or true or false OR the wildcard %.
The actual query printed out is as follows:
SELECT trees.* , price.10mm , price.20mm FROM trees INNER JOIN price ON
trees.ID=price.treeid where trees.primarycolor like '%' and trees.secondarycolor like '%'
and trees.fallcolor like '%' and trees.droughtresistant like '%' and trees.height like
'%' and trees.spread like '%' and trees.trunkcolor like '%' and trees.flowering like '%'
and trees.fruitproducing like '%'
My issue is that even with all the wildcards in the WHERE CLAUSE, the query returns no results.
I am using PHP and HTML - not sure what versions, most likely the newest possible.
I want to use the % as the * to select all that are of any type in the database.
Please help me Good people of the Internet, and glory shall be yours :D
If no filters are chosen, don't use a WHERE clause at all. This will run faster and be more efficient on the server, which you want anyways. Keep all your queries as simple as possible to avoid load where not needed. Only add on the WHERE clause to your query if a choice is made. Don't use LIKE clauses that still return all the records. Building your WHERE clause dynamically based on the inputs is the way to go. Little more work, but way better in the long run.
Nick - http://www.meltedjoystick.com
I have two questions regarding my script and searching. I have this script:
$searchTerms = explode(' ', $varSearch);
$searchTermBits = array();
foreach($searchTerms as $term){
$term = trim($term);
if(!empty($term)){
$searchTermBits[] = "column1 LIKE '%".$term."%'";
}
}
$sql = mysql_query("SELECT * FROM table WHERE ".implode(' OR ', $searchTermBits)."");
I have a column1 with a data name "rock cheer climbing here"
If I type in "rock climb" this data shows. Thats perfect, but if I just type "Rocks", it doesn't show. Why is that?
Also, How would I add another "column2" for the keyword to search into?
Thank you!
Searching that string for "rocks" doesn't work, because the string "rocks" doesn't exist in the data. Looking at it, it makes sense to you, because you know that the plural of "rock" is "rocks", but the database doesn't know that.
One option you could try is removing the S from search terms, but you run into other issues with that - for example, the plural of "berry" is "berries", and if you remove the S, you'll be searching for "berrie" which doesn't get you any further.
You can add more search terms by adding more lines like
$searchTermBits[] = "column1 LIKE '%".$term."%'";
and replacing ".$term." with what you want to search for. For example,
$searchTermBits[] = "column1 LIKE '%climb%'";
One other thing to note... as written, your code is susceptible to SQL injection. Take this for example... What if the site visitor types in the search term '; DROP TABLE tablename; You've just had your data wiped out.
What you should do is modify your searchTermBits[] line to look like:
$searchTermBits[] = "column1 LIKE '%" . mysql_real_escape_string($term) . "%'";
That will prevent any nastiness from harming your data.
Assuming the data you gave is accurate, it shouldn't match because you're using "Rocks" and the word in the string is "rock". By default mysql doesn't do case sensitive matching, so it's probably not the case.
Also, to avoid sql injection, you absolutely should be using mysql_real_escape_string to escape your content.
Adding a second column would be pretty easy as well. Just add two entries to your array for every search term, one for column1 and one for column2.
Your column1 data rock cheer climbing here your search criteria %Rocks% it doesn't fit at all as rocks is not in your column1 data
you can add column2 as you do for column1 then put it all together by using an AND operator (column1 LIKE "%rock%" OR column1 LIKE "%climb%") AND (column2 LIKE "%rope%" OR column2 LIKE "%powder%")
TIPS:
If your table/schema are using xx_xx_ci collation (then this is mean case insensitive,mysql doesn't care case sensitive) but if other then you need to make sure that the search term must be case sensitive(mysql do case sensitive).