Improving my PHP search of a MySQL database? - php

I'm building a website that has records that need to be searched through. I just realized that the search function is too precise. It's a recipe website, so for example, if the user types "keylime pie", the recipe named "Key Lime Pie" won't show up in the results. I'm not sure if there's a script that I can get for this, but I'd really appreciate some help.
Here's my current query:
SELECT * FROM `recipes` WHERE
`recipe_title` LIKE '%$search%' // Key Lime Pie
OR `recipe_summary` LIKE '%$search%' // I love key lime pie
OR `recipe_categories` LIKE '%$search%' //desserts, pies
//... etc
Thanks!

Like queries will soon take down your MySQL database.
If it's not too complicated for you, use Sphinx for searching on mysql it will give you nice results based on keyword density and keyword weight etc. And it's really really fast.

Your best bet may be to look into full text searching. MySQL only supports full text search in MyISAM by default, but if you're running MySQL 5.6 or later, you can do it in InnoDB, at well.
Alternatively, you can run dedicated full text search tools such as Lucene or Sphinx. These are more sophisticated tools that include things like relevance ranking, and may even be able to handle spelling differences/errors (depending on the tool).

Not sure about those double quotes, try this:
SELECT * FROM `recipes` WHERE (`recipe_title` LIKE '%$search%'
OR `recipe_summary` LIKE '%$search%'
OR `recipe_categories` LIKE '%$search%')

I have found it useful to add a related table where you add all the keywords pertaining to a specific entry.
So if the name of your recipe is "Key Lime Pie" you could have one of the keywords be "KeyLime Pie". That allows for typos by the user or allows you to add obvious related terms that people might look for.
This very simple problem could also be solved by adding a column in your recipes table where you store the name (or other short fields) without spaces (and maybe without punctuation). So for instance the name "Key Lime Pie" would be also stored as "KeyLimePie".
Now you can find the recipe by searching for the name "keylime pie": first remove the spaces (and maybe also the punctuation) from the search term. "keylime pie" becomes "keylimepie". That way your query will return the "Key Lime Pie" entry.
You can also expand your search by adding partial LIKE specifically for the beginning or the end of the searched term.
Assuming you store the title without spaces in a column called recipe_title_no_space
"SELECT * FROM recipes
WHERE
recipe_title_no_space LIKE '%".$search."'
OR
recipe_title_no_space LIKE '".$search."%'"
That way you will fin a match for "keylime" and also for "pie keylime". Be careful that this does not return too many/poor results in some cases.

There are a few ways of customising search results as stated above, but if you wanted to experiment...
$searchArray = explode(' ', $search);
$idArray = array();
for($i=0; $i<count($searchArray) $i++) {
$query = "SELECT * FROM table WHERE recipe_title LIKE '%$search%'
AND recipe_summary LIKE '%$search%' AND recipe_categories LIKE '%$search%'";
$result = mysqli_query($link, $query);
$row = mysqli_fetch_array($result);
$idArray = array_merge($idArray, $row['id']);
}
$mostMatches = array_count_values($idArray);
foreach($mostMatches as $key => $value) {
if($value > 1) {$popularResultId[] = $key;}
}
for($i=0; $i<count($popularResultId)$i++) {
mysqli_data_seek($result, $popularResultId[$i]);
$row = mysqli_fetch_array($result);
echo "Title: {$row['recipe_title']}\n\r";
echo "Summary: {$row['recipe_summary']}\r\n";
echo "category: {$row['recipe_categories']}\r\n";
}
If my theory is correct you should be able to tune the results by changing the "if($value > 1)" up or down, you could even have it auto adjust depending on how big your database got if you wanted.

Related

MYSQL PHP check a like statement on more one column at once

I have query I have came up with for a search bar. I know I can use OR to split the statement to check different columns:
$query = mysql_query('SELECT * FROM PRODUCTS WHERE cats LIKE "%'.$s.'%" OR sub_cats LIKE "%'.$s.'%"');
But if I want to check more than 2 or three columns is the re a way doing something like this to speed things up a bit:
$query_string = explode('&&=',$_GET['q']);
$db_str = 'SELECT * FROM products WHERE name, cats, sub_cats, desc, brand ';
for($x = 0; $x < count($query_string); $x++){
$enc = mysql_real_escape_string($query_string[$x]);
if($x == (count($query_string) -1)){
$db_str .= 'LIKE "%'.$enc.'%"';
}else{
$db_str .= 'LIKE "%'.$enc.'%" ';
}
}
$query = mysql_query($db_str);
I normally used POST for search bars but I fancy giving GET a go it looks more user friendly to me.
I don't know if this would speed things up (you would have to test on your data). However, the following is an alternative with only one comparison:
WHERE concat_ws('|', name, cats, sub_cats, desc, brand) LIKE '%$enc%'
Because your like has a wildcard at the beginning, the query does not use regular indexes. So this version should not slow things down much, if at all.
The answer to your performance problem, though, may be a full text index.
Your code will not work like that, even ignoring the deprecated API. You would need to do something like this:
$searchFields = ['name', 'cats', 'sub_cats', 'desc', 'brand'];
$searchTerm = mysql_real_escape_string($query_string[$x]);
$filters = implode(' OR ', array_map(function($field) use ($searchTerm) {
return "$field LIKE '%{$searchTerm}%'";
}, $searchFields));
$query = mysql_query('SELECT * FROM products WHERE '.$filters);
It's far from optimal though, and will quickly become slow as database and userbase grow. It is however exactly how for example phpMyAdmin implements its database-wide searches. If your database is larger than a few records, or is likely to attract more than a handful of searches simultaneously, you should implement a smarter solution like full text search or implement an external search engine like Apache SOLR, Amazon CloudSearch and the like - relational databases like MySQL on their own just aren't good at messy searching nor were they ever meant to be.
As for this remark:
I normally used POST for search bars but I fancy giving GET a go it
looks more user friendly to me.
This isn't really the good argument to choose. As a rule:
GET is for repeatable requests, that the user can F5 on as much as they want.
POST is for non-repeatable requests, that are usually considered harmful if (overly) repeated.
If your name is Google and you have a gazillion servers all over the world, use GET. If you already know your search has bad performance, stick with POST and don't tempt the lions.

How do I search Full text with partial matches?

I have a table, with not many rows, and neither many columns. I am doing a Full text search on 3 columns.
My code is
$search_input = trim($_GET['s']);
$search = mysql_real_escape_string($search_input);
$search = '+'.str_replace(' ', '* +', $search).'*';
$sql = "SELECT * FROM table WHERE
MATCH(def, pqr, xyz) AGAINST ('$search' IN BOOLEAN MODE)";
$result = mysql_query($sql);
I can correctly search for terms like abcdefgh, which are present as ... abcdefgh ....
But I am receiving empty set with search terms like abc, where in table entry is present something like abc-123, and also terms like abcdefghs. (notice this is plural of above)
Clearly I need to implement partial search, or something like that.
But how do I implement such a search? Any better way to do a entire table search on user input?
Do mention anything I am doing incorrectly.
EDIT : By adding * after each word, now I am able to also search for abcde, but above problems remains.
Do you mean you don't get results for 3 letter combinations? If so, you might be hitting the mysql index length (which is usually set to 3)
More info here - http://dev.mysql.com/doc/refman/5.1/en/fulltext-fine-tuning.html

how to implement the an effective search algorithm when using php and a mysql database?

I'm new to web design, especially backend design so I have a few questions about implementing a search function in PHP. I already set up a MySQL connection but I don't know how to access specific rows in the MySQL table. Also is the similar text function implemented correctly considering I want to return results that are nearly the same as the search term? Right now, I can only return results that are the exact same or it gives "no result." For example, if I search "tex" it would return results containing "text"? I realize that there are a lot of mistakes in my coding and logic, so please help if possible. Event is the name of the row I am trying to access.
$input = $_POST["searchevent"];
while ($events = mysql_fetch_row($Event)) {
$eventname = $events[1];
$eventid = $events[0];
$diff = similar_text($input, $event, $hold)
if ($hold == '100') {
echo $eventname;
break;
else
echo "no result";
}
Thank you.
I've noticed some of the comments mentioned more efficient ways of performing the search than with the "similar text" function, if I were to use the LIKE function, how would it be implemented?
A couple of different ways of doing this:
The faster one (performance wise) is:
select * FROM Table where keyword LIKE '%value%'
The trick in this one is the placement of the % which is a wildcard, saying either search everything that ends or begins with this value.
A more flexible but (slightly) slower one could be the REGEXP function:
Select * FROM Table WHERE keyword REGEXP 'value'
This is using the power of regular expressions, so you could get as elaborate as you wanted with it. However, leaving as above gives you a "poor man's Google" of sorts, allowing the search to be bits and pieces of overall fields.
The sticky part comes in if you're trying to search names. For example, either would find the name "smith" if you searched SMI. However, neither would find "Jon Smith" if there was a first and last name field separated. So, you'd have to do some concatenation for the search to find either Jon OR Smith OR Jon Smith OR Smith, Jon. It can really snowball from there.
Of course, if you're doing some sort of advanced search, you'll have to condition your query accordingly. So, for instance, if you wanted to search first, last, address, then your query would have to test for each:
SELECT * FROM table WHERE first LIKE '%value%' OR last LIKE '%value%' OR address LIKE '%value'
Look at below example :
$word2compare = "stupid";
$words = array(
'stupid',
'stu and pid',
'hello',
'foobar',
'stpid',
'upid',
'stuuupid',
'sstuuupiiid',
);
while(list($id, $str) = each($words)){
similar_text($str, $word2compare, $percent);
if($percent > 90) // Change percentage value to 80,70,60 and see changes
print "Comparing '$word2compare' with '$str': ";
}
You can check with $percent parameter for how strong match you want to apply.

having trouble search through mysql database

I have two questions regarding my script and searching. I have this script:
$searchTerms = explode(' ', $varSearch);
$searchTermBits = array();
foreach($searchTerms as $term){
$term = trim($term);
if(!empty($term)){
$searchTermBits[] = "column1 LIKE '%".$term."%'";
}
}
$sql = mysql_query("SELECT * FROM table WHERE ".implode(' OR ', $searchTermBits)."");
I have a column1 with a data name "rock cheer climbing here"
If I type in "rock climb" this data shows. Thats perfect, but if I just type "Rocks", it doesn't show. Why is that?
Also, How would I add another "column2" for the keyword to search into?
Thank you!
Searching that string for "rocks" doesn't work, because the string "rocks" doesn't exist in the data. Looking at it, it makes sense to you, because you know that the plural of "rock" is "rocks", but the database doesn't know that.
One option you could try is removing the S from search terms, but you run into other issues with that - for example, the plural of "berry" is "berries", and if you remove the S, you'll be searching for "berrie" which doesn't get you any further.
You can add more search terms by adding more lines like
$searchTermBits[] = "column1 LIKE '%".$term."%'";
and replacing ".$term." with what you want to search for. For example,
$searchTermBits[] = "column1 LIKE '%climb%'";
One other thing to note... as written, your code is susceptible to SQL injection. Take this for example... What if the site visitor types in the search term '; DROP TABLE tablename; You've just had your data wiped out.
What you should do is modify your searchTermBits[] line to look like:
$searchTermBits[] = "column1 LIKE '%" . mysql_real_escape_string($term) . "%'";
That will prevent any nastiness from harming your data.
Assuming the data you gave is accurate, it shouldn't match because you're using "Rocks" and the word in the string is "rock". By default mysql doesn't do case sensitive matching, so it's probably not the case.
Also, to avoid sql injection, you absolutely should be using mysql_real_escape_string to escape your content.
Adding a second column would be pretty easy as well. Just add two entries to your array for every search term, one for column1 and one for column2.
Your column1 data rock cheer climbing here your search criteria %Rocks% it doesn't fit at all as rocks is not in your column1 data
you can add column2 as you do for column1 then put it all together by using an AND operator (column1 LIKE "%rock%" OR column1 LIKE "%climb%") AND (column2 LIKE "%rope%" OR column2 LIKE "%powder%")
TIPS:
If your table/schema are using xx_xx_ci collation (then this is mean case insensitive,mysql doesn't care case sensitive) but if other then you need to make sure that the search term must be case sensitive(mysql do case sensitive).

MySQL/PHP Search Efficiency

I'm trying to create a small search for my site. I've tried using full-text index search, but I could never get it to work. Here is what I've come up with:
if(isset($_GET['search'])) {
$search = str_replace('-', ' ', $_GET['search']);
$result = array();
$titles = mysql_query("SELECT title FROM Entries WHERE title LIKE '%$search%'");
while($row = mysql_fetch_assoc($titles)) {
$result[] = $row['title'];
}
$tags = mysql_query("SELECT title FROM Entries WHERE tags LIKE '%$search%'");
while($row = mysql_fetch_assoc($tags)) {
$result[] = $row['title'];
}
$text = mysql_query("SELECT title FROM Entries WHERE entry LIKE '%$search%'");
while($row = mysql_fetch_assoc($text)) {
$result[] = $row['title'];
}
$result = array_unique($result);
}
So basically, it searches through all the titles, body-text, and tags of all the entries in the DB. This works decently well, but I'm just wondering how efficient would it be? This would only be for a small blog, too. Either way I'm just wondering if this could be made any more efficient.
There's no way to make LIKE '%pattern%' queries efficient. Once you get a nontrivial amount of data, using those wildcard queries performs hundreds or thousands of times slower than using a fulltext indexing solution.
You should look at the presentation I did for MySQL University:
http://www.slideshare.net/billkarwin/practical-full-text-search-with-my-sql
Here's how to get it to work:
First make sure your table uses the MyISAM storage engine. MySQL FULLTEXT indexes support only MyISAM tables. (edit 11/1/2012: MySQL 5.6 is introducing a FULLTEXT index type for InnoDB tables.)
ALTER TABLE Entries ENGINE=MyISAM;
Create a fulltext index.
CREATE FULLTEXT INDEX searchindex ON Entries(title, tags, entry);
Search it!
$search = mysql_real_escape_string($search);
$titles = mysql_query("SELECT title FROM Entries
WHERE MATCH(title, tags, entry) AGAINST('$search')");
while($row = mysql_fetch_assoc($titles)) {
$result[] = $row['title'];
}
Note that the columns you name in the MATCH clause must be the same columns in the same order as those you declared in the fulltext index definition. Otherwise it won't work.
I've tried using full-text index search, but I could never get it to work... I'm just wondering if this could be made any more efficient.
This is exactly like saying, "I couldn't figure out how to use this chainsaw, so I decided to cut down this redwood tree with a pocketknife. How can I make that work as well as the chainsaw?"
Regarding your comment about searching for words that match more than 50% of the rows.
The MySQL manual says this:
Users who need to bypass the 50% limitation can use the boolean search mode; see Section 11.8.2, “Boolean Full-Text Searches”.
And this:
The 50% threshold for natural language
searches is determined by the
particular weighting scheme chosen. To
disable it, look for the following
line in storage/myisam/ftdefs.h:
#define GWS_IN_USE GWS_PROB
Change that line to this:
#define GWS_IN_USE GWS_FREQ
Then recompile MySQL. There is no need
to rebuild the indexes in this case.
Also, you might be searching for stopwords. These are words that are ignored by the fulltext search because they're too common. Words like "the" and so on. See http://dev.mysql.com/doc/refman/5.1/en/fulltext-stopwords.html
Using LIKE is NOT fulltext.
You need to use ... WHERE MATCH(column) AGAINST('the query') in order to access a fulltext search.
MySQL Full-text search works -- I would look into it and debug it rather than trying to do this. Doing 3 separate MySQL queries will not be anywhere near as efficient.
If you want to try to make that much efficient you could separate the LIKE statements in one query with OR between them.

Categories