I have database with list of similar words. I need to get similar words and I am trying this approach to get every similar word:
I am using LIKE on mysql query to get similar words.
It's not enough to use LIKE, so I make every possible string with % to get more similar results. I don't know and I can't find if there is any LIKE alternative to find much more relevant queries.
So for example to find similar words like "EL", I am using this query:
SELECT * FROM `words` WHERE word LIKE 'el' OR word LIKE '%el' OR word LIKE '%el%' OR word LIKE '%e%l%'
And it returns only one result, which is not what I want to. However if I would use multi_query and multiple queries like:
SELECT * FROM `words` WHERE word LIKE 'el';
SELECT * FROM `words` WHERE word LIKE '%el';
SELECT * FROM `words` WHERE word LIKE '%el%';
SELECT * FROM `words` WHERE word LIKE '%e%l%';
To fetch these I use:
if ($con->multi_query($query)) {
do {
/* almacenar primer juego de resultados */
if ($result = $con->store_result()) {
while ($row = $result->fetch_row()) {
printf("%s\n", $row[0]);
}
$result->free();
}
/* mostrar divisor */
if ($con->more_results()) {
printf("-----------------\n");
}
} while ($con->next_result());
}
/* cerrar conexión */
$con->close();
These are (example/similar) results I get with it:
el,espinel,el,wheels,espinel,wheels
It would get all the possible results. Of course I would need to filter the duplicates, but I would get them all.
For single query I use:
$result = $con->query($query);
$row = mysqli_fetch_row($result);
However I think multiple queries and filtering would take more time then single query, so I am looking for a way to get all the results on single query or even better without creating all possible variations of string.
As mentioned already, you only need to run the last query. In order to get the desired order of the results (relevance), you'd need to implement that logic via php.
$keyword = 'Your search term';
function getRelevance($value,$keyword){
$value = strtolower($value);
$keyword = strtolower($keyword);
$index = strpos($value, $keyword);
$word_index = strpos($value, ' '.$keyword);
if($index==0) // first word starts with keyword
return 3;
else if($word_index!==false) // any word starts with keyword
return 2;
else if($index!==false) // keyword matches anywhere
return 1;
else
return 0;
}
$keyword = mysqli_real_escape_string($con, $keyword); // prevent SQL injection
$res = $con->query("SELECT * FROM words WHERE `word` LIKE '%$keyword%'");
$words = array();
while($line = $res->fetch_assoc()){
$line['relevance'] = getRelevance($line['word'],$keyword); // assign relevance value based on the "el" query and the elements word
$words[] = $line;
}
function compareRelevance($a,$b){
return $b['relevance'] - $a['relevance'];
}
usort($words,'compareRelevance');
This will fetch all matching entries from the 'words' table and sort them based on the search term.
Related
I was trying to make a search engine so far I had done with it and the feature I lack in it was, when user queries a long sentence or two or three words with on not in line-wise in my DB is not going to show and it shows 0 results.
So far it is working fine with the one-word search it is doing a great job with that.
public function getResultsHtml($page, $pageSize, $term){
$fromLimit = ($page - 1) * $pageSize;
//page 1: (1-1) * 20
//page 2:(2-1) * 20
$query = $this->con->prepare("SELECT *
FROM sites WHERE title LIKE :term
OR url LIKE :term
OR keywords LIKE :term
OR description LIKE :term
ORDER BY clicks DESC
LIMIT :fromLimit,:pageSize");
$searchTerm = "%" . $term ."%";
$query->bindParam(":term",$searchTerm);
$query->bindParam(":fromLimit",$fromLimit,PDO::PARAM_INT);
$query->bindParam(":pageSize",$pageSize,PDO::PARAM_INT);
$query->execute();
$resultsHtml ="<div class='siteResults'>";
while($row = $query->fetch(PDO::FETCH_ASSOC)){
$id = $row["id"];
$url = $row["url"];
$title = $row["title"];
$description = $row["description"];
$title = $this->trimField($title,55);
$description = $this->trimField($description,230);
$resultsHtml .="<div class='resultContainer'>
<h3 class='title'>
<a class='result' href='$url' data-linkId='$id'>
$title
</a>
</h3>
<span class='url'>$url</span>
<span class='description'>$description</span>
</div>";
}
$resultsHtml .= "</div>";
return $resultsHtml;
}
So when user searches apple it is retrieving the data and we can see the search result in "apple store search" in the first image.But in second image when we search "apple search" it should be able to show us the "apple store search".
first image
Second image
First you need to breakdown your $term into separated words. With European languages, you can simply use explode:
<?php
$terms = explode(' ', $term);
// lets say your $term is 'apple orange lemon banana'
// $terms would be ['apple', 'orange', 'lemon', 'banana']
Prepare Terms List for Binding
Then, I'd build a key-value array for the terms binding:
<?php
// build a term list with term key (:term{number}) and associated term
// for binding
$term_params = [];
foreach ($terms as $key => $term) {
$term_params[":term{$key}"] = "%{$term}%";
}
// $term_params would be:
// [
// ':term0' => '%apple%',
// ':term1' => '%orange%',
// ':term2' => '%lemon%',
// ':term3' => '%banana%',
// ]
Prepare the Where Clause in SQL for Binding
Now, supposed the logics is all the terms need to show up in either of the target fields:
<?php
// a list of fields to search from
$fields = ['title', 'url', 'keywords', 'description'];
// build a where clause SQL based on the terms and fields
$or_where_clauses = [];
foreach (array_keys($term_params) as $term_key) {
$or_where_clauses[] = implode(' OR ', array_map(function ($field) use ($term_key) {
return "{$field} LIKE {$term_key}";
}, $fields));
}
$where_clauses_sql = implode(' AND ', array_map(function ($term_clause) {
// quote each term clause with bracket for proper OR logic to work
return '(' . $term_clause . ')';
}, $or_where_clauses);
The resulting where clauses sql would be:
(title like :term0 OR url like :term0 OR keywords like :term0 OR description like :term0)
AND
(title like :term1 OR url like :term1 OR keywords like :term1 OR description like :term1)
AND
...
...
(title like :termN OR url like :termN OR keywords like :termN OR description like :termN)
Putting Everything Together
I can then build the SQL string and bind the terms to the query accordingly:
<?php
// prepare the select statement
$query = $this->con->prepare("SELECT *
FROM sites WHERE {$where_clauses_sql}
ORDER BY clicks DESC
LIMIT :fromLimit,:pageSize");
// bind the terms
foreach ($term_params as $key => $value) {
$query->bindParam($key, $value);
}
$query->bindParam(":fromLimit", $fromLimit, PDO::PARAM_INT);
$query->bindParam(":pageSize", $pageSize, PDO::PARAM_INT);
$query->execute();
// ...
Shortcomings and Potential Solution
Please note that this approach does not understand the number of occurrence of terms in the field. And the way to separate words is far from perfect. For example:
It didn't handle punctuation marks.
It couldn't search for plural terms with the singular, or visa verse.
It could be matching part of a word by mistake (e.g. search "sing" and match "dressing").
It couldn't deal with CJK or languages without space).
You can use software like Elastic Search for better feature. With plugin and proper config, you can give even sort results with relevance, or give different field a different importance in the search process etc.
But that is an entirely different software than SQL server to use. You'd need to learn and plan how to index your contents there as well as just saving your data to SQL.
we dont know how u get search words from user, but as i guess u get them as an array. so you can try below code:
<?php
...
$textSearch = "";
for($i = 0 ; $i< count($userInputs);$i++){
if($i !== count($userInputs) -1){
$userInputData = $userInputs[$i];
$textSearch .= "'%$userInputData%' OR";
}else{
$textSearch .= "'%$userInputData%'";
}
}
and put $textSearch into your query.
remember , $userInputs is an array that u had before.
UPDATE
as your images shown, you can add $userInput = explode(" ",$textFromUser) in very begin of given code.
<?php
include"configration.php";
?>
<?php
$query = $_GET['query'];
$min_length = 1;
//echo $query;exit();
if (strlen($query) >= $min_length) { // if query length is more or equal minimum length then
//echo "success";exit();
$query = htmlspecialchars($query);
$query = mysqli_real_escape_string($conn, $query);
$sql = "SELECT * FROM table2
WHERE title LIKE '%".$query."%' order by date DESC";
$raw_results = mysqli_query($conn, $sql) or die(mysql_error());
if (mysqli_num_rows($raw_results) > 0) { // if one or more rows are returned do following
while ($res = mysqli_fetch_array($raw_results)) { ?>
<?php echo $res['title'] ?> // Place where result comes ..
<?php }
}
}
?>
This is code works fine but search in this way
For Example Title is: you are vary nice boy but lazy
When I search by:
You are vary ............. result shows ..
vary nice boy ............. result shows ..
vary lazy, or boy lazy or vary lazy .. result not shows ..
Plz some one help me in this and how to show searched query in title ..
<title> Searched Query ...</title>
LIKE '%boy lazy%' will show the Of the cases where anything can be before boy lazy and anything can be after boy lazy, but boy lazy will be together.
In your case, one approach can be, you can explode your $query, and then use multiple LIKE queries to create sql query. Example:
<?php
//$conn = mysqli_connect("localhost","your user","your pass","db");
$query = $_GET['query'];
$min_length = 1;
//echo $query;exit();
if (strlen($query) >= $min_length) { // if query length is more or equal minimum length then
//echo "success";exit();
$query = htmlspecialchars($query);
$query = mysqli_real_escape_string($conn, $query);
$searchKeys = explode(' ',$query);
$sql = "SELECT * from table2 where title ";
foreach ($searchKeys as $key) {
$sql.= "LIKE '%".$key."%' AND title ";
}
$sql = substr($sql, 0, -10);
//$sql.="ORDER BY date DESC;";
$raw_results = mysqli_query($conn, $sql) or die(mysql_error());
if (mysqli_num_rows($raw_results) > 0) { // if one or more rows are returned do following
while ($res = mysqli_fetch_assoc($raw_results)) {
echo $res['title']."\n";
}
}
}
When you search title LIKE "%vary lazy%", you will get records that contain the string "vary lazy" preceeded and followed by any other or no character sequences. If you want to match strings that contain the words - I should better say, the character sequences - "vary" and "lazy" in that specific order you should use:
title LIKE "%vary%lazy%"
However, this will also match "varylazy", "varying lazytown characters".
Assuming you generally intend to use queries as you mentioned, i.e. each word is separated by a space character and you want to see if those words appear in a text in specifically that order, you could write something like this:
$query = $_GET["query"];
$query = '%'.str_replace(' ', '%', $query).'%';
//... MySQL stuff
Please be aware that the code above is very specific to your needs. I wouldn't use it as a general purpose approach for processing query strings, e.g. having multiple spaces between words would result in multiple consequent % in your SQL query - I'm not even sure if that is allowed. However, under the constraints described, this code should work just fine.
I have in mysql database table column keywords there are csv keywords like "hotel, new hotel, good hotel".
Now when user enter hotel it works(select data) but not for hotels(it shouldn't). Now I want user enter hotels then it should also match hotel keyword.
In-short with suffix search should work. currently i implemented following.
$queried = trim(mysqli_real_escape_string($con,$_POST['query']));
$keys = explode(" ",$queried);
$sql = 'SELECT name FROM image WHERE keyword LIKE "%$queried%"';
foreach($keys as $k){
$k= trim(mysqli_real_escape_string($con,$k));
if(count($keys) > 1)
{
$sql .= ' OR keyword LIKE "%$k%" ';
}
}
you'd have to (additionally) ask whether the search term contains any of the words in the rows. Currently you're doing the opposite (which is fine for the opposite situation, so don't get rid of it)
Something like:
$sql = 'SELECT name FROM image WHERE $queried LIKE "%" + keyword + "%"';
(Apologies if MySQL syntax isn't quite right, not used it for a while).
It might occasionally throw up unwanted things though, e.g. if the user wrote "aparthotel" it'd still return "hotel", you may or may not want that. Or it could even something entirely irrelevant depending on the words involved.
Once you get onto anything more complex than that though, you're probably into the realms of search engines and natural language processing.
i did this way it's not what i want but it works for my criteria.
$suffix = array('','s','es','ing','ment'); // suffix you want to ad
$sql = 'SELECT name FROM image WHERE keyword LIKE "%$queried%"';
foreach($keys as $k)
{
$k= trim(mysqli_real_escape_string($con,$k));
for ($i=1; $i < sizeof($suf) ; $i++)
{
if(substr($k, (-1 * strlen($suf[$i])))==$suf[$i])
{
$wp=substr( $k, 0, (-1 * $i));
}
}
if($wp!="")
{
$sql .= " OR keyword LIKE '%$k%' OR keyword LIKE '%$wp%' ";
}
}
i have a variable and an user_name i want to search on a string(function_description) of the user_name for it
whats wrong with this :
$function_keywords = mysql_real_escape_string($_POST['function_keywords']);
if($function_keywords=="" || empty($function_keywords)){
redirect("show.php?functions=PHP");
}
//trim whitespace from the stored variable
$trimmed = trim($function_keywords);
//separate key-phrases into keywords
$trimmed_keywords = explode(" ",$trimmed);
// Build SQL Query for each keyword entered
foreach ($trimmed_keywords as $trimm){
// MySQL "MATCH" is used for full-text searching.
//this code is ebv weird , should check out soon!
$query = "SELECT *
FROM functions
WHERE isEnabled=1 AND isPrivate=0
AND function_description LIKE '{$trimm}'
AND user_name='{$user_name}'
";
// Execute the query to get number of rows that contain search kewords
$results=mysql_query ($query,$connection);
as far as "like" syntax goes you have to use the '%' symbol. if you query for
select * from table where column like '%yourkeyword%'
then it returns any rows with 'yourkeyword' inside the table column.
your statement will be true only if the column = 'yourkeyword'
That's highly inefficient. If someone puts in 5 keywords, you'd be running the search 5 times and getting 5 sets of results. Try something more along these lines:
$words = $_POST['function_keywords'];
if ($words == '') {
... abort ...
}
$parts = trim(explode(' ', $words));
$clauses = array();
foreach($parts as $part) {
$clauses[] = "function_description LIKE '%" . mysql_real_escape_string($part) . "%'";
}
$clause = implode(' OR ' , $clauses);
$sql = "SELECT .... WHERE (isEnabled=1) AND (isPrivate=1) AND (user_name='$user_name') AND ($clause)";
$result = mysql_query($sql) or die(mysql_error());
This'll build up a long series of or statements for each keyword specified, and run the whole thing as a single query.
To see if the function_description contains the keyword you need to use '%' which stands for anything much the way '*' does in unix. Try function_description LIKE '%{$trimm}%'
Quick MYSQL/PHP question. I'm using a "not-so-strict" search query as a fallback if no results are found with a normal search query, to the tune of:
foreach($find_array as $word) {
clauses[] = "(firstname SOUNDS LIKE '$word%' OR lastname SOUNDS LIKE '$word%')";
}
if (!empty($clauses)) $filter='('.implode(' AND ', $clauses).')';
$query = "SELECT * FROM table WHERE $filter";
Now, I'm using PHP to highlight the results, like:
foreach ($find_array as $term_to_highlight){
foreach ($result as $key => $result_string){
$result[$key]=highlight_stuff($result_string, $term_to_highlight);
}
}
But this method falls on its ass when I don't know what to highlight. Is there any way to find out what the "sound-alike" match is when running that mysql query?
That is to say, if someone searches for "Joan" I want it to highlight "John" instead.
Note that SOUNDS LIKE does not work as you think it does. It is not equivalent to LIKE in MySQL, as it does not support the % wildcard.
This means your query will not find "John David" when searching for "John". This might be acceptable if this is just your fallback, but it is not ideal.
So here is a different suggestion (that might need improvement); first use PHPs soundex() function to find the soundex of the keyword you are looking for.
$soundex = soundex($word);
$soundexPrefix = substr($soundex, 0, 2); // first two characters of soundex
$sql = "SELECT lastname, firstname ".
"FROM table WHERE SOUNDEX(lastname) LIKE '$soundexPrefix%' ".
"OR SOUNDEX(firstname) LIKE '$soundexPrefix%'";
Now you'll have a list of firstnames and lastnames that has a vague similarity in sounding (this might be a lot entries, and you might want to increase the length of the soundex prefix you use for your search). You can then calculate the Levenshtein distance between the soundex of each word and your search term, and sort by that.
Second, you should look at parameterized queries in MySQL, to avoid SQL injection bugs.
The SOUND LIKE condition just compares the SOUNDEX key of both words, and you can use the PHP soundex() function to generate the same key.
So, if you found a matching row and needed to find out which word to highlight, you can fetch both the firstname and lastname, and then use PHP to find which one matches and highlight just that word.
I made this code just to try this out. (Had to test my theory xD)
<?php
// A space seperated string of keywords, presumably from a search box somewhere.
$search_string = 'John Doe';
// Create a data array to contain the keywords and their matches.
// Keywords are grouped by their soundex keys.
$data = array();
foreach(explode(' ', $search_string) as $_word) {
$data[soundex($_word)]['keywords'][] = $_word;
}
// Execute a query to find all rows matching the soundex keys for the words.
$soundex_list = "'". implode("','", array_keys($data)) ."'";
$sql = "SELECT id, firstname, lastname
FROM sounds_like
WHERE SOUNDEX(firstname) IN({$soundex_list})
OR SOUNDEX(lastname) IN({$soundex_list})";
$sql_result = $dbLink->query($sql);
// Add the matches to their respective soundex key in the data array.
// This checks which word matched, the first or last name, and tags
// that word as the match so it can be highlighted later.
if($sql_result) {
while($_row = $sql_result->fetch_assoc()) {
foreach($data as $_soundex => &$_elem) {
if(soundex($_row['firstname']) == $_soundex) {
$_row['matches'] = 'firstname';
$_elem['matches'][] = $_row;
}
else if(soundex($_row['lastname']) == $_soundex) {
$_row['matches'] = 'lastname';
$_elem['matches'][] = $_row;
}
}
}
}
// Print the results as a simple text list.
header('content-type: text/plain');
echo "-- Possible results --\n";
foreach($data as $_group) {
// Print the keywords for this group's soundex key.
$keyword_list = "'". implode("', '", $_group['keywords']) ."'";
echo "For keywords: {$keyword_list}\n";
// Print all the matches for this group, if any.
if(isset($_group['matches']) && count($_group['matches']) > 0) {
foreach($_group['matches'] as $_match) {
// Highlight the matching word by encapsulatin it in dashes.
if($_match['matches'] == 'firstname') {
$_match['firstname'] = "-{$_match['firstname']}-";
}
else {
$_match['lastname'] = "-{$_match['lastname']}-";
}
echo " #{$_match['id']}: {$_match['firstname']} {$_match['lastname']}\n";
}
}
else {
echo " No matches.\n";
}
}
?>
A more generalized function, to pull out the matching soundex word from a strings could look like:
<?php
/**
* Attempts to find the first word in the $heystack that is a soundex
* match for the $needle.
*/
function find_soundex_match($heystack, $needle) {
$words = explode(' ', $heystack);
$needle_soundex = soundex($needle);
foreach($words as $_word) {
if(soundex($_word) == $needle_soundex) {
return $_word;
}
}
return false;
}
?>
Which, if I am understanding it correctly, could be used in your previously posted code as:
foreach ($find_array as $term_to_highlight){
foreach ($result as $key => $result_string){
$match_to_highlight = find_soundex_match($result_string, $term_to_highlight);
$result[$key]=highlight_stuff($result_string, $match_to_highlight);
}
}
This wouldn't be as efficient tho, as the more targeted code in the first snippet.