I have a table with columns named: "ID, Track Name, City, County" among others. I'm looking for a way for a user to query the database and get results back relating to those 4 columns, with the most relevant results appearing at the top.
Say there was a row with: "1, Track Number One, Taunton, Somerset"
and a row with: "2, Circuit Number Two, Taunton, Somerset"
and the user searched "circuit in taunton"
Obviously I would like both rows to show, but I would like the second row to appear at the top because it has matched 2 keywords.
This is the current code I am using but it is only working on one column or another and if the user queried "karting in taunton" it would not work.
$searchterm = htmlspecialchars($getArray['searchterm']);
$searchterm = $mysqli->real_escape_string($searchterm);
$terms = explode(' ', $searchterm);
$bits = array();
$cityBits = array();
$countyBits = array();
foreach ($terms as $term) {
$bits[] = "name LIKE '%".mysql_real_escape_string($term)."%'";
}
foreach ($terms as $cityTerms) {
$cityBits[] = "city LIKE '%".mysql_real_escape_string($cityTerms)."%'";
}
foreach ($terms as $countyTerms) {
$countyBits[] = "county LIKE '%".mysql_real_escape_string($countyTerms)."%'";
}
$this->query = "SELECT * FROM tracks WHERE (".implode(' AND ', $bits).")
union SELECT * FROM tracks WHERE (".implode(' AND ', $cityBits).")
union SELECT * FROM tracks WHERE (".implode(' AND ', $countyBits).")";
I know this code is not ideal but it all I've managed to get working at the moment.
Firstly, your requirement is a much better match for full text searching than "like" comparisons. As soon as you get this working, you'll want to boost "Taunton" over "Aunton" and then deal with different spellings, wildcards, and then you'll notice that the wildcards in your like statement slow the whole thing down. Full text searching does all that for you, and is pretty quick.
The specific answer to your question is a nasty one; I'm reluctant to give you code PHP code because I can't easily test it. I've created a little SQLFiddle to show how you can do it in SQL, though.
In essence, you need to turn your "hits" on individual searches into a derived table, and count each row's number of entries; the more times it turns up in your derived table, the more likely it is to be a "good" hit.
In one of the places i've been working at, i had to do a search engine to search songs into a database (so basically what you're trying to do). The solution i came up with was : when you insert a song into the database, you also put some relevant keywords which should make the song appear when you type them.
So what's happening when someone search for let's say : MJ, my code was searching for exact matches into the artist name / song title. If i had results, i was displaying them by alphabetical number or by whatever the user wanted to sort them by.
If no results were found (and here's the interesting part for you) my function was doing a query into the keywords of each songs. Once i had all the results and especially their associated keywords into an array i was just counting the number of keywords matching the query for each result and only then i could sort by relevance.
Hope this helps you, i guess you're only missing a "keywords" row in your db.
Your query is setup to return 2 identical rows if 2 matches are found. You might reword it to return the row once, along with an ORDER BY clause that counts the matches.
"SELECT * FROM tracks WHERE (".implode(' AND ', $bits).")
OR (".implode(' AND ', $cityBits).")
OR (".implode(' AND ', $countyBits).")
ORDER BY ((".implode(' AND ', $bits).")
+ (".implode(' AND ', $cityBits).")
+ (".implode(' AND ', $countyBits).")) DESC"
This isn't exact, because I'm not sure why you are imploding with "AND" rather than "OR", but you get the jist.
Related
I am implementing a search feature for my project. I am using a FULL TEXT SEARCH query to derive accurate results to User. I am beginner in PHP programming and I do not have enough information about FULL TEXT SEARCH.
This is my query:
$sql = $conn->prepare("SELECT *, MATCH(title, keyword) AGAINST(? IN BOOLEAN MODE) AS relevance FROM table ORDER BY relevance DESC LIMIT 20");
$sql->bind_param("s", $q);
$sql->execute();
$rs = $sql->get_result();
This query works good but this is only showing old results first instead of accurate results, and second thing is this query is not working correctly when the length of keyword is not more than 1 (e.g. keyword = Google).
Please do not give suggestions about Elastic search, Sphinx,
Algolia etc.
When MATCH() is used in a WHERE clause, the rows returned are automatically sorted with the highest relevance first.
So all you have to do is, remove the match from select and put it in where condition.
Source: https://dev.mysql.com/doc/refman/8.0/en/fulltext-natural-language.html
Why are you not using the sql like operator, I am providing you the example for multiple words in column named product in table named products
$db=mysqli_connect('localhost','root','','project');
$search=$_GET['userinput'];
$searcharray = explode(' ', $search);
$searchpdo=array();
$finalstate="";
foreach ( $searcharray as $ind=> $query){
$sql=array();
$exp='%'.$query.'%';
array_push($sql,"(title LIKE ?)");
array_push($searchpdo,$exp);
array_push($sql,"(keywords LIKE ?)");
array_push($searchpdo,$exp);
if($finalstate==""){
$finalstate = "(".implode(" OR ",$sql).")";
}
else{
$finalstate = $finalstate." AND "."(".implode(" OR ",$sql).")";
}
}
$stmt = $db->prepare("SELECT * FROM products WHERE (".$finalstate.") ");
$types=str_repeat('s',count($searchpdo));
$stmt->bind_param($types,...$searchpdo);
$stmt->execute();
$result = $stmt->get_result();
This will provide you the correct result with single word or multiple words
I think you have to tweak you query little bit and you would get desired results as under:
$sql = mysql_query("SELECT * FROM
patient_db WHERE
MATCH ( Name, id_number )
AGAINST ('+firstWord +SecondWord +ThirdWord' IN BOOLEAN MODE);");
and if you want to do exact search:
$sql = mysql_query("SELECT *
FROM patient_db
WHERE MATCH ( Name, id_number )
AGAINST ('"Exact phrase/Words"' IN BOOLEAN MODE);");
I had also posted the same answer in SO post somewhere but didn't know the post
There are multiple aspect to your question
If available, use mysql client to run the query instead of PHP first, until your query is ready to the like you want
If you recent documents (record) to show up on top of the search result, you need to change your ORDER BY clause. Currently, it is supposed to return the closest match (i.e. by relevance).
You need to strike a balance between relevance and recency (not clear how you define this) in your custom logic. A simple example that prioritize last week over last month and last month over the rest:
SELECT
....
, DATEDIFF (ItemDate, CURDATE() ) ItemAgeInDays
ORDER BY
relevance
* 100
* CASE
WHEN ItemAgeInDays BETWEEN 0 AND 7 --- last week
THEN 20
WHEN ItemAgeInDays BETWEEN 0 AND 30 --- last month
THEN 10
ELSE 1
END
DESC
You say single word item cannot be searched. In BOOLEAN MODE, you build a boolean logic for your search and such it uses special characters for that. For example +apple means 'apple' must exist. It is possible your single word might be conflicting with these characters.
Please review this reference, it explains the BOOLEAN MODE in great detail.
https://dev.mysql.com/doc/refman/8.0/en/fulltext-boolean.html
You say the query is not returning correct result. FULL TEXT search searches for your login in each document(row) and finds how many times it appears in each document. It then offset that by how many times your search appears in ALL documents. This means it prioritizes records where your search appears much more than the average. If your search is not distinguishing enough, it might seem not correct if most documents are similar to each in terms of that search. See the above link for more details.
BOOLEAN MODE does not sort the result by relevance by default. You need to add ORDER BY yourself, which you already did. Just wanted to note it here for others
Before I start, here is the sample format of the database:
id date title content
--------------------------------------------------------------
1 YYYY-MM-DD some text here lots of stuff to search here
2 YYYY-MM-DD some text here lots of stuff to search here
3 YYYY-MM-DD some text here lots of stuff to search here
I currently have something set up to get the id of a row, then assign the date, title, and content to PHP variables to display in different places on a page (it's a blog, these are posts).
Now I'm trying to make a function that will search the content for each string in an array (the code already splits words entered into the search box by spaces into an array, so entering cat hotdog suit will give [0]cat [1]hotdog [2]suit).
I have a loop set up that performs a query on each of the strings:
foreach ($queryArray as $queryArrayString) {
// query the content column for $queryArrayString
}
However, I can't figure out what query I should use to find the id of the post that the string occurs in. I'd like to assign it to a variable ($results).
After reversing the array, so higher ids (newer results) show up first, I'd like to find the titles of each id and store it like this:
id (let's say it is 3) => matching title
id (let's say it is 1) => matching title
Is this possible? I've tried several different queries, and none worked, but one took 2 minutes and loads of memory to attempt.
Any help would be appreciated.
EDIT: I've created a text index of the title and content columns (I'd like to search both), shown in this image.
Is it possible to search this? Like I've said in the comments, I'd just need it to return the corresponding id and title somehow. (Title for result link text, id for link destination: [title] is the format I'm using
In ideal world, there are complex solutions available like Apache SOLR or Elasticsearch for indexing records and return efficiently when multiple text-search-terms are provided. Luckily, you can achieve such results by trying something like this (not tested)..
$searchTerms = "cat hotdog suit";
$searchArray = explode(' ',$searchTerms);
$conditions = array();
$foreach (searchArray as $searchTerm){
$conditions[] = 'title-content LIKE %'.searchTerm.'%';
}
$query = '
SELECT
tableName.id,
CONCAT_WS(' ', tableName.title,tableName.content) as title-content
FROM
tableName
WHERE
'.implode(' OR ', $conditions);
I'm building a small internal phone number extension list.
In the database I have two tables, a people table, and a numbers table. The relationship is a one-to-many respectively.
I want to display the results in a single HTML table with one person per row but if they have multiple numbers it shows those in a single column with a rowspan on the person row to compensate.
Now, to get the results from the database to work with, I can either do:
(pseudocode)
SELECT id, name
FROM people
foreach result as row {
SELECT number
FROM numbers
WHERE numbers.person_id = row['id']
}
This would mean that I'm doing one query to get all users, but then if I have 100 users, I'm performing 100 additional queries to get the numbers for each user.
Instead I could do it like this:
(pseudocode)
SELECT number, person_id
FROM numbers
SELECT id, name
FROM people
foreach people as person {
echo name
foreach numbers as number {
if (number.id = person.id) {
echo number
}
}
}
So, essentially it is doing the exact same thing except instead I do two queries to get all the results into arrays and then loop through the arrays to format my tables.
Which method should I be using or is there a better way to do this?
The common way is to do a regular JOIN:
SELECT id, name, number
FROM people, numbers
WHERE people.id = numbers.person_id;
You can either add an ORDER BY to get the numbers in order, or you could create an array with a single pass over the resultset, and then loop through that array.
You can also consider a GROUP_CONCAT to concatenate all the numbers for the same person:
SELECT id, name, GROUP_CONCAT(number SEPARATOR ', ')
FROM people, numbers
WHERE people.id = numbers.person_id
GROUP BY people.id;
Since you are even asking this question: I cannot stress the fact that you should pick up an introductory book on database design. It helped me wonders to learn the theories behind relational databases.
You probably want to execute just one query. Something like
select people.id, people.name, group_concat(numbers.number)
from people
inner join numbers on numbers.id = people.id
group by people.id, people.name
order by people.name
Then you can loop over the result set with simple php code.
It depends, and you may have to time it to find out. Doing multiple queries is a lot of network turns if your database is on a different computer than your web server, so often this takes longer. However, if your database server is on the same computer as your web server, this might not be an issue.
Also consider the time it will take to look up the number in the array. As an array you are doing a linear order O(N) search. If you can put it in a hash, the lookup will be much faster, and the two query approach may be faster, but not if you spend a lot of time looking up the answer in your array.
Using a join to get it into one query, may be fastest, as the numbers will be associated with the people, depending on your container structure you are storing the data into to be accessed in your foreach loop.
Use a stored procedure or function to retrive the data, don't wite the sql in your programm
You should do neither. You can do one query (join) over the tables:
select name, number
from people p, numbers n
where p.id = n.person_id
order by name, number;
and then just one loop:
$link = mysqli_connect(...);
$query = 'select name, number from people p, numbers n where p.id = n.person_id order by name, number';
if ($result = mysqli_query($link, $query)) {
$person = '';
while ($row = mysqli_fetch_array($result, MYSQLI_ASSOC)) {
if ($row['name'] !== $person) {
$person = $row['name'];
echo $person;
}
echo $row['number'];
}
}
I have a simple search feature that will search a table for a hit. Basically the one flaw of this search is that it tries to match exactly the string. I want it to take a string of say "large red bird" and search "large", "red" and "bird" separately against the table. here's my search query...
$result = mysql_query("SELECT * FROM files
WHERE (tags LIKE '%$search_ar%' OR
name LIKE '%$search_ar%' OR
company LIKE '%$search_ar%' OR
brand LIKE '%$search_ar%')
$str_thera $str_global $str_branded $str_medium $str_files");
any Ideas? thanks
Edit
OK here's my updated query but it doesnt return anything.
$result = mysql_query("SELECT * FROM files
WHERE MATCH(tags, name, company, brand)
AGAINST ('$seach_ar' IN boolean MODE)");
A fulltext search will indeed help, given you do have a lot of data to search by. A good reference for the full text can be found here: http://forge.mysql.com/w/images/c/c5/Fulltext.pdf
The reason I say a lot of data, or a fair amount, is that if the search results yields above a certain percentage of returned rows to total rows, nothing is returned.
If you want to continue using the LIKE method, you can. You just have to seperate the words (explode) and the join them in the sql query using AND:
...(tags LIKE '%$search_ar[0]%' AND tags LIKE '%$search_ar[1]%') OR ....
In a fashion like that. This method can get overly complicated especially if say, you want to return matches which has any of the words and not all of them. So yea, it will take some customization to do and to automate, but it is possible.
If, for whatever reason, you can't use full text searching (nice idea, Nick and Brad, but only MyISAM supports it, and from what I hear, it's not really all that good), it's not too hard to rig up basic searching with explode() and implode():
$search_arr = explode(' ',$search_str);
$search_str = implode('%',$search_arr);
//query where fields like $search_arr
Or:
$search_arr = explode(' ', $search_str);
$sql = 'select * from files where ';
foreach($search_arr = $term){
$sql .= "tags like '%{$term}%' or " //fill in rest of fields
}
As a side note: If you plan on growing your searching and can't use MySQL's full text search, I recommend checking out one of the search servers, such as Solr or Lucene.
I have a row that has keywords in this way (keyword1, keyword2, keyword3) separated by commas as shown. When a user signs in, you know that he wants information about (keyword1, keyword3). Now, I have another table that has bunch of information related to different keywords, this table has a row called (keywords) which indicates if this information is suitable for which keyword. How do I render for the user the information he needs depending on the keywords.
In other words, if the user wants information about (keyword3, keyword1) how do I go to the (information) table and find all the information that has the word (keyword1) or the word (keyword3) in the row (keyword)?
Sorry if this is complicated (my explanation) but I tried my best to explain it.
Thank you in advance :)
Please normalize your table, and be very suspicious about comma separated values in a single field. The more flexible setup would be:
USERS
id
name
KEYWORDS
id (or just use name as primary key)
name
...
USER_KEYWORDS
user_id
keyword_id
INFORMATION
id
data
INFORMATION_KEYWORDS
information_id
keyword_id
Your resulting query would be something like:
SELECT information.data
FROM users
JOIN user_keywords ON user_keywords.user_id = users.id
JOIN information_keywords ON information_keywords.keyword_id = user_keywords.keyword_id
JOIN information ON information_keywords.information_id = information.id
WHERE users.id = <id>
Use the MySQL FIND_IN_SET function in your SQL query, like this:
FROM TABLE t
WHERE FIND_IN_SET(t.keywords, 'keyword1, keyword3') > 0
First rule of database design: store a single value per row.
In other words, you should modify your database to store each keyword in a separate row.
Have you considered changing the data model?
Currently you have a mini-table inside one column. That is, the keyword column contains multiple values, separated by comma's. You are going to have a hard time matching those keywords.
A better solution would be to make a table UserKeywords:
User Keyword
Paul snacks
Paul food
Paul beer
Kelly snacks
Kelly wine
This way, you can join this table against the information and user tables.
Sjoerd is right. A relational database setup is definitely the best solution.
I'm not sure about MySQL FIND_IN_SET() but you can parse out your user keywords and compose a string so your search query would end up as something like this:
SELECT * FROM `information` WHERE MATCH(`keyword`) AGAINST('keyword1 keyword3' IN BOOLEAN MODE)
In this case your match against should return if it matches either one.
You should keep in mind that the column "keyword" has to be full text indexed for match against to work AND that match against only works for the minimum number of characters defined in the php.ini config on your server. In most cases the default is 4 characters.
In other words, your keywords have to be 4 characters or more to use match against.
That being said, you can set up your query string to do something like this:
$user_keywords = array('keyword1', 'keyword3');
if(count($user_keywords) == 1)
{
$word = $user_keywords[0];
if(strlen($word) >= 4)
{
$query_str = "MATCH(`keyword`) AGAINST ('".$word."' IN BOOLEAN MODE)";
}
else
{
$query_str = "`keyword` LIKE '%".$word."%'";
}
}
else
{
$query_str = "";
foreach($user_keywords AS $key)
{
$query_str .= "`keyword` = '".$key."' OR ";
}
$query_str = substr($query_str, 0, -3);
}
$sql = "SELECT * FROM `information` WHERE ".$query_str;
Either way you do it, you really should begin with converting your database to a relational setup unless you have no choice - like you're using third party packages that you did not code and it would take forever to convert.
Also, the % are in there as an example of partial match. If you need more explanation, let us know.
Did you mean LIKE query ?? Check Out http://dev.mysql.com/doc/refman/4.1/en/pattern-matching.html
When doing keyword type queries (hopefully doing the one-per-line method explained by sjoerd, I'm a big fan of the infrequently used REGEXP feature of MySql. Similar to like, it can do partial searches of all characters in a cell for similar data without all the special characters required to search various parts of the query. Think of it as Google for MySql
an example:
SELECT * from TABLE WHERE value REGEXP 'foo'