php and mysql live search - php

I'm currently working on a live search that displays results directly from a mysql db.
The code works, but not really as i want it.
Let's start with an example so that it is easier to understand:
My database has 5 columns:
id, link, description, try, keywords
The script that runs the ajax request on key up is the following:
$("#searchid").keyup(function () {
var searchid = encodeURIComponent($.trim($(this).val()));
var dataString = 'search=' + searchid;
if (searchid != '') {
$.ajax({
type: "POST",
url: "results.php",
data: dataString,
cache: false,
success: function (html) {
$("#result").html(html).show();
}
});
}
return false;
});
});
on the results.php file looks like this:
if ($db->connect_errno > 0) {
die('Unable to connect to database [' . $db->connect_error . ']');
}
if ($_REQUEST) {
$q = $_REQUEST['search'];
$sql_res = "select link, description, resources, keyword from _db where description like '%$q%' or keyword like '%$q%'";
$result = mysqli_query($db, $sql_res) or die(mysqli_error($db));
if (mysqli_num_rows($result) == 0) {
$display = '<div id="explainMessage" class="explainMessage">Sorry, no results found</div>';
echo $display;
} else {
while ($row = $result->fetch_assoc()) {
$link = $row['link'];
$description = $row['description'];
$keyword = $row['keyword'];
$b_description = '<strong>' . $q . '</strong>';
$b_keyword = '<strong>' . $q . '</strong>';
$final_description = str_ireplace($q, $b_description, $description);
$final_keyword = str_ireplace($q, $b_keyword, $keyword);
$display = '<div class="results" id="dbResults">
<div>
<div class="center"><span class="">Description :</span><span class="displayResult">' . $final_description . '</span></div>
<div class="right"><span class="">Keyword :</span><span class="displayResult">' . $final_keyword . '</span></div>
</div>
<hr>
</div>
</div>';
echo $display;
}
}
}
now, let's say that i have this row in my DB:
id = 1
link = google.com
description = it's google
totry = 0
keywords: google, test, search
if i type in the search bar:
google, test
i have the right result, but if i type:
test, google
i have no results, as obviously the order is wrong.
So basically, what o'd like to achieve is something a bit more like "tags", so that i can search for the right keywords without having to use the right order.
Can i do it with my current code (if yes, how?) or i need to change something?
thanks in advance for any suggestion.
PS: I know this is not the best way to read from a DB as it has some security issues, i'm going to change it later as this is an old script that i wrote ages ago, i'm more interested in have this to work properly, and i'm going to change method after.

Normalize your schema
The rules of relational database are very simple (at least the first three).
keywords: google, test, search
...breaks the second rule. Each keyword should be in its own row in a related table. Then you can simply write your query as....
SELECT link, description, resources, keyword
FROM _db
INNER JOIN keywords
ON _db.id=keywords.db_id
WHERE keyword.value IN (" . atomize($q) . ")
(where atomize explodes the query string, applies mysqli_escape_paramter() to each entry to avoid breaking your code, encloses each term in single quotes and concatenates the result).
Alternatively you could use MySQL's full text indexing which does this for you transparently.
Although hurricane makes some good points in his/her answer, they do not mention that none of the solutions proposed there does not scale to handle large volumes of data with any efficiency (decomposing the field into a new table/using full text indexing does).

Untested code but modify according to your needs,
$q = $_REQUEST['search'];
$q_comma = explode(",", $q);
$where_in_set = '';
$count = count($q_comma);
foreach( $q_comma as $q)
{
$counter++;
if($counter == $count) {
$where_in_set .= "FIND_IN_SET('$q','keywords')";
}else {
$where_in_set .= "FIND_IN_SET('$q','keywords') OR ";
}
}
$sql_res = "select link, description, resources, keyword from _db where $where_in_set or description like '%$q%'";

There are 2 solutions I can think of:
Use fulltext index and search.
You can split the search string into words in php for example using explode() and serach for the words not in a single serach criteria, but in separate ones. This latter one can be very resource intensive, since you are seraching in multiple fields.

LIKE '%google, test%' will match id=1 but not '%google,test%' (no space between coma) nor '%google test%' (space delimiter) nor '%test, google%'. Put each keyword as separate table or you can split input keywords into several single keyword and use OR operator such as LIKE 'google%' OR LIKE 'test%'

Not an ideal solution, but instead of treating your search datastring as one element, you can have php treat it as an array of keywords separated by a comma (by using explode). You'd then build a query depending on how many keywords were sent.
For example, using "google, test" your query would be:
$sql_res = "select link, description, resources, keyword from _db where (description like '%$q1%' or keyword like '%$q1%') AND (description like '%$q2%' or keyword like '%$q2%')";
Where $q1 and $q2 are "google" and "test".

First of all as you say it is not a good way to do it. I think you are writing a autocompleter.
Seperators for words
"google, test" or "test, google" is a attached words. First you need to define a seperator for users. Usually it is a whitespace ' '.
When you define it you need to split words.
$words = explode(" ",$q);
// now you get two words "google," and "test"
Then you need to create a sql which gives you multiple search chance.
There are a lot example in MySQL LIKE IN()?
Now you get your result.
Text similarity
Select all result from db and in a while search a text from another text. It gives you a dobule point for similarity. Best result is your result.
Php Similarity Example
Important Info
If you ask my opinion don't use it like that bcs it is very expensive. Use autocompleters on html side. Here is an example

Related

PHP MySqli show result for similar terms (Keyword)

Let me explain fast what i want to do!
I want to show similar rows from my database by a PHP term.
I have a table called "games" and a column called "title" that titles are looks like "Rockstar - GTA V".
So i want to remove all words after dash and use new string as keyword to search in database.
My CMS use this code to show post title inside the loop:
$_smarty_tpl->tpl_vars['game']->value['title']
I just found a code to convert "Rockstar - GTA V" to "Rockstar":
<?php $mygame = strstr($_smarty_tpl->tpl_vars['game']->value['title'], '-', true); echo($mygame); ?>
When i put this code in my "Single template file", it work fine and trim the title as i want and it work good in every game's single page.
So i want to make a section in single page to display all games made by that company (i mean that trimmed word from title). I tried some codes and nothing! This is what i tried:
<?php
$connect = mysqli_connect("localhost", "dbname", "dbpass", "dbuser");
$connect->set_charset('utf8mb4');
mysqli_set_charset($link, 'utf8mb4');
$gamecompany = strstr($_smarty_tpl->tpl_vars['game']->value['title'], '-', true);
$query = 'SELECT * FROM games WHERE title = "'.$gamecompany.'" ORDER BY game_id ASC LIMIT 50';
$result = mysqli_query($connect, $query);
if(mysqli_num_rows($result) > 0)
{
$output .= '<div class="list">';
while($row = mysqli_fetch_array($result))
{
$output .= '<li class="game">'.$row["title"].'</li>';
}
$output .= '</div>';
echo $output;
}
else
{
echo 'Nothing Found';
}
?>
So i used $gamecompany to trim and get a game's company and use it as a keyword in query. But everytime it just show "Nothing Found". When i have some games with keyword "Rockstar" in my database But it won't display that and just pass the conditions statement and can't show nothing.
Tried another keywords (Directly in my code) but won't work!
And one note: My titles are in "Arabic" language and it should be UTF8. Is this my problem? or just a wrong coding?
Using LIKE you can find all occurences with 'Rockstar', but to be safe, convert it to lower case and remove any extra spaces that might occur. Also, lets protect ourselves from SQL attacks with a prepared statement.
$gamecompany = strtolower(trim(strstr($_smarty_tpl->tpl_vars['game']->value['title'], '-', true))); // put it in lower case, trim any excess white space
$query = 'SELECT * FROM games WHERE LOWER(title) LIKE ? ORDER BY game_id ASC LIMIT 50';
$stmt = $conn->prepare($query);
$value = "%$gamecompany%"; // The % allows us to find any titles that have our search string in them
$stmt->bind_param("s", $value);
$stmt->execute();
$result = $stmt->get_result();
For you requirement
title = "'.$gamecompany.'"
is not going to work. You'll need to either use likewise search or full-text search
Likewise
title like '$gamecompany'
Full-Text - For full-text to work, you'll need to have full-text index for that column
MATCH (title) AGAINST (:gamecompany IN NATURAL LANGUAGE MODE)
You can create Full-text index like this
ALTER TABLE games ADD FULLTEXT(title)
Try using the LIKE keyword inside the query , and for the Arabic part make sure both the web app and the database uses the same encoding , i once had this problem and when both of them followed the same encode it worked out.

How to detect specific keyword from text in a large DB?

I have had at this issue for a day now. From PHP + MYSQL angle. I but because of the amount of data, most all scripts, that I've tried have timed out.
So we have two tables:
People with the row name - about 4000 unique entries
Texts with the row message - about 24 000 entries
Messages have their own format, that names get put into [] tags, like so: [Jenna].
Sadly, not all entries from Texts are correctly formatted. However I do have alot of names in People. So I want to parse trough the Texts->message's and see if any names from People is matched. Of course I do not want to match [Somename], since its already tagged.
Ultimately, the goal is to then do an UPDATE query, so the freshly matched message would be then formatted correctly with [] tag. I don't know if, this could be achieved inside the same single SQL query?!
This is a regex example on, what I want to detect and explanation on what is going on inside preg_match_all(): https://regex101.com/r/cQ6gK5/1
This is what I tried, as advanced MySQL is not my strongest side:
<?
function GetPeople () {
global $DB;
$results = $DB->query("SELECT `name` FROM People");
while ($result = $DB->fetch_array($results)) {
$return[] = $result['name'];
}
return implode('|', $return);
}
$people = GetPeople();
echo '<table><tr><th>Message raw</th><th>Matches</th>';
$results = $DB->query("SELECT `message` FROM Texts WHERE `message` NOT REGEXP '\[(.+?)\]'");
while ($result = $DB->fetch_array($results)) {
if (preg_match_all('/(?:(?:^|[\s])(' . $people . ')[\s|\n])/i', $result['message'], $matches)) {
echo '<tr><td>' . $result['message'] . '</td><td><pre>'; print_r($matches); echo '</pre></td></tr>';
}
}
echo '</table>';
I have indexed out the name and message in MySQL, because I assume, that makes it easier to search. And I imagine, that all this could be done without the php matching and only with SQL query alone. Sadly, I could never get it so optimized as it should be on my own. Any help is highly appreciated, thank you.
You could try something like this:
SELECT texts.message
FROM texts
JOIN people on texts.message LIKE CONCAT('%', people.name, '%');
This will join the two tables and then perform a like comparison based on the 'names' column in the 'people' table.

Database search like google [duplicate]

This question already has answers here:
Google-like Search Engine in PHP/mySQL [closed]
(9 answers)
Closed 1 year ago.
I currently have a search option on my PHP+MYSQL website.
The MYSQL query is currently something like "SELECT pageurl WHERE name LIKE '%$query%'.
The reason I posted here is because I noticed that if the name of one of my products is "Blue Bike" and someone looks for "Bike Blue", no results are returned.
I am looking for a solution to this because I know that if I type on google same word, something appears.
I was thinking to create a PHP function to mix up all the words from the query if the query is having 4 or fewer words, generating around 24 queries.
Is there an easier solution to this?
Thanks for your time
As to not let this go without a working answer:
<?php
$search = 'this is my search';
$searchSplit = explode(' ', $search);
$searchQueryItems = array();
foreach ($searchSplit as $searchTerm) {
/*
* NOTE: Check out the DB connections escaping part
* below for the one you should use.
*/
$searchQueryItems[] = "name LIKE '%" . mysqli_real_escape_string($searchTerm) . "%'";
}
$query = 'SELECT pageurl FROM names' . (!empty($searchQueryItems) ? ' WHERE ' . implode(' AND ', $searchQueryItems) : '');
?>
DB connections escaping
mysqli_:
Keep using mysqli_real_escape_string or use $mysqli->real_escape_string($searchTerm).
mysql_:
if you use mysql_ you should use mysql_real_escape_string($searchTerm) (and think about changing as it's deprecated).
PDO:
If you use PDO, you should use trim($pdo->quote($searchTerm), "'").
use full text search instead of like
full text search based on indexed text and is very faster and beter than using like.
see this article for more information about full text search
What you are looking for is fulltext search.
Try Sphinx, it is very fast and integrates well with MySQL.
Sphinx website
I wrote a function that approaches Google's operation taking into account the double quotes for the elements to search as a whole block. It does NOT take into account the - or * instructions.
table: MySQL table to consider
cols: array of column to parse
searchParams: search to process. For example: red mustang "Florida 90210"
function naturalQueryConstructor($table, $cols, $searchParams) {
// Basic processing and controls
$searchParams = strip_tags($searchParams);
if( (!$table) or (!is_array($cols)) or (!$searchParams) ) {
return NULL;
}
// Start query
$query = "SELECT * FROM $table WHERE ";
// Explode search criteria taking into account the double quotes
$searchParams = str_getcsv($searchParams, ' ');
// Query writing
foreach($searchParams as $param) {
if(strpos($param, ' ') or (strlen($param)<4)) {
// Elements with space were between double quotes and must be processed with LIKE.
// Also for the elements with less than 4 characters. (red and "Florida 90210")
$query .= "(";
// Add each column
foreach($cols as $col) {
if($col) {
$query .= $col." LIKE '%".$param."%' OR ";
}
}
// Remove last ' OR ' sequence
$query = substr($query, 0, strlen($query)-4);
// Following criteria will added with an AND
$query .= ") AND ";
} else {
// Other criteria processed with MATCH AGAINST (mustang)
$query .= "(MATCH (";
foreach($cols as $col) {
if($col) {
$query .= $col.",";
}
}
// Remove the last ,
$query = substr($query, 0, strlen($query)-1);
// Following criteria will added with an AND
$query .= ") AGAINST ('".$param."' IN NATURAL LANGUAGE MODE)) AND ";
}
}
// Remove last ' AND ' sequence
$query = substr($query, 0, strlen($query)-5);
return $query;
}
Thanks to the stackoverflow community where I found parts of this function!
To have a google like search you'd need many database and index nodes, crazy algorithms.. now you come up with a SELECT LIKE ... lol :D
MySQL is slow in searching, you'd need fulltext and index set properly (MyISAM or Aria Engine). Combinations or different entities to search for are almost not implementable properly AND fast.
I'd suggest to setup an Elasticsearch server which is based on Apache's Lucene.
This searchs very fast and is easy to maintain. And you would not have to care about SQL injection and can still use the mysql server fast.
Elasticsearch (or other Lucene based search engines like SolR) can easily be installed on any server because they are written in Java.
Good documentation:
http://www.elasticsearch.org/guide/en/elasticsearch/client/php-api/current/
I would do an explode first:
$queryArray = explode(" ", $query);
and then generate the SQL query something like:
for ($i=0; $i< count($queryArray); $i++) {
$filter += " LIKE '%" + $queryArray[$i] + "%' AND" ;
}
$filter = rtrim ($filter, " AND");
$sql = "SELECT pageurl FROM ... WHERE name " + $filter
(note: haven't tested/run this code)

Specialized Search Query Refinement for Auto-Complete function

I am doing a query for an autocomplete function on a mysql table that has many instances of similar titles, generally things like different years, such as '2010 Chevrolet Lumina' or 'Chevrolet Lumina 2009', etc.
The query I am currently using is:
$result = mysql_query("SELECT * FROM products WHERE MATCH (name) AGAINST ('$mystring') LIMIT 10", $db);
The $mystring variable gets built as folows:
$queryString = addslashes($_REQUEST['queryString']);
if(strlen($queryString) > 0) {
$array = explode(' ', $queryString);
foreach($array as $var){
$ctr++;
if($ctr == '1'){
$mystring = '"' . $var . '"';
}
else {
$mystring .= ' "' . $var . '"';
}
}
}
What I need to be able to do is somehow group things so only one version of a very similar actually shows in the autosuggest dropdown, leaving room for other products with chevrolet in them as well. Currently it is showing all 10 spots filled with the same product with different years, options, etc.
This one should give some of you brainiacs a good workout :)
I think the best way to do this would be to create a new field on the products table, something like classification. All the models would be entered with the same classification (e.g. "Chevrolet"). You could then still MATCH AGAINST name, but GROUP BY classification. Assuming you are using MySQL you can cheat a little and get away with selecting values and matching against values that you are not grouping by. Technically in SQL this gives undefined results and many SQL engines will not even let you try to do this, but MySQL lets you do it -- and it returns a more-or-less random sample that matches. So, for example, if you did the above query, grouped by classification, only one model (picked pretty much at random) will show up in the auto-completer.

Searching Database PHP/MYSQL Question

Right now I'm just using a simple
WHERE name LIKE '%$ser%'
But I'm running into an issue - say the search is Testing 123 and the "name" is Testing, it's not coming back with any results. Know any way to fix it? Am I doing something wrong?
If you want to search for 'Testing' or '123' use OR:
WHERE (name LIKE '%Testing%' OR name LIKE '%123%')
Note however that this will be very slow as no index can be used and it may return some results you didn't want (like "4123"). Depending on your needs, using a full text search or an external database indexing product like Lucene might be a better option.
That's how LIKE works - it returns rows that completely contain the search string, and, if you use "%" optionally contain something else.
If you want to see if the field is contained in a string, you can do it this way:
SELECT * FROM `Table` WHERE "Testing 123" LIKE CONCAT("%",`name`,"%")
As Scott mentioned, you cannot check to see if the search contains the column value, it works the other way round.
so if $ser = "testing" and table has a row name = testing 123 it will return
For what you're trying to do you'll need to tokenize the search query into terms and perform an OR search with each of them or better still check out mysql full text search for a much better approach
After the variable $ser is replaced, the query is:
WHERE name LIKE '%Testing 123%'
You should build the query separating by words:
WHERE name LIKE '%$word[1]%$word[2]%'
not efficient (as your example) but working as you want:
WHERE name LIKE '%$ser%' OR '$ser' LIKE CONCAT('%', name, '%')
As mentioned by Mark and others, a full text search method may be better if possible.
However, you can split the search string on word boundary and use OR logic—but check for the whole string first, then offer the option to widen the search:
NOTE: Input sanitization and preparation not shown.
1. Query with:
$sql_where = "WHERE name LIKE '%$ser%'";
2. If zero results are returned, ask user if they would like to query each word individually.
3. If user requests an 'each word' search, query with:
$sql_where = get_sql_where($ser);
(Working) Example Code Below:
$ser = 'Testing 123';
$msg = '';
function get_sql_where($ser){
global $msg;
$sql_where = '';
$sql_where_or = '';
$ser = preg_replace("/[[:blank:]]+/"," ", trim($ser)); //replace consecutive spaces with single space
$search_words = explode(" ", $ser);
if($search_words[0] == ''){
$msg = 'Search quested was blank.';
}else{
$msg = 'Search results for any of the following words:' . implode(', ', $search_words);
$sql_where = "WHERE name LIKE '%$ser%'";
foreach($search_words as $word){
$sql_where_or .= " OR name LIKE '%$word%'";
}
}
return $sql_where . $sql_where_or;
}
$sql_where = get_sql_where($ser);
//Run query using $sql_where string

Categories