How to make a full text search engine with php and mysql? - php

I have created a php search engine by using an open source code. I have created a database and a table with 5 columns:
title, description, keywords, link, date
The code I'm using is the php code below:
<?php
$keywords = $_GET ['input'];
$keywords = "Led Zeppelin";
$words = explode(" ",$keyword);
foreach ($words as $w) {
$keyword_parts .= "+" . $w ." ";
}
$keyword_parts = substr($keyword_parts,0,-1);
$query = "SELECT
*,
MATCH (keywords,title) AGAINST ('" . $keywords . "') as score
FROM search
WHERE MATCH (keywords,title) AGAINST ('" . $keyword_parts . "' IN BOOLEAN MODE)
ORDER BY score DESC";
//connect
mysql_connect ("localhost", "root", "password");
mysql_select_db("intranet");
$query = mysql_query($query);
$numrows = mysql_num_rows ($query);
if($numrows >0){
while ($row = mysql_fetch_assoc($query)){
$id = $row ['id'];
$title = $row ['title'];
$description = $row ['description'];
$keywords = $row ['keywords'];
$link = $row ['link'];
$date = $row ['date'];
echo "<h2><a href='$link'>$title</a><h2/>
$description <br /><br />";
}
}
else
echo "No results found for \"<b>$each</b>\" ";
//disconnect
mysql_close();
?>
The search script works fine, but the only problem is that it searches by keyword which increases the search result, but makes it hard to find what you want because a lot of keywords on different entries match.
Now, I was reading online about full text search ("Full-text searching is performed using MATCH() ... AGAINST syntax. MATCH()") but I don't know how to apply that to my code.

This is what I've used in the past. Also, the column you're doing a MATCH on must be altered to allow FULLTEXT:
$keyword = "Led Zeppelin";
$words = explode(" ",$keyword);
$keyword_parts = '';
foreach ($words as $w) {
$keyword_parts .= "+" . $w ." ";
}
$keyword_parts = substr($keyword_parts,0,-1);
$query = "SELECT
*,
MATCH (column1,column2) AGAINST ('" . $keyword . "') as score
FROM table
WHERE MATCH (column1,column2) AGAINST ('" . $keyword_parts . "' IN BOOLEAN MODE)
ORDER BY score DESC";

Related

Search query array value binding not working

I'm working on a search query and i hit a little bump... So as you see in the code below, i'm adding values to a array to execute it later in the script, but it's not really working... So when i var_dumped all of this, it returned like it is supposed to but the :q was not changed to the value which was entered in the link.
$query = "SELECT * FROM articles";
$columnsQuery = [];
$values = [];
if(isset($_GET['q']) && !empty($_GET['q']))
{
$columnsQuery[] = " WHERE MATCH (title) AGAINST (':q' IN NATURAL LANGUAGE MODE)";
$values[":q"] = $_GET['q'];
}
$fullQuery = $query . implode(" ", $columnsQuery)
. " ORDER BY id DESC"
. " LIMIT {$paginator->getLimitSQL()}";
$getArticles = $db->prepare($fullQuery)->execute($values);
$query = "SELECT * FROM articles";
$columnsQuery = [];
$values = [];
if(isset($_GET['q']) && !empty($_GET['q']))
{
$columnsQuery[] = " WHERE MATCH (title) AGAINST (':q' IN NATURAL LANGUAGE MODE)";
$values["q"] = $_GET['q']; // TRY WITHOUT COLON
}
$fullQuery = $query . implode(" ", $columnsQuery)
. " ORDER BY id DESC"
. " LIMIT {$paginator->getLimitSQL()}";
$getArticles = $db->prepare($fullQuery)->execute($values);
You should not use colon in the place of $values["q"] = $_GET['q'];
$query = "SELECT * FROM articles";
$columnsQuery = [];
$values = [];
if(isset($_GET['q']) && !empty($_GET['q']))
{
$columnsQuery[] = " WHERE MATCH (title) AGAINST (':q' IN NATURAL LANGUAGE MODE)";
$values["q"] = $_GET['q']; // TRY WITHOUT COLON
}
$fullQuery = $query . implode(" ", $columnsQuery)
. " ORDER BY id DESC"
. " LIMIT {$paginator->getLimitSQL()}";
$getArticles = $db->prepare($fullQuery)->execute($values);
$query = "SELECT * FROM articles";
$values = array();
if(!empty($_GET['q'])) {
$query .= " WHERE MATCH (title) AGAINST (q IN NATURAL LANGUAGE MODE)";
$db->bindParam(':q', $_GET['q']);
}
$fullQuery = $query . " ORDER BY id DESC" . " LIMIT {$paginator->getLimitSQL()}"
$getArticles = $db->prepare($fullQuery)->execute();
So after a while i figured it out, You're not supposed to use parameters while binding in the query, and like #Poiz pointed out i shouldnt use colons in the array either
Thx to everyone who tried helping :)

Output of DISTINCT and <> from SQL Query issue

I am obtaining some values from an array and making a match against these values in an SQL Query.
The code for this is as follows:
foreach($files as $ex){
$search = substr($ex,3,4);
echo $search . '<br>';
echo '<br>';
$sql = 'SELECT DISTINCT `pdb_code` FROM pdb WHERE `pdb_code` <> "' . $search . '" LIMIT 4';
}
$result = mysql_query($sql) or die(mysql_error());
while($row = mysql_fetch_array($result)){
echo 'SQL' . $row['pdb_code'] .'<br>';
$pdb[] = $row['pdb_code'];
}
The issue that I am having is that the <> seems not to be working.. I have even tried using the != operator, but still having the same issue.
The output of $search from the array are :
101m
102l
102m
103l
The output of SQL from the query is still:
101m
102l
102m
103l
Your code doesn't seem that logical, as you generate numerous SQL statements and then just execute the last one.
However I assume what you want to do is take a list of files, extract a string from each file name and then list all the pdb_code values from the table which are not already in the string.
If so something like this would do it. It takes each file name, extracts the sub string and escapes it, putting the result into an array. Then it builds one query, imploding the array to use in a NOT IN clause:-
<?php
$search_array = array();
foreach($files as $ex)
{
$search = substr($ex,3,4);
echo $search . '<br>';
echo '<br>';
$search_array[] = mysql_real_escape_string($search);
}
if (count($search_array) > 0)
{
$sql = "SELECT DISTINCT `pdb_code` FROM pdb WHERE `pdb_code` NOT IN ('" . implode("','", $search_array) . "') LIMIT 4";
$result = mysql_query($sql) or die(mysql_error());
while($row = mysql_fetch_array($result))
{
echo 'SQL' . $row['pdb_code'] .'<br>';
$pdb[] = $row['pdb_code'];
}
}
You have to use not in:
SELECT * FROM table_name WHERE column_name NOT IN(value1, value2...)
Try this:
$searchIds = implode(',',$search);
$sql = "SELECT DISTINCT `pdb_code` FROM pdb WHERE `pdb_code` NOT IN ('$searchIds') LIMIT 4";

How can I scan all my fields in MySQL for form input

In my script below, the user inputs a form and rows are returned from a MYSQL table if rows are similar to inputted by the user. I am building a search engine and everything is based on rank. But I want to be able to adjust the code below to see how many times the word 'iPad' for example comes up with the row fields, which are 'title', 'description', 'keywords' and 'link'. If so, I want that row to return higher than say a row that has a higher id, but only mentions iPad once in all of the fields combined.
My code is below:
Terms together query:
$query = " SELECT * FROM scan WHERE ";
$terms = array_map('mysql_real_escape_string', $terms);
$i = 0;
foreach ($terms as $each) {
if ($i++ !== 0){
$query .= " AND ";
}
$query .= "title LIKE '%{$each}%' OR link LIKE '%{$each}%' OR keywords LIKE '%{$each}%' OR description LIKE '%{$each}%' ";
}
$query = mysql_query($query) or die('MySQL Query Error: ' . mysql_error( $connect ));
echo '<p class="time">Qlick showed your results in ' . number_format($secs,2) . ' seconds.</p>';
$numrows = mysql_num_rows($query);
if ($numrows > 0) {
while ($row = mysql_fetch_assoc($query)) {
$id = $row['id'];
$title = $row['title'];
$description = $row['description'];
$keywords = $row['keywords'];
$link = $row['link'];
$rank = $row['rank'];
Seperate Terms Query
$query = " SELECT * FROM scan WHERE ";
$terms = array_map('mysql_real_escape_string', $terms);
$i = 0;
foreach ($terms as $each) {
if ($i++ !== 0){
$query .= " OR ";
}
$query .= "title LIKE '%{$each}%' OR link LIKE '%{$each}%' OR keywords LIKE '%{$each}%' OR description LIKE '%{$each}%' ";
}
// Don't append the ORDER BY until after the loop
$query = mysql_query($query) or die('MySQL Query Error: ' . mysql_error( $connect ));
$numrows = mysql_num_rows($query);
if ($numrows > 0) {
while ($row = mysql_fetch_assoc($query)) {
$id = $row['id'];
$title = $row['title'];
$description = $row['description'];
$keywords = $row['keywords'];
$link = $row['link'];
$rank = $row['rank'];
I'd try to do this using an auxiliary field on which to run a FULLTEXT query, in which you would save all textual data:
http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
The alternative is to run the filtering in MySQL and the ranking in PHP. You might squeeze some performance by running a single LIKE on the concatenated field.
By the way, your code above lacks parentheses in the LIKE, so that the results won't be correct: you mustn't ask WHERE field1 LIKE 'x' OR field2 LIKE 'x' AND field1 LIKE 'y' OR..., you must state WHERE (field1 LIKE 'x' OR field2 LIKE 'x') AND (field1 LIKE 'y' OR...).
// Here we search for ALL terms (all must be present at least once)
// use ' OR ' to ask that at least one term must be present once.
$where = array();
foreach($terms as $term)
$where[] = "( CONCAT(title,'|',link,'|',keywords) LIKE '%{$term}%')";
$query .= ' WHERE ' . '('.implode(' AND ', $where).')';
Now in the OR case you could do a simple ranking on the number of matched terms (with AND the number is always the total number of terms):
$select_fields[] '(' . implode ('+', $where) . ') AS ranking';
Otherwise in SQL you would need recourse to a really ugly hack:
(LENGTH(
REPLACE(CONCAT(title,'|',link,'|',keywords),'{$term}','')
) - LENGTH(CONCAT(title,'|',link,'|',keywords)))/LENGTH('{$term}');
This above calculates the difference between the total length of the text where the search is to be done and the total length of the same text, with search string removed. The difference is of course proportional to how many times the search string is there: if the string is 8 characters long, a difference of 32 would mean that it is present four times. Dividing the length difference by the length of the term, we obtain the number of hits.
The problem is that for several terms you have to complicate the query enormously, and it might be really expensive to run:
$select_fields = array('*');
$where = array();
$rank = array();
foreach($terms as $term)
{
// assume $term is NOT QUOTED
$search = mysql_real_escape_string($term);
$concat = "CONCAT(title,'|',link,'|',keywords)";
$where[] = "(${concat} LIKE '%{$search}%')";
$rank[] = "(LENGTH(REPLACE(${concat},'{$search}',''))
- LENGTH(${concat}))/LENGTH('{$search}')";
}
$select_fields[] = "(".implode(",", $rank).") AS ranking";
$query .= "SELECT " . implode(',', $select_fields)
. ' FROM scan WHERE (' . implode(' AND ', $where) . ')';

Pass set of array into MySQL query as parameters

what am trying to do here is get data from mysql generated checkboxes. The check boxes data are in array . So for all the check boxes selected I want to use each as a parameter in a query to get more info from another databastablee.
Sample Code Below
if (isset($_POST['submitCourseCode'])) {
//GET ARRAY FROM DATABASE GENERATED CHECKBOXES
$aElective = $_POST['electiveModules'];
foreach($aElective as $snode) {
echo "$snode <br />";
}
//PASSING EACH DATA FROM ARRAY INTO QUERY
$Query = "SELECT ID,title,credits
FROM module
WHERE ID IN('" . implode("', '", $aElective) ."')";
$Result = mysql_query($Query)
or die ("Query failed: " . mysql_error() . " Actual query: " . $Query);
while ($Row = mysql_fetch_array($Result)) {
$id = htmlentities($Row['ID']);
$title = htmlentities($Row['title']);
$credits = $Row['credits'];
echo "<ul>" . $id . " " . $title . " " . $credits . "</ul>";
}
}
var_dump($Query);
var_dump($Result);
var_dump($Row);
Screenshot of my result
Am guessing something is happening with my query probably because of the implode function but everything seems fine in my query.Any suggestions on what am doing wrong?
You need to trim array elements using array_map('trim', $aElective) as:
$Query = "SELECT ID,title,credits
FROM module
WHERE ID IN('" . implode("','", array_map('trim', $aElective)) ."')";
There is a problem with white spaces before your IDs in IN clause.
Try to add a str_replace() call
$Query = "SELECT ID,title,credits
FROM module
WHERE ID IN('" . str_replace(" ", "", implode("', '", $aElective)) ."')";

array_unique question

I have a search engine type website. It takes the users input, stores the query as $q, explodes the query and searches the database. It then displays the results with the name and web address of each result.
For example, if i searched for "computer programming"... Stack Overflow, stackoverflow.com would be my result. However, it displays twice. (once for computer, and once for programming.)
I tried to solve this with the array_unique function, and it does not work.
any help would be appreciated.
// trim whitespace
$trimmed = trim($q);
// seperate key-phrases
$trimmed_array = explode(" ", $trimmed);
// remove duplicates
$clean_array = array_unique($trimmed_array);
//query dataabase
foreach ($clean_array as $trimm){
$query = mysql_query("SELECT * FROM forumlist WHERE `keys` LIKE '%" . mysql_real_escape_string($trimm) . "%' ORDER BY rating DESC, total_ratings DESC") or die(mysql_error());
Thank you!
//query dataabase
$query = 'SELECT * FROM forumlist ';
$where = array();
foreach ($clean_array as $trimm){
$where[] = " `keys` LIKE '%" . mysql_real_escape_string($trimm) . "%' ";
}
if(!empty($where)){
$query .= " WHERE ". implode(' OR ', $where);
}
$query .= " ORDER BY rating DESC, total_ratings DESC";
$result = mysql_query($query) or die(mysql_error());

Categories