How to make search autocomplete faster? - php

Question

I have implemented a basic auto-complete feature using jQuery autocomplete. I am querying DB every time which is making auto-complete thing quite slow. I am looking for ways to make it faster much like Quora.
Here is the code from front-end:
<script type="text/javascript">
var URL2 = '<?php e(SITE_URL); ?>fronts/searchKeywords';
jQuery(document).ready(function(){
var CityKeyword = jQuery('#CityKeyword');
CityKeyword.autocomplete({
minLength : 1,
source : URL2
});
});
</script>
Here is the code from server side:
function searchKeywords(){
if ($this->RequestHandler->isAjax() ) {
$this->loadModel('Expertise_area');
Configure::write ( 'debug',0);
$this->autoRender=false;
$expertise=$this->Expertise_area->find('all',array(
'conditions'=>array('Expertise_area.autocomplete_text LIKE'=>'%'.$_GET['term'].'%'),
'fields' => array('DISTINCT (Expertise_area.autocomplete_text) AS autocomplete_text'),
'limit'=>5
));
$i=0;
if(!empty($expertise)){
$len = strlen($_GET['term']);
foreach($expertise as $valueproductname){
$pos = stripos($valueproductname['Expertise_area']['autocomplete_text'],$_GET['term']);
$keyvalue = "";
if($pos == 0) {
$keyvalue= "<strong>".substr($valueproductname['Expertise_area']['autocomplete_text'],$pos,$len)."</strong>"
.substr($valueproductname['Expertise_area']['autocomplete_text'],$len);
}else {
$keyvalue= substr($valueproductname['Expertise_area']['autocomplete_text'],0,$pos)."<strong>"
.substr($valueproductname['Expertise_area']['autocomplete_text'],$pos,$len)."</strong>"
.substr($valueproductname['Expertise_area']['autocomplete_text'],$pos+$len);
}
$response[$i]['value']=$valueproductname['Expertise_area']['autocomplete_text'];
$response[$i]['label']="<span class=\"username\">".$keyvalue."</span>";
$i++;
}
echo json_encode($response);
}else{
}
}
}
I have researched a bit and so far following solutions are worth looking at:
Query data on page load and store it in COOKIE to be used in future.
Implement some caching mechanism (memcache??). But my website is on Cakephp which does it internal cahcing if I am right. So will it be worth to go in this direction.
Use some third party indexing mechanism like Solr, Lucene etc. Don't know much about this.
Implement a much complex "Prefix Search" myself
What is the right way to go about it? Please help me out here.

Answer 1

I've never tried this but will be doing it soon for a project I'm working on.
I always considered the possibility of during the initial page load recieveing some AJAX (or perhaps just including it in the page) the top 10 words for each alphabet letter.. e.g.
A - apples, anoraks, alaska, angela, aha, air, arrgh, any, alpha, america
B - butter, bob etc.....
This way when user presses A-Z you can instantly provide them with 10 of the most popular keywords without any further requests, as you already have them stored in an array in the JS.
I'm not sure of size/memory usage but this could be extended further to handle the first 2 letters, e.g. AA, AB, AC.....BA, BB, BC.... ZA, ZB, ZZ... of course many combinations such as words starting with ZZ won't have any data unless it's a music site and it's ZZ Top! This means it probably won't take up so much memory or bandwidth to send this data during initial page load. Only when the user types the 3rd letter do you need to do any further data lookups/transfers.
You auto-update this data every day, week or whatever depending on site usage and the most popular searches.

Answer 2

I am adding a solution to my question which I figured out after a lot of research.
Problem was:
I was using Ajax to fetch keywords from database every time a user changes text in search box
I was doing a wild card search to match search item within entire strings and not just starting of keywords for ex. "dev" would return "social development", "development" etc
Solution:
I have a fixed array of keywords (200) which is not going to increase exponentially in near future. So, instead of doing complex indexing I am currently sending all keywords in an array.
I am sending this data in an array on page load since it is small. If it becomes large, I will fetch it in background via some ajax in different indexed arrays.
I am using jQuery's Autocomplete widget to do rest of thing for me.
For highlighting search item, I am using a hack by working around __renderItem. (Copied from Stackoverflow. Thanks to that!!)
Code:
function monkeyPatchAutocomplete() { //Hack to color search item
jQuery.ui.autocomplete.prototype._renderItem = function( ul, item) {
var re = new RegExp("(?![^&;]+;)(?!<[^<>]*)(" + this.term + ")(?![^<>]*>)(?![^&;]+;)", "gi");
var t = item.label.replace(re,"<span style='font-weight:bold;color:#434343;'>" +
"$&" +
"</span>");
return jQuery( "<li></li>" )
.data( "item.autocomplete", item )
.append( "<a>" + t + "</a>" )
.appendTo( ul );
};
}
function getKeywords(){
//Function that returns list of keywords. I am using an array since my data is small.
//This function can be modified to fetch data in whatever way one want.
//I intend to use indexed arrays in future if my data becomes large.
var allKeywords = <?php echo json_encode($allKeywords); ?>;
return allKeywords;
}
jQuery(document).ready(function(){
monkeyPatchAutocomplete();
var CityKeyword = jQuery('#CityKeyword');
CityKeyword.autocomplete({
minLength : 1,
source : getKeywords()
});
});