I'm trying to write a search script for my website. And i'm having two issues to battle with.
1) In a situation whereby the user types a sentence in the search box, how can I handle that? Should I just go on and take it like a single string and search for it in my desired table? If I do wont it affect the effectiveness of the website.? And most time return no result..
2) If i'm to search in several tables how can I acheive this.
Thanks for you time and patience.
Example:
Assuming i have the following tables in my db
about Table
about_id
about_head
about_content
about_tags
about_created
about_cs Table
about_cs_id
about_cs_head
about_cs_content
about_cs_tags
about_cs_created
home Table
home_id
home_head
home_content
home_tags
home_created
I'm planning to have different criteria for the users to base their search on..
Search By Date: ( Date picker... Date will be queried against the date a particular article was created. In all the tables )
Search by Tags: ( A single (one or two) word search to search all the tables with a tag column in it )
Search Main Article: ( this is where the user will be allowed to enter any type of search query (string) and it'll be search for in the all the tables that have a content field in it )
So if the User enters a word like "The Most Famous Church In South Bend"....
Do I just search for thw whole string
LIKE % ' .$searchString. ' %
or is there a way I can break it down and search for each word seperately?
Information is not clear enough. I am assuming you need site wide search.
In that case, I would suggest you to go with php lucene implementation Zend_Lucene
[EDIT]
Since you have to search very few tables, I would suggest you to go with full text search.
here is very good tutorial to move on.
http://devzone.zend.com/26/using-mysql-full-text-searching/
Related
I have a search on my website and im trying to show a list of the most popular search terms on my site it sort of works but it isn't matching the strings close enough.
This is what i'm currently using:
$sql = "SELECT * FROM db WHERE r_name LIKE '%".$searchname."%' OR r_number like '%".$searchname."%'
However if a user searches for say Game Name and another searches for Game Name (Reviews) it will add 2 entries into my database, Is there a way to do a similarity test before entering the entry ?
yes, there is a way but it requires you to fiddle with it a little. there is a search called
SOUNDEX, which will match things that are very close. It might not be a perfect solution to your question, but it is definitely something that might get you started in the right direction.
SELECT * FROM db WHERE SOUNDEX( db.r_name ) LIKE SOUNDEX( '{$searchname}' );
I believe that if you have an entry 'lowercasexd' , and do soundex like('what is lowercasexd?'), it will find the entry that's associated with 'lowercasexd'.
Be aware that this type of search take a little while to run compare to '=' searches on indexed databases(on my database it does about 5-6k entries per second) so it is NOT recommended for anything big. If you want a near-perfect solution, I suggest you read about google's search mechanism, and look up some search engine source code if your project is significant enough.
I have a large INNODB database with over 2 million products on it. The 'products' table has the following fields: id,title,description,category.
There is also a MyISAM table called 'category' that contains a list of all categories used on the website. This has the following fields: id,name,keywords,parentid.
My question is more about the logic rather than code, but what I am trying to achieve is as follows:
When a user lists a new product on the site, as they are typing the description it should try to work out what category to put the product in (with good accuracy).
I tried this initially by using MySQL MATCH() to match the entered title against a list of keywords in the category table, but this was far from accurate.
A better idea seems to be to match the user entered title against titles for products already in the database, grouping them by the category they are in and then sorting them by the largest group. However, on an INNODB database I obviously can't use fulltext, and with 2mill items I think it would be pretty slow anyway?
How would you do it - I guess it would need to be a similar way to how stackoverflow displays similar questions?
A fulltext index on 2 million records is a valid option, if you are running on a decent server. The inital indexing will take a while, that's for sure, but searches should be reasonably fast, MySQL can take it.
InnoDB supports fulltext indexes as of v5.6.4. You should consider upgrading.
If upgrading is not an option, please see this previous answer of mine where I suggest a workaround.
For your use case, you may want to take a look at the WITH QUERY EXPANSION option:
It works by performing the search twice, where the search phrase for the second search is the original search phrase concatenated with the few most highly relevant documents from the first search. Thus, if one of these documents contains the word “databases” and the word “MySQL”, the second search finds the documents that contain the word “MySQL” even if they do not contain the word “database”
I want to create an autosuggest for a fulltext search with AJAX, PHP & MySQL.
I am looking for the right way to implement the backend. While the user is typing, the input field should give him suggests. Suggests should be generated from text entrys in a table.
Some information for this entrys: They are stored in fulltext, generated from PDF with 3-4 pages each. There not more than 100 entrys for now and will reach a maximum of 2000 in the next few years.
If the user starts to type, the word he is typing should be completed with a word which is stored in the DB, sorted by occurrences descending. Next step is to suggest combinations with other words, witch have a high occurrence in the entrys matching the first word. Surely you can compare it to Google autosuggest.
I am thinking about 3 different ways to implement this:
Generate an index via cronjob, witch counts occurrences of words and combinations over night. The user searches on this index.
I do a live search within the entrys with an 'LIKE "%search%"' function. Then I look for the word after the this and GROUP them by occurrence.
I create a logfile for all user searches, and look for good combinations like in 1), so the search gets more intelligent with each search action.
What is the best way to start with this? The search should be fast and performant.
Is there a better possibility I did not think about?
I'd use mysql's MATCH() AGAINST() (http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html), eg:
SELECT *
FROM table
WHERE MATCH(column) AGAINST('search')
ORDER BY MATCH(column) AGAINST('search')
Another advantage is that you could further tweak the importance of words being searched for (if neccessary), like:
MATCH(column) AGAINST('>important <lessimportant') IN BOOLEAN MODE
Or say that certain words of the search term are to be required, whilst others may not be present in the result, eg:
MATCH(column) AGAINST('+required -prohibited') IN BOOLEAN MODE
I think, the idea no 1 is the best. By the way, dont't forget to eliminate stopwords from autosuggest (an, the, by, ...).
In my web application there will be several users. and they have their own contents uploaded to my webapp. For each content they upload it has a title, description and tags(keywords). I can write a search script to search for content or user name. but they keywords when they have given with a spelling mistake it doesn't return any result. For example if there is a user named "Michael" in the database and the search query was "Micheal" i should get "Did you mean to search for 'Michael'" which is none other than a search suggestion.
Also this suggestion should be for the contents uploaded by the user. An user may keep their content's title as "Michael's activities May 2011" and suggestions should be generated for individual words.
You could use SOUNDEX to search for similar-sounding names, like that:
SELECT * FROM users WHERE SOUNDEX(name) = SOUNDEX(:input)
or like that
SELECT * FROM users WHERE name SOUNDS_LIKE :input
(which is completely equivalent)
Edit: if you need to use an algorithm other than Soundex, as Martin Hohenberg suggested, you would need to add an extra column to your table, called, for example, sound_equivalent. (This is actually a more efficient solution as this column can be indexed). The request would then be:
SELECT * FROM users WHERE sound_equivalent = :input_sound_equivalent
The content of the sound_equivalent column can then be generated with a PHP algorithm, and inserted in the table with the rest of user parameters.
You can also use the php library pspell to get suggestions if you have no search results.
Maybe create a database of the most common words (like: dog, house, city, numbers, water, internet). Don't need to make it big (<10000 words).
Then when you explode the search term, check the "word" database for words LIKE the search terms. Then just echo out the suggestions.
My site acts like a search engine where people enter search queries on the main page. I wanted to make a trending / recent feature where each query gets recorded into a mysql database, then from that data, calculates which searches are being searched the most, and then displayed back on the page labeled as trending searches. Also, under that, I would like "recent searches" which simply displays the last 5 or so searches.
Honestly, I have no experience with mysql. I don't even know how to move data from my site to mysql. Any help would be appreciated. I searched and searched these questions and google, but didn't find anything. Thanks!
First of all, you need to CREATE a DATABASE, in which you want a table with a timestamp and the keyword that's been searched. (CREATE TABLE)
Then you want to store each keyword access into this table (INSERT INTO ... VALUES ...)
Then you can select the top key words by creating a SELECT query with a "GROUP BY keyword", ORDER ing by COUNT(*) (the number of occurrences of a keyword)
This is a bit vague, but you'll need to go through a number of steps so I've uppercased the terms you'd need to google for each step. Do come back if you run into complications in any of those steps!