My site acts like a search engine where people enter search queries on the main page. I wanted to make a trending / recent feature where each query gets recorded into a mysql database, then from that data, calculates which searches are being searched the most, and then displayed back on the page labeled as trending searches. Also, under that, I would like "recent searches" which simply displays the last 5 or so searches.
Honestly, I have no experience with mysql. I don't even know how to move data from my site to mysql. Any help would be appreciated. I searched and searched these questions and google, but didn't find anything. Thanks!
First of all, you need to CREATE a DATABASE, in which you want a table with a timestamp and the keyword that's been searched. (CREATE TABLE)
Then you want to store each keyword access into this table (INSERT INTO ... VALUES ...)
Then you can select the top key words by creating a SELECT query with a "GROUP BY keyword", ORDER ing by COUNT(*) (the number of occurrences of a keyword)
This is a bit vague, but you'll need to go through a number of steps so I've uppercased the terms you'd need to google for each step. Do come back if you run into complications in any of those steps!
Related
As i am a junior PHP Developer growing day by day stuck in a performance problem described here:
I am making a search engine in PHP ,my database has one table with 41 column and million's of rows obviously it is a very large dataset. In index.php i have a form for searching data.When user enters search keyword and hit submit the action is on search.php with results.The query is like this.
SELECT * FROM TABLE WHERE product_description LIKE '%mobile%' ORDER BY id ASC LIMIT 10
This is the first query.After result shows i have to run 4 other query like this:
SELECT DISTINCT(weight_u) as weight from TABLE WHERE product_description LIKE '%mobile%'
SELECT DISTINCT(country_unit) as country_unit from TABLE WHERE product_description LIKE '%mobile%'
SELECT DISTINCT(country) as country from TABLE WHERE product_description LIKE '%mobile%'
SELECT DISTINCT(hs_code) as hscode from TABLE WHERE product_description LIKE '%mobile%'
These queries are for FILTERS ,the problem is this when i submit search button ,all queries are running simultaneously at the cost of Performance issue,its very slow.
Is there any other method to fetch weight,country,country_unit,hs_code speeder or how can achieve it.
The same functionality is implemented here,Where the filter bar comes after table is filled with data,How i can achieve it .Please help
Full Functionality implemented here.
I have tried to explain my full problem ,if there is any mistake please let me know i will improve the question,i am also new to stackoverflow.
Firstly - are you sure this code is working as you expect it? The first query retrieves 10 records matching your search term. Those records might have duplicate weight_u, country_unit, country or hs_code values, so when you then execute the next 4 queries for your filter, it's entirely possible that you will get values back which are not in the first query, so the filter might not make sense.
if that's true, I would create the filter values in your client code (PHP)- finding the unique values in 10 records is going to be quick and easy, and reduces the number of database round trips.
Finally, the biggest improvement you can make is to use MySQL's fulltext searching features. The reason your app is slow is because your search terms cannot use an index - you're wild-carding the start as well as the end. It's like searching the phonebook for people whose name contains "ishra" - you have to look at every record to check for a match. Fulltext search indexes are designed for this - they also help with fuzzy matching.
I'll give you some tips that will show useful in many situations when querying a large dataset, or mostly any dataset.
If you can list the fields you want instead of querying for '*' is a better practice. The weight of this increases as you have more columns and more rows.
Always try to use the PK's to look for the data. The more specific the filter, the less it will cost.
An index in this kind of situation would come pretty handy, as it will make the search more agile.
LIKE queries are generally pretty slow and resource heavy, and more in your situation. So again, the more specific you are, the better it will get.
Also add, that if you just want to retrieve data from this tables again and again, maybe a VIEW would fit nicely.
Those are just some tips that came to my mind to ease your problem.
Hope it helps.
I am working on implementing search in my Social Networking website and I have this problem.
Say, User "A" is searching for "John" and if that user is a friend of User "A" or friend of friends or friend of friend of friend... recursion level goes till 100, I will be able to find it with search.
What if User "A" has just signed up and has no friends and he is searching for "John" and I have no proper info I can use to filter records? I will have to search my entire database for "John" (ofcourse I am limiting the total search result to 5 using MySQL LIMIT clause).
Is this method efficient or is there anything else I can do to avoid this problem?
And also, First Name, Middle Name, Last Name cannot be set as index since they are not unique. So, I am searching a non-indexed column (without preindexing (Google does preindexing I think) ) and I am using MySQL LIKE for this. So, what should I do about it when considering improving performance?
Using MySQL, PHP. Thanks in advance
Here are some of the problems I see with what you are doing..
By using limit 5 on your Query you are always going to show the first or last 5 John's that the query finds (based on the query).. I would request new users to add their 1st few friends by email (which is unique) or first and last name (non unique)
once the new user has multiple friends you will be able to provide better search results and will be able to avoid always showing the same people to everyone who uses the same keyword/name
First of, I'm pretty new to this site and coding in general so please explain in simple terms as I'm still learning! Thanks
Ok, so I've got a database of results. These are 1-6 ratings. I've already created the ability to retrieve certain results (user, group, all).
But now I'm wanting to alongside retrieving the group and all results to display at the top of the results a mean for each question.
So to start I'm wanting something like this I believe.
SELECT sum(r1), sum(r2), sum(r3) so on,
FROM table
This is where I get confused.
I think I'd need a variable to contain these and then another that counts the amount of entries to divide the total of r1 hence the mean.
Any ideas?..
To calculate a mean, use the AVG function, e.g.
SELECT AVG(r1), AVG(r2)
FROM table
See the MySQL docs.
I have a large INNODB database with over 2 million products on it. The 'products' table has the following fields: id,title,description,category.
There is also a MyISAM table called 'category' that contains a list of all categories used on the website. This has the following fields: id,name,keywords,parentid.
My question is more about the logic rather than code, but what I am trying to achieve is as follows:
When a user lists a new product on the site, as they are typing the description it should try to work out what category to put the product in (with good accuracy).
I tried this initially by using MySQL MATCH() to match the entered title against a list of keywords in the category table, but this was far from accurate.
A better idea seems to be to match the user entered title against titles for products already in the database, grouping them by the category they are in and then sorting them by the largest group. However, on an INNODB database I obviously can't use fulltext, and with 2mill items I think it would be pretty slow anyway?
How would you do it - I guess it would need to be a similar way to how stackoverflow displays similar questions?
A fulltext index on 2 million records is a valid option, if you are running on a decent server. The inital indexing will take a while, that's for sure, but searches should be reasonably fast, MySQL can take it.
InnoDB supports fulltext indexes as of v5.6.4. You should consider upgrading.
If upgrading is not an option, please see this previous answer of mine where I suggest a workaround.
For your use case, you may want to take a look at the WITH QUERY EXPANSION option:
It works by performing the search twice, where the search phrase for the second search is the original search phrase concatenated with the few most highly relevant documents from the first search. Thus, if one of these documents contains the word “databases” and the word “MySQL”, the second search finds the documents that contain the word “MySQL” even if they do not contain the word “database”
I'm trying to write a search script for my website. And i'm having two issues to battle with.
1) In a situation whereby the user types a sentence in the search box, how can I handle that? Should I just go on and take it like a single string and search for it in my desired table? If I do wont it affect the effectiveness of the website.? And most time return no result..
2) If i'm to search in several tables how can I acheive this.
Thanks for you time and patience.
Example:
Assuming i have the following tables in my db
about Table
about_id
about_head
about_content
about_tags
about_created
about_cs Table
about_cs_id
about_cs_head
about_cs_content
about_cs_tags
about_cs_created
home Table
home_id
home_head
home_content
home_tags
home_created
I'm planning to have different criteria for the users to base their search on..
Search By Date: ( Date picker... Date will be queried against the date a particular article was created. In all the tables )
Search by Tags: ( A single (one or two) word search to search all the tables with a tag column in it )
Search Main Article: ( this is where the user will be allowed to enter any type of search query (string) and it'll be search for in the all the tables that have a content field in it )
So if the User enters a word like "The Most Famous Church In South Bend"....
Do I just search for thw whole string
LIKE % ' .$searchString. ' %
or is there a way I can break it down and search for each word seperately?
Information is not clear enough. I am assuming you need site wide search.
In that case, I would suggest you to go with php lucene implementation Zend_Lucene
[EDIT]
Since you have to search very few tables, I would suggest you to go with full text search.
here is very good tutorial to move on.
http://devzone.zend.com/26/using-mysql-full-text-searching/