I'm making a search engine which creates a query depending on what's chosen on the search page. Since the query is limited I can not just include anything in it, which is where I have to do the rest with IF clauses and query while-loops. So, there's no age field in the table, but a birthdate field, so I use an IF function to check if the age is correct and then prints out the username(s) in a while loop.
This works alright, but I also need to add two more fields into the search which are as follows: County and City, right now working on the County selection of the page. What I can't figure out by myself is the logic behind how I'm supposed to manage to print out the users that fit all the required fields without having 1000 IF ELSE's.
I thought of SELECTing and filtering out all the correct zip codes to the county/region chosen, then put it in an array, and then validate the output query while-loop against it, but that didn't work so well either.
In my database I have 3 tables which look like this:
county_table
id, name_of_county
municipial_table
id, county_id, municipial_name
zip_code_table
zip, zip_place_name, municipial_id
These are pre-made for my country. So, given the zip code of the user, I will have to do two different SELECT queries to connect it to the county_table (zip->municipial->county).
So basically, what I'm trying to say; I want the search engine to output the users that have the correct data, this depending if the age, region and city fields are selected. They need to be independent and not like:
if($age>X){
if($county==Y){
if($city==Z){
-OUTPUT RESULTS HERE-
} } }
Now, the problem with this is: What if one of the fields are not requested in the search? Say, the age? The county number? The city? I think what I need are non-nested and independent IF blocks, but I'm not sure how to set it up correctly.
Help very much appreciated, thank you alot.
My re-inventing the wheel? Unless you're making a search engine for educational purpose, use something that's already been tested and optimized. See this related question.
.
Related
As i am a junior PHP Developer growing day by day stuck in a performance problem described here:
I am making a search engine in PHP ,my database has one table with 41 column and million's of rows obviously it is a very large dataset. In index.php i have a form for searching data.When user enters search keyword and hit submit the action is on search.php with results.The query is like this.
SELECT * FROM TABLE WHERE product_description LIKE '%mobile%' ORDER BY id ASC LIMIT 10
This is the first query.After result shows i have to run 4 other query like this:
SELECT DISTINCT(weight_u) as weight from TABLE WHERE product_description LIKE '%mobile%'
SELECT DISTINCT(country_unit) as country_unit from TABLE WHERE product_description LIKE '%mobile%'
SELECT DISTINCT(country) as country from TABLE WHERE product_description LIKE '%mobile%'
SELECT DISTINCT(hs_code) as hscode from TABLE WHERE product_description LIKE '%mobile%'
These queries are for FILTERS ,the problem is this when i submit search button ,all queries are running simultaneously at the cost of Performance issue,its very slow.
Is there any other method to fetch weight,country,country_unit,hs_code speeder or how can achieve it.
The same functionality is implemented here,Where the filter bar comes after table is filled with data,How i can achieve it .Please help
Full Functionality implemented here.
I have tried to explain my full problem ,if there is any mistake please let me know i will improve the question,i am also new to stackoverflow.
Firstly - are you sure this code is working as you expect it? The first query retrieves 10 records matching your search term. Those records might have duplicate weight_u, country_unit, country or hs_code values, so when you then execute the next 4 queries for your filter, it's entirely possible that you will get values back which are not in the first query, so the filter might not make sense.
if that's true, I would create the filter values in your client code (PHP)- finding the unique values in 10 records is going to be quick and easy, and reduces the number of database round trips.
Finally, the biggest improvement you can make is to use MySQL's fulltext searching features. The reason your app is slow is because your search terms cannot use an index - you're wild-carding the start as well as the end. It's like searching the phonebook for people whose name contains "ishra" - you have to look at every record to check for a match. Fulltext search indexes are designed for this - they also help with fuzzy matching.
I'll give you some tips that will show useful in many situations when querying a large dataset, or mostly any dataset.
If you can list the fields you want instead of querying for '*' is a better practice. The weight of this increases as you have more columns and more rows.
Always try to use the PK's to look for the data. The more specific the filter, the less it will cost.
An index in this kind of situation would come pretty handy, as it will make the search more agile.
LIKE queries are generally pretty slow and resource heavy, and more in your situation. So again, the more specific you are, the better it will get.
Also add, that if you just want to retrieve data from this tables again and again, maybe a VIEW would fit nicely.
Those are just some tips that came to my mind to ease your problem.
Hope it helps.
First of, I'm pretty new to this site and coding in general so please explain in simple terms as I'm still learning! Thanks
Ok, so I've got a database of results. These are 1-6 ratings. I've already created the ability to retrieve certain results (user, group, all).
But now I'm wanting to alongside retrieving the group and all results to display at the top of the results a mean for each question.
So to start I'm wanting something like this I believe.
SELECT sum(r1), sum(r2), sum(r3) so on,
FROM table
This is where I get confused.
I think I'd need a variable to contain these and then another that counts the amount of entries to divide the total of r1 hence the mean.
Any ideas?..
To calculate a mean, use the AVG function, e.g.
SELECT AVG(r1), AVG(r2)
FROM table
See the MySQL docs.
Well I'm having a problem mainly caused by bad structure in database. I'm coding this for a company whose code is quite messy and the table is very large so I don't think it's an option to fix the structure.
Anyway, my issue is that I'm trying to somehow group a value that won't be alone in the string...
They are storing values separated with commas... So it would be like
field: "category" value: 'var1, var2, var3'
And I will search using this query:
SELECT name, category
FROM companies
WHERE (MATCH(name, category) AGAINST ('$search' IN BOOLEAN MODE)
OR category LIKE '$search%')
It would match with for example var2 (it's not limited to 3 variables though, can be solo or many more) and I'd split it manually in PHP, no problem. Although I will not get enough matches, I want e.g. 10 matches by different searches. To be more specific I'm making an autosuggest feature, which means I will for example want to match "moto%" with motorbike, motor alone or whatever but I keep getting the same values, like there'd be a couple of 100 of results that contains "motorbike" and I don't know how to filter them, as I'm not able to use GROUP BY due to bad db structure...
I found this: T-SQL - GROUP BY with LIKE - is this possible?
It SEEMED as something that would be a solution, but as far as I've tried I could not get it work with what I wanted.
So I'm wondering which solutions there are... If there are ABSOLUTELY no way of working this around I might probably have to fix the db structure (but this really has to be the last option)
Start taking steps to make database structure proper. Make an extra table and fill it with split values.
Then you can use proper queries to select the data you need. Both you and next developer will have less troubles with this project in the future, not mentioning queries speed gain.
I am not sure why i cannot write a comment, but maybe you can try this:
SELECT name, category FROM companies WHERE category LIKE '$search%' or LOCATE('search', category)>0;
That would look if in category appears any of your 'search' value.
I would have to agree that you should make the database right. It'll save you much trouble and time later. However, using SELECT DISTINCT may fix your immediate issue.
In my web application there will be several users. and they have their own contents uploaded to my webapp. For each content they upload it has a title, description and tags(keywords). I can write a search script to search for content or user name. but they keywords when they have given with a spelling mistake it doesn't return any result. For example if there is a user named "Michael" in the database and the search query was "Micheal" i should get "Did you mean to search for 'Michael'" which is none other than a search suggestion.
Also this suggestion should be for the contents uploaded by the user. An user may keep their content's title as "Michael's activities May 2011" and suggestions should be generated for individual words.
You could use SOUNDEX to search for similar-sounding names, like that:
SELECT * FROM users WHERE SOUNDEX(name) = SOUNDEX(:input)
or like that
SELECT * FROM users WHERE name SOUNDS_LIKE :input
(which is completely equivalent)
Edit: if you need to use an algorithm other than Soundex, as Martin Hohenberg suggested, you would need to add an extra column to your table, called, for example, sound_equivalent. (This is actually a more efficient solution as this column can be indexed). The request would then be:
SELECT * FROM users WHERE sound_equivalent = :input_sound_equivalent
The content of the sound_equivalent column can then be generated with a PHP algorithm, and inserted in the table with the rest of user parameters.
You can also use the php library pspell to get suggestions if you have no search results.
Maybe create a database of the most common words (like: dog, house, city, numbers, water, internet). Don't need to make it big (<10000 words).
Then when you explode the search term, check the "word" database for words LIKE the search terms. Then just echo out the suggestions.
I currently work for a social networking website.
My boss recently had the idea to show search results by random instead of normal results (registration date). The problem with that is simple and obvious: if you go from one page to another, it's going to show you different results each time as the list is randomized each time.
I had the idea to store results in database+cookies something like this:
Cookie containing a serialized version of the $_POST request (needed if we want to do a re-sort)
A table which would serve as the base for the search id => searches (id,user_id, creation_date)
A table which would store the results and their order => searches_results (search_id, order, user_id)
Flow chart would look like something like that:
After each searches I store the "where" into a cookie or session
Then I erase the previous search in "searches"
Then I delete previous results in "searches_results"
Then I insert a row into "searches" for the key
Then I insert each user row into "searches_results"
And finally I redirect the user to somethink like ?search_id=[search_key]
There is a big flaw here : performances .... it is definetly possible to make the system OR down OR very slow.
Any idea what would be the best to structure this ?
What if instead of ordering randomly, you ordered by some function where the order is known and repeatable, just non-obvious? You could seed such a function with some data from the search query to make it be even less obvious that it repeats. This way, you can page back and forth through your results and always get what you expect. Music players use this sort of function for their shuffle feature (so that if you click back, you get the previous song, and if you click next again, you're back where you started). I'm sure you can divine some function to accomplish this... bitwise XORing ID values with some constant (from the query) and then sorting by the resulting number might be sufficient. I chose XOR arbitrarily because it's a trivially simple function that will get you repeatable and non-obvious results.
Hum maybe, but doesn't the xor operator only say if it is an OR exclusive ? I mean, there is no mathematical operation here, as far as I know of tho.
Sorry, I know this doesn't help, but I don't understand why your boss would want this?
I know that if I search for a person on a social network, then I want the results to be ordered by relevance and relevance only. I think that randomized results would just be frustrating for the user, but maybe that's just me.
For example, if I search for "John Smith", then first first batch of results better be people named "John Smith". Then show me similar names near the end of the results. I don't want to search for "John Smith" and get "Jon Smithers" as my second result.
Well, I'm with Matt in asking "Why?"
I think rmeador has a good suggestion as well. You could randomly sort by a different field or some sort of algorithm. Just from the permutations of DESC / ASC on last updated or some other result field.
Other option would be to do an initial search the first time and return only related ID's and then store the full ID's string in the database and each subsequent page is then a lookup against those ID's.
My two cents.
I can see a scenario where a randomized result set is useful but not for searching but for browsing profiles or artists or local events. It offers more exposure to those that wouldn't show up in a traditionally directed search.