I am doing a query for an autocomplete function on a mysql table that has many instances of similar titles, generally things like different years, such as '2010 Chevrolet Lumina' or 'Chevrolet Lumina 2009', etc.
The query I am currently using is:
$result = mysql_query("SELECT * FROM products WHERE MATCH (name) AGAINST ('$mystring') LIMIT 10", $db);
The $mystring variable gets built as folows:
$queryString = addslashes($_REQUEST['queryString']);
if(strlen($queryString) > 0) {
$array = explode(' ', $queryString);
foreach($array as $var){
$ctr++;
if($ctr == '1'){
$mystring = '"' . $var . '"';
}
else {
$mystring .= ' "' . $var . '"';
}
}
}
What I need to be able to do is somehow group things so only one version of a very similar actually shows in the autosuggest dropdown, leaving room for other products with chevrolet in them as well. Currently it is showing all 10 spots filled with the same product with different years, options, etc.
This one should give some of you brainiacs a good workout :)
I think the best way to do this would be to create a new field on the products table, something like classification. All the models would be entered with the same classification (e.g. "Chevrolet"). You could then still MATCH AGAINST name, but GROUP BY classification. Assuming you are using MySQL you can cheat a little and get away with selecting values and matching against values that you are not grouping by. Technically in SQL this gives undefined results and many SQL engines will not even let you try to do this, but MySQL lets you do it -- and it returns a more-or-less random sample that matches. So, for example, if you did the above query, grouped by classification, only one model (picked pretty much at random) will show up in the auto-completer.
Related
I am trying to design an application and part of it is to show users new articles in different categories after the last visit of the user to the webapp. To this I use MySql and have a table that keeps track of last visits and I can query the table to get a php array like below:
$array =[[user1,category1, datetime1],[user1,category2, datetime2],[user1,category3,datetime3]];
Where user is the user id and datetime is the visited datetime and category is the article category.
Having the setup above, I am trying to get new articles from the article table where the publish date is after user last visited to categories.
I can achieve this by multiple OR in a query like below, however it is not really a good and nice looking query, and probable not scalable. Is there any other way of doing this which is simpler and faster?
$multiwhere=[];
foreach($array as $a){
$multiwhere[]="select article_id from articles where category=".$a[1]." and publish_date>".$a[2];
}
And the final query would be like this:
"Select * from articles where article_id in (".implode(" or ".$multiwhere.")";
I deeply appreciate any suggestion to improve the query above.
Your query is almost correct, apart from the fact that you first retrieve all the article_id you want, and then use them to query for those articles. You can do that in one step, like so:
$multiwhere = [];
foreach ($array as $a) {
$multiwhere[] = "(category = " . $a[1] . " AND publish_date >= " . $a[2] .")";
}
$query = "SELECT * FROM articles";
if (count($multiwhere) > 0) {
$query = " WHERE " . implode(" OR ", $multiwhere);
}
One query will do.
I kept the way you use the $array, but it looks weird to me. Especially around publish_date. I cannot change that because I don't know the type of the field. And, of course, $array is quite a bad name. It tells you what the type of the variable is, not what it contains, as it should. A better name would be: $lastCategoryVisits, or something like that. Your loop should look something like this:
foreach ($lastCategoryVisits as $lastCategoryVisit) {
$category = $lastCategoryVisit["category"];
$lastVisit = $lastCategoryVisit["lastVisit"];
$QueryConditions[] = "(category = '$category' AND publish_date >= '$lastVisit')";
}
Don't be afraid to write out what your code actually does. It might be a bit longer, but now you can see what is going on. This will not slow down the execution of your code at all.
Finally, it would be better to always use prepared statements to prevent the possibility of SQL-injection. If you get into the habit of always doing this you don't have to use excuses like: "It is not important in this project.", "I'll to it later when the code works." or "The data for this query doesn't come from an user.".
I'm currently working on a live search that displays results directly from a mysql db.
The code works, but not really as i want it.
Let's start with an example so that it is easier to understand:
My database has 5 columns:
id, link, description, try, keywords
The script that runs the ajax request on key up is the following:
$("#searchid").keyup(function () {
var searchid = encodeURIComponent($.trim($(this).val()));
var dataString = 'search=' + searchid;
if (searchid != '') {
$.ajax({
type: "POST",
url: "results.php",
data: dataString,
cache: false,
success: function (html) {
$("#result").html(html).show();
}
});
}
return false;
});
});
on the results.php file looks like this:
if ($db->connect_errno > 0) {
die('Unable to connect to database [' . $db->connect_error . ']');
}
if ($_REQUEST) {
$q = $_REQUEST['search'];
$sql_res = "select link, description, resources, keyword from _db where description like '%$q%' or keyword like '%$q%'";
$result = mysqli_query($db, $sql_res) or die(mysqli_error($db));
if (mysqli_num_rows($result) == 0) {
$display = '<div id="explainMessage" class="explainMessage">Sorry, no results found</div>';
echo $display;
} else {
while ($row = $result->fetch_assoc()) {
$link = $row['link'];
$description = $row['description'];
$keyword = $row['keyword'];
$b_description = '<strong>' . $q . '</strong>';
$b_keyword = '<strong>' . $q . '</strong>';
$final_description = str_ireplace($q, $b_description, $description);
$final_keyword = str_ireplace($q, $b_keyword, $keyword);
$display = '<div class="results" id="dbResults">
<div>
<div class="center"><span class="">Description :</span><span class="displayResult">' . $final_description . '</span></div>
<div class="right"><span class="">Keyword :</span><span class="displayResult">' . $final_keyword . '</span></div>
</div>
<hr>
</div>
</div>';
echo $display;
}
}
}
now, let's say that i have this row in my DB:
id = 1
link = google.com
description = it's google
totry = 0
keywords: google, test, search
if i type in the search bar:
google, test
i have the right result, but if i type:
test, google
i have no results, as obviously the order is wrong.
So basically, what o'd like to achieve is something a bit more like "tags", so that i can search for the right keywords without having to use the right order.
Can i do it with my current code (if yes, how?) or i need to change something?
thanks in advance for any suggestion.
PS: I know this is not the best way to read from a DB as it has some security issues, i'm going to change it later as this is an old script that i wrote ages ago, i'm more interested in have this to work properly, and i'm going to change method after.
Normalize your schema
The rules of relational database are very simple (at least the first three).
keywords: google, test, search
...breaks the second rule. Each keyword should be in its own row in a related table. Then you can simply write your query as....
SELECT link, description, resources, keyword
FROM _db
INNER JOIN keywords
ON _db.id=keywords.db_id
WHERE keyword.value IN (" . atomize($q) . ")
(where atomize explodes the query string, applies mysqli_escape_paramter() to each entry to avoid breaking your code, encloses each term in single quotes and concatenates the result).
Alternatively you could use MySQL's full text indexing which does this for you transparently.
Although hurricane makes some good points in his/her answer, they do not mention that none of the solutions proposed there does not scale to handle large volumes of data with any efficiency (decomposing the field into a new table/using full text indexing does).
Untested code but modify according to your needs,
$q = $_REQUEST['search'];
$q_comma = explode(",", $q);
$where_in_set = '';
$count = count($q_comma);
foreach( $q_comma as $q)
{
$counter++;
if($counter == $count) {
$where_in_set .= "FIND_IN_SET('$q','keywords')";
}else {
$where_in_set .= "FIND_IN_SET('$q','keywords') OR ";
}
}
$sql_res = "select link, description, resources, keyword from _db where $where_in_set or description like '%$q%'";
There are 2 solutions I can think of:
Use fulltext index and search.
You can split the search string into words in php for example using explode() and serach for the words not in a single serach criteria, but in separate ones. This latter one can be very resource intensive, since you are seraching in multiple fields.
LIKE '%google, test%' will match id=1 but not '%google,test%' (no space between coma) nor '%google test%' (space delimiter) nor '%test, google%'. Put each keyword as separate table or you can split input keywords into several single keyword and use OR operator such as LIKE 'google%' OR LIKE 'test%'
Not an ideal solution, but instead of treating your search datastring as one element, you can have php treat it as an array of keywords separated by a comma (by using explode). You'd then build a query depending on how many keywords were sent.
For example, using "google, test" your query would be:
$sql_res = "select link, description, resources, keyword from _db where (description like '%$q1%' or keyword like '%$q1%') AND (description like '%$q2%' or keyword like '%$q2%')";
Where $q1 and $q2 are "google" and "test".
First of all as you say it is not a good way to do it. I think you are writing a autocompleter.
Seperators for words
"google, test" or "test, google" is a attached words. First you need to define a seperator for users. Usually it is a whitespace ' '.
When you define it you need to split words.
$words = explode(" ",$q);
// now you get two words "google," and "test"
Then you need to create a sql which gives you multiple search chance.
There are a lot example in MySQL LIKE IN()?
Now you get your result.
Text similarity
Select all result from db and in a while search a text from another text. It gives you a dobule point for similarity. Best result is your result.
Php Similarity Example
Important Info
If you ask my opinion don't use it like that bcs it is very expensive. Use autocompleters on html side. Here is an example
I have had at this issue for a day now. From PHP + MYSQL angle. I but because of the amount of data, most all scripts, that I've tried have timed out.
So we have two tables:
People with the row name - about 4000 unique entries
Texts with the row message - about 24 000 entries
Messages have their own format, that names get put into [] tags, like so: [Jenna].
Sadly, not all entries from Texts are correctly formatted. However I do have alot of names in People. So I want to parse trough the Texts->message's and see if any names from People is matched. Of course I do not want to match [Somename], since its already tagged.
Ultimately, the goal is to then do an UPDATE query, so the freshly matched message would be then formatted correctly with [] tag. I don't know if, this could be achieved inside the same single SQL query?!
This is a regex example on, what I want to detect and explanation on what is going on inside preg_match_all(): https://regex101.com/r/cQ6gK5/1
This is what I tried, as advanced MySQL is not my strongest side:
<?
function GetPeople () {
global $DB;
$results = $DB->query("SELECT `name` FROM People");
while ($result = $DB->fetch_array($results)) {
$return[] = $result['name'];
}
return implode('|', $return);
}
$people = GetPeople();
echo '<table><tr><th>Message raw</th><th>Matches</th>';
$results = $DB->query("SELECT `message` FROM Texts WHERE `message` NOT REGEXP '\[(.+?)\]'");
while ($result = $DB->fetch_array($results)) {
if (preg_match_all('/(?:(?:^|[\s])(' . $people . ')[\s|\n])/i', $result['message'], $matches)) {
echo '<tr><td>' . $result['message'] . '</td><td><pre>'; print_r($matches); echo '</pre></td></tr>';
}
}
echo '</table>';
I have indexed out the name and message in MySQL, because I assume, that makes it easier to search. And I imagine, that all this could be done without the php matching and only with SQL query alone. Sadly, I could never get it so optimized as it should be on my own. Any help is highly appreciated, thank you.
You could try something like this:
SELECT texts.message
FROM texts
JOIN people on texts.message LIKE CONCAT('%', people.name, '%');
This will join the two tables and then perform a like comparison based on the 'names' column in the 'people' table.
I'm trying to create a search engine in php and mysql. The search engine should be able to accept multiple value and display all possible result, i checked this thread php/mysql search for multiple values, but i need to use global variable at the place where LIKE '$search%' is. This is how my sql statement looks like,
SELECT name FROM product WHERE name LIKE '%$search%
the variable search is declared correctly,now everything works fine when i search specifically, such as Gold chain will show Gold chain.But when i search another name together such Gold Chain Shirt,where shirt is another product's name,the result is not showing. How should i change my sql command to get multiple result from multiple value searched? I'm very sorry i did not tell earlier that i was asked to do it in 3 tier programming.
There's a decent article here which will give you a decent introduction to searching MySQL with PHP, but basically what you want to do is split your search phrase in to parts and then use them in the MySQL query. For instance:
<?php
$search = 'Gold Chain Shirt';
$bits = explode(' ', $search);
$sql = "SELECT name FROM product WHERE name LIKE '%" . implode("%' OR name LIKE '%", $bits) . "%'";
The above will generate this query:
SELECT name FROM product WHERE name LIKE '%Gold%' OR name LIKE '%Chain%' OR name LIKE '%Shirt%'
You really need to look at FULLTEXT indexing, then read about FULLTEXT Query Expressions.
If I understood you correctly, you want to search all items that have value "Gold Chain" or "Shirt". In this case, as you tag the question as "php", you could do this by changing the $search as the whole WHERE clause. I do this such way (example with showing different conditions to explain the idea):
$search_array=array('name LIKE "%Gold Chain%"', 'price > 5');
$where=implode($search_array,' OR '); // you might wish ' AND '
some_function_to_query('SELECT name FROM product WHERE '.$where);
You could improve your search by doing :
'%$search' to search only from the beginning of the String to get more results.
Than, if you wanted to search each word from a sentence, you could do like that :
$search = 'Gold Chain Shirt';
$searches = explode(' ', $search);
$query = "SELECT * FROM product WHERE name ";
foreach($searches as $word) {
$query .= "LIKE '%{$word}' OR ";
}
I'm using WordPress, but this question is more pertaining to the SQL involved. I'll gladly move it if I need to.
I'm working on http://www.libertyguide.com/jobs and I'm trying to alter the filtering mechanics. Currently it's a global OR query.
Anyways, I have three filtering lists, and I'm storing what's selected into three strings (interests, type, experience) in the following way:
"( $wpdb->terms.slug = 'webdevelopment' OR $wpdb->terms.slug = 'journalism' OR ... ) AND"
It's populated by whatever is selected in my filtering lists.
When it comes down to it, I have this as a basic query (I'm leaving out the LEFT JOINS):
Before:
SELECT * FROM $wpdb->posts WHERE ($wpdb->terms.slug = 'fromlist1'
OR $wpdb->terms.slug = 'fromlist2' OR $wpdb->terms.slug = 'fromlist3')
AND $wpdb->term_taxonomy.taxonomy = 'jobtype'...
After:
SELECT * FROM $wpdb->posts WHERE
($wpdb->terms.slug = 'fromlist1' OR $wpdb->terms.slug = 'fromlist1again')
AND ($wpdb->terms.slug = 'fromlist2' OR $wpdb->terms.slug = 'fromlist2again')
AND ($wpdb->terms.slug = 'fromlist3' OR $wpdb->terms.slug = 'fromlist3again')
AND $wpdb->term_taxonomy.taxonomy = 'jobtype'...
So essentially I want to go from an
OR filter
to
an AND filter with OR filtering inbetween.
My new filtering only works when one item overall is selected, but returns nothing when I select more than one thing (that I know would match up with a few posts).
I've thought through the logic and I don't see anything wrong with it. I know nothing is wrong with anything else, so it has to be the query itself.
Any step in the right direction would be greatly appreciated. Thanks!
UPDATE
From the confusion, basically I have this:
"SELECT ...... WHERE $terms ..."
but I WANT
"SELECT ....... WHERE $interests AND $type AND $experience"
I don't want to have it filter $interest[1] OR $interest[2] OR $type[1] OR $experience[1], but instead want it to filter ($interest[1] OR $interest[2]) AND ($type[1]) AND ($experience[1])
I hope this makes more sense
*UPDATE 2*
Here's and example:
In my interests list, I select for example three things: WebDevelopment, Academia, Journalism.
In my type list, I choose two things: Fulltime, Parttime
In my experience list, I choose three things: Earlycareer, Midcareer, Latecareer.
When I run my query, I want to make sure that each record has AT LEAST one of each of the three lists. Possible Results: (WebDevelopment, Parttime, Midcareer), (Academia, Fulltime, Earlycareer, Midcareer).
NOT A RESULT: (Journalism, Earlycareer) - missing fulltime or parttime
I really hope this clears it up more. I'm willing to give compensation if I can get this working correctly.
Okay, I'll take a shot at this:
SELECT * FROM $wpdb->posts WHERE
(
$wpdb->terms.slug IN ('$interest1', '$interest2') AND
$wpdb->terms.slug IN ('$type1', '$type2') AND
$wpdb->terms.slug IN ('$exp1', '$exp2')
)
AND $wpdb->term_taxonomy.taxonomy = 'jobtype'
The IN keyword will return true if any member of the set matches.
I think you're looking for a WHERE category IN (comma, seperated, list, of, values) that you can generate dynamically from the form. If you combine it with the other categories, you can require them to select something from each with...
WHERE category1 IN (a, comma, seperated, list, of, values)
AND category2 IN (another, list, of, values)
AND ...
Which will only return a value if there is something selected from each category and will return nothing if any of the selection lists are empty; actually it may well kick out an error, so I would also generate the query dynamically if there is any content whatsoever for a given category.
if (!empty($arrayOfCategory1)) {
//sanitize input logic here
$Category[1] = 'category1 IN ('. implode(', ', $arrayOfCategory1) .')';
} else {
$Category[1] = '';
}
You concatenate the resultant string together and build the query with that. The WHERE 1=1 trick is problematic because if nothing is chosen, everything in the database will match, so I strongly recommend going through the process of adding the AND operators properly.
EDIT: it occurs to me that if you build the conditional statements as an array, you can implode those with ' AND ' and get the query in a fairly small number of lines of code.
Sort of confused a bit by what you are saying but if I wanted to build a filter I would be dynamically generating the SQL query based on the submitted filter values. Something like:
$sql = "SELECT * FROM $wpdb->posts WHERE 1=1";
if ( !empty($interest) ) { // they ticked the interested in ??? checkbox
$sql .= " AND $wpdb->terms.slug = $interest"
}
Obviously you will need to filter and escape any values that have been submitted.