MySQL query to search keywords ordered by relative match - php

I am creating an online system that matches bank statements (description, value, type, etc) to purchasers (names, addresses) and would like some feedback on the way I am currently doing this:
$array = array("keywords", "taken", "from",
"bank", "statements", "using", "explode",
"function", "using", "a", "space");
$i = 0;
$r = array(); //r = relevant
while($i<count($array)) {
$keyword = $array[$i];
$get = mysql_query("SELECT `id` FROM `users`, `addresses`
LEFT JOIN `users`.`id` = `addresses`.`user`
WHERE (`users`.`frame` LIKE '%$keyword%'
OR `users`.`lname` LIKE '%$keyword%')
OR ((`addresses`.`address` LIKE '%$keyword%'
OR `addresses`.`town` LIKE '%$keyword%')
OR (`addresses`.`county` LIKE '%$keyword%'
OR `postcode`.`town` LIKE '%$keyword%'));");
if(mysql_num_rows($get)) {
while($fetch = mysql_fetch_array($get)) {
list($var) = $fetch;
push_array($r, $var);
}
}
$i++;
}
//count the IDs that match within the relative array to see
//which is most relative to the search
Is there a better way of doing this as well as keeping the execute time to an absolute minimum? The database will be in the 10s of thousands when it's finished.

It would be better to build a keyword table that ties each keyword to a user, e.g.:
keywords (keyword, user) + UNIQUE(keyword, user)
keyword1 12
keyword1 14
keyword2 3
After you populate the keywords table from the data you wish to search on, the query becomes much more optimal:
SELECT users.*
FROM keywords
INNER JOIN users ON users.id = keywords.user
WHERE keyword IN ('keyword1', 'keyword2')
Of course, you need to maintain this table when you make changes to the user or address table (insert, update, delete).

Related

PHP/SQL: How to concatenate/combine columns value into one row

I have this php script called title, where it is supposed to list movie details of those movies with the title matching the inputed substring. The expected output is supposed to be like in the link/picture below. I have trouble with concatenating the genres of each movies since one movie can have many genres. I have tried using the concat(), array_to_string() but still fails.
mkSQL() constructs "safe" SQL query strings by taking a query template
string and filling in printf-like slots in the template with values
supplied in subsequent arguments. The function takes a variable number
of arguments; the first is always a query template string, with the
following arguments corresponding exactly to the slots in the
template. E.g.
$id = 3012345;
$q1 = mkSQL("select * from R where id = %d",$id);
would create the query strings:
$q1: "select * from R where id = 12345"
Below are the codes, any helps and tips will be greatly appreciated, thanks!
This is the Genre Table Schema
CREATE TABLE Genre (
movie_id integer REFERENCES Movie(id),
genre GenreType,
primary key (movie_id,genre));
#!/usr/bin/php
<?php
// include the common PHP code file
require("a2.php");
$db = pg_connect("dbname=mydb");
// Check arguments
if (count($argv) < 2) exit("$usage\n");
// Get the return results
$val = $argv[1];
$q = "select m.title, m.year, m.content_rating, r.imdb_score, array_to_string(array(select g.genre FROM Genre g where g.movie_id = m.id),',')
-- concat(select g.genre FROM Genre g where g.movie_id = m.id
from Movie m JOIN Rating r ON r.movie_id = m.id
where m.title ilike %p
order by m.year, r.imdb_score desc, m.title asc";
$r = pg_query($db, mkSQL($q, $val));
// Iterate through the results and print
$i = 1;
while ($t = pg_fetch_array($r)) {
echo "$i. $t[0] ($t[1], $t[2], $t[3]) [$t[4]]\n";
$i++;
}
?>
The expected output is supposed to be in this format
Change your query like,
SELECT CONCAT(m.title, ' (', m.year, ', ', m.content_rating, ',', r.imdb_score, ') [', (SELECT array_to_string(array_agg(g.genre), ',') FROM Genre g WHERE g.movie_id = m.id), ']') movie_title
FROM Movie m JOIN Rating r ON r.movie_id = m.id
WHERE m.title ilike %p
ORDER BY m.year, r.imdb_score desc, m.title ASC
Here, I have concat all columns into one and given it an alias movie_title. You will get the movie name as per your specified format.
For achieving this, you can use the group_concat function in your mysql script.
This will concatenate your respective column via comma(,).

Select a fixed number of records from a particular user in a sql result

I have 2 tables - users and articles.
users:
user_id (int)
name (varchar)
articles:
article_id (int)
user_id (int)
title (varchar)
description (text)
In my application I need to display 20 RANDOM articles on a page.
My query is like this:
SELECT a.title
, a.description
, u.name
FROM articles a
JOIN users u
USING (user_id)
ORDER
BY RAND()
LIMIT 20
A user can have any number of articles in the database.
Now the problem is sometimes out of 20 results, there are like 9-10 articles from one single user.
I want those 20 records on the page to not contain more than 3 (or say 4) articles from a particular user.
Can I achieve this through SQL query. I am using PHP and MySQL.
Thanks for your help.
You could try this?
SELECT * FROM
(
SELECT B.* FROM
(
SELECT A.*, ROW_NUMBER() OVER (PARTITION BY A.USER_ID ORDER BY A.R) USER_ROW_NUMBER
FROM
(
SELECT a.title, a.description, u.name, RND() r FROM articles a
INNER JOIN users u USING (user_id)
) A
) B
WHERE B.USER_ROW_NUMBER<=4
) C
ORDER BY RAND() LIMIT 20
Mmm, intresting I don't think this is possible through a pure sql query.
My best idea would be to have an array of the articles that you'll eventually display query the database and use the standard SELECT * FROM Articles ORDER BY RAND() LIMIT 20
The go through them, making sure that you have indeed got 20 articles and no one has breached the rules of 3/4 per user.
Have another array of users to exclude, perhaps using their user id as an index and value of a count.
As you go through add them to your final array, if you find any user that hits you rule add them to the array.
Keep running the random query, excluding users and articles until you hit your desired amount.
Let me try some code (it's been a while since I did php)
$finalArray = [];
$userArray = [];
while(count($finalArray) < 20) {
$query = "SELECT * FROM Articles ";
if(count($finalArray) > 0) {
$query = $query . " WHERE articleID NOT IN(".$finalArray.")";
$query = $query . " AND userID NOT IN (".$userArray.filter(>4).")";
}
$query = $query . " ORDER BY Rand()";
$result = mysql_query($query);
foreach($row = mysql_fetch_array($result)) {
if(in_array($finalArray,$row) == false) {
$finalArray[] = $row;
}
if(in_array($userArray,$row[userId]) == false) {
$userArray[$row[userId]] = 1;
}
else {
$userArray[$row[userId]] = $userArray[$row[userId]] + 1;
}
}

Search query and ordering by matches with my sql

Hi am currently stuck here, been doing research on how to write a mysql statement for a flexible search for products and order them by relevance on a project am working on have seen a few but wasn't helpful please i need help on how to make it work, my current method doesn't work, here it is.
User types in search field and submits "iPad 3rd Generation".
My script breaks the string into words like so.
$termsExploded = array_unique(explode(' ', $term));
No i use php to create an sql query based on the number of words found.
$i = 0;
foreach ($termsExploded as $word) {
if (strlen($word)>1) {
if ($i == 0) {
$where_query = $where_query." name LIKE '%".$word."%'";
}
else{
$where_query = $where_query." OR name LIKE '%".$word."%'";
}
$i++;
}
}
The where query variable now looks like this.
name Like '%ipad%' Or name Like '%3rd%' Or name Like '%Generation%'
Now search for the products ids like so.
$IDs = "SELECT DISTINCT id FROM store_items WHERE".$where_query;
I now create a second where query based on the IDs returned like so
$where_query_s = null;
$i = 0;
foreach ($IDs as $result) {
$returnID = $result->id;
if ($i == 0) {
$where_query_s = $where_query_s." id = ".$returnID."";
}
else{
$where_query_s = $where_query_s." OR id = ".$returnID."";
}
$i++;
};
Now i select the products again based on the distinct IDs returned like so
$items = "SELECT * FROM store_items WHERE".$where_query_s;
Now this works to get the products but how can i sort it based on best match?
Assuming you want to order by the number of matches then build up another string as follows:-
ORDER BY IF(name Like '%ipad%', 1, 0) + IF(name Like '%3rd%', 1, 0) + IF(name Like '%Generation%', 1, 0) DESC
But this will be slow, and takes no account of indexing to improve performance nor of plural / singular (ie, it someone searches for 'flies' it won't rank 'fly' properly).
To put that more into code:-
$where_query = array();
$order_query = array();
foreach ($termsExploded as $word)
{
if (strlen($word)>1)
{
$where_query[] = " name LIKE '%".$word."%'"
$order_query[] = " IF(name Like '%".$word."%', 1, 0)"
}
}
$IDs = "SELECT DISTINCT id FROM store_items WHERE ".implode(' OR ', $where_query)." ORDER BY ".implode(' + ', $order_query)." DESC";
Arrange for your query to look like this:
select field1, field2, etc, count(*) records
from store_items
where blah blah blah
group by field1, field2, etc
order by records desc
If the table is MyISAM based or if it is InnoDB and the version is Mysql 5.6 or greater, then you can use full text search (see http://dev.mysql.com/doc/refman/5.6/en/fulltext-search.html)
effectively you want a query similar to
SELECT * FROM store_items WHERE MATCH (name) AGAINST ('iPad 3rd Generation')
ORDER BY MATCH (name) AGAINST ('iPad 3rd Generation')

Combine several SQL querys to one

I know this is probably really simple but I have tried to find some similar examples and failed.
The problem is that I would like to list 8 random images from a gallery in a database, sorted by added date. And I have managed to do this, but only with several iterating query's that becomes really slow. So if someone would be so kind to teach me about combining them for faster speed, I guess UNION is the way to go? Here is my working (but slooow code)
<?php
$latestPictures = mysql_query("SELECT pictureID, addedDate FROM picture ORDER BY addedDate DESC LIMIT 8");
$latestConcerts = mysql_query("SELECT concertID, addedDate FROM concert WHERE pictureID is null ORDER BY addedDate DESC LIMIT 8");
// Add concerts and pictures to array
while($curFestival = mysql_fetch_object($latestPictures))
{
$array[$curFestival->addedDate] = "p" . $curFestival->pictureID;
}
while($curConcert = mysql_fetch_object($latestConcerts))
{
$array[$curConcert->addedDate] = "c" . $curConcert->concertID;
}
// Order array by key
krsort($array);
$latestArray = array_slice($array, 0, 8);
foreach($latestArray as $key => $value) {
$type = substr($value, 0, 1);
$ID = substr($value, 1);
// If type == picture
if($type == 'p')
{
$picturesPicturesID = mysql_query("SELECT concertID, name FROM photo WHERE concertID IN(SELECT concertID FROM concert WHERE pictureID = $ID) ORDER BY photoID");
// Get random picture
$curRandomPicture = rand(0, (mysql_num_rows($picturesPicturesID) - 1));
$curPictureConcertID = mysql_result($picturesPicturesID, $curRandomPicture, "concertID");
$curPictureName = mysql_result($picturesPicturesID, $curRandomPicture, "name");
$curPicture = mysql_fetch_object(mysql_query("SELECT c.URL, p.name FROM concert c, picture p WHERE p.pictureID = c.pictureID AND c.concertID = $curPictureConcertID"));
echo "Some image";
}
// If type == concert
if($type == 'c')
{
$concertPicturesID = mysql_query("SELECT concertID, name FROM photo WHERE concertID = $ID ORDER BY photoID");
// Get random picture
$curRandomPicture = rand(0, (mysql_num_rows($concertPicturesID) - 1));
$curPictureConcertID = mysql_result($concertPicturesID, $curRandomPicture, "concertID");
$curPictureName = mysql_result($concertPicturesID, $curRandomPicture, "name");
$curPicture = mysql_fetch_object(mysql_query("SELECT URL, name FROM concert WHERE concertID = $curPictureConcertID"));
echo "Some image";
}
}
?>
I realized that I forgot to include the tables, here they are:
TABLE OF photo:
PKEY: photoID
FKEY: concertID
name
TABLE OF concert:
PKEY: concertID
FKEY: pictureID
name
URL
addedDate
TABLE OF picture
PKEY: pictureID
name
date
So every post is part of TABLE photo AND concert, but only some is part of picture witch is only used sometimes to group differens albums together. When they are grouped together I whant a random name post from that grouping ID (picture) and if they are by them self a random name post from there (concert).
Accckkk! (In the infamous words of Bill the Cat.)
It's hard to figure out what result set your code really wants from the database.
This query isn't the most efficient, but it will return 8 random rows from the photos table, with those rows ordered by addedDate:
SELECT r.*, c.*
FROM (SELECT p.*
FROM photo p
WHERE p.concertid IS NOT NULL
ORDER BY RAND()
LIMIT 0,8
) r
JOIN concert c
ON c.concertid = p.concertid
ORDER BY r.addedDate ASC
If you have a really large photo table, this is going to be slow, because that RAND() function has to get called for every single row in the table, and MYSQL has to produce a temporary result set (a copy of the table) and then sort it on that derived column.
(NOTE: I'm assuming here that it's the photo table that you want to return "random" rows from, and I'm assuming that concertid is the primary key on the concert table, and a foreign key from the photo table. It's apparent that you have three tables... concert, picture and photo, but it's not clear which columns are the primary keys and which columns are the foreign keys, so it's likely I have it wrong.)
(NOTE: replace the p.* and c.* with a list of expressions you actually want to return.)
There are more efficient approaches to returning a single random row. In your case, you want exactly eight rows, and you presumably don't want to return a duplicate.

mysql php match multiple keywords

I have three tables which are currently structured in the following way
Table: Images
image_id
image_title
...
Table: Keywords
keyword_id
keyword
Table: Image_Keyword
image_id
keyword_id
With this structure, I'm able to search if any images match any keywords using joins and or statements - however I would like to be able to retrieve images that have multiple keywords matches e.g. "keyword = ('red' or 'dress') and 'night'" - which would return all images that had either 'red' or 'dress' in them, alongside night.
Ideally I want to allow the user to be able to specify the AND and OR commands in the search box, which is why I have so far opted out of making separate joins for each new keyword - however I'm not sure how to proceed with the structuring of the query.
Currently I have the following, without the 'and' implementation:
SELECT i.* FROM images i
JOIN image_keyword ik ON i.id = ik.image_id
JOIN keywords k ON k.id = ik.keyword_id
WHERE k.keyword IN ('night','red')
Any help on how to go about creating the 'and' portion of this query would be greatly appreciated! Thanks kindly,
Dan
// UPDATE
So it looks as if I am going to have to do it by creating joins for each 'AND' request that I need to sort out - however I have an extension on the requirements now...
I have two other tables which follow the following structure
Table ImageData
id
image_id
caption_id
...
Table Caption
id
data (text)
In this instance, I would want to search for the keywords ('red','dress' and 'night'), using the same 'AND' and 'OR' capability as before, but also return the image if the text matches (using the same rules) in the caption data field. I would assume I potentially use an OR after the 'keyword' search, and then use a fulltext search on the caption, however I don't know if there is a cleaner way of combining the two, maybe even as two separate queries and then choosing the distinct results - which might allow for instances where the AND is successful in the keywords, and the OR is successful in the caption.
Any thoughts would be fantastic
Thanks again
I think what you will end up is this -
One INNER JOIN for all your ORs.
One INNER JOIN each for all your ands.
For example -
SELECT i.* FROM images i
INNER JOIN image_keyword ik ON i.id = ik.image_id
INNER JOIN keywords kOR ON kOR.id = ik.keyword_id AND (kOR.keyword IN ('dress', 'red'))
INNER JOIN keywords kAND1 ON kAND1.id = ik.keyword_id AND kAND1.keyword = 'night'
PHP script would look something like.
$orKeywords = arrya('dress', 'red', 'white');
$andKeywords = array('night', 'day');
$orJoin = '';
$andJoin = '';
if(count($orKeywords) > 0)
{
$orCondition = "'".implode("', '", $orKeywords)."'";
$orJoin = " INNER JOIN keywords kOR ON kOR.id = ik.keyword_id AND kOR.keyword IN ($orCondition) ";
}
if(count($andKeywords) > 0)
{
$cnt = 1;
foreach($andKeywords as $keyword)
{
$andJoin .= " INNER JOIN keywords kAND{$cnt} ON kAND{$cnt}.id = ik.keyword_id AND kAND{$cnt}.keyword = '$keyword' ";$cnt++;
}
}
$sql = "SELECT i.* FROM images i
INNER JOIN image_keyword ik ON i.id = ik.image_id
$orJoin
$andJoin";
You get the idea..
I would just generate the WHERE part of the query in PHP script, like this:
<?php
$entered_keywords = array('night','red');
$logic = 'OR'; // or 'AND'
$sql_where = implode(' '.$logic.' ', "k.keyword='$entered_keywords'"); //don't forget the escaping here!
$sql = 'SELECT i.* FROM images i
JOIN image_keyword ik ON i.id = ik.image_id
JOIN keywords k ON k.id = ik.keyword_id
WHERE '.$sql_where;
?>

Categories