I'm building a searchfunctionallity where a user can select the different libraries to search in with the provided keywords. Also the AND operator is supported. My problem is that when I try to search for information it gets somehow multiplied or it gets increased exponential??
I'm using a MySQL database and PHP to process the results.
A sample query looks like this, sample tablenames taken for readability:
SELECT table1.id AS table1_id,
table2.id AS table2_id,
table1.*,
table2.*
FROM table1, table2
WHERE (table1.column1 LIKE '%key%' OR table1.column2 LIKE '%key%') OR
(table2.column1 LIKE '%key%' OR table2.column2 LIKE '%key%')
Or when a user provides keywords like 'key AND yek' the queries will look like:
SELECT table1.id AS table1_id,
table2.id AS table2_id,
table1.*,
table2.*
FROM table1, table2
WHERE ( (table1.column1 LIKE '%key%' AND table1.column2 LIKE '%key%') OR
(table1.colum1 LIKE '%yek%' AND table1.colum2 LIKE '%yek%') )
OR
(( table2.column1 LIKE '%key%' AND table2.column2 LIKE '%key%') OR
(table2.colum1 LIKE '%yek%' AND table2.colum2 LIKE '%yek%') )
I've searched arround and came accross UNION, but that won't do the trick. Firstly because of the tables I'm querying are not equal and it's a pretty expensive query. So also because the searchfunctionallity will be used often it's not an option.
In fact I'm querying 5 tables and per table between 2 or 4 columns. I set up a testrow in one of the tables with a unique value so I should get 1 row as result but in fact I'm getting a lot more. When I select all 5 tables I'm getting over 10K resultrows. So my guess is the result gets multiplied some way or another. I've played with the operators AND and OR but so far without any luck.
Is there someone who can help me further in my 'quest' to build a userfriendly searchengine in a webapp?
you should take a look at a real full text search engine, there is many around:
lucene
sphinx
Mysql: MySQL has a full text search engine but it's kind of slow so I don't advise you to use it apart if you want a quick solution
couchDB: that's not really a full text search engine but in your case it might helps but you might be able to create a DB from your current MySQL database and use that only for search purpose.
depending how your table are done, you might want to create a table with all the information you need.
A join will solve the problem of your dataset appearing to 'multiply'.
http://en.wikipedia.org/wiki/Join_%28SQL%29
This requires a field from Table1 to also be present in Table2 and used as a foreign key.
For eg, Table1 and Table2 both have ProductID:
Select * From Table1 Left Outer Join Table2 on Table1.ProductID = Table2.ProductID
Some suggestions:
I would suggest using (INNER?) JOIN instead of SELECT FROM table1, table2. That will narrow the number of tables. This should limit the output considerably.
Look into the DISTINCT keyword (or parallels like GROUP BY), it should further limit the output
DON'T rule out UNION. I don't know about MySQL, but in Oracle's SQL it is often two to three times faster than using an "OR". If you have
-- Admittedly, this query is a bit defective, but I'm trying to demonstrate.
SELECT
table1.data
FROM
table1, table2
WHERE
table1.data = 7
UNION
SELECT
table1.data
FROM
table1, table2
WHERE
table1.data = 8;
that will be far faster than using:
SELECT
table1.data, table2.*
FROM
table1, table2
WHERE
table1.data = 7 OR table1.data = 8;
Overall, I think that you'll see better results using something like this (this is Oracle syntax):
SELECT
ti.id,
t1.data,
t2.*
FROM
table1 t1
JOIN
table2 t2
ON(
t1.id = t2.id
)
WHERE
table1.data LIKE '%foo%'
UNION
SELECT
ti.id,
t1.data,
t2.*
FROM
table1 t1
JOIN
table2 t2
ON(
t1.id = t2.id
)
WHERE
table1.data LIKE '%bar%';
------- EDIT -------
It has been protested that "JOIN will not work as the tables are completely different". So long as there exists some key which exists that can relate one of the tables to another, JOIN will be able to create (effectively) an "uber-table" which has access to all of the data of both tables. If this index does not exist, then you'll probably need some system to concatenate the information outside of the SQL query.
I found the fulltext solutions to 'heavy' for my situation. I've created a query-array where queries per table are stored and later on I've used them for displaying the data I wanted. Everybody thanks for their effort!
I think what you want is to use UNION, which can combine the result of different SELECT queries.
(SELECT id FROM table1 WHERE (table1.column1 LIKE '%key%' OR table1.column2 LIKE '%key%'))
UNION
(SELECT id FROM table2 WHERE (table2.column1 LIKE '%key%' OR table2.column2 LIKE '%key%'))
Related
table1 (id, name)
table2 (id, name)
Query:
SELECT name
FROM table2
-- that are not in table1 already
SELECT t1.name
FROM table1 t1
LEFT JOIN table2 t2 ON t2.name = t1.name
WHERE t2.name IS NULL
Q: What is happening here?
A: Conceptually, we select all rows from table1 and for each row we attempt to find a row in table2 with the same value for the name column. If there is no such row, we just leave the table2 portion of our result empty for that row. Then we constrain our selection by picking only those rows in the result where the matching row does not exist. Finally, We ignore all fields from our result except for the name column (the one we are sure that exists, from table1).
While it may not be the most performant method possible in all cases, it should work in basically every database engine ever that attempts to implement ANSI 92 SQL
You can either do
SELECT name
FROM table2
WHERE name NOT IN
(SELECT name
FROM table1)
or
SELECT name
FROM table2
WHERE NOT EXISTS
(SELECT *
FROM table1
WHERE table1.name = table2.name)
See this question for 3 techniques to accomplish this
I don't have enough rep points to vote up froadie's answer. But I have to disagree with the comments on Kris's answer. The following answer:
SELECT name
FROM table2
WHERE name NOT IN
(SELECT name
FROM table1)
Is FAR more efficient in practice. I don't know why, but I'm running it against 800k+ records and the difference is tremendous with the advantage given to the 2nd answer posted above. Just my $0.02.
SELECT <column_list>
FROM TABLEA a
LEFTJOIN TABLEB b
ON a.Key = b.Key
WHERE b.Key IS NULL;
https://www.cloudways.com/blog/how-to-join-two-tables-mysql/
This is pure set theory which you can achieve with the minus operation.
select id, name from table1
minus
select id, name from table2
Here's what worked best for me.
SELECT *
FROM #T1
EXCEPT
SELECT a.*
FROM #T1 a
JOIN #T2 b ON a.ID = b.ID
This was more than twice as fast as any other method I tried.
Watch out for pitfalls. If the field Name in Table1 contain Nulls you are in for surprises.
Better is:
SELECT name
FROM table2
WHERE name NOT IN
(SELECT ISNULL(name ,'')
FROM table1)
You can use EXCEPT in mssql or MINUS in oracle, they are identical according to :
http://blog.sqlauthority.com/2008/08/07/sql-server-except-clause-in-sql-server-is-similar-to-minus-clause-in-oracle/
That work sharp for me
SELECT *
FROM [dbo].[table1] t1
LEFT JOIN [dbo].[table2] t2 ON t1.[t1_ID] = t2.[t2_ID]
WHERE t2.[t2_ID] IS NULL
You can use following query structure :
SELECT t1.name FROM table1 t1 JOIN table2 t2 ON t2.fk_id != t1.id;
table1 :
id
name
1
Amit
2
Sagar
table2 :
id
fk_id
email
1
1
amit#ma.com
Output:
name
Sagar
All the above queries are incredibly slow on big tables. A change of strategy is needed. Here there is the code I used for a DB of mine, you can transliterate changing the fields and table names.
This is the strategy: you create two implicit temporary tables and make a union of them.
The first temporary table comes from a selection of all the rows of the first original table the fields of which you wanna control that are NOT present in the second original table.
The second implicit temporary table contains all the rows of the two original tables that have a match on identical values of the column/field you wanna control.
The result of the union is a table that has more than one row with the same control field value in case there is a match for that value on the two original tables (one coming from the first select, the second coming from the second select) and just one row with the control column value in case of the value of the first original table not matching any value of the second original table.
You group and count. When the count is 1 there is not match and, finally, you select just the rows with the count equal to 1.
Seems not elegant, but it is orders of magnitude faster than all the above solutions.
IMPORTANT NOTE: enable the INDEX on the columns to be checked.
SELECT name, source, id
FROM
(
SELECT name, "active_ingredients" as source, active_ingredients.id as id
FROM active_ingredients
UNION ALL
SELECT active_ingredients.name as name, "UNII_database" as source, temp_active_ingredients_aliases.id as id
FROM active_ingredients
INNER JOIN temp_active_ingredients_aliases ON temp_active_ingredients_aliases.alias_name = active_ingredients.name
) tbl
GROUP BY name
HAVING count(*) = 1
ORDER BY name
See query:
SELECT * FROM Table1 WHERE
id NOT IN (SELECT
e.id
FROM
Table1 e
INNER JOIN
Table2 s ON e.id = s.id);
Conceptually would be: Fetching the matching records in subquery and then in main query fetching the records which are not in subquery.
First define alias of table like t1 and t2.
After that get record of second table.
After that match that record using where condition:
SELECT name FROM table2 as t2
WHERE NOT EXISTS (SELECT * FROM table1 as t1 WHERE t1.name = t2.name)
I'm going to repost (since I'm not cool enough yet to comment) in the correct answer....in case anyone else thought it needed better explaining.
SELECT temp_table_1.name
FROM original_table_1 temp_table_1
LEFT JOIN original_table_2 temp_table_2 ON temp_table_2.name = temp_table_1.name
WHERE temp_table_2.name IS NULL
And I've seen syntax in FROM needing commas between table names in mySQL but in sqlLite it seemed to prefer the space.
The bottom line is when you use bad variable names it leaves questions. My variables should make more sense. And someone should explain why we need a comma or no comma.
I tried all solutions above but they did not work in my case. The following query worked for me.
SELECT NAME
FROM table_1
WHERE NAME NOT IN
(SELECT a.NAME
FROM table_1 AS a
LEFT JOIN table_2 AS b
ON a.NAME = b.NAME
WHERE any further condition);
What I want to do is to query three separate tables into one row which is identified by a unique reference. I don't really have full understanding of the Join clause as it seems to require some sort of related data from each table.
I know I can go about this the long way round, but can not afford to lose even a little efficiency. Any help would be greatly appreciated.
Table Structure
package_id int(8),
client_id int(8),
unique reference varchar (40)
Each of the tables have essentially the same structure. I just need to know how to query all three, for 1 row.
If you have few tables that are sharing the same or similar definition, you can use union or union all to treat them as one. This query will return rows from each table having requested reference. I've included OriginTable info in case your code will need to refer to original table for update or something else.
select 'TableA' OriginTable,
package_id,
client_id
from TableA
where reference = ?
union all
select 'TableB' OriginTable,
package_id,
client_id
from TableB
where reference = ?
union all
select 'TableC' OriginTable,
package_id,
client_id
from TableC
where reference = ?
You might extend select list with other columns, provided that they have the same data type, or are implicitly convertible to data type from first select.
Let's say you have 3 tables :
table1, table2 and table3 with structure
package_id int(8),
client_id int(8),
unique reference varchar (40)
Let's assume that column reference is unique key.
Then you can use this:
SELECT t1.exists_row ,t2.exists_row ,t3.exists_row FROM
(
(SELECT COUNT(1) as exists_row FROM table1 t1 WHERE
t1.reference = #reference ) t1,
(SELECT COUNT(1) as exists_row FROM table1 t2 WHERE
t2.reference = #reference ) t2,
(SELECT COUNT(1) as exists_row FROM table1 t3 WHERE
t3.reference = #reference ) t3
) a
;
Replace #reference with actual value of unique key
or when you provide output of
SHOW CREATE TABLE
I can rewrite SQL with actual query
It is entirely possible to create a join between tables using a where clause. In fact this is often what I do as I find it leads to clearer information of what you are actually doing, and if you don't get the results you expect you can debug it bit by bit.
That said however a join is certainly a lot quicker to write!
Please bear in mind I'm a bi rusty on SQL so I may have missed remembered, and I'm not going to include any code as you haven't said what DBMS you are using as they all have slightly different code.
The thing to remember is that the join functions on a column with the same data (and type) within it.
It is much easier if each table has the 'joining' field named the same, then it should be a matter of
join on <nameOfField>
However if you wish to use field that have different names in the different tables you will need to list the fully qualified names. ie tableName.FieldName
If you are having trouble with natural, inner and outer, left and right, you need to think of a venn diagram with the natural being the point of commonality between the tables. If you are using only 2 tables inner and outer are equivalent to left and right (with each table being a single circle in the venn diagram) and left and right being the order of the tables in your list in the main part of your select (the first being the left and the second being the right).
When you add a third table this is where you can select any of the cross over section using these keywords.
Again however I have always found it easier to do a primary select and create a temp table, then perform my next join using this temp table (so effectively only need to use natural or left and right again). Again I find this easier to debug.
The best thing is to experiment and see what you get in return. Without a diagram of your tables this is the best I can offer.
in brief...
nested selects where field = (select from table where field = )
and temp tables
are (I think) easier to debug... but do take more writting !
David.
array_of_tables[]; // contain name of each table
foreach(array_of_tables as $val)
{
$query="select * from `$val` where $condition "; // $conditon
$result=mysqli_query($connection,$query);
$result_row[]=mysqli_fetch_assoc($result); // if only one row going to return form each table
//check resulting array ,for your row
}
SELECT * FROM table1 t1 JOIN table2 t2 ON (t2.unique = t1.unique) JOIN table3 t3 ON (t3.unique = t1.unique) WHERE t1.unique = '?';
You could use a JOIN like this, assuming all three tables have the same unique column.
I have one query that INNER JOINs Table A with Table B and Table C, and I have another query that INNER JOINs Table A with Table D. I could achieve what I want to do by merging the 2 results, removing duplicates and ordering them in PHP code, but I want to know if MySQL has this functionality, I also think it'd be faster and easier to code once I understand it. Essentially, I want to have the results from Query 1 OR from Query 2. Perhaps the following will help:
Query 1:
SELECT pkAssociation, associations.strNameEn, associations.strNameFr
FROM associations
INNER JOIN eventassociations ON fkAssociation = pkAssociation
INNER JOIN events on fkEvent = pkEvent
WHERE events.enumDeleted = 'no'
GROUP BY pkAssociation
Query 2:
SELECT pkAssociation, associations.strNameEn, associations.strNameFr
FROM associations
INNER JOIN associationprograms AS aprogs ON aprogs.fkAssociation = associations.pkAssociation
GROUP by pkAssociation
The tables don't have anything else of relevance that don't show up in the query. I'm sorry if I'm not asking this correctly, I don't even know how to ask a question about this properly. If the column names or sample data is needed, then I can provide some. Sorry for the inconvenience and long post.
You want the UNION DISTINCT statement, placed between the two queries. This will combine the result sets from both and remove duplicates. You can then place your ORDER BY clause after all UNIONS (if you have one).
For example:
SELECT col1, col2 FROM tableA
UNION DISTINCT
SELECT col1, col2 FROM tableB
ORDER BY col2 DESC
I have a mysql table(table1) which has the following row:
topic_id: 1
topics: programming
description: A programming language is an artificial language designed to...
I have another table(table2) with this row:
desc_id: 1
description: In mathematics and computer science, an algorithm is an effective...
topics: mathematics, computer science, programming
What I'm looking to do is to run a query to compare the two topics fields and let me know which topics exist in table2 that don't exist in table1.
For instance, comparing the two above I'd like to run a query to let me know that topics mathematics and computer science don't exist in table1.
I would use a subquery, but it can also be done with innerjoins :
SELECT *
FROM `table2`
WHERE `topics` NOT IN (
SELECT DISTINCT(topics)
FROM `table1`
)
you can try NOT IN
i.e.
SELECT topics FROM table2 where topics NOT IN( select topics from table1)
If you normalized your table2 so that the topics list is in a separate sub-table, this would be a trivial query. As it stands now, it's difficult as by default mysql won't see those seperate topics in table2.topics as discrete topics. It's just a long string that happens to have commas in there.
Thankfully, MySQL has the find_in_set() function, which can help out immensely, but this function isn't available elsewhere. Not having access to your dataset, I'm just guessing here, but this should do the trick:
SELECT table1.topics, count(table1.topic_id) AS cnt
FROM table1
LEFT JOIN table2.topics ON FIND_IN_SET(table1.topics, table2.topics) = 0
GROUP BY table1.topics
HAVING cnt = 0
Basically, join the tables wherever the table1 topic is NOT in a table2 topic and count how many times the table1 topic shows up like this. If it shows up zero times, then it's present in at least one record in table2.
normalize by creating a third table, one that links table 2 to table 1 with a many to many relationship.
Table_1
id, etc
Table_2
id, etc
Table_3
id, table1_id, table2_id
you could then use simple joins to create a query that will pull the relavent data
SELECT * FROM Table_1 LEFT JOIN Table_3 ON Table_1.id = Table_3.table1_id WHERE Table_3.table2_id = $table2_id
This will pull all topics for the course.
What I want to do is get the data from two different tables (table1 and table2) where row1 = 'test' in both of the tables
You'll want to use an INNER JOIN here - something along these lines (can't tell you for sure since you didn't give the structure of your tables)...
SELECT * FROM thread t
INNER JOIN post_display pd ON pd.threadid = t.threadid
WHERE t.threadid = 2
ORDER BY t.threadid DESC
Note: SELECT * can be very bad if you're selecting a bunch of fields you're never going to need. Once you have the query working, narrow down your select to the specific fields you're looking to work with.
More info on JOIN syntax for MySQL is available here: http://dev.mysql.com/doc/refman/5.1/en/join.html
I'm not quite sure what you're asking, but if you want to fetch columns from multiple tables at once (and it sounds like you're saying rows when you mean columns) you probably want a JOIN, which is an SQL feature
I am not getting what you are asking about.. but.. i can give u suggestion on you asked question.. u can try this.. have a look
SELECT * FROM table1 t1
INNER JOIN table2 t2 ON t1.id = t2.t1id
WHERE t1.row1 like 'test' AND t2.row like 'row';