Alright so I'll try to explain it as simple as possible; consider that I have two database tables (MySQL Server / MariaDB, database-related tasks coded in procedural style in PHP using prepared statements):
in one, I have a column of datatype JSON, whose content corresponds to sth like {name1:info,name2:info}
In another one, I have simple non-json records, having a structure like:
name | status
------+--------
name1 | statusX
------+--------
name2 | statusY
My Goal: I need to retrieve the name2 from table 1), but I also need to retrieve the status of the person having that same name (which in this case is statusY). Note that, for the retrieval of name2, I cannot rely on indexes of the json object (name2 may be the first key of the json object).
How I would do it so far:
A) Get the name2 from table 1) in a first query, sanitize it, and
B) use it in the second query which then correctly retrieves the statusY
Both statements A) and B) are parametrized prepared sql statements, triggered by an AJAX Call at a regular interval (AJAX Polling).
Given that these database queries are thus executed frequently, I want them to be executed as fast as possible, and thus ideally reduce my two queries above to a single one. My problem: I need the result of statement A) to execute statement B), so I cannot summarize the two queries into a single prepared statement, as prepared statements cannot contain multiple sql statements. The best solution to reach what I want is create a stored procedure like:
SET #name = SELECT ..... FROM table_1; SELECT .... FROM table_2;
And then execute it as parametrized prepared statement; is that correct?
I'm not at all experienced with stored procedures in MySQL Server, didn't really need them yet, but they seem to be the only solution if you want to wrap > 1 sql statements into a single prepared statement. Is this assumption, and thus the conclusion that I gotta create a stored procedure to reach what I want, correct?
IMPORTANT NOTE: I don't know the name I need to query. From the two names of the json column of table 1), I only know the other name. In other words, I have one name of a person X, and I want to get the status of all the persons which have been associated with that person X in table 1), while the status of each person is listed in table 2), to avoid to have duplicate status storage in the DB. ATM, I retrieve the other names of each relation record from DB 1) by using a conditional statement saying sth like
UPDATE
See added answer below, got it working.
You can query JSON data type with MySQL (if version > 5.7), and thus you can simply do everything with a single query
Give this a try
SELECT t1.name1, t1.name2, t2.status
FROM
(
SELECT JSON_EXTRACT(your_json_column, "$.name1") AS name1,
JSON_EXTRACT(your_json_column, "$.name2") AS name2
FROM table1
WHERE JSON_EXTRACT(your_json_column, "$.name1") = 'info'
) t1
INNER JOIN table2 t2 ON t2.`name`=t1.name2
Adapt the name of your_json_column. Also I assumed that you wanted to search the name2 of a specific name1, thus my WHERE clause, remove it if it was a false assumption.
Okay got it working, pretty much thanks to the solution proposed by Thomas G and some hints of JNevill (cheers guys!):
SELECT t1.info1, t1.info2, t1.info3, t1.other_name, t2.status FROM (
SELECT
field1 AS info1,
field2 AS info2,
field3 AS info3,
CASE
WHEN JSON_VALUE(JSON_KEYS(json_names_column),"$[0]") = 'name1'
THEN JSON_VALUE(JSON_KEYS(json_names_column),"$[1]")
ELSE JSON_VALUE(JSON_KEYS(json_names_column),"$[0]")
END
AS other_name
FROM table1
WHERE id = 345
) t1 INNER JOIN table2 t2 ON t1.other_name = t2.name;
Note that I used JSON_VALUE(JSON_KEYS()) instead of JSON_EXTRACT, to only return the needed name as name data of t1, and because I don't know the name to retrieve before the query, so I cannot use the WHEREclause proposed by Thomas G.
Related
This question is in relation to the answer of [another question][1] that I've posted a year ago.
Basically, I need to translate the answer in PHP and I don't even know where to start. Should I use 6 queries? Should I concatenate every query into 1 query? Should I use mysqli_multi_query?
I just need some advice, tips, and I will do the rest, I will do the research needed on how to achieve this in PHP.
This is the query that works perfectly and that I need to translate in PHP:
-- Query 1
CREATE TEMPORARY TABLE t (
ID INT AUTO_INCREMENT PRIMARY KEY
)
SELECT
w.work_id,
w.name wname,
r.sort_name rsortname,
CONCAT(r.seo_url, '.', r.recording_id) as rurl
FROM
WORK AS w
JOIN recording AS r ON w.work_id = r.work_id
JOIN `release` AS rl ON r.release_id = rl.release_id
WHERE
r.is_performer = 1
AND r.is_video = 0
ORDER BY
w.work_id,
- rl.released_year DESC,
- rl.released_month DESC,
- rl.released_day DESC,
rl.release_id;
-- Query 2
CREATE TEMPORARY TABLE x
SELECT
MIN(ID) AS ID
FROM
t
GROUP BY
work_id;
-- Query 3
SELECT
work_id,
wname,
rurl
FROM
x
JOIN t ON x.ID = t.ID
ORDER BY
rsortname;
-- Query 4, 5
DROP TEMPORARY TABLE t;
DROP TEMPORARY TABLE x;
Should I use 6 queries?
Yes, and if these queries depend on variable from PHP then you should use prepared statements to execute them.
Should I concatenate every query into 1 query?
Definitely not. They are not one query, they are all separate and you should execute them separately.
Should I use mysqli_multi_query?
Never! Forget that this function even exists. It is difficult to use and completely unnecessary. Just use prepared statements or command-line interface in MySQL for administrative tasks.
I have a problem about my code, this is my code on connecting to database, selecting some unique data and sum on these unique data from another table, nothing problem with my code, thats all okay, but it taking so long time on querying.
<?php
include "koneksi.php";
$no=1;
$arqury=odbc_exec($koneksi, "SELECT DISTINCT NIP_AR,NAMA_AR FROM USRBPS.MASTERFILE");
while($ar=odbc_fetch_array($arqury)){
$total=0;
$ambilqury=odbc_exec($koneksi, "SELECT NPWP FROM USRBPS.MASTERFILE WHERE NIP_AR='$ar[NIP_AR]'");
while($ambil=odbc_fetch_array($ambilqury)){
$testqury=mysql_query("SELECT SUM(jumlah_bayar) as PENERIMAAN FROM mpnruteng WHERE npwp='$ambil[NPWP]'");
$test=mysql_fetch_array($testqury);
$total += $test[PENERIMAAN];
}
if($ar[NIP_AR]==""){
echo "<tr><td>$no</td><td colspan=2>UNASSIGN</td><td>$total</td>";
}
else{
echo "<tr><td>$no</td><td>$ar[NAMA_AR]</td><td>$ar[NIP_AR]</td><td>$total</td></tr>";
}
$no++;
}
?>
on simply is being like this,
|Name |num_se
---------------
|andre |1111
|john |2222
|simon |3333
|andre |4444
|andre |5555
|simon |6666
|john |7777
|num_se |Total
---------------
|1111 |12
|2222 |15
|3333 |10
|4444 |8
|5555 |20
|6666 |18
|7777 |22
So, what i need is get the sum of 'Total', from each 'Name'. What i want to get is, list of the name "Uniquely" (in this example is Andre, John, and Simon) with each sum of "Total" that get from num_se.
sorry my english is bad, but i hope you're understand.
It would have been pertinent to explain you were working against two separate data sources up front.
It's slow because you're attempting to loop over your resultsets in PHP, calling individual queries to return single rows from another table, which is very inefficient.
Let the database handle the relationship between your two related tables by using a JOIN , and then let it handle to aggregation of your grand total, not just the individual totals.
The following query will get your total for each distinct pair of nip_ar and nama_ar:
SELECT
t1.NIP_AR,
t1.NAMA_AR,
SUM(t2.jumlah_bayar) as PENERIMAAN
FROM
USRBPS.MASTERFILE t1
INNER JOIN mpnruteng t2
ON t2.npwp = t1.npwp
GROUP BY
1, 2
But you're rolling this up into one grand $total anyhow, so this will get that for you:
SELECT
SUM(PENERIMAAN) as PENERIMAAN
FROM
(
SELECT
t1.NIP_AR,
t1.NAMA_AR,
SUM(t2.jumlah_bayar) as PENERIMAAN
FROM
USRBPS.MASTERFILE t1
INNER JOIN mpnruteng t2
ON t2.npwp = t1.npwp
GROUP BY
1, 2
)
You can configure a link to your mySql server from your Oracles server. Refer to Using Heterogenous Service Agents - chapter 4. Setting up access to non-Oracle systems. My understanding is that you cannot link to Oracle from mySql.
This will allow you to run the above queries, on your Oracle instance. Although you'll need to update your table names to fully qualify them with the datasource names.
If I understand correctly, this query may help
SELECT t1.Name, sum(t2.Total)
FROM (table1 t1 LEFT JOIN table2 t2 ON t1.num_se = t2.num_se)
GROUP BY t1.Name
Modify the table name and column name based on your case.
Update
I don't know why you use two separated data source. However, I think this approaches will improve the efficiency of your code.
First, get the list of all num_se belong to each name by this query:
"SELECT Name, GROUP_CONCAT(num_se) as nums FROM table1 GROUP BY Name"
Now your result array ('$ar') will have an element like this "1111,2222" with the key nums
Second, use this query to get the sum for each name
"SELECT SUM(Total) as total FROM Table2 WHERE num_se IN (" . $ar['nums'] . ")"
And you will get the total for each name without needing a second loop.
Remember to use escaping techniques to be sure your queries is safe.
If you're on Windows, I have noticed that using localhost instead of 127.0.0.1 slows down the connection to the database.
Before giving any solution I wanted to share the reason why your query is too slow. If you see your code carefully you are opening database connection for reach record in the first query result. The major cost is going in opening and running the query against the database . SO imagine if you have 1000 records in the first result , you will open the database connection 1000 times , that will make it very very slow. Use the Inner join or subquery as stated by Trung . Or if not possible using stored procedure and putting the logic inside the SP will also help you to gain the performance.Create a Stored procedure and pass the input parameter as comma separated Ids what you are getting from another database , Inside the SP use the comma separated Ids to either loop and do the select statement. You can use the temp table to do so. The advantage is that you will open the Database connection only once.
I am fairly new to MSSQL and have never used In-Memory Tables and am not 100% sure if this is what I need.
I have a result set from a query (which cannot be amended) and I loop through and display each row of data. For one field of the result set I need to perform a query to get the relevant display data for that field. The issue is I may have to potentially call this query 1000's of times within the loop depending on how many rows there are in the result set.
Can anyone advise on ways to do this efficiently? I have heard of In Memory tables and am wondering if this is what I need? If so where do I start? Or do I simply store in an array or something?
Any advice much appreciated.
Declare #Test_Memory_Table Table
(
/* Fields needed for lookup; use same datatypes as schema */
IndexOrPkFieldName Numeric(19,0),
Field1 varchar(255),
Field2 date
)
Insert into #Test_Memory_Table Select t2.Field1 as field1, t1.field2 as Field2, CONVERT(char(10),t3.Field3, 101) as Field3
From Table1 t1
INNER JOIN Table2 t2 on t1.pkId = t2.pkId and isnull(t2.IS_ACTIVE,0) = 1 and ISNULL(t2.TYPE, 0) > 0
INNER JOIN Table3 t3 on t2.pkId = t3.pkId
select * from #Test_Memory_Table
Just test the query in SSMS and look at the plan to see how long it takes to return the memory table query versus querying the table directly. remember that ssms can be faster than production because of hints defaulted in ssms (e.g. arithabort), but may not be set that way when querying through .net client. If your tables are small I would expect the difference to be marginal.
I'm trying to write a statement that will check 2 tables and compare the 2 and only show the users who are not on table2. For example:
table1 has user1, user2, user3, and user4 AND
table2 has user1, user3, and user4 then it would only show user2 in the list.
I was able to write something as basic as to only show member_id='1' but I don't how to make it work the way I would like. I'm hoping you understand my question. I'll include the short code that deals with this issue...
$sql->db_Select(DB_TABLE_ROSTER_MEMBERS, "*", "member_id='1' AND
(member_status like 'Team Member%'
OR member_status like 'Squad Leader%'
OR member_status like 'Recruiter%'
OR member_status like 'New Recruit%'$customArgs)
ORDER BY member_application_date DESC");
the syntax for this can be located here- http://wiki.e107.org/index.php?title=Db#db_Select.28.29
I'm guessing I just need to make another select DB and just have it included in the above so it would be something like
member_id != $member_id
where $member_id would be something like $member_id = $row['member_id'];
You need to make a LEFT JOIN and check where table2.member_id IS NULL.
First, AND is an operator ;)
Second, there are many relational operators that imply logical AND e.g. intersect, join, product (SQL CROSS JOIN), restriction (SQL WHERE, extension (SQL AS clause) and matching/difference (SQL MATCH or EXISTS or IN).
Third, from your first sentence I rather think you are looking for relational operators that imply logical NOT i.e. non-existential qualification. Again, there are many options e.g. minus (SQL EXCEPT), not matching/semi-difference (SQL NOT EXISTS or NOT IN but bizarrely not NOT MATCH!); personally, I find OUTER JOIN testing for nulls to be the most odd (not to mention not relational)!
Fourth, your relational/SQL product of choice will likely not support and/or equally optimize all the different options from a given relational algebra or SQL Standard.
My personal choice for semi-difference is NOT EXISTS: the name clearly implies non-existential qualification, the predicates are located close together in SQL code, is widely supported and usually well optimized:
SELECT *
FROM table1
WHERE NOT EXISTS (
SELECT *
FROM table2
WHERE table2.member_id = table1.member_id -- predicates close together
);
If you had data in table 1 that you need to use to return data in table 2 for each row returned in table 1. What is more efficient to use a set of querys in PHP one inbedded in the while loop of the other or an SQL function within a query?
for example:
$qry = mysql_query(SELECT date FROM table1)
while($res = mysql_fetch_array($qry))
{
$qry = mysql_query("SELECT name FROM table2 WHERE date=$res['date']")
}
or to do this as a function that returns the Id from table1 within the query.
A (LEFT / RIGHT) JOIN?
Unless I've misunderstood the question...
I think you're looking for JOIN sql syntax. If you have 2 tables: messages and author and you want to return messages with authors. Then you can write following SQL statement:
SELECT m.body, a.name FROM message m
LEFT JOIN author a ON (a.id=m.author_id)
This will return message body with corresponding author name
Table author:
id - primary key
name - name of the author
Table message:
body - text of the message
author_id - id of the author
UPD1:
This will be faster then looping each message and select an author. Because JOIN allows you to return all data in single query (not N x queries when using for loop).
UPD2:
With your tables the query will look like:
SELECT t1.date, t2.name FROM table1 t1 LEFT JOIN table2 t2 ON (t2.date=t1.date)
It depends on if the data is easier to find during the while loop or in the query itself.
So if the DB has to base the sub-query on the result of each row in the main query, and there are 1000 total rows and 100 results in the main query, it has to check all of the rows 100 times, so that's 100,000 sub-queries it runs.
So think it terms of the number of results of the main query. If your while loop has to query the DB 100 times while the DB can do it much faster and efficiently in one query, that's better. But if you want a subset of answers that you can say 'query only based on the last set of results' the while loop is better.
What is more efficient to use
a set of querys in PHP one inbedded in the while loop of the other
or
an SQL function within a query
Seems you answered your question yourself, didn't you?
Every query you send to the dbms has to be sent over the network, parsed, analyzed then executed. You may want to minimize the number of queries sent to the db.
There may be exceptions, for example if the processing of the data requires operations which the dbms is not capable of.