how to optimize database connections for nested queries? - php

Question

I am using nested queries to retrieve information from multiple tables. I need advice on optimizing this php code.
This function creates an object.
public function conn($query){
$mysqli = new mysqli('test','test','test','test');
$result = $mysqli->query("SET NAMES utf8");
$result = $mysqli->query("set character_set_client='utf8'");
$result = $mysqli->query("set collation_connection='utf8_general_ci'");
$result = $mysqli->query($query);
$mysqli->close();
return $result;
}
This code uses that function.
$connect = $this->conn("SELECT * FROM Table LIMIT 100000");
while($i = $connect->fetch_assoc()){
$name = $i["name"];
$connect2 = $this->conn("SELECT * FROM Names WHERE Name = '$name'");
if($connect2 ->num_rows > 0){
echo $name.'<br>';
}
}
Need recommendations for a connection to the database.
In the while loop, as you see, I am checking for the presence of $name in other table. But I am opening and closing a connection every time through the loop. And this will be 100001 connection opens and closes.
Is it possible to open a connection to the database only once?
P.S.: The SQL is an example - Please don't suggest changes there, because I am trying to figure out how to handle the repeated queries, not optimize the SQL.

Answer 1

Connection objects are reusable. Make a connection, then use it to make as many queries as you want. Close each query (that is, each result set) when you're done with it, then close the connection at the end of the run.
Closing a connection is a network operation, so it takes a while. Closing a query is mostly an in-memory operation, so it is faster.
In your example, you're using nested queries (more on that in a moment). Your code should end up looking something like this pseudocode:
public function getconn(){
$mysqli = new mysqli('test','test','test','test');
$mysqli->query("SET NAMES utf8");
$mysqli->query("set character_set_client='utf8'");
$mysqli->query("set collation_connection='utf8_general_ci'");
return $mysqli; /* return the connection handle */
}
$conn1 = getconn();
$conn2 = getconn();
$resultset1 = $conn1->query("SELECT * FROM Table LIMIT 100000");
while($i = $resultset1->fetch_assoc()){
$name = $i["name"];
$resultset2 = $conn2->query("SELECT * FROM Names WHERE Name = '$name'");
if($resultset2->num_rows > 0){
echo $name.'<br>';
}
$resultset2->close();
}
$resultset1->close();
$conn1->close();
$conn2->close();
(Please note; I haven't debugged this code.)
To take this optimization one step further, you should consider using a prepared statement for the query inside the while loop. Here's documentation on that http://php.net/manual/en/mysqli-stmt.fetch.php.
$conn1 = getconn();
$conn2 = getconn();
/* create a prepared statement with placeholder parameter ? */
$stmt = $mysqli->prepare("SELECT * FROM Name WHERE Name = ?"));
$name = '';
$name_out = '';
$stmt->bind_param("s", $name);
$stmt->bind_result($name_out);
$resultset1 = $conn1->query("SELECT * FROM Table LIMIT 100000");
while($i = $resultset1->fetch_assoc()){
$name = $i["name"];
$resultset2 = $stmt->execute(); /* run query with bound parameter */
if ($stmt_fetch() ( {
echo $name.'<br>';
}
$resultset2->close();
}
$resultset1->close();
$conn1->close();
$conn2->close();
(Please note; I haven't debugged this code either.)
Now, it's possible your pair of queries are just an example to show a set of nested queries. If so, that's fine. But, you are performing this task (retrieve 100K names) in an almost unimaginably inefficient way. You've said you don't want anybody to rewrite this query, but I am sorry, I can't just let this one pass.
This code would do a far more streamlined job.
$conn = getconn();
$q = "SELECT t.name FROM Table t JOIN Name n ON t.name = n.name LIMIT 100000";
$resultset = $conn->query($q);
while($i = $resultset->fetch_assoc()){
$name = $i["name"];
echo $name.'<br>';
}
$resultset->close();
$conn->close();
It's more efficient for two reasons. First, it doesn't use SELECT *, which ends up sending all sorts of data over the network from your MySQL server to your php program, just to throw it away.
Second, it doesn't use the nested queries. Instead, the JOIN query pulls all the name columns from Table that have a matching name column in Names.