Fixing performance issues when looping through MySQL results in PHP - php

I'm working on a virtual game board style game in which players get points on a certain area of the board. (Take it easy on me now as I only do this as a hobby so I might be doing this in the worst way possible)
I have 3 tables. One stores all the player information (eg. id, screenname). A second stores all the Area information (eg. id, x, y) and a third stores how many points each player has in each area (eg. id, playerid, areaid, points). In order to create a "leaderboard" I'm looping through all the players, then within that loop I'm also looping through all the areas, then finally within that second loop, I get the leader of that area and see if that matches the current player in the first loop, if so I increment a counter, then store it into an array. (See code below with some commenting)
I looked into MySQL caching, but I dont have access to a lot of the server options, as well as would like to keep as much of the results as live as possible, so caching may not be the right way to go.
My question is whether or not I'm doing this properly. Currently there is only around 10 players, and approx. 500 areas. I'm finding the below script already takes about 5-8 seconds to run. Potentially there could be millions of areas, so such a long delay in processing could be catastrophic (for the leaderboard anyway). Am I going about this the right way, and/or is there a better way to do this?
<?php
$leaders = array();
//Loop through all the players
$sql = "SELECT * FROM players";
$result = mysqli_query($con, $sql) or die(mysqli_error($con));
while ($row = mysqli_fetch_array($result)) {
//save player information into variables
$playerId = $row['id'];
$playerScreenName = $row['screenname'];
//Reset the area counter
$AreaCount = 0;
$leader = array();
//Loop through all areas
$sql2 = "SELECT * FROM areas";
$result2 = mysqli_query($con, $sql2) or die(mysqli_error($con));
while ($row2 = mysqli_fetch_array($result2)) {
$areaId = $row2['id'];
//Get the player with the most points in that area
$sql3 = "SELECT * FROM points WHERE areaid='$areaId' ORDER BY totalpoints DESC LIMIT 1";
$result3 = mysqli_query($con, $sql3) or die(mysqli_error($con));
while ($row3 = mysqli_fetch_array($result3)) {
$leaderOfArea = $row3['playerid'];
//See if the leader of the area is the same player we are looping through
if ($playerId == $leaderOfArea) {
//if it is, then increment the counter
$AreaCount++;
}
}
}
//Store the leader information into an array to be output later
$leader['screenname'] = $playerScreenName;
$leader['areacount'] = $AreaCount;
$leaders[] = $leader;
}
// sort leaders by score
usort($leaders, 'compare_areacount');
?>

There is overhead of opening database connections, and when you do it in a loop, you exacerbate the problem (and then when you add a loop inside of that, you make it that much worse). Instead, restructure it as one query using a Join or Subquery.
This post has some specifics.

I think this code can help. You might have to change the sql queries with appropriate column names you need.
<?php
$leaders = array();
//Loop through all the players
$sql = "SELECT * FROM players";
$result = mysqli_query($con, $sql) or die(mysqli_error($con));
$players = array();
while ($row = mysqli_fetch_array($result)) {
//save player information into variables
$players[$row['id']] = array($row['screenname'], 0);
// number 0 will be the count of how many times this player is the leader
}
$sql = "SELECT Area.id, Area.name, (SELECT Pts.playerid FROM `points`"
. " AS Pts WHERE Pts.areaid=Area.id ORDER BY totalpoints"
. " DESC LIMIT 1) AS `leader_id` FROM `areas` AS Area";
$result = mysqli_query($con, $sql) or die(mysqli_error($con));
$areas = array();
while ($row = mysqli_fetch_row($result)) {
$areas[$row[0]] = $row;
// $row[2] will contain leader_id
// index 1 corresponds to the second element in player values array
$players[$row[2]][1]++;
}
// now if you want to print:
foreach ($areas as $area_id => $area) {
echo "Area id: " . $area_id . ", name: " . $area[1] . ", leader_id: " . $area[2] . "<br /><br />";
}
foreach ($players as $player_id => $player) {
echo "Player id: " . $player_id . ", name: " . $player[0] . ", No of areas this player is a leader of: " . $player[1] . "<br /><br />";
}

Related

How to count amount of students per university

I have a SQL database of university students, with the following details:
Table_name: register
Column_names: position, tertinst
The data in the database will look something like this:
Coach..........UCT
Athlete........Tukkies
Official.......University of JHB
Athlete........Tukkies
Athlete........Tshwane Tech
Manager........UCT
I need to count the amount of students who are athletes(position), per university(tertinst) and the output has to be something like this:
UCT.....................735
University of Jhb.......668
Tukkies.................886
this is my attempt:
$positionx = 'Athletes';
include_once 'core/includes/db_connect.php';
$sql = "SELECT tertinst, COUNT(position) FROM register WHERE position = '$positionx' GROUP BY tertinst ";
$result = $conn->query($sql);
while ($row = mysql_fetch_array($result)) {
echo $row['COUNT(tertinst)'] . '......' . $row['COUNT(position)'];
}
$conn->close();
I get no result when the code is executed and I have searched for a different solution for hours, without success.
I made a few mistakes, and corrected the syntax in the sql count, as well as the echo. Here is the solution to my problem:
<?php
include_once 'core/includes/db_connect.php';
$sql = "SELECT tertinst, COUNT(*) total FROM register WHERE position = 'Athletes' GROUP BY tertinst ";
$result = $conn->query($sql);
while ($row = $result->fetch_assoc()) {
$tertinst = $row["tertinst"];
echo $tertinst . '......' . $row['total'] . '<br>';
}
$conn->close();

PHP mysql query issue when filtering from array and variables

On my webpage (html, php), I'm trying to get it such that users may select multiple checkboxes, each checkbox effectively filters what results the users see. Info is pulled from the database (MySQL) based on the values in different columns. As shown below, one column is Joint_1, another column is Position.
Effective code that WORKS for filtering (very static, not practical to use obviously) is this:
$sql = "SELECT * FROM `Table_Name` WHERE (Joint_1=\'region1\' OR
Joint_1=\'region2\' OR
Joint_1=\'region3\' OR
Joint_1=\'region4\') AND
(Position=\'position1\' OR
Position=\'position2\' OR
Position=\'position3\')";
$result = $conn->query($sql);
if($result->num_rows > 0) {
while($row = $result->fetch_assoc()) {
echo $row["Common_Name1"] . "<br>";
}
} else {
echo "0 results";
}
Below code is attempts at above code, but using arrays, which does NOT work.
$regions =
array('region1', 'region2', 'region3', 'region4');
$position = array('position1', 'position2', 'position3');
$sql = "SELECT * FROM `Table_Name` WHERE (Joint_1=\'. $regions[0] .\' OR
Joint_1=\'. $regions[1] .\' OR
Joint_1=\'. $regions[2] .\' OR
Joint_1=\'. $regions[3] .\') AND
(Position=\'. $position[0] .\' OR
Position=\'. $position[0] .\' OR
Position=\'. $position[0] .\')";
Above code provides results of '0 results.'
I've attempted to perform this numerous times, with additional NON-FUNCTIONAL CODE also below (below attempting to filter based on only 1 column as I have obviously not mastered the code to approach filtering based on 2 columns).
$sqlregion = array();
foreach ($_POST['region'] as $reg) {
$sqlreg[] = "'" . mysql_real_escape_string($reg) . "'";
}
$sql = "SELECT * FROM 'Exercises' WHERE Joint_1 IN (" . implode(",",
$sqlreg) . ");";
$result=$conn->query($sql);
if($result->num_rows > 0) {
while($row = $result->fetch_assoc()) {
echo $row["Common_Name1"] . "<br>";
}
}
Any help is appreciated! Thanks.
You can construct this query from arrays, you can use the IN clause in mysql to specify multiple OR values
$regions = array('region1', 'region2', 'region3', 'region4');
$position = array('position1', 'position2', 'position3');
$regions = implode(",", $regions); // Converts array to string with comma as a separator
$position = implode(",", $position);
$sql = "SELECT * FROM `Table_Name` WHERE Joint_1 IN ($regions) AND Position IN ($position)";
echo $sql;
This gives you a query like this
SELECT * FROM Table_Name WHERE Joint_1 IN (region1,region2,region3,region4) AND Position IN (position1,position2,position3)
add in your own LIMIT GROUP BY or ORDER BY parameters to suit your needs

How to loop for every value in table AND find average

I am a beginner programmer and I am trying to find the average user rating of vendor 2.
My steps are as follows:
1. User 'Jimmy' gives a rating of 3 to vendor 2 (ratings are out of 5)
2.TABLE vendoratings is updated
3. ratingstot.php then calculates the total sum of ratings and the number of responses for ALL vendors before updating TABLE vendortotalratings shown below
4.User 'Jimmy' clicks on 'View average user rating'
5. The values totalratings and totalno are retrieved and divided in Javascript
var average= totalratings/totalno
6. totalno is displayed to user Jimmy. END
Question
1. I need help forming the for or while loop in ratingstot.php to calculate both ratings and no. of responses belonging to vendor X before inserting them in vendortotalratings table for every vendor.
ratingstot.php
<?php
header("Access-Control-Allow-Origin: *");
header("Content-Type: application/json; charset=UTF-8");
error_reporting(E_ERROR);
try{
//Database connection
$conn = new mysqli("localhost", "XXXXXXXX_XXX", "XXXXXXXX", "XXXXXXXX");
//Unsure how to loop this to make vendor new value every loop
for($i=0; $i<=6; $i++){
$vendor = ??
//Calculate sum of ratings from table ratings
$result = $conn->query("SELECT SUM(ratings) FROM ratings WHERE vendorid = '".$vendorid."' ");
$row = mysqli_fetch_array($result);
$totalratings = $row[0];
//Calculate no. of responses (by counting no. of rows)
$result1 = $conn->query("SELECT * FROM ratings WHERE vendorid = '" . $vendorid."' ");
$totalno = mysqli_num_rows($result);
//inserting the results into the table
$query = " UPDATE vendortotalratings SET ";
$query .= " totalratings = '". $totalratings ."', totalno='".$totalno."' ";
$query .= " WHERE vendorid = '". $vendorid ."'";
$result2 = $conn->query($query);
}
echo($outp);
}
catch(Exception $e) {
$json_out = "[".json_encode(array("result"=>0))."]";
echo $json_out;
}
?>
I have no idea how to loop this, are there any easier steps to calculate average of ratings for each vendors?
Instead of all these complicated things, you can simplify the required solution like this:
(Assumption: I'm assuming that vendorid, ratings, totalratings and totalno columns are of type INT)
Use the below statement/query to get the totalratings and totalno corresponding to each vendorid.
$result = $conn->query("SELECT vendorid, SUM(ratings) as totalratings, COUNT(userid) as totalno FROM vendorratings GROUP BY vendorid");
Now loop through the $result result set using while() loop.
while($row = $result->fetch_array()){
...
}
In each iteration of above while() loop, check if the vendorid value already exists or not. If it exists, then UPDATE the row with new totalratings and totalno, otherwise INSERT a new row comprising of vendorid, totalratings and totalno.
while($row = $result->fetch_array()){
$res = $conn->query("SELECT vendorid FROM vendortotalratings WHERE vendorid = " . $row['vendorid']);
if($res->num_rows){
// Update the existing row
$conn->query("UPDATE vendortotalratings SET totalratings = ".$row['totalratings'].", totalno = ".$row['totalno']." WHERE vendorid = ".$row['vendorid']);
echo "Affected rows: " . $conn->affected_rows . '<br />';
}else{
// Insert a new row
$res = $conn->query("INSERT INTO vendortotalratings VALUES(".$row['vendorid'].", ".$row['totalratings'].",".$row['totalno'].")");
if($res) echo "New row inserted <br />";
}
}
So the complete code of try-catch block would be like this:
// your code
try{
//Database connection
$conn = new mysqli("localhost", "XXXXXXXX_XXX", "XXXXXXXX", "XXXXXXXX");
$result = $conn->query("SELECT vendorid, SUM(ratings) as totalratings, COUNT(userid) as totalno FROM vendorratings GROUP BY vendorid");
while($row = $result->fetch_array()){
$res = $conn->query("SELECT vendorid FROM vendortotalratings WHERE vendorid = " . $row['vendorid']);
if($res->num_rows){
// Update the existing row
$conn->query("UPDATE vendortotalratings SET totalratings = ".$row['totalratings'].", totalno = ".$row['totalno']." WHERE vendorid = ".$row['vendorid']);
echo "Affected rows: " . $conn->affected_rows . '<br />';
}else{
// Insert a new row
$res = $conn->query("INSERT INTO vendortotalratings VALUES(".$row['vendorid'].", ".$row['totalratings'].",".$row['totalno'].")");
if($res) echo "New row inserted <br />";
}
}
}catch(Exception $e) {
$json_out = json_encode(array("result"=>0));
echo $json_out;
}
The question doesn't clarify how the records are first inserted in vendortotalratings table. So, assuming that there is already a record in this table for each vendor, you don't have to write a whole new loop.
Updating vendortotalratings:
SQL can take care of calculating the total ratings and their counts in a single query which can then replace the loop that you have.
UPDATE vendortotalratings vtr
INNER JOIN
(
SELECT vendorid, SUM(ratings) AS sumratings, COUNT(ratings) AS countratings
FROM vendoratings
GROUP BY vendorid
) vr
ON vtr.vendorid = vr.vendorid
SET
vtr.totalratings = vtr.totalratings + vr.sumratings
,vtr.totalno = vtr.totalno + vr.countratings
Computing averages:
As for your second question, to compute the average, you could run the following query which will give you the run-time result:
SELECT vendorid, totalratings, totalno,
CAST((totalratings/totalno) AS DECIMAL(10, 2)) AS avgrating
FROM vendortotalratings;
The variable avgrating can be accessed directly in PHP by using $row['avgrating'] if you're fetching an associative array from the results, or by using the appropriate index number, which in this case should be $row[3]

How can i simplify this php mysql count code and reduce queries?

I have database with 8 different product category for download.
pic, app, ebo, tem, des, cod, mus, cat
I'd like to count clients total downloaded products and total downloads of each product category.
Maximum daily limit downloads for category product is 3.
When user log in should see how many downloads remain.
Here is working code
$query = "SELECT COUNT(*) as sum FROM service_downloads where client_id like '$client'";
$result = mysql_query($query) or die(mysql_error());
// Print out result
while($row = mysql_fetch_array($result))
{
echo "You have downloaded". $row['sum'] ." products.";
echo "<br />";
}
$query = "SELECT COUNT(*) as sum FROM service_downloads where client_id like '$client' and product like 'pic'";
$result = mysql_query($query) or die(mysql_error());
// Print out result
while($row = mysql_fetch_array($result))
{
echo "". $row['sum'] ." downloaded pictures";
$leftovers = 3 - $row['sum'];
echo " $leftovers pictures remain for download";
echo "<br />";
}
$query = "SELECT COUNT(*) as sum FROM service_downloads where client_id like '$client' and product like 'app'";
$result = mysql_query($query) or die(mysql_error());
// Print out result
while($row = mysql_fetch_array($result))
{
echo "". $row['sum'] ."downloaded applications";
$leftovers = 3 - $row['sum'];
echo " $leftovers applications remain for download.";
echo "<br />";
}
$query = "SELECT CO.... This procedure repeat eight times for different product category.
result
You have downloaded 12 products.
3 downloaded pictures 0 pictures remain for download.
1 downloaded applications 2 applications remain for download.
3 downl.......
You could use a GROUP BY statement to group your results.
SELECT COUNT(`Product`) AS `Sum`, `Product`
FROM `service_downloads`
WHERE `client_id` = '<client_id>'
GROUP BY `Product`
Then you can use one while statement to loop through each product:
// Print out result
while($row = mysql_fetch_array($result))
{
echo "". $row['Sum'] ."downloaded " . $row['Product'];
$leftovers = 3 - $row['Sum'];
echo " $leftovers " . $row['Product'] . " remain for download.";
echo "<br />";
}
$query = "SELECT COUNT(*) as sum,product FROM service_downloads where client_id like '$client' GROUP BY product";
$result = mysql_query($query) or die(mysql_error());
// Print out result
while($row = mysql_fetch_array($result))
{
echo "You have downloaded". $row['sum'] ." ".$row['product'];
echo "<br />";
}
This should work
You should breakdown the quantity of downloads per category in one query:
SELECT product,COUNT(*)
FROM service_downloads
WHERE client_id like '$client';
I also don't think you need to use LIKE; you probably want to use =
You can get a single result set with all the sums in it with this query.
SELECT COUNT(*) as sum, product
FROM service_downloads
WHERE client_id = '$client'
AND PRODUCT IN ('pic', 'app', 'abc', 'def', 'ghi')
GROUP BY product WITH ROLLUP
ORDER BY product NULLS FIRST
This will give you one row for each specific product and a summary (rollup) row with a NULL value in the product column.
If this query takes a long time create an index on (client, product) and it should go pretty fast.
If you are showing this data frequently, which is what it sounds like, then you should have a separate table that represents those SUMs and is index by CLIENT_ID.
You can then increment/decrement that value each time you add a new entry.
For example, when you add a new row to service_downloads with an entry in 'pic' for CLIENT_ID 1, then you would also increment this shortcut table:
UPDATE service_counts SET pic=pic+1 WHERE client_id=1;

max(id) and limit 10, but use them in different places

I have two tables, posts and sections. I want to get the last 10 posts WHERE section = 1,
but use the 10 results in different places. I make a function:
function sectionposts($section_id){
mysql_set_charset('utf8');
$maxpost1 ="SELECT max(id) from posts WHERE section_id = $section_id ORDER BY ID DESC LIMIT 20";
$maxpost12 =mysql_query($maxpost1);
while ($maxpost_rows = mysql_fetch_array($maxpost12 ,MYSQL_BOTH)){
$maxpost2 = $maxpost_rows[0];
}
$query = "SELECT * FROM posts WHERE id = $maxpost2";
$query2 = mysql_query($query);
return $query2;
}
$query2 = sectionposts(6);
while ($rows = mysql_fetch_array($query2)){
echo $rows['title'] . "<br/>" . "<br/>";
echo $rows['id'] . "<br/>" . "<br/>";
echo $rows['image_section'] . "<br/>";
echo $rows['subject'] . "<br/>";
echo $rows['image_post'] . "<br/>";
}
How can it take these ten results but use them in different places, and keep them arranged from one to ten.
this was the old case and i solve it but i found another problem, that, if the client had deleted a post as id = 800 "so there aren't id = 800 in DB" so when i get the max id minus $NUM from it, and this operation must be equal id = 800, so i have a programing mistake here, how can i take care of something like that.
function getmax_id_with_minus ($minus){
mysql_set_charset('utf8');
$maxid ="SELECT max(id) FROM posts";
$maxid1 =mysql_query($maxid);
while ($maxid_row = mysql_fetch_array($maxid1)){
$maxid_id = $maxid_row['0'];
$maxid_minus = $maxid_id - $minus;
}
$selectedpost1 = "SELECT * FROM posts WHERE id = $maxid_minus";
$query_selectedpost =mysql_query($selectedpost1);
return ($query_selectedpost);
}
<?php
$ss = getmax_id_with_minus (8);
while ($rows = mysql_fetch_assoc($ss)){
$main_post_1 = $rows;
?>
anyway "really" thanks again :) !
A few thoughts regarding posted code:
First and foremost, you should stop using mysql_ functions as they are being deprecated and are vulnerable to SQL injection.
$maxpost1 ="SELECT max(id) from posts WHERE section_id = $section_id ORDER BY ID DESC LIMIT 20";
When you SELECT MAX, MIN, COUNT, AVG ... functions that only return a single row, you do not need an ORDER BY or a LIMIT.
Given that you are only asking for the MAX(id), you can save work by combining your queries like so:
SELECT * FROM posts
WHERE id = (SELECT MAX(id) from posts WHERE section_id = $section_id)
If I'm understanding what you're trying to do (please correct me if I'm wrong), your function would look something like:
function sectionposts($section_id) {
$link = mysqli_connect("localhost", "my_user", "my_password", "world");
$stmt = mysqli_prepare($link, "SELECT title, id, image_section, subject, image_post FROM posts "
. "WHERE section_id = ? ORDER BY id DESC LIMIT 10");
mysqli_stmt_bind_param($stmt, $section_id);
return mysqli_query($link, $stmt)
}
$result = sectionposts(6);
while ($row = mysqli_fetch_assoc($result)) {
echo $rows['title'] . "<br /><br />";
echo $rows['id'] . "<br /><br />";
echo $rows['image_section'] . "<br />";
echo $rows['subject'] . "<br />";
echo $rows['image_post'] . "<br />";
}
Try this instead, to save yourself a lot of pointless code:
$sql = "SELECT * FROM posts WHERE section_id=$section_id HAVING bar=MAX(bar);"
$result = mysql_query($sql) or die(mysql_error());
$row = mysql_fetch_assoc($result);
echo ...;
echo ...;
The having clause lets you find the max record in a single operation, without the inherent raciness of your two-query version. And unless you allow multiple records with the same IDs to pollute your tables, removing the while() loops also makes things far more legible.
Seems like you want to store them in an array.
$posts = array(); //put this before the while loop.
$posts[] = $row; //put this in the while loop

Categories