Exhausting Memory - Tried Fixing Loops, Still Does Not Work - php

I'm currently experiencing a memory usage issue - but I cannot figure out where. I've tried replacing some of my foreach loops with for loops or by issuing another query to the DB, but I am still gettting the same error - "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 72 bytes) in on line 109". Can anyone provide some insight as to what may be causing the issue? Thank you!
Code after #Patrick 's answer:
$participating_swimmers = array();
$event_standings = array();
$qualifying_times = array();
$events = array();
$current_event = '';
$select_times_sql = "SELECT event, time, name, year, team, time_standard, date_swum
FROM demo_times_table
WHERE sex = 'M' AND (time_standard = 'A' OR time_standard = 'B')
ORDER BY event, time ASC";
$select_times_query = mysql_query($select_times_sql);
//Create array with the current line's swimmer's info
while ($swimmer_info = mysql_fetch_assoc($select_times_query)) {
if($current_event != $swimmer_info['event']){
$events[] = $current_event = $swimmer_info['event'];
}
//Create array with the current line's swimmer's info
$swimmer_info["time"] = $select_times_row['time'];
$swimmer_info["name"] = $select_times_row['name'];
$swimmer_info["year"] = $select_times_row['year'];
$swimmer_info["team"] = $select_times_row['team'];
$swimmer_info["time_standard"] = $select_times_row['time_standard'];
$swimmer_info["date_swum"] = $select_times_row['date_swum'];
//Create "Top 8" list - if more than 8 A cuts, take them all
if (($swimmer_info["time_standard"] == "A") || ($swimmer_info["time_standard"] == "B")) {
//Check if there are 8 or less entries in the current event, or if the swim is an A cut
if ((count($event_standings[$current_event]) < 8) || ($swimmer_info["time_standard"] == "A")) {
//Add swimmer to the list of invites
$event_standings[$current_event][] = $swimmer_info;
//Keep only the identifying information about the swimmer
$condensed_swimmer_info["name"] = $swimmer_info["name"];
$condensed_swimmer_info["year"] = $swimmer_info["year"];
$condensed_swimmer_info["team"] = $swimmer_info["team"];
//Check if swimmers name already appears in list
if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
//It is a unique user - add them to the list
$participating_swimmers[] = $condensed_swimmer_info;
}
} else {
//Add the qualifying time that did not fit into the list to a list of qualifying times
$qualifying_times[$current_event][] = $swimmer_info;
}
}
}
//Sort each array of times in descending order
arsort($event_standings);
arsort($qualifying_times);
$num_of_swimmers = count($participating_swimmers);
while ($num_of_swimmers < 80) {
foreach ($events as $loe) {
$num_of_qualifying_times = count($qualifying_times[$loe]);
$swimmer_info = $qualifying_times[$loe][$num_of_qualifying_times-1];
$event_standings[$loe][] = $swimmer_info;
//Keep only the identifying information about the swimmer
$condensed_swimmer_info["name"] = $swimmer_info["name"];
$condensed_swimmer_info["year"] = $swimmer_info["year"];
$condensed_swimmer_info["team"] = $swimmer_info["team"];
//Check if swimmers name already appears in list
if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
//It is a unique user - add them to the list
$participating_swimmers[] = $condensed_swimmer_info;
}
//Remove time from array of qualifying times
unset($qualifying_times[$loe][$num_of_qualifying_times-1]);
}
$new_num_of_swimmers = count($participating_swimmers);
if($num_of_swimmers == $new_num_of_swimmers) break;
else $num_of_swimmers = $new_num_of_swimmers;
}
arsort($event_standings);
arsort($qualifying_times);
foreach($event_standings as $loe => $event_swimmer) {
echo "<h1>",$loe,"</h1><br />";
foreach ($event_swimmer as $es) {
echo $es["time"]," ",$es["name"]," ",$es["team"],"<br />";
}
}

Large data in database is the problem 95% !
- try using limit x,y in your queries , and put those queries in some loop .
- see http://php.net/manual/en/function.mysql-free-result.php it might help
<?php
$participating_swimmers = array();
$event_standings = array();
$qualifying_times = array();
$select_times_sql = "SELECT *
FROM demo_times_table
WHERE sex = 'M'
ORDER BY time ASC";
$select_times_query = mysql_query($select_times_sql);
while ($select_times_row = mysql_fetch_assoc($select_times_query)) {
//Create array with the current line's swimmer's info
$swimmer_info["time"] = $select_times_row['time'];
$swimmer_info["name"] = $select_times_row['name'];
$swimmer_info["year"] = $select_times_row['year'];
$swimmer_info["team"] = $select_times_row['team'];
$swimmer_info["time_standard"] = $select_times_row['time_standard'];
$swimmer_info["date_swum"] = $select_times_row['date_swum'];
//Create "Top 8" list - if more than 8 A cuts, take them all
if (($swimmer_info["time_standard"] == "A") || ($swimmer_info["time_standard"] == "B")) {
//Check if there are 8 or less entries in the current event, or if the swim is an A cut
if ((count($event_standings[$current_event]) < 8) || ($swimmer_info["time_standard"] == "A")) {
//Add swimmer to the list of invites
$event_standings[$current_event][] = $swimmer_info;
//Keep only the identifying information about the swimmer
$condensed_swimmer_info["name"] = $swimmer_info["name"];
$condensed_swimmer_info["year"] = $swimmer_info["year"];
$condensed_swimmer_info["team"] = $swimmer_info["team"];
//Check if swimmers name already appears in list
if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
//It is a unique user - add them to the list
$participating_swimmers[] = $condensed_swimmer_info;
}
} else {
//Add the qualifying time that did not fit into the list to a list of qualifying times
$qualifying_times[$current_event][] = $swimmer_info;
}
}
}
mysql_free_result($select_times_query);
//Sort each array of times in descending order
arsort($event_standings);
arsort($qualifying_times);
$num_of_swimmers = count($participating_swimmers);
$sql = "SELECT DISTINCT(event)
FROM demo_times_table
WHERE sex = 'M' limit 80";
$query = mysql_query($sql);
while ($row = mysql_fetch_assoc($query)) {
$loe = $row['event'];
$num_of_qualifying_times = count($qualifying_times[$loe]);
$event_standings[$loe][] = $qualifying_times[$loe][$num_of_qualifying_times-1];
//Keep only the identifying information about the swimmer
$condensed_swimmer_info["name"] = $qualifying_times[$loe][$num_of_qualifying_times]["name"];
$condensed_swimmer_info["year"] = $qualifying_times[$loe][$num_of_qualifying_times]["year"];
$condensed_swimmer_info["team"] = $qualifying_times[$loe][$num_of_qualifying_times]["team"];
//Check if swimmers name already appears in list
if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
//It is a unique user - add them to the list
$participating_swimmers[] = $condensed_swimmer_info;
}
//Remove time from array of qualifying times
unset($qualifying_times[$loe][$num_of_qualifying_times-1]);
}
$num_of_swimmers = count($participating_swimmers);
mysql_free_result($query);
arsort($event_standings);
arsort($qualifying_times);
$sql = "SELECT DISTINCT(event)
FROM demo_times_table
WHERE sex = 'M'";
$query = mysql_query($sql);
while ($row = mysql_fetch_assoc($query)) {
$loe = $row['event'];
echo "<h1>".$loe."</h1><br />";
foreach ($event_standings[$loe] as $es) {
echo $es["time"]." ".$es["name"]." ".$es["team"]."<br />";
}
}
/*
foreach ($participating_swimmers as $ps) {
echo $ps["name"]."<br /><br />";
}
echo "<br /><br />";
*/
?>

Instead of doing a query within a query, with the potential for logic holes that create a never-ending loop, you can condense it into a single query. For your first block, both loops are just checking for the current event and the sex of the participant, right? So:
$result = SELECT * FROM <my database> WHERE sex = 'M' ORDER BY time ASC
Then you can pull whichever rows you want during the while ($row = mysql_fetch_assoc($result)) loop and compensate for non-distinct values in another way.
One other source of the never-ending loop could be in the block where you sort the qualifying times. You're using "unset" after each, which could be getting the pointer stuck. You could try adding array_values($qualifying_times) after the unset to reindex the array.

Frist off, lose the SELECT DISTINCT like so:
$events = array()
$current_event = '';
$select_times_sql = "SELECT event, time, name, year, team, time_standard, date_swum
FROM demo_times_table
WHERE sex = 'M' AND (time_standard = 'A' OR time_standard = 'B')
ORDER BY event, time ASC";
$select_times_query = mysql_query($select_times_sql);
//Create array with the current line's swimmer's info
while ($swimmer_info = mysql_fetch_assoc($select_times_query)) {
if($current_event != swimmer_info['event']){
$events[] = $current_event = $swimmer_info['event'];
}
//Create "Top 8" list - if more than 8 A cuts, take them all
//Check if there are 8 or less entries in the current event, or if the swim is an A cut
This also loses a bit of redundant code, and can speed up the final output (note the commas in the echo statements - the string doesn't need to be concatenated before it is spat out)
foreach($event_standings as $loe => $event_swimmer) {
echo "<h1>",$loe,"</h1><br />";
foreach ($event_swimmer as $es) {
echo $es["time"]," ",$es["name"]," ",$es["team"],"<br />";
}
}
The final problem lies in the second while loop, where the info being put into $condensed_swimmer_info doesn't have the -1 in place, thus is always blank, and $num_of_swimmers never rises to more than 1 over its original value:
while ($num_of_swimmers < 80) {
foreach ($events as $loe) {
$loe = $row['event'];
$num_of_qualifying_times = count($qualifying_times[$loe]);
$swimmer_info = $qualifying_times[$loe][$num_of_qualifying_times-1];
$event_standings[$loe][] = $swimmer_info;
//Keep only the identifying information about the swimmer
$condensed_swimmer_info["name"] = $swimmer_info["name"];
$condensed_swimmer_info["year"] = $swimmer_info["year"];
$condensed_swimmer_info["team"] = $swimmer_info["team"];
//Check if swimmers name already appears in list
if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
//It is a unique user - add them to the list
$participating_swimmers[] = $condensed_swimmer_info;
}
//Remove time from array of qualifying times
unset($qualifying_times[$loe][$num_of_qualifying_times-1]);
}
$new_num_of_swimmers = count($participating_swimmers);
if($num_of_swimmers == $new_num_of_swimmers) break;
else $num_of_swimmers = $new_num_of_swimmers;
}

Related

Sql statement into an array causing duplicates

this is my first time using the site. I need some help. When I run this code and get the output I get duplicated lines of data. So for instance a member maybe on the list 3 - 4 times depending on how many years he has gone to the events. I need to somehow combine any similar data into one row, if that makes sense? So John Doe should not be on the list 7 times just once with all the years he has gone.
$query_mem = "SELECT m.id, m.prefix, m.first_name, m.middle_name, m.last_name, m.suffix, m.member_class, m.jurisdiction_associated,
event.year,
m_event.attend
FROM members AS m, event AS event, members_event AS m_event $where
AND m.member_class IN (1,3)
AND m.id = m_event.codigo_usuario AND m_event.type_event = event.codigo_inscricao
ORDER BY event.year, m.jurisdiction_associated, m.last_name, m.first_name";
$retorno_mem = $conexao->query($query_mem);
if(($conexao->errorCode() == '0') && ($retorno_mem->rowCount() > 0))
{
while($registros_mem = $retorno_mem->fetch(PDO::FETCH_ASSOC))
{
$prefix=$fname=$mname=$lname=$suffix=$address_1=$address_2=$city=$state=$zip=$country=$org=$campaign=NULL;
$member_id = ($registros_mem["id"]);
$prefix = ($registros_mem["prefix"]);
$fname = ($registros_mem["first_name"]);
$mname = ($registros_mem["middle_name"]);
$lname = ($registros_mem["last_name"]);
$suffix = ($registros_mem["suffix"]);
$class = getMemberClassName($conexao,$registros_mem["member_class"]);
$jurisdiction = getJurisdictionName($conexao,$registros_mem["jurisdiction_associated"]."000");
$year = ($registros_mem["year"]);
$attendance = getMemberAttendanceName($conexao,$registros_mem["attend"]);
$data1[] = array(
''.utf8_encode((int)$member_id).'',
''.utf8_encode($prefix).'',
''.utf8_encode($fname).'',
''.utf8_encode($mname).'',
''.utf8_encode($lname).'',
''.utf8_encode($suffix).'',
''.utf8_encode($class).'',
''.utf8_encode($jurisdiction).'',
''.utf8_encode($year).'',
''.utf8_encode($attendance).'',
);
}
}
You can use the member's ID as the array key to see if he's already been, and if so just add to the year field. Something like this:
while($registros_mem = $retorno_mem->fetch(PDO::FETCH_ASSOC))
{
if ( ! array_key_exists( $registros_mem['id'], $data1)
{
// We need to make year an array so it can hold multiple years.
$registros_mem['year'] = [$registros_mem['year']];
$data1[$registros_mem['id']] = $registros_mem;
} else {
$data1[$registros_mem['id']]['year'][] = $registros_mem['year'];
}
}

How to avoid exponential slowdown in PHP/MYSQL?

I'm the owner of an online browser based game that has around 300 players signed up. I've written a script to detect cheaters, but the issue is that the number of queries in said script will grow exponentially.
It works like this:
Send a query that gets player's information.
Inside of the query, run another query that gets the information of every player.
So basically I am running a query that gets every player's name and information, and inside of that query I run another query to get the information from every other player besides themself. I use this to compare and delete cheaters.
The issue is, since I have 300 players, I have to run 300 queries per player. That's 90,000 queries. If I reach 1,000 players, it would be 1,000,000 queries. There has to be a better way to do this.
My code:
<?php
require '../connect.php';
$rulerinfo = $conn->query("SELECT id, rulername, nationname, alliance, email, dateregister, user_agent, lastseen, password FROM players");
while ($rulerinfo2 = $rulerinfo->fetch_assoc()) {
$id = $rulerinfo2['id'];
$rulername = $rulerinfo2['rulername'];
$nationname = $rulerinfo2['nationname'];
$alliance = $rulerinfo2['alliance'];
$email = $rulerinfo2['email'];
$dateregister = $rulerinfo2['dateregister'];
$useragent = $rulerinfo2['user_agent'];
$lastseen = $rulerinfo2['lastseen'];
$password = $rulerinfo2['password'];
$playerinfo = $conn->query("SELECT id, rulername, nationname, alliance, email, dateregister, user_agent, lastseen, password FROM players WHERE id != '$id'");
while ($playerinfo2 = $playerinfo->fetch_assoc()) {
$id2 = $playerinfo2['id'];
$rulername2 = $playerinfo2['rulername'];
$nationname2 = $playerinfo2['nationname'];
$alliance2 = $playerinfo2['alliance'];
$email2 = $playerinfo2['email'];
$dateregister2 = $playerinfo2['dateregister'];
$useragent2 = $playerinfo2['user_agent'];
$lastseen2 = $playerinfo2['lastseen'];
$password2 = $playerinfo2['password'];
$rulerdistance = levenshtein($rulername, $rulername2);
$nationdistance = levenshtein($nationname, $nationname2);
$emaildistance = levenshtein($email, $email2);
$agentdistance = levenshtein($useragent, $useragent2) / 2;
$totaldistance = $rulerdistance + $nationdistance + $emaildistance + $agentdistance;
if ($password == $password2) {
$totaldistance = $totaldistance - 20;
}
if ($totaldistance < 0) {
$totaldistance = 0;
}
}
}
?>
You should only do the query once, put it in an array and work with it from there. I don't see the need to make almost the same query twice. Loop in your array a second time and just check if the id is not the same as the current.
$res = $conn->query("SELECT id, rulername, nationname, alliance, email, dateregister, user_agent, lastseen, password FROM players");
$array=array();
while ($row = $res->fetch_assoc()) {
$array[] = $row;
}
for($i=0; $i<count($array);$i++) {
for($j=0; $j<count($array); $j++) {
if ($i != $j) {
// Call your functions
$rulerdistance = levenshtein($array[$i]['rulername'], $array[$j]['rulername']);
...
}
}
}

PHP Recursion -- Increment query variables

I have a database where each row contains a unique userid number, and a referralid number that links them to the person that referred them to the site. I am trying to create a recursive function that will search the database as long as it continues to find users, and perform several calculations as it goes. Below is what I have so far - I am getting stuck on trying to create incrementing query variables - PLEASE HELP!
function findrepsloop($refidnum, $num){
echo "FINDREPSLOOP STARTED<br><br>";
if(strlen($refidnum) > 0){
echo "FIRST STRLEN IF LOOP PASSED: $refidnum<br><br>";
$totalreps++;
$query{$num} = "SELECT * FROM user WHERE (refidnum = '$refidnum' && active = 'YES') ";
$result{$num} = mysql_query($query{$num});
$line{$num} = mysql_fetch_array($result{$num}, MYSQL_NUM);
$total{$num} = mysql_num_rows($result{$num});
echo "QUERY $num found $total{$num} rows in the database. LINE 0: '$line{$num}[0]<br><br>";
$totaltemp = 0;
if($total{$num} > 2){ $totaltemp = $total{$num} / 3; $totaltemp = floor($totaltemp); }else{}
$numberofthrees = $numberofthrees + $totaltemp;
while(strlen($line{$num}[0]) > 0){
$refidnum = $line{$num}[0];
$num++;
findrepsloop($refidnum, $num);
$line{$num} = mysql_fetch_array($result{$num}, MYSQL_NUM);
}
}else{
}
}

php sql find and insert in empty slot

I have a game script thing set up, and when it creates a new character I want it to find an empty address for that players house.
The two relevant table fields it inserts are 'city' and 'number'. The 'city' is a random number out of 10, and the 'number' can be 1-250.
What it needs to do though is make sure there's not already an entry with the 2 random numbers it finds in the 'HOUSES' table, and if there is, then change the numbers. Repeat until it finds an 'address' not in use, then insert it.
I have a method set up to do this, but I know it's shoddy- there's probably some more logical and easier way. Any ideas?
UPDATE
Here's my current code:
$found = 0;
while ($found == 0) {
$num = (rand()%250)+1; $city = (rand()%10)+1;
$sql_result2 = mysql_query("SELECT * FROM houses WHERE city='$city' AND number='$num'", $db);
if (mysql_num_rows($sql_result2) == 0) { $found = 1; }
}
You can either do this in PHP as you do or by using a MySQL trigger.
If you stick to the PHP way, then instead of generating a number every time, do something like this
$found = 0;
$cityarr = array();
$numberarr = array();
//create the cityarr
for($i=1; $i<=10;$i++)
$cityarr[] = i;
//create the numberarr
for($i=1; $i<=250;$i++)
$numberarr[] = i;
//shuffle the arrays
shuffle($cityarr);
shuffle($numberarr);
//iterate until you find n unused one
foreach($cityarr as $city) {
foreach($numberarr as $num) {
$sql_result2 = mysql_query("SELECT * FROM houses
WHERE city='$city' AND number='$num'", $db);
if (mysql_num_rows($sql_result2) == 0) {
$found = 1;
break;
}
}
if($found) break;
}
this way you don't check the same value more than once, and you still check randomly.
But you should really consider fetching all your records before the loops, so you only have one query. That would also increase the performance a lot.
like
$taken = array();
for($i=1; $i<=10;$i++)
$taken[i] = array();
$records = mysql_query("SELECT * FROM houses", $db);
while($rec = mysql_fetch_assoc($records)) {
$taken[$rec['city']][] = $rec['number'];
}
for($i=1; $i<=10;$i++)
$cityarr[] = i;
for($i=1; $i<=250;$i++)
$numberarr[] = i;
foreach($cityarr as $city) {
foreach($numberarr as $num) {
if(in_array($num, $taken[]) {
$cityNotTaken = $city;
$numberNotTaken = $number;
$found = 1;
break;
}
}
if($found) break;
}
echo 'City ' . $cityNotTaken . ' number ' . $numberNotTaken . ' is not taken!';
I would go with this method :-)
Doing it the way you say can cause problems when there is only a couple (or even 1 left). It could take ages for the script to find an empty house.
What I recommend doing is insert all 2500 records in the database (combo 1-10 with 1-250) and mark with it if it's empty or not (or create a combo table with user <> house) and match it on that.
With MySQL you can select a random entry from the database witch is empty within no-time!
Because it's only 2500 records, you can do ORDER BY RAND() LIMIT 1 to get a random row. I don't recommend this when you have much more records.

Stumped in the middle of a PHP loop

Here's what I've got so far-
$awards_sql_1 = mysql_query('SELECT * FROM categories WHERE section_id = 1') or die(mysql_error());
$awards_sql_2 = mysql_query('SELECT * FROM categories WHERE section_id = 2') or die(mysql_error());
$awards_sql_3 = mysql_query('SELECT * FROM categories WHERE section_id = 3') or die(mysql_error());
$awards_sql_4 = mysql_query('SELECT * FROM categories WHERE section_id = 4') or die(mysql_error());
$loop = 1;
while($row_sections = mysql_fetch_array($sections_query)) {
$category = 1;
echo "<h3>" . $row_sections['section_name'] . " (Loop# $loop)</h3>";
while($categories = mysql_fetch_array(${"awards_sql_{$loop}"})) {
${"winners_sql_{$loop}"} = mysql_query("SELECT * FROM 2009_RKR_bestof WHERE section = $loop && category = $category ORDER BY result_level ASC") or die(mysql_error());
echo "<h4><strong>{$categories['category_name']}</strong></h4>";
echo "<ul class=\"winners\">";
>> while($winners = mysql_fetch_array(${"winners_sql_{$loop}"})) {
switch ($winners['result_level']) {
case 1: $result_level = "Platinum"; break;
case 2: $result_level = "Gold"; break;
case 3: $result_level = "Silver"; break;
}
if (isset($winners['url'])) { $anchor = ""; $close = ""; }
echo "<li>$anchor{$winners['winner']}$close ($result_level)</li>";
unset($anchor);
unset($close);
}
echo "</ul>";
$category++;
}
$loop++;
}
Where I'm getting stumped, is I'm getting this thing to loop through correctly, my loop counter ($loop) is working, but when it gets time to spit out the actual reward recipients after the first loop through winners, it's only producing the category titles, the list-items are not getting looped out.
I added a little pointer to where I think the problem begins or centers around (>>).
My guess is I need to maybe unset a var somewhere, but I don't know, I can't see it.
I'm with KM - you're displaying a single page and with your loops, you've got a LOT of queries happening at once - what if 1,000 people hit that page at the same time? ouch...
Maybe consider a larger query (with some repeated data) and loop through it once?
For example:
SELECT
section_name,
category_name,
result_level,
url,
winner
FROM 2009_RKR_bestof
INNER JOIN categories ON 2009_RKR_bestof.category = categories.id
INNER JOIN sections ON 2009_RKR_bestof.section = sections.id
ORDER BY section_name,category_name ASC
In your loop, you can do checks to determine if you're in a new section (category/whatever):
//pseudo-code
$current_section = "";
while($stuff = mysql_fetch_array($sql))
{
if ($current_section == "")
{
$current_section = $stuff["section_name"];
}
if ($current_section == $stuff["section_name"])
{
//keep going in your loop
}
else
{
//we've gotten to a new section - so close your html and start a new section
}
}
You get the idea..
My guess would be that it is a data problem. It isn't having trouble reading the titles, only the winners. If it iterated once, I would check the data, and ensure that winners_sql_2 - winnders_sql_4 are getting actual data. Perhaps add an echo winners_sql_2 line, to output the contents of the query, and ensure the query is framed correctly.

Categories