How to search a PHP PDO AllResults Array - php

I can't figure out how to efficiently get SQL data for a Room/Rates/Dates=Amount...
First I load all the "RateData" for a date range with a PDO select. There are many rows, for many rooms, each with many rates... maybe, or maybe it is all empty except a couple of Amounts. It needs to display $0 for missing dates, so next...
I load the Rooms with PDO and loop through them, and for each room I load the Rates with PDO and loop through them (not a ton of rates per room, and not a ton of rooms, but possibly a very long date-range).
So then I loop through the date range and add $0 to the giant UI grid of Amounts by Rate/Date, nested under each Room. I have to do this anyway, as I also have a ton of logic on what to display in the parent Room row that averages the Rates and such.
So what I need to do is instead of using $0, I need to see if the Room/Rate/Date exists in RateData...
$RateAmount = 0;
$RateDataRow = $RateData.filter('Room=1 && Rate=1 && Date=2022-10-01');
If ($RateDataRow exists) {$RateAmount = $RateDataRow['Amount']}
How to I write the above sudocode in PHP?
The only alternative I can think of would be to do 1000's of SQL calls to populate the grid... which seems bad. Maybe it is not that bad though if PDO caches and doesn't actually query the DB for each grid cell. Please advise. Thanks.
I tried this:
$currcost = 0;
//if $ratedata exists for currdate + currrate + currroom
function ratematch($row)
{if (($row['RoomType_Code']=$currrate)
&& ($row['RatePlan_Code']=$currroom)
&& ($row[ 'Rate_Date']=$currdate->format("Y-m-d")))
{return 1;}
else {return 0;}
}
$match = array_filter($ratedata, 'ratematch');
if (!empty($match)) {$currcost = $match['Rate_Amount'];}
But got an error about redeclaring a function. I have to redeclare it because it is in a loop of currdate under a loop of currrate under a loop of currroom (about 1000 cells).

I made it work, I just have to manually loop through the RateData...
//Get Amount from DB
$currcost = 0;
foreach($ratedata as $datarow)
{if (($datarow['RoomType_Code']==$currroom)
&& ($datarow['RatePlan_Code']==$currrate)
&& ($datarow[ 'Rate_Date']==$currdate->format("Y-m-d")))
{$currcost = $datarow['Rate_Amount'];
break;
}
}
If anyone knows a more efficient way to "query" a previous query without a trip to the SQL server, please post about it. Looping through the fetchAll 1000's times seems bad, but not as bad as doing 1000's of WHERE queries on the SQL Server.
I suppose I could make a 3-dimentional array of $0, then just loop through the RateData once to update that array, and finally loop through the 3-dimentional array to do my other calculations (average rate per room per week sat-sun). Sounds hard though.

Related

PHP Foreach loop too slow when applied on large arrays

So, basically, i have to loop thought an array of 25000 items, then compare each item with another array's ID and if the ID's from the first array and the second match then create another array of the matched items. That looks something like this.
foreach ($all_games as $game) {
foreach ($user_games_ids as $user_game_id) {
if ($user_game_id == $game["appid"]) {
$game_details['title'][] = $game["title"];
$game_details['price'][] = $game["price"];
$game_details['image'][] = $game["image_url"];
$game_details['appid'][] = $game["appid"];
}
}
}
I tested this loop with only 2500 records from the first array ($all_games) and about 2000 records from the second array ($user_games_ids) and as far as i figured, it takes about 10 seconds for the execution of that chunk of code, only the loops execution. Is that normal? Should that take that long or I'm I approaching the issue from the wrong side? Is there a way to reduce that time? Because when i apply that code to 25000 records that time will significantly increase.
Any help is appreciated,
Thanks.
EDIT: So there is no confusion, I can't use the DB query to improve the performance, although, i have added all 25000 games to the database, i can't do the same for the user games ids. There is no way that i know to get all of the users through that API i'm accessing, and even there is, that would be really a lot of users. I get user games ids on the fly when a user enters it's ID in the form and based on that i use file_get_contents to obtain those ids and then cross reference them with the database that stores all games. Again, that might not be the best way, but only one i could think of at this point.
If you re-index the $game array by appid using array_column(), then you can reduce it to one loop and just check if the data isset...
$game = array_column($game,null,"appid");
foreach ($user_games_ids as $user_game_id) {
if (isset( $game[$user_game_id])) {
$game_details['title'][] = $game[$user_game_id]["title"];
$game_details['price'][] = $game[$user_game_id]["price"];
$game_details['image'][] = $game[$user_game_id]["image_url"];
$game_details['appid'][] = $game[$user_game_id]["appid"];
}
}

mysql | PHP | Join within own table

i dont know if i am doing right or wrong, please dont judge me...
what i am trying to do is that if a record belongs to parent then it will have parent id assosiated with it.. let me show you my table schema below.
i have two columns
ItemCategoryID &
ItemParentCategoryID
Let Suppose a record on ItemCategoryID =4 belongs to ItemCategoryID =2 then the column ItemParentCategoryID on ID 4 will have the ID of ItemCategoryID.
I mean a loop with in its own table..
but problem is how to run the select query :P
I mean show all the parents and childs respective to their parents..
This is often a lazy design choise. Ideally you want a table for these relations or/and a set number of depths. If a parent_id's parent can have it's own parent_id, this means a potential infinite depth.
MySQL isn't a big fan of infinite nesting depths. But php don't mind. Either run multiple queryies in a loop such as Nil'z's1, or consider fetching all rows and sorting them out in arrays in php. Last solution is nice if you pretty much always get all rows, thus making MySQL filtering obsolete.
Lastly, consider if you could have a more ideal approach to this in your database structure. Don't be afraid to use more than one table for this.
This can be a strong performance thief in the future. An uncontrollable amount of mysql queries each time the page loads can easily get out of hands.
Try this:
function all_categories(){
$data = array();
$first = $this->db->select('itemParentCategoryId')->group_by('itemParentCategoryId')->get('table')->result_array();
if( isset( $first ) && is_array( $first ) && count( $first ) > 0 ){
foreach( $first as $key => $each ){
$second = $this->db->select('itemCategoryId, categoryName')->where_in('itemParentCategoryId', $each['itemParentCategoryId'])->get('table')->result_array();
$data[$key]['itemParentCategoryId'] = $each['itemParentCategoryId'];
$data[$key]['subs'] = $second;
}
}
print_r( $data );
}
I don't think you want/can to do this in your query since you can nest a long way.
You should make a getChilds function that calls itself when you retrieve a category. This way you can nest more than 2 levels.
function getCategory()
{
// Retrieve the category
// Get childs
$childs = $this->getCategoryByParent($categoryId);
}
function getCategorysByParent($parentId)
{
// Get category
// Get childs again.
}
MySQL does not support recursive queries. It is possible to emulate recursive queries through recursive calls to a stored procedure, but this is hackish and sub-optimal.
There are other ways to organise your data, these structures allow very efficient querying.
This question comes up so often I can't even be bothered to complain about your inability to use Google or SO search, or to offer a wordy explanation.
Here - use this library I made: http://codebyjeff.com/blog/2012/10/nested-data-with-mahana-hierarchy-library so you don't bring down your database

php array_unique combined with array_filter, incorrect output

In a database table are multiple entries related to a customer. Each entry will have time_1, time_2, time_3, time_4, time_5 and time_6 fields. I have looped through the entries and used this to combine these times to create a round cycle for a day:
while($row=mysql_fetch_assoc($script_check))
{
$prerounds = array_filter($row);
$rounds = array_unique(array_merge($prerounds));
}
This works however if one entry has times 1-4 set and the second entry has times 1-3 set the fourth time will be totally removed thus only times 1-3 are put into the array. How can I combine these fields without losing data when some are blank? This strategy so far has been to strip the $row of all blanks then merge it together to create one set of round times related to that customer for that day. Thanks.
As the OP requested:
Basically, your main problem is you're re-initializing $rounds on each iteration, you're not merging any existing array with a new one.
$rounds = array();
while($row = $pdo->fetch(PDO::FETCH_ASSOC))
{
$rounds = array_merge($rounds, array_unique(array_filter($row)));
}
Hmz... apart from mysql_* being deprecated (that's why I used PDO in my snippet), you should really check your logic... perhaps a decent query might be more suitable, or a simple set of if's. It's even likely to be faster that way

Timing out while updating MySQL with PHP from a CSV

I need to come up with a way to make a large task faster to beat the timeout.
I have very limited access to the server due to the restrictions of the hosting company.
I have a system set up where a cron visits a PHP file that grabs a csv that contains data on some products. The csv does not contain all of the fields that the product would have. Just a handful of essential ones.
I've read a fair number of articles on timeouts and handling csv's and currently (in an attempt to shave time) I have made a table (let's call it csv_data) to hold the csv data. I have a script that truncates the csv_data table then inserts data from the csv so each night the latest recordset from the csv is in that table (the csv file gets updated nightly). So far, no timeout problems..the task only takes about 4-5 seconds.
The timeouts occur when I have to sift through the data to make updates to the products table. The steps that it is running right now is like this
1. Get the sku from csv_data table (that holds thousands of records)
2. Select * from Products where products.sku = csv.sku (products table also holds thousands of records to loop through)
3. Get numrows.
If numrows<0{no record in products, so skip}.
If numrows>1{duplicate entries, don't change anything, but later on report the sku}
If numrows==1{Update selected fields in the products table with csv data}
4. Go to the next record in csv_data all over again
(I figured outlining the process is shorter and easier than dropping in the code.)
I looked into MySQl views and stored procedures but I am not skilled enough in it to know if it will handle the 'if' statement portion.
Is there anything I can do to make this faster to avoid the timeouts?
edit:
I should mention that set_time_limit(0); isn't doing it. And if it helps, the server uses IIS7 and fastcgi
Thanks for your help.
Update after using suggestions from Jakob and Shawn:
I'm doing something wrong. The speed is definitely faster and the csv sku is incrementing,
but when I tried to implement Shawn's solution; the query is giving me a PHP Warning: mysql_result() expects parameter 1 to be resource, boolean error.
Can you help me spot what I am doing wrong?
Here is the section of code:
$csvdata="SELECT * FROM csv_update";
$csvdata_result=mysql_query($csvdata);
mysql_query($csvdata);
$csvdata_num = mysql_num_rows($csvdata_result);
$i=0;
while($i<$csvdata_num){
$csv_code=#mysql_result($csvdata_result,$i,"skucode");
$datacheck=NULL;
$datacheck=substr($csv_code,0,1);
if($datacheck>='0' && $datacheck<='9'){
$csv_price=#mysql_result($csvdata_result,$i,"price");
$csv_retail=#mysql_result($csvdata_result,$i,"retail");
$csv_stock=#mysql_result($csvdata_result,$i,"stock");
$csv_weight=#mysql_result($csvdata_result,$i,"weight");
$csv_manufacturer=#mysql_result($csvdata_result,$i,"manufacturer");
$csv_misc1=#mysql_result($csvdata_result,$i,"misc1");
$csv_misc2=#mysql_result($csvdata_result,$i,"misc2");
$csv_selectlist=#mysql_result($csvdata_result,$i,"selectlist");
$csv_level5=#mysql_result($csvdata_result,$i,"level5");
$csv_frontpage=#mysql_result($csvdata_result,$i,"frontpage");
$csv_level3=#mysql_result($csvdata_result,$i,"level3");
$csv_minquantity=#mysql_result($csvdata_result,$i,"minquantity");
$csv_quantity1=#mysql_result($csvdata_result,$i,"quantity1");
$csv_discount1=#mysql_result($csvdata_result,$i,"discount1");
$csv_quantity2=#mysql_result($csvdata_result,$i,"quantity2");
$csv_discount2=#mysql_result($csvdata_result,$i,"discount2");
$csv_quantity3=#mysql_result($csvdata_result,$i,"quantity3");
$csv_discount3=#mysql_result($csvdata_result,$i,"discount3");
$count_check="SELECT COUNT(*) AS totalCount FROM products WHERE skucode = '$csv_code'";
$count_result=mysql_query($count_check);
mysql_query($count_check);
$totalCount=#mysql_result($count_result,0,'totalCount');
$loopCount = ceil($totalCount / 25);
for($j = 0; $j < $loopCount; $j++){
$prod_check="SELECT skucode FROM products WHERE skucode = '$csv_code' LIMIT ($loopCount*25), 25;";
$prodresult=mysql_query($prod_check);
mysql_query($prod_check);
$prodnum =#mysql_num_rows($prodresult);
$prod_id=#mysql_result($prodresult,0,"catalogid");
if($prodnum<1){
echo "NOT FOUND:$csv_code<br>";
$count_sku_not_found=$count_sku_not_found+1;
$list_sku_not_found=$list_sku_not_found." $csv_code";}
if($prodnum>1){
echo "DUPLICATE:$csv_ccode<br>";
$count_duplicate_skus=$count_duplicate_skus+1;
$list_duplicate_skus=$list_duplicate_skus." $csv_code";}
if ($prodnum==1){
///This prevents an overwrite from happening if the csv file doesn't produce properly
if ($csv_price!="" OR $csv_price!=NULL)
{$sql_price='price="'.$csv_price.'"';}
if ($csv_retail!="" OR $csv_retail!=NULL)
{$sql_retail=',retail="'.$csv_retail.'"';}
if ($csv_stock!="" OR $csv_stock!=NULL)
{$sql_stock=',stock="'.$csv_stock.'"';}
if ($csv_weight!="" OR $csv_weight!=NULL)
{$sql_weight=',weight="'.$csv_weight.'"';}
if ($csv_manufacturer!="" OR $csv_manufacturer!=NULL)
{$sql_manufacturer=',manufacturer="'.$csv_manufacturer.'"';}
if ($csv_misc1!="" OR $csv_misc1!=NULL)
{$sql_misc1=',misc1="'.$csv_misc1.'"';}
if ($csv_misc2!="" OR $csv_misc2!=NULL)
{$sql_pother2=',pother2="'.$csv_misc2.'"';}
if ($csv_selectlist!="" OR $csv_selectlist!=NULL)
{$sql_selectlist=',selectlist="'.$csv_selectlist.'"';}
if ($csv_level5!="" OR $csv_level5!=NULL)
{$sql_level5=',level5="'.$csv_level5.'"';}
if ($csv_frontpage!="" OR $csv_frontpage!=NULL)
{$sql_frontpage=',frontpage="'.$csv_frontpage.'"';}
$import="UPDATE products SET $sql_price $sql_retail $sql_stock $sql_weight $sql_manufacturer $sql_misc1 $sql_misc2 $sql_selectlist $sql_level5 $sql_frontpage $sql_in_stock WHERE skucode='$csv_code'";
mysql_query($import) or die(mysql_error("error updating in products table"));
echo "Update ".$csv_code." successful ($i)<br>";
$count_success_update_skus=$count_success_update_skus+1;
$list_success_update_skus=$list_success_update_skus." $csv_code";
//empty out variables
$sql_price='';
$sql_retail='';
$sql_stock='';
$sql_weight='';
$sql_manufacturer='';
$sql_misc1='';
$sql_misc2='';
$sql_selectlist='';
$sql_level5='';
$sql_frontpage='';
$sql_in_stock='';
$prodnum=0;
}
}
$i++;
}
Is it timing out before the first row is returned or is it between rows during the read? One good practice bit would be to handle your query in chunks; do a count first to see how many records you are dealing with for the SKU, the loop through smaller chunks (the size of these chunks would depend on how many things you have to do with each row). Your updated workflow would look more like this:
Get next SKU from CSV
Get a total count: SELECT COUNT(*) AS totalCount FROM products WHERE products.sku = csv.sku
Determine chunk size (using 25 for this demo)
loopCount = ceil(totalCount / 25)
Loop through all results using a loop like this: for($i = 0; $i < loopCount; $i++)
Inside your loop you should be running a query like this: SELECT * FROM products WHERE products.sku = csv.sku LIMIT (loopCount*25), 25
You will want to use a constant order for your SELECT chunks; your unique ID would probably be best.
I think you can solve this problem with cron. http://en.wikipedia.org/wiki/Cron . It has never had timeout.

get max value in php (instead of mysql)

I have two msyql tables, Badges and Events. I use a join to find all the events and return the badge info for that event (title & description) using the following code:
SELECT COUNT(Badges.badge_ID) AS
badge_count,title,Badges.description
FROM Badges JOIN Events ON
Badges.badge_id=Events.badge_id GROUP
BY title ASC
In addition to the counts, I need to know the value of the event with the most entries. I thought I'd do this in php with the max() function, but I had trouble getting that to work correctly. So, I decided I could get the same result by modifying the above query by using "ORDER BY badgecount DESC LIMIT 1," which returns an array of a single element, whose value is the highest count total of all the events.
While this solution works well for me, I'm curious if it is taking more resources to make 2 calls to the server (b/c I'm now using two queries) instead of working it out in php. If I did do it in php, how could I get the max value of a particular item in an associative array (it would be nice to be able to return the key and the value, if possible)?
EDIT:
OK, it's amazing what a few hours of rest will do for the mind. I opened up my code this morning, and made a simple modification to the code, which worked out for me. I simply created a variable on the count field and, if the new one was greater than the old one, changed it to the new value (see the "if" statement in the following code):
if ( $c > $highestCount ) {
$highestCount = $c; }
This might again lead to a "religious war", but I would go with the two queries version. To me it is cleaner to have data handling in the database as much as possible. In the long run, query caching, etc.. would even out the overhead caused by the extra query.
Anyway, to get the max in PHP, you simply need to iterate over your $results array:
getMax($results) {
if (count($results) == 0) {
return NULL;
}
$max = reset($results);
for($results as $elem) {
if ($max < $elem) { // need to do specific comparison here
$max = $elem;
}
}
return $max;
}

Categories