Page not available when using SplFixedArray - php

I'm getting "Page not available" if I run the following code:
namespace Database;
class Table extends \Database\Connection {
// ...
/**
* Execute query and return result
* #param type $query
* #param type $sqlWildcards
* #return SplFixedArray ResultSet
*/
public static function query($query, $sqlWildcards = array()) {
// Do some query stuff
// $stmt is a PHP PDO Statement
// Additional result manipulation
$rowCount = 0;
$rsStorage = new \SplFixedArray(500000);
while ($row = $stmt->fetch(\PDO::FETCH_ASSOC)) {
$modRow = static::modifyRow($row);
if ($modRow !== false) {
$rsStorage[$rowCount] = $modRow;
$rowCount++;
}
}
// Resize
$rsStorage->setSize($rowCount);
}
}
Method modifyRow($row):
/**
* Gives the possibility to modify a result for child classes
* #param array $row
* #return array
*/
protected static function modifyRow($row) {
return $row;
}
But if I do the following:
while ($row = $stmt->fetch(\PDO::FETCH_ASSOC)) {
//$modRow = static::modifyRow($row);
//if ($modRow !== false) {
$rsStorage[$rowCount] = $modRow;
$rowCount++;
//}
}
Everything works fine!
Edit: The test above makes no sense, I forgot to set $modRow to $row - this fails as well.
Another hint: if I do something like this
while ($row = $stmt->fetch(\PDO::FETCH_ASSOC)) {
$modRow = static::modifyRow($row);
if ($modRow !== false) {
$rsStorage[$rowCount] = $modRow;
$rowCount++;
}
// Works
if($rowCount == 5000) { break; }
// Crashes (Page not available
if($rowCount == 10000) { break; }
}
Error reporting is activated but there's no internal server error, though. Just "page not available"
Edit: 9742 is the magic number. When $rowCount hits 9743 it crashes - doesn't matter if I test it from 0 to 9743 or 9744 to x
Edit2: If I use array instead of SplFixedArray everything's fine. Does not make sense since splfixedarray is more memory efficient than array()
Edit3: I wrote the memory usage into a file until the script crashes, it reaches a peak of 54MB - server limit is 128MB
Edit4: Apache log shows me a 'zend_mm_heap corrupted'
Never had this before, also google could not help me
Any idea?

Related

CodeIgniter DataMapper crash

I inherit old cms build on CodeIgniter 2.0.2 with DataMapper ORM 1.8.0 and the website is with ads. When the ads were under a few thousands it was working fine. Now the ads are close to 20 thousands and it just crash. And there is not error log. The problem is on homepage and sometimes on ad page. I can't figure it out why. I had to edit this function located in "./application/libraries/datamapper.php" so it won't crash.
/**
* Process Query
*
* Converts a query result into an array of objects.
* Also updates this object
*
* #ignore
* #param CI_DB_result $query
*/
protected function _process_query($query)
{
if ($query->num_rows() > 0)
{
// Populate all with records as objects
$this->all = array();
$this->_to_object($this, $query->row());
// don't bother recreating the first item.
$index = ($this->all_array_uses_ids && isset($this->id)) ? $this->id : 0;
$this->all[$index] = $this->get_clone();
if($query->num_rows() > 1)
{
$model = get_class($this);
$first = TRUE;
foreach ($query->result() as $key =>$row)
{
if($first)
{
$first = FALSE;
continue;
}
$item = new $model();
$this->_to_object($item, $row);
if($this->all_array_uses_ids && isset($item->id))
{
$this->all[$item->id] = $item;
} //HERE IS THE EDIT
else if($key < 12)
{
$this->all[] = $item;
}
}
}
// remove instantiations
$this->_instantiations = NULL;
// free large queries
if($query->num_rows() > $this->free_result_threshold)
{
$query->free_result();
}
}
else
{
// Refresh stored values is called by _to_object normally
$this->_refresh_stored_values();
}
}
On line "HERE IS THE EDIT" , I add if the key is less than 12 to add then to $this->all . If I put more than 12 , it crashes.
My question is how can I overcome this? Has anyone come a cross this? On the server I use php 7 and etc. It is as modern server.
I set:
public $free_result_threshold = 50;
public $production_cache = FALSE; //I turn it on but it didn't help
Any idea would be helpful.

How I can close the connection between PHP & Mysql immediately after accessing

May wabsite has faced max_user_connections error
After contacting the website hosting provider they said I have to close the connection immediately after accessing the database using mysql_close()
My question is that how I can close the connection if my php file is :
*/
if (!defined('QA_VERSION')) { // don't allow this page to be requested directly from browser
header('Location: ../');
exit;
}
/**
* Indicates to the Q2A database layer that database connections are permitted fro this point forwards
* (before this point, some plugins may not have had a chance to override some database access functions).
*/
function qa_db_allow_connect()
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
global $qa_db_allow_connect;
$qa_db_allow_connect=true;
}
/**
* Connect to the Q2A database, select the right database, optionally install the $failhandler (and call it if necessary).
* Uses mysqli as of Q2A 1.7.
*/
function qa_db_connect($failhandler=null)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
global $qa_db_connection, $qa_db_fail_handler, $qa_db_allow_connect;
if (!$qa_db_allow_connect)
qa_fatal_error('It appears that a plugin is trying to access the database, but this is not allowed until Q2A initialization is complete.');
if (isset($failhandler))
$qa_db_fail_handler = $failhandler; // set this even if connection already opened
if ($qa_db_connection instanceof mysqli)
return;
// in mysqli we connect and select database in constructor
if (QA_PERSISTENT_CONN_DB)
$db = new mysqli('p:'.QA_FINAL_MYSQL_HOSTNAME, QA_FINAL_MYSQL_USERNAME, QA_FINAL_MYSQL_PASSWORD, QA_FINAL_MYSQL_DATABASE);
else
$db = new mysqli(QA_FINAL_MYSQL_HOSTNAME, QA_FINAL_MYSQL_USERNAME, QA_FINAL_MYSQL_PASSWORD, QA_FINAL_MYSQL_DATABASE);
// must use procedural `mysqli_connect_error` here prior to 5.2.9
$conn_error = mysqli_connect_error();
if ($conn_error)
qa_db_fail_error('connect', $db->connect_errno, $conn_error);
// From Q2A 1.5, we explicitly set the character encoding of the MySQL connection, instead of using lots of "SELECT BINARY col"-style queries.
// Testing showed that overhead is minimal, so this seems worth trading off against the benefit of more straightforward queries, especially
// for plugin developers.
if (!$db->set_charset('utf8'))
qa_db_fail_error('set_charset', $db->errno, $db->error);
qa_report_process_stage('db_connected');
$qa_db_connection=$db;
}
/**
* If a DB error occurs, call the installed fail handler (if any) otherwise report error and exit immediately.
*/
function qa_db_fail_error($type, $errno=null, $error=null, $query=null)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
global $qa_db_fail_handler;
#error_log('PHP Question2Answer MySQL '.$type.' error '.$errno.': '.$error.(isset($query) ? (' - Query: '.$query) : ''));
if (function_exists($qa_db_fail_handler))
$qa_db_fail_handler($type, $errno, $error, $query);
else {
echo '<hr><font color="red">Database '.htmlspecialchars($type.' error '.$errno).'<p>'.nl2br(htmlspecialchars($error."\n\n".$query));
qa_exit('error');
}
}
/**
* Return the current connection to the Q2A database, connecting if necessary and $connect is true.
*/
function qa_db_connection($connect=true)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
global $qa_db_connection;
if ($connect && !($qa_db_connection instanceof mysqli)) {
qa_db_connect();
if (!($qa_db_connection instanceof mysqli))
qa_fatal_error('Failed to connect to database');
}
return $qa_db_connection;
}
/**
* Disconnect from the Q2A database.
*/
function qa_db_disconnect()
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
global $qa_db_connection;
if ($qa_db_connection instanceof mysqli) {
qa_report_process_stage('db_disconnect');
if (!QA_PERSISTENT_CONN_DB) {
if (!$qa_db_connection->close())
qa_fatal_error('Database disconnect failed');
}
$qa_db_connection=null;
}
}
/**
* Run the raw $query, call the global failure handler if necessary, otherwise return the result resource.
* If appropriate, also track the resources used by database queries, and the queries themselves, for performance debugging.
*/
function qa_db_query_raw($query)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
if (QA_DEBUG_PERFORMANCE) {
global $qa_usage;
// time the query
$oldtime = array_sum(explode(' ', microtime()));
$result = qa_db_query_execute($query);
$usedtime = array_sum(explode(' ', microtime())) - $oldtime;
// fetch counts
$gotrows = $gotcolumns = null;
if ($result instanceof mysqli_result) {
$gotrows = $result->num_rows;
$gotcolumns = $result->field_count;
}
$qa_usage->logDatabaseQuery($query, $usedtime, $gotrows, $gotcolumns);
}
else
$result = qa_db_query_execute($query);
// #error_log('Question2Answer MySQL query: '.$query);
if ($result === false) {
$db = qa_db_connection();
qa_db_fail_error('query', $db->errno, $db->error, $query);
}
return $result;
}
/**
* Lower-level function to execute a query, which automatically retries if there is a MySQL deadlock error.
*/
function qa_db_query_execute($query)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
$db = qa_db_connection();
for ($attempt = 0; $attempt < 100; $attempt++) {
$result = $db->query($query);
if ($result === false && $db->errno == 1213)
usleep(10000); // deal with InnoDB deadlock errors by waiting 0.01s then retrying
else
break;
}
return $result;
}
/**
* Return $string escaped for use in queries to the Q2A database (to which a connection must have been made).
*/
function qa_db_escape_string($string)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
$db = qa_db_connection();
return $db->real_escape_string($string);
}
/**
* Return $argument escaped for MySQL. Add quotes around it if $alwaysquote is true or it's not numeric.
* If $argument is an array, return a comma-separated list of escaped elements, with or without $arraybrackets.
*/
function qa_db_argument_to_mysql($argument, $alwaysquote, $arraybrackets=false)
{
if (is_array($argument)) {
$parts=array();
foreach ($argument as $subargument)
$parts[] = qa_db_argument_to_mysql($subargument, $alwaysquote, true);
if ($arraybrackets)
$result = '('.implode(',', $parts).')';
else
$result = implode(',', $parts);
}
elseif (isset($argument)) {
if ($alwaysquote || !is_numeric($argument))
$result = "'".qa_db_escape_string($argument)."'";
else
$result = qa_db_escape_string($argument);
}
else
$result = 'NULL';
return $result;
}
/**
* Return the full name (with prefix) of database table $rawname, usually if it used after a ^ symbol.
*/
function qa_db_add_table_prefix($rawname)
{
if (qa_to_override(__FUNCTION__)) { $args=func_get_args(); return qa_call_override(__FUNCTION__, $args); }
$prefix = QA_MYSQL_TABLE_PREFIX;
if (defined('QA_MYSQL_USERS_PREFIX')) {
switch (strtolower($rawname)) {
case 'users':
case 'userlogins':
case 'userprofile':
case 'userfields':
case 'messages':
case 'cookies':
case 'blobs':
case 'cache':
case 'userlogins_ibfk_1': // also special cases for constraint names
case 'userprofile_ibfk_1':
$prefix = QA_MYSQL_USERS_PREFIX;
break;
}
}
return $prefix.$rawname;
}
/**
* Callback function to add table prefixes, as used in qa_db_apply_sub().
*/
function qa_db_prefix_callback($matches)
{
return qa_db_add_table_prefix($matches[1]);
}
/**
* Substitute ^, $ and # symbols in $query. ^ symbols are replaced with the table prefix set in qa-config.php.
* $ and # symbols are replaced in order by the corresponding element in $arguments (if the element is an array,
* it is converted recursively into comma-separated list). Each element in $arguments is escaped.
* $ is replaced by the argument in quotes (even if it's a number), # only adds quotes if the argument is non-numeric.
* It's important to use $ when matching a textual column since MySQL won't use indexes to compare text against numbers.
*/
function qa_db_apply_sub($query, $arguments)
{
$query = preg_replace_callback('/\^([A-Za-z_0-9]+)/', 'qa_db_prefix_callback', $query);
if (!is_array($arguments))
return $query;
$countargs = count($arguments);
$offset = 0;
for ($argument = 0; $argument < $countargs; $argument++) {
$stringpos = strpos($query, '$', $offset);
$numberpos = strpos($query, '#', $offset);
if ($stringpos === false || ($numberpos !== false && $numberpos < $stringpos)) {
$alwaysquote = false;
$position = $numberpos;
}
else {
$alwaysquote = true;
$position = $stringpos;
}
if (!is_numeric($position))
qa_fatal_error('Insufficient parameters in query: '.$query);
$value = qa_db_argument_to_mysql($arguments[$argument], $alwaysquote);
$query = substr_replace($query, $value, $position, 1);
$offset = $position + strlen($value); // allows inserting strings which contain #/$ character
}
return $query;
}
/**
* Run $query after substituting ^, # and $ symbols, and return the result resource (or call fail handler).
*/
function qa_db_query_sub($query) // arguments for substitution retrieved using func_get_args()
{
$funcargs=func_get_args();
return qa_db_query_raw(qa_db_apply_sub($query, array_slice($funcargs, 1)));
}
/**
* Return the number of rows in $result. (Simple wrapper for mysqli_result::num_rows.)
*/
function qa_db_num_rows($result)
{
if ($result instanceof mysqli_result)
return $result->num_rows;
return 0;
}
/**
* Return the value of the auto-increment column for the last inserted row.
*/
function qa_db_last_insert_id()
{
$db = qa_db_connection();
return $db->insert_id;
}
/**
* Return the number of rows affected by the last query.
*/
function qa_db_affected_rows()
{
$db = qa_db_connection();
return $db->affected_rows;
}
/**
* For the previous INSERT ... ON DUPLICATE KEY UPDATE query, return whether an insert operation took place.
*/
function qa_db_insert_on_duplicate_inserted()
{
return (qa_db_affected_rows() == 1);
}
/**
* Return a random integer (as a string) for use in a BIGINT column.
* Actual limit is 18,446,744,073,709,551,615 - we aim for 18,446,743,999,999,999,999.
*/
function qa_db_random_bigint()
{
return sprintf('%d%06d%06d', mt_rand(1, 18446743), mt_rand(0, 999999), mt_rand(0, 999999));
}
/**
* Return an array of the names of all tables in the Q2A database, converted to lower case.
* No longer used by Q2A and shouldn't be needed.
*/
function qa_db_list_tables_lc()
{
return array_map('strtolower', qa_db_list_tables());
}
/**
* Return an array of the names of all tables in the Q2A database.
*/
function qa_db_list_tables()
{
return qa_db_read_all_values(qa_db_query_raw('SHOW TABLES'));
}
/*
The selectspec array can contain the elements below. See qa-db-selects.php for lots of examples.
I bet you have your code set to use persistent connections for MySQL. This can result in open but idle connections, maxing out whatever your (most likely shared) hosting provider allocates for you.
You have a QA_PERSISTENT_CONN_DB constant in your code that determines your connection mode. You will find it defined as true or false somewhere in a configuration file. (It's not defined in your code above). Ensure it is set to false and see if the problem is resolved. When persistent connections are off, the code above takes care of closing the connections. You don't need to add it in.
If you're interested in knowing more about persistent connections, this answer covers a lot of ground; and there's more in the manual:
Persistent connections are links that do not close when the execution of your script ends.
Next time, make sure you read through your code to see if the functionality is already there (in this case, it's inside function qa_db_disconnect()), and if it's simply a matter of configuration.

AJAX calls return results in a (seemingly) synchronous manner

All,
Background of how problem was detected
My question concerns the performance of a web app, mainly the index page. I noticed the problem when I was giving a demonstration at a local branch of my company that has slow internet (I don't know the exact speeds, or ping rate) judged by the fact that Google took about 10 seconds to load. My index page took ~10-20 times longer to load. I was under the assumption that my app did most of the work on the server side (as php is making all of the database queries...). But this led me to look at the network tool of Chrome and see the latency times of these 4 divs being loaded by ajax (I'll elaborate in a bit). Interestingly, the scripts being called appear to run sequentially, but not necessarily in the order I invoked the ajax calls (sometimes they do, other times they don't).
What are these divs / ajax requests?
Here is a code snippets of a request:
Yii::app()->clientScript->registerScript('leftDiv', '
$( "#left_dash" ).load(
"'.$this->createUrl("/site/page?view=leftDashLoad") .'",
function(){
$("#left_dash p a").click(function() {
$(this).parent().parent().find("div.scroll100").slideUp();
$(this).parent().next().stop(false, false).slideDown();
});
$("p:first-child").next().slideDown();
}
);
' );
Here is the page requested:
$this->widget('widgets.ScrollList',array(
'condition'=>
function($muddJob,$scrollList)
{
$job = $muddJob->Job;; //returns a job or empty array
if(!empty($job) )
{
if( $muddJob->uploadArtwork == null && $muddJob->uploadData == null ) {
array_push($scrollList->_models,$job);
$scrollList->columnValues = array($muddJob->jobDescription,$muddJob->dropDate1);
return true;
}
}
return false;
},
'columns' => array('col1'=>"MuddJob#",'col2'=>"Desc",'col3'=>"Dealer Name"),
'name'=> "Print New Ticket",
'muddJobs' => $currentExchanges->getCurrentMuddExchanges(),
)
);
Imagine that page (the page that ajax has called) having 6 similar declarations that create widgets. The goal is to return html to put back in place of a loading gif on the index page.
Here is the scroll widget:
<?php
Yii::import('widgets.ScrollListBase');
include_once Yii::app()->extensionPath . "/BusinessDay.php";
class ScrollList extends ScrollListBase
{
private $_content;
public $columns = array();
public $columnValues;
private $_listInfo;
public $name;
public $_models = array();
public $condition;
public $muddJobs; //object to pass
public $jobsMailingTodayArray = array();
public function init()
{
//$this->init();
$this->_listInfo = $this->generateListInfo($this->columns);
//$muddJobs = $this->getCurrentMuddExchanges();
$listInfo = $this->newScrollList($this->muddJobs);
$contents = $this->createContent($listInfo,$this->name);
$this->_content = $contents[0];
// $this->_fullTableContent = $contents[1];
//$this->_listInfo = $contents[2];
}
public function run()
{
//if($this->data['isVisible'])
echo $this->_content;
Yii::app()->session["exploded_content_{$this->name}"] = $this->_models;
}
private function newScrollList($muddJobs)
{
$listInfo = $this->_listInfo;
$tempCount = 0;
foreach($muddJobs as $muddJob)
{
$condition = $this->condition;
if($condition($muddJob,$this) && empty($this->jobsMailingTodayArray) ) //if no job exists for the muddExchange...
{
$tempArray = $this->createWidgetLinks($tempCount,$listInfo,$muddJob,$this->columnValues);
$listInfo = $tempArray[0];
$tempCount = $tempArray[1];
}
elseif ( !empty($this->jobsMailingTodayArray ) )
{
foreach ($this->jobsMailingTodayArray as $jobMailingToday) //change to for loop over the length of the jobsMailingToday
{
$tempArray = $this->createWidgetLinks($tempCount,$listInfo,$muddJob,$this->columnValues);
$listInfo = $tempArray[0];
$tempCount = $tempArray[1];
}
$this->jobsMailingTodayArray = array();
}
}
return array($listInfo,$tempCount);
}
}
?>
Here is it's parent:
<?php
class ScrollListBase extends CWidget
{
private $content = "<p>";
private $divDeclaration = "<div class='scroll100'>\n<table class='quickInfoTable'>\n<thead>\n";
private $headTag = "<th>";
private $headTagClose = "</th>\n";
private $theadTagClose = "</thead>\n";
private $bodyTag = "<tbody>\n";
private $listInfo = "<div class='scroll100'>\n<table class='quickInfoTable'>\n<thead>\n<th>Job#</th>\n<th>Package#</th>\n<th>Entry Date</th>\n</thead>\n<tbody>\n";
/**
* Initializes the widget.
*/
public function createContent($listInfo,$name)
{
$largeHref = Yii::app()->request->baseUrl . '/index.php/site/fullTableView';
$this->content .= "<span class='badge' >{$listInfo[1]} </span> <a href='#'>{$name} </a> <a href='$largeHref/Name/{$name}'> <small>(view larger)</small> </a> </p>";
if( $listInfo[1] > 0 )
{
// $this->fullTable .= substr($listInfo[0],22);
// $this->fullTableContent= $this->fullContent .= $this->fullTable . "</tbody>\n</table>\n</div>";
$this->content .= $listInfo[0] . "</tbody>\n</table>\n</div>";
}
return array($this->content);
}
//Helper Methods
/**
*
* #param type $attributeArray. send an accociative array
* #return type = either a job or an empty array
*/
protected function getJobByAttributes($attributeArray)
{
return Jobs::model()->with('MuddExchange')->findByAttributes($attributeArray);
}
protected function createWidgetLinks($tempCount,$listInfo,$muddJob,$columnValues,$url="/MuddExchange/")
{
$tempCount++;
$viewIndex = $muddJob->exchange_id;
$model = $muddJob;
$job = $muddJob->Job;
if ( isset($job ))
{
$model = $job;
$url = "/Jobs/";
$viewIndex = $model->job_id;
}
$link = CHtml::link("$model->jobNumber",array("{$url}{$viewIndex}"));
$listInfo .= "<tr>\n<td>$link</td>\n";
foreach ($columnValues as $columnValue)
{
$listInfo .= "<td>{$columnValue}</td>\n";
}
$listInfo .= "</tr>";
return array($listInfo,$tempCount);
}
protected function getListInfo()
{
return $this->listInfo;
}
/**
* Takes an array of strings to generate the column names for a particular list.
* #param array $heads
* #return string
*
*/
protected function generateListInfo($heads)
{
//<th>Job#</th>\n<th>Package#</th>\n<th>Entry Date</th>\n</thead>\n<tbody>\n";
$htmlScrollStart = $this->divDeclaration;
foreach ($heads as $tableColumn => $name)
{
$htmlScrollStart .= $this->headTag . $name . $this->headTagClose;
}
$htmlScrollStart .= $this->theadTagClose . $this->bodyTag;
return $htmlScrollStart;
}
public function calculateDueDate($jobsMailDate,$job)
{
$package = PackageSchedule::model()->findByAttributes(array('package_id'=>$job->packageID));
$projectedDays = $package->projected_days_before_mail_date;
$dropDate1 = $jobsMailDate->projected_mail_date;
$dropDate = wrapBusinessDay($dropDate1); //use this for actual command...
$toSec = 24*60*60;
$dayInt =0;
$secDropDate = strtotime($dropDate1);
do{
$dayInt +=1;
$daysInSec = ($dayInt) * $toSec ;
$secGuessDueDate = $secDropDate - $daysInSec;
$dueDate = date('Y-m-d',$secGuessDueDate);
$difference = $dropDate->difference($dueDate);
}while( $difference != $projectedDays);
return $dueDate;
}
}
?>
Why I think this behavior is odd
The whole slow internet thing is a beast in and of itself, but I don't think that is in the scope of StackOverflow. I'm more concerned about the loading of these divs. The div that loads last, i.e., takes on average 1.5 to 2 seconds, is an ajax request to a page that creates a single widget. The logic behind it is here:
<?php
include_once Yii::app()->extensionPath . "/CurrentExchanges.php";
$currentExchanges = Yii::app()->session['currentExchanges'];
$this->layout = 'barebones';
$this->widget('widgets.ScrollList',array(
'condition'=>
function($muddJob,$scrollList)
{
if ($muddJob->dropDate1 != null && $muddJob->dropDate1 != '0000-00-00')
{
$job = $muddJob->Job;;
if(!empty($job) && $job->packageID != null) //if job exists for the muddExchange and has a package
{
if($job->uploadArtwork == null )
{
$jobsMailDate = JobsMailDate::model()->findByAttributes(array("job_id"=>$job->job_id,'sequence_num'=>1));
if(!empty($jobsMailDate))
{
$calculatedDueDate = $scrollList->calculateDueDate($jobsMailDate,$job);
if (strtotime($calculatedDueDate) <= strtotime(date("Y-m-d")) )
{
array_push($scrollList->_models , $job);
$scrollList->columnValues = array($muddJob->jobDescription,$muddJob->dropDate1,$jobsMailDate->projected_mail_date);
return true;
}
}
}
}
}
return false;
},
'columns' => array('col1'=>"MuddJob#",'col2'=>"Desc",'col3'=>"Drop Date", 'col4' =>'Projected Drop Date'),
'name'=> "Artwork Due Today",
'muddJobs' => $currentExchanges->getCurrentMuddExchanges(),
)
);
?>
The calculateduedate method makes 2 additional calls to the server.
What I'm failing to understand is why the left div (with the most proccessing to do) is usually the first to return and the artLoad is usually the last to load (by a substantial difference). Here are some times returned by chromes network tool:
leftDashLoad: 475ms
rightDashLoad: 593ms
dataLoad: 825ms
artLoad: 1.41s
dataLoad: 453ms
rightDashLoad: 660ms
leftDashLoad: 919ms
artLoad: 1.51s
rightDashLoad: 559ms
leftDashLoad: 1.17s
dataLoad: 1.65s
artLoad: 2.01s
I just can't fathom why the left/right dashloads return so much faster than the artLoad. The code for artLoad and dataLoad are nearly identical save the actual comparison (the one if statement). If this were truly asynchronous, I'd expect the order to be art/dataLoad, rightDashLoad and leftDashLoad based purely on the amounts of computation done in each page. Perhaps the server isn't multithreading, or there is some weird configuration, but if that were the case, I don't see why the effects of the loading would be hit so hard by slow internet.
If I have overlooked something obvious or failed to use google appropriately, I do apologize. Thanks for any help you can offer!
Language/other tech info
The app was developed using the Yii framework. PHP 5.3. Apache Server. INNODB Tables. Server is hosted in dreamhost.
Update
I've changed the view page so that the ajax calls are now calling a controller action. It seems to have made the loading times more similar (asynchronous?) on my local dev environment, but really slowed it down on the QA environment (hosted on dreamhost...). Here is screen shot of the local network tools info:
dev environment
and the qa site (note, that the databases have about the same amounts of data...)
qa environment
Thoughts? It seems to me that my original problem may be solved, as the return times (on my local dev) look more like I expect them to.
Also, my own ignorance as to how NetBeans debugs was playing a part in this synchronous loading thing as xdebug is using a session. I believe this was forcing the ajax calls to wait their turn.
Thanks to #Rowan for helping me diagnose this strange behavior. PHP was trying to request a session before the session was closed in hopes to prevent a race hazard. There were session requests in my code, and there was a session started by my IDE (NetBeans). Removing all references to sessions in the ajax called pages and having ajax call actions that use renderPartial() proved to return the expected behavior (and much better code IMO). Let me know how to properly thank users in terms of StackOverflow (can I up comments, or what is there available? ). Thank you all!

joomla 2.5 pagination always set start limit to 20

I developed my own joomla 2.5 custom component for displaying data table in front-end.It contain filtering,paging and sorting.When navigate via paging it always shows only first 20.
Is there any way to override limit of a query which generate on function getListQuery().
My populateState method is
protected function populateState($ordering = null, $direction = null) {
// Initialise variables.
$app = JFactory::getApplication();
$search = $this->getUserStateFromRequest($this->context . '.filter.search', 'filter_search');
$filter_order = $this->getUserStateFromRequest($this->context . '.filter_order', 'filter_order');
//$filter_order = JRequest::getCmd('filter_order');
$filter_order_Dir = $this->getUserStateFromRequest($this->context . '.filter_order_Dir', 'filter_order_Dir');
//$filter_order_Dir = JRequest::getCmd('filter_order_Dir');
'filter_region', '');
$this->setState('filter_order', $filter_order);
$this->setState('filter_order_Dir', $filter_order_Dir);
// List state information
$limit = $app->getUserStateFromRequest('global.list.limit', 'limit', $app->getCfg('list_limit'));
$this->setState('list.limit', $limit);
$limitstart = JRequest::getVar('limitstart', 0, '', 'int');
$this->setState('list.start', $limitstart);
parent::populateState();
}
Constructor method is
function __construct() {
parent::__construct();
//Get configuration
$app = JFactory::getApplication();
$config = JFactory::getConfig();
// Get the pagination request variables
$this->setState('limit', $app->getUserStateFromRequest('com_jointcm.limit', 'limit', $config->getValue('config.list_limit'), 'int'));
$this->setState('limitstart', JRequest::getVar('limitstart', 0, '', 'int'));
}
List query method is
protected function getListQuery() {
// Create a new query object.
$db = JFactory::getDBO();
$query = $db->getQuery(true);
//code goes here...
..............
return $query;
}
After some digging around and taking a look at the source code of the JModelList class, I realized that problem is with
\libraries\joomla\application\component\modellist.php file ,method name public function getItems(),line number 115.
I changed it to
public function getItems()
{
// Get a storage key.
$store = $this->getStoreId();
// Try to load the data from internal storage.
if (isset($this->cache[$store]))
{
return $this->cache[$store];
}
// Load the list items.
$query = $this->_getListQuery();
//$items = $this->_getList($query, $this->getStart(), $this->getState('list.limit'));
$items = $this->_getList($query, $this->getState('limitstart'), $this->getState('list.limit'));
// Check for a database error.
if ($this->_db->getErrorNum())
{
$this->setError($this->_db->getErrorMsg());
return false;
}
// Add the items to the internal cache.
$this->cache[$store] = $items;
return $this->cache[$store];
}
Change was
$items = $this->_getList($query, $this->getStart(), $this->getState('list.limit'));
to
$items = $this->_getList($query, $this->getState('limitstart'), $this->getState('list.limit'));
It works fine.
In JModelList's getItems() the default method uses getStart() which in turn uses your models getQuery() to get a count of the number of items returned by your query, via _getListCount($query) which in turn calls the particular database adaptors version of getNumRows()). That value is used in the calculation in getStart(), if you have a large complicated query and don't really need to use the fancy getStart() implementation you can just override it in your model (i.e. your version of the JModelList class)
e.g. for our components model's for the front end which have rather complicated $query's returned by getListQuery, in their most basic implementation they do something similar to this:
public function getStart()
{
return $this->getState('list.start');
}
If you don't override it the default JModelList getStart() is invoked which looks like this:
/**
* Method to get the starting number of items for the data set.
*
* #return integer The starting number of items available in the data set.
*
* #since 11.1
*/
public function getStart()
{
$store = $this->getStoreId('getstart');
// Try to load the data from internal storage.
if (isset($this->cache[$store]))
{
return $this->cache[$store];
}
$start = $this->getState('list.start');
$limit = $this->getState('list.limit');
$total = $this->getTotal();
if ($start > $total - $limit)
{
$start = max(0, (int) (ceil($total / $limit) - 1) * $limit);
}
// Add the total to the internal cache.
$this->cache[$store] = $start;
return $this->cache[$store];
}
But, this probably isn't the problem area, it's more likely in your populateState(). At the end of populateState() you call parent::populateState() (if was called at the beginning it wouldn't be overwriting results of your method).
You seem to be duplicating the work done by the parent::populateState() which is probably redundant, looking at JModelList's implementation you will see this:
protected function populateState($ordering = null, $direction = null)
{
// If the context is set, assume that stateful lists are used.
if ($this->context)
{
$app = JFactory::getApplication();
$value = $app->getUserStateFromRequest('global.list.limit', 'limit', $app->getCfg('list_limit'), 'uint');
$limit = $value;
$this->setState('list.limit', $limit);
$value = $app->getUserStateFromRequest($this->context . '.limitstart', 'limitstart', 0);
$limitstart = ($limit != 0 ? (floor($value / $limit) * $limit) : 0);
$this->setState('list.start', $limitstart);
// Check if the ordering field is in the white list, otherwise use the incoming value.
$value = $app->getUserStateFromRequest($this->context . '.ordercol', 'filter_order', $ordering);
if (!in_array($value, $this->filter_fields))
{
$value = $ordering;
$app->setUserState($this->context . '.ordercol', $value);
}
$this->setState('list.ordering', $value);
// Check if the ordering direction is valid, otherwise use the incoming value.
$value = $app->getUserStateFromRequest($this->context . '.orderdirn', 'filter_order_Dir', $direction);
if (!in_array(strtoupper($value), array('ASC', 'DESC', '')))
{
$value = $direction;
$app->setUserState($this->context . '.orderdirn', $value);
}
$this->setState('list.direction', $value);
}
else
{
$this->setState('list.start', 0);
$this->state->set('list.limit', 0);
}
}
The most obvious condition in the parent::populateState() that causes list.start to be set to 0 is the very first line, which checks your object context value, it may be that something is going wrong there and your objects context value is equating to false. (I can't see context defined anywhere... so, it will try an guess/build a context value for you in __construct()).
However, it may also be in the way in which getUserSateFromRequest() is processing the values returned from your request, it's hard to tell with the code available.
You can add limit like this $query->limit('0,40'); in getListQuery() function
Do you have list_limit defined in your component options? If not, then add a new parameter to your component options and call it list_limit. This will allow you to set your pagination limit to what ever you want in the component options.

PHP Turn recursive array function into a class

I have a simple recursive array function that looks like this:
function recursive_array($results) {
global $DBH;
if (count($results)) {
echo $res - > Fname;
foreach($results as $res) {
$STH = $DBH - > query("SELECT FID,FParentID,Fname FROM list WHERE FParentID = ".$res - > FID."");
$fquerycount = $STH - > rowCount();
$STH - > setFetchMode(PDO::FETCH_OBJ);
recursive_array($STH);
}
}
}
$FID = isset($_GET['FID']) ? $_GET[' FID'] : 0;
$STH = $DBH - > query("SELECT FID,FParentID,Fname FROM list WHERE FParentID ='0' ");
$STH - > setFetchMode(PDO::FETCH_OBJ);
recursive_array($STH);
I also have created a simple query class that looks like this:
class queryloop {
function __construct($args) {
global $DBH;
$table = $args['tbl'];
if (array_key_exists('orderby', $args)): $orderby = 'ORDER BY '.$args['orderby'];
else: $orderby = '';endif;
if (array_key_exists('groupby', $args)): $groupby = 'GROUP BY '.$args['groupby'];
else: $groupby = '';endif;
if (array_key_exists('start', $args)): unset($orderby);$start = $args['start'].' , ';
else: $start = '';endif;
if (array_key_exists('limit', $args)): $limit = 'LIMIT '.$start.' '.$args['limit'];
else: $limit = '';endif;
// UNSET the previously used array keys so they are not use again to create the query string
unset($args['tbl']);
unset($args['orderby']);
unset($args['groupby']);
unset($args['start']);
unset($args['limit']);
// Checks if args still an array after UNSET above. If not empty create the query string
if (!empty($args)): foreach($args as $k = > $v): $querystr. = 'AND '.$k.' = \''.$v.'\'';endforeach;
// If args array empty return empty query string
else: $querystr = '';endif;$STH = $DBH - > query("SELECT * FROM ".$table." WHERE key = '".KEY."' ".$querystr." ".$groupby." ".$orderby." ".$limit." ");
if ($STH): $STH - > setFetchMode(PDO::FETCH_OBJ);
while ($row = $STH - > fetch()): foreach($row as $key = > $val):
// check if value is numeric //
if (is_numeric($row - > $key)): $data[$row - > ID][$key] = $row - > $key;
// check if value is array //
elseif(is_array($row - > $key)): $data[$row - > ID][$key] = $row - > $key;
// check if value is not numeric or array convert to html entities //
else: $data[$row - > ID][$key] = htmlentities($row - > $key);endif;endforeach;endwhile;$this - > data = json_encode($data); // return json array if data
else: $this - > data = ''; // return 'null' if no data
endif;
}
}
$args = array('tbl' = > 'atable', 'limit' = > '5', 'start' = > '200', 'orderby' = > 'ID DESC');
$loop = new queryloop($args) // run the loop etc.
How do I turn my recursive array into something like the class queryloop so that I can "pull out" json endoded data I know that this (below) is totally wrong but what ever I do I cannot get a correctly formed json array or even anything to return form my attempted class below. Help would be much appreciate. Thanks in advance.
class recloop {
function __construct() {}
function recursive_array($results) {
global $DBH;
if (count($results)) {
foreach($results as $res) {
echo $res - > Name;
$STH = $DBH - > query("SELECT * FROM atable WHERE ParentID = ".$res - > ID."");
$fquerycount = $STH - > rowCount();
$STH - > setFetchMode(PDO::FETCH_OBJ);
recursive_array($STH);
}
}
}
function recursive_start() {
global $DBH;
$ID = isset($_GET['ID']) ? $_GET['ID'] : 0;
$STH = $DBH - > query("SELECT * FROM atable WHERE ParentID = '".$ID."' ");
$STH - > setFetchMode(PDO::FETCH_OBJ);
recursive_array($STH);
}
}
How do I turn my recursive array into something like the class queryloop so that I can "pull out" json endoded data I know that this (below) is totally wrong but what ever I do I cannot get a correctly formed json array or even anything to return form my attempted class below. Help would be much appreciate. Thanks in advance.
To answer your question, I would say it's not specific if you encapsulate your routines into objects or not that much, but that you take care that each object is there for a sole purpose. For example:
One object is fetching the data from the database.
One object/composite/array is the data-structure, representing the data.
One object or function is taking over the job to convert/encode the data into json.
Within your code I see that you right now are only running SQL-queries. The data fetched from the database server is not stored into a return variable at all, it get's directly consumed while being recursively processed. I assume you do this for debugging reasons.
So the actual question is, what do you want to do? You write that you want to encode an object into json output, which is perfectly possible with json_encodeDocs, however I think you refer to some specific data, like the entity (data) of the most parentId or something.
Following is some mock-up code based on your code for reading purposes (not tested, must not match your needs) that can provide all parent objects of that one specified by ID by using recursion. The recursion has been criticised because this can result in running a lot of queries - and additionally there is risk to create an endless loop which will result in a recursion stack overflow - your program crashes then.
To handle that alternatively, this is bound to the database design (which should be done before the design of the code, and I don't know your database design nor what you actually want to do, so I can't add assumptions for that). So the following code takes care of already queried objects only while still using recursion as the strategy to query your database.
For the actual data-structure I opted for an array of plain old PHP objects, keyed by the ID field from the database (which I assume that it exists per record):
/**
* HTTP Get Parameter (Input)
*/
class HTTPGetParameter {
private $name;
private $default;
public function __construct($name, $default = '') {
$this->name = (string) $name;
$this->default = (string) $default;
}
/**
* #return string
*/
public function getValue()
{
return isset($_GET[$name]) ? $_GET[$name] : $this->default;
}
/**
* #return int
*/
public function getValueInt()
{
return (int) $this->getValue();
}
/**
* #link http://www.php.net/manual/en/language.oop5.magic.php#language.oop5.magic.tostring
*/
public function __toString()
{
return $this->getValue();
}
}
/**
* Data Provider
*/
class PDODataProvider
{
private $pdo;
public function __construct(PDO $pdo)
{
$this->pdo = $pdo;
}
/**
* #return array
*/
public function findAllATableParents($id)
{
return $this->findAllOn('atable', 'ParentID', $id);
}
public function findAllBTableParents($id)
{
return $this->findAllOn('btable', 'ParentID', $id);
}
private function findAllOn($table, $field, $id)
{
$id = (int) $id;
$objects = array();
$sql = sprintf("SELECT * FROM %s WHERE %s = '%d'", $table, $field, $id);
$pdoStatement = $this->pdo->query($sql);
$pdoStatement->setFetchMode(PDO::FETCH_OBJ);
foreach($pdoStatement as $parent)
{
$parentId = $parent->ID;
# parents that had been queried are skipped
if (isset($objects[$parentId]))
continue;
$objects[$parentId] = $parent;
# add parent objects by recursion
$objects += $this->findAllParents($parentId);
}
return $objects;
}
}
/**
* main
*/
$data = new PDODataProvider($DBH);
$id = new HTTPGetParameter('ID', 0);
$objects = $data->findAllParents($id->getValueInt());
echo json_encode($objects);
I hope this example is helpful for you to answer your question.

Categories