can't see describe queries in zend framework db profiler - php

I have zend database profiler set up and turned on in my development site. I can see all the queries except for the DESCRIBE queries, which I know it should be running each time I ask for a new table object. I'm using something like this to look at the queries:
$db = Zend_Registry::get('db');
$profiler = new Zend_Db_Profiler();
$profiler->setEnabled(true);
$db->setProfiler($profiler);
$i = 1;
$output = 'PROFILE FOR: '.$_SERVER['REQUEST_URI'] . "\n";
foreach ($profiler->getQueryProfiles() as $query) {
$output .= "Query ".$i++.": ".$query->getQuery(). "\n";
}
$output .= 'Average query length: ' . $totalTime / $queryCount .
' seconds' . "\n";
$output .= 'Queries per second: ' . $queryCount / $totalTime . "\n";
$output .= 'Longest query length: ' . $longestTime . "\n";
$output .= "Longest query: \n" . $longestQuery . "\n\n";
file_put_contents('/tmp/zend_profiler.log', $output, FILE_APPEND);
}
Not sure why I can't see the describe queries. Has anyone else run into this issue?

Maybe you have activated the metadata cache for you database connection (which usually is a very good thing especially in production). In that case the DESCRIBE queries won't be executed until the cache expires.
Verify it by executing this line somewhere in application after the bootstrap
if (null != Zend_Db_Table_Abstract::getDefaultMetadataCache()) {
echo "Cache is active, describe queries not run";
} else {
echo "Cache is not active";
}

Related

PHP and Child Processes

So I've been developing PHP for a bit now and have been attempting to troubleshoot why exec with a redirection of standard IO is still hanging the main thread.
exec(escapeshellcmd("php ".getcwd()."/orderProcessing.php" . " " . $Prepared . " " . escapeshellarg(35). "> /dev/null 2>&1 &"));
The code provided is what I've been trialing with, and I can't seem to get it to not hang until the other script completes. The reason I am doing this is in this paticular case, processing an order can take upto 30 seconds, and I don't want the user on the frontend to wait for that.
Can I only spawn child processes with php-fpm?
Is there something I have misconfigured?
Am I just misunderstanding how child processes work?
Setup is:
Centos 7 with Apache HTTPD and PHP 8.1.11
Any help appreciated!
Yes, it is possible in PHP. With proc_open() function, you can send command without wait another process. With handle opened stream, you can catch process status and check it consistently. For example:
$max = 5;
$streams = [];
$command = 'php ' . __DIR__ . '/../../process runSomeProcessCommand';
while (true) {
echo 'Working :' . count($streams) . "\n";
for ($i = 0; $i < $max; $i++) {
if (!isset($streams[$i])) {
$descriptor = [
0 => ['pipe', 'r'],
1 => ['pipe', 'w'],
2 => ['pipe', 'w']
];
echo "proc opened $i \n";
$streams[$i]['proc'] = proc_open($command . ":$i", $descriptor, $pipes);
$streams[$i]['pipes'] = $pipes;
$streams[$i]['ttl'] = time();
usleep(200000);
} else {
$info = proc_get_status($streams[$i]['proc']);
if ($info['running'] === false) {
echo "Finished $i . TTL: (" . (time() - $streams[$i]['ttl']) . " sec.).Response: " . stream_get_contents($streams[$i]['pipes'][1]) . " \n";
fclose($streams[$i]['pipes'][1]);
if ($error = stream_get_contents($streams[$i]['pipes'][2])) {
echo "Error for $i. Error: $error " . PHP_EOL;
fclose($streams[$i]['pipes'][2]);
}
proc_close($streams[$i]['proc']);
unset($streams[$i]);
} else {
echo "Running process (PID {$info['pid']}) - $i: \n";
}
# $return_value = proc_close($streams[$i]);
}
}
sleep(3);
}
There are 5 threads in same time which are doesn't wait one another.

laravel mysql to query builder

i'm relatively new to laravel and i'm having issues when trying to convert this function to laravel's query builder. This is the function i've been given which also runs a python script to decrypt the database.
Using the documentation from laravel.com you can do something like this:
function call($unitId)
{
$pfContact = DB::table('PFContact')
->where('UnitID', $unitId)
->latest() // Order by created_at
->first([ // Only retrieve these columns
'Send',
'Receive',
'Core',
'lock'
]);
$pfReadings = DB::table('PFReadings')
->get();
$rowCount = $pfReadings->count();
foreach ($pfReadings as $i => $reading) {
echo $i < count($reading) / $rowCount;
foreach ($reading as $column => $value) {
echo shell_exec(
'python3 enc.py ' . $value
. ' ' . $pfContact->Send
. ' ' . $pfContact->Receive
. ' ' . $pfContact->Core
. ' ' . $pfContact->lock . ' '
. $unitId . ' l'
) . '~';
}
echo ';';
}
}
And although I do not know what arguments this pyhton script needs you should really think this through. Why would you use PHP for this and not just handle everything from the python (or php) side because this looks over complicated to me.

ModX Revolution Caching Dynamicly Generated Placeholders

How can I cache (using ModX's cacheManager), the dynamic placeholders i am generating here:
// recursive function to generate our un-ordered list of menu items
if(!function_exists('GenerateMenu')){
function GenerateMenu($level, $parent = 0, $wc, $wi, $we, $cs, $sc, $cl){
try {
$lvl = ++$level;
global $modx;
// Check for #1, should this be cached, #2 does it already exist in the cache
$cached = $modx->cacheManager->get('Navigator');
if($sc && isset($cached)){
// need to get the placeholders from cache - here somehow!
return $cached;
}else{
// get the site start
$siteStartId = $modx->getOption('site_start');
// Set our initial rows array
$rows = array();
// Run our query to get our menu items
$sql = 'Select `id`, `menutitle`, `uri`, `longtitle`, `parent`, `link_attributes`, `class_key`, `content`, `alias`, `introtext`
From `' . $modx->getOption(xPDO::OPT_TABLE_PREFIX) . 'site_content`
Where `deleted` = 0 AND `hidemenu` = 0 AND `published` = 1 AND `parent` = :parent
Order by `parent`, `menuindex`';
$query = new xPDOCriteria($modx, $sql, array(':parent' => $parent));
if ($query->stmt && $query->stmt->execute()) {
$rows = $query->stmt->fetchAll(PDO::FETCH_ASSOC);
}
// do some cleanup
unset($query, $sql);
// make sure we have some rows, and then build our html for the menu
if($rows){
// grab a count of our results
$rCt = count($rows);
$cls = ($lvl > 1) ? 'sub-item-' . $lvl : 'main-item-' . $lvl;
$ret .= ' <ul class="' . $cls . '" id="' . $cls . '-' . $parent . '">' . "\r\n";
for($i = 0; $i < $rCt; ++$i){
// if this resource is a WebLink, show the content in it, as the URL for the href tag, otherwise, use the resource's URI
$url = ($rows[$i]['class_key'] == 'modWebLink') ? $rows[$i]['content'] : '/' . $rows[$i]['uri'];
// Check for the site's start id, if true, show a "blank" link, otherwise show the $url
$showUrl = ($siteStartId == $rows[$i]['id']) ? '/' : $url;
$la = (strlen($rows[$i]['link_attributes']) > 0) ? ' ' . $rows[$i]['link_attributes'] : null;
// Set some dynamic placeholders, they can only be used ont he pages that contain this snippet
$modx->toPlaceholders(array('Title-' . $rows[$i]['id'] => $rows[$i]['longtitle'],
'MenuTitle-' . $rows[$i]['id'] => $rows[$i]['menutitle'],
'URL-' . $rows[$i]['id'] => $showUrl),
'link');
$ret .= ' <li class="' . $cls . '" id="' . $rows[$i]['alias'] . '">' . "\r\n";
$ret .= ' <a href="' . $showUrl . '" title="' . $rows[$i]['longtitle'] . '"' . $la . '>' . $rows[$i]['menutitle'] . '</a>' . "\r\n";
$ret .= GenerateMenu($lvl, $rows[$i]['id']);
// Check for a snippet, and render it
$it = $rows[$i]['introtext'];
if($cs && IsSnippet($it)){
// if we find a snippet in the Summary field, run it, and attach it to our output
preg_match('/\[\[!?(.*)\]\]/', $it, $sm);
$ret .= $modx->runSnippet($sm[1]);
// clean up
unset($sm);
}
$ret .= ' </li>' . "\r\n";
}
$ret .= ' </ul>' . "\r\n";
}
// clean up
unset($rows);
// Check to see if we should cache it, if so, set it to cache, and apply the length of time it should be cached for: defaults to 2 hours
if($sc){
// NEED TO SET THE PLACEHOLDERS TO CACHE SOMEHOW
$modx->cacheManager->set('Navigator', $ret, $cl);
}
// return the menu
return $ret;
}
} catch(Exception $e) {
// If there was an error, make sure to write it out to the modX Error Log
$modx->log(modX::LOG_LEVEL_ERROR, '[Navigator] Error: ' . $e->getMessage());
return null;
}
}
}
The easiest solution may be pdoTools which allows you to establish caching at run time.
http://www.shawnwilkerson.com/modx/tags/pdotools/
Also, I do not believe resource placeholders are cached which is the best place to have your items cahced:
case '+':
$tagName= substr($tagName, 1 + $tokenOffset);
$element= new modPlaceholderTag($this->modx);
$element->set('name', $tagName);
$element->setTag($outerTag);
$elementOutput= $element->process($tagPropString);
break;
From lines 455-461 of https://github.com/modxcms/revolution/blob/master/core/model/modx/modparser.class.php#L455
You may notice the other tag types have:
$element->setCacheable($cacheable);
I cover the parser in Appendix D of my book. I found some issues in it in 2011 which Jason corrected.
Move toPlaceholders() to the end of your script and instead cache the array of placeholder data:
// attempt to retrieve placeholders from cache
$placeholders = $modx->cacheManager->get('Navigator');
// if not in cache, run your snippet logic and generate the data
if (empty($placeholders))) {
$placeholders = array(
'myplaceholder' => 'placeholder data'
);
$modx->cacheManager->set('Navigator', $placeholders, $cl);
}
// set placeholders for use by MODX
$modx->toPlaceholders($placeholders);

php script executes but no output

I have gone through all the similar questions and nothing fits the bill.
I am running a big script, which ran on chron on an old server but failed on the new so I am working on and testing in browser.
I have two functions, one pulls properties from the database, and then runs them through another which converts the price into 4 currencies, and if the value is different updates the row. The functions are as follows:
<?php
function convert_price($fore_currency, $aft_currency, $amount)
{
echo "going into convert<br/>";
$url = "http://www.currency.me.uk/remote/ER-ERC-AJAX.php?ConvertFrom=" . $fore_currency .
"&ConvertTo=" . $aft_currency . "&amount=" . $amount;
if (!is_int((int)file_get_contents($url))) {
//echo "Failed on convert<br/>";
return false;
} else {
//echo "Conversion done";
return (int)file_get_contents($url);
}
}
function run_conversion($refno = '', $output = false)
{
global $wpdb;
$currencies = array("GBP", "EUR", "TRY", "USD");
$q = "SELECT * FROM Properties ";
$q .= (!empty($refno)) ? " WHERE Refno='" . $refno . "'" : "";
$rows = $wpdb->get_results($wpdb->prepare($q), ARRAY_A);
$currencies = array("USD", "GBP", "TRY", "EUR");
//$wpdb->show_errors();
echo "in Run Conversion " . "<br/>";
foreach ($rows as $row) {
echo "In ROw <br/>";
foreach ($currencies as $currency) {
if ($currency != $row['Currency'] && $row['Price'] != 0) {
$currfield = $currency . "_Price";
$newprice = convert_price($row['Currency'], $currency, $row['Price']);
echo "Old Price Was " . $row['Price'] . " New Price Is " . $newprice . "<br/>";
if ($newprice) {
if ($row[$currfield] != $newprice) {
$newq = "UPDATE Properties SET DateUpdated = '" . date("Y-m-d h:i:s") . "', " .
$currfield . "=" . $newprice . " WHERE Refno='" . $row['Refno'] . "'";
$newr = $wpdb->query($newq);
if ($output) {
echo $newq . " executed <br/>";
} else {
echo "query failed " . $wpdb->print_error();
}
} else {
if ($output) {
echo "No need to update " . $row['Refno'] . " with " . $newprice . "<br/>";
}
}
} else {
echo "Currency conversion failed";
}
}
}
}
}
?>
I then run the process from a seperate file for the sake of chron like so:
require($_SERVER['DOCUMENT_ROOT'] . "/functions.php"); // page containing functions
run_conversion('',true);
If I limit the mysql query to 100 properties it runs fine and I get all the output in a nice stream. But when I try to run it in full the script completes (I can see from rows updated in db) but no output. I have tried upping the memory allowance but no joy. Any ideas gratefully received. Also, I get a 500 error when changing from Apache Module to CGI. Any ideas on that also well received as ideally I would like to turn site onto fastCGI.
You're running out of memory most likely. Probably in your DB layer. Your best bet is to unset your iterative values at the end of each loop:
foreach ( $rows as $row ) {
...
foreach ( $currencies as $currency ) {
...
unset( $currency, $newr, $currfield, $newprice );
}
unset( $row );
}
This will definitely help.
As another answer suggests you may be running out of memory, but if you make an HTTP call everytime the loop is running, and you have 100+ currencies, this may also cause the problem. Try limit it to 5 currencies for example.
If this is indeed the problem I would split the process, so a chunk of currencies have their respective value updated per cron job, instead of everything.

Access to Access Transfer

I have asked a couple of questions on here in a rush, and I am getting nowhere fast, I keep changing things and just ending up with a different problem, and no closer to having a clue what is causing it. I am a PHP MYSQL man, and I am having to work with Access via the COM class.
Basically the client has multiple servers, each with an access database, and each with a CMS, the tables should contain the same data and have slipped out of Sync. It is my job to come up with a way of resyncing them.
I have came to bringing the data out into an Array, serializing it and saving it to a file on the main server (the one all should sync to) and then on the other servers, downloading the file, unserializing it and item by item, checking if it is in the DB, and if not inserting it. The insert is failing. I have tried building an sql query for each item and doing $this->conn->Execute(); and that is failing row by row on silly things, so I am now trying this:
function syncCMS()
{
$this->output['msg'] .= "<p><b>The following properties were added to the database on ." . $this->hsite . "</b></p>";
$this->get_field_names();
$this->make_connection(); //assigns connection to $this->conn
$rs = new COM('ADODB.Recordset');
$rs->CursorType = 2;
$rs->CursorLocation = 1;
$rs->LockType = 4;
$rs->Open($this->table_name,$this->conn);
foreach ($this->awayPropertyDetails as $key => $property)
{
$this->check_for_property($property['pname']);
if (!$this->property_exists || $this->mode == "fullSync")
{
unset($values);
$bfields = array("pshow","rent","best", "oda1", "oda2", "oda3", "oda4", "oda5", "oda6", "odap", "topool","tomountain","tofitness","tosauna"); //stores the yes/no values
$q = "INSERT INTO " . $this->table_name . " (" . $this->dbfields . ") VALUES (";
foreach ($property as $k => $value)
{
if ($k == "Kimlik") {
$value = null;
}
if ($k == "tarih")
{
$value = date("d/m/Y");
$value = "'" . $value . "'";
}
if (in_array($k,$bfields))
{
if ($value == "")
{
$value = 'FALSE';
}
else
{
$value = 'TRUE';
}
}
$rs->fields->$k = $value;
$this->output['msg'] .= $property['pname'] . " added";
}
$rs->BatchUpdate();
$rs->Close();
$this->conn->Close();
//$this->download_images($property['OBJECT_NR'],$k);
//$this->output['msg'] .= "<p>Images added</p>";
}
}
$message .= "</ul>";
$this->output['msg'] .= $message;
$this->sendOutput();
//print_r($property);*/
}
And getting this:
Fatal error: Uncaught exception 'com_exception' with message 'Unable to lookup `Kimlik': Unknown name. ' in D:\inetpub...
Kimlik is the name of the first field, the Auto Number field, I took it out and the problem shifted to the second field.
I got there with:
$sql = "SELECT * FROM " . $this->table_name;
if (!$this->property_exists || $this->mode == "fullSync")
{
$this->make_connection();
$rs->Open($sql,$this->conn);
$rs->addnew();
foreach($rs->Fields as $field)
{
if ($field->name != "Kimlik")
{
$rs->Fields[$field->name] = $property[$field->name];
}
}
$rs->Update();
$rs->Close();
$this->conn->Close();
$msg = $this->download_images($property['OBJECT_NR'],$k);
$this->output['msg'] .= $property['pname'] . " added<br/>" . $msg . "<br/><br/>";
}

Categories