So I've been developing PHP for a bit now and have been attempting to troubleshoot why exec with a redirection of standard IO is still hanging the main thread.
exec(escapeshellcmd("php ".getcwd()."/orderProcessing.php" . " " . $Prepared . " " . escapeshellarg(35). "> /dev/null 2>&1 &"));
The code provided is what I've been trialing with, and I can't seem to get it to not hang until the other script completes. The reason I am doing this is in this paticular case, processing an order can take upto 30 seconds, and I don't want the user on the frontend to wait for that.
Can I only spawn child processes with php-fpm?
Is there something I have misconfigured?
Am I just misunderstanding how child processes work?
Setup is:
Centos 7 with Apache HTTPD and PHP 8.1.11
Any help appreciated!
Yes, it is possible in PHP. With proc_open() function, you can send command without wait another process. With handle opened stream, you can catch process status and check it consistently. For example:
$max = 5;
$streams = [];
$command = 'php ' . __DIR__ . '/../../process runSomeProcessCommand';
while (true) {
echo 'Working :' . count($streams) . "\n";
for ($i = 0; $i < $max; $i++) {
if (!isset($streams[$i])) {
$descriptor = [
0 => ['pipe', 'r'],
1 => ['pipe', 'w'],
2 => ['pipe', 'w']
];
echo "proc opened $i \n";
$streams[$i]['proc'] = proc_open($command . ":$i", $descriptor, $pipes);
$streams[$i]['pipes'] = $pipes;
$streams[$i]['ttl'] = time();
usleep(200000);
} else {
$info = proc_get_status($streams[$i]['proc']);
if ($info['running'] === false) {
echo "Finished $i . TTL: (" . (time() - $streams[$i]['ttl']) . " sec.).Response: " . stream_get_contents($streams[$i]['pipes'][1]) . " \n";
fclose($streams[$i]['pipes'][1]);
if ($error = stream_get_contents($streams[$i]['pipes'][2])) {
echo "Error for $i. Error: $error " . PHP_EOL;
fclose($streams[$i]['pipes'][2]);
}
proc_close($streams[$i]['proc']);
unset($streams[$i]);
} else {
echo "Running process (PID {$info['pid']}) - $i: \n";
}
# $return_value = proc_close($streams[$i]);
}
}
sleep(3);
}
There are 5 threads in same time which are doesn't wait one another.
Related
I have a question about a PHP script that I wrote to connect to a collect statistics from IRC servers, as a test as I am new to PHP and still learning.
Here is the script:
<?php
set_time_limit(0);
$servers = array(
"irc.icq.com",
"irc.quakenet.org"
);
function get_statistics ($server, $port) {
$nick = 'IRCDir' . rand(1000, 9999);
$irc = fsockopen($server, $port);
fputs($irc,"USER $nick 0 * :$nick\n");
fputs($irc,"NICK $nick\n");
while ($data = fgets($irc, 128)) {
$ex = explode(' ', $data);
if (isset($ex[0]) && $ex[0] == "PING") {
fputs($irc, "PONG ".$ex[1]."\n");
}
if (count($ex) > 0) {
if (isset($ex[1]) && $ex[1] == "001") {
$network = $ex[6];
echo date('h:i:s') . " network: " . $server;
echo "\n";
}
if (isset($ex[1]) && $ex[1] == "002") {
$server = str_replace(',', '', $ex[6]);
echo date('h:i:s') . " server: " . $server;
echo "\n";
}
if (isset($ex[1]) && $ex[1] == "251") {
$users = $ex[5] + $ex[8];
$servers = $ex[11];
echo date('h:i:s') . " users: " . $users;
echo "\n";
echo date('h:i:s') . " servers: " . $servers;
echo "\n";
}
if (isset($ex[1]) && $ex[1] == "252") {
$ircops = $ex[3];
echo date('h:i:s') . " ircops: " . $ircops;
echo "\n";
}
if (isset($ex[1]) && $ex[1] == "254") {
$channels = $ex[3];
echo date('h:i:s') . " channels: " . $channels;
echo "\n";
}
}
}
fclose($irc);
}
foreach ($servers as $server) {
echo date('h:i:s') . " getting statistics for " . $server;
echo "\n";
get_statistics($server, '6667');
}
exit;
?>
Here is the output:
root#li140-48:~# php bot.php
11:04:23 getting statistics for irc.freenode.net
11:04:24 network: irc.freenode.net
11:04:24 server: sendak.freenode.net
11:04:24 users: 90601
11:04:24 servers: 26
11:04:24 ircops: 22
11:04:24 channels: 50958
11:05:25 getting statistics for irc.icq.com
11:05:26 network: irc.icq.com
11:05:26 server: irc-k01a.orange.icq.com
11:05:26 users: 2671
11:05:26 servers: 3
11:05:26 ircops: 16
11:05:26 channels: 810
11:06:26 getting statistics for irc.quakenet.org
11:06:28 network: irc.quakenet.org
11:06:28 server: blacklotus.ca.us.quakenet.org
11:06:28 users: 37648
11:06:28 servers: 40
11:06:28 ircops: 67
11:06:28 channels: 26711
root#li140-48:~#
My question is what causes the delay between the servers being indexed?
At 10.24:19 it has completed the index of irc.icq.com so at this point it should disconnect and immediately index the next server in the array, but instead of that it is waiting exactly a minute before doing so, but I have not mentioned a minute in the script?
Hoping some PHP gurus can lend a hand!
60 seconds is the default socket timeout. You're continuing to try an read from the stream after you've already got all the information you're going to get - up until the timeout is finally reached.
I have the problem that in a PHP application Gearman jobs sometimes are passed to more than one worker. I could reduce a code to reproduce it into one file. Now I am not sure if this is a bug in Gearman or a bug in the pecl library or maybe in my code.
Here is the code to reproduce the error:
#!/usr/bin/php
<?php
// Try 'standard', 'exception' or 'exception-sleep'.
$sWorker = 'exception';
// Detect run mode "client" or "worker".
if (!isset($argv[1]))
$sMode = 'client';
else
$sMode = 'worker-' . $sWorker;
$sLogFilePath = __DIR__ . '/log.txt';
switch ($sMode) {
case 'client':
// Remove all queued test jobs and quit if there are test workers running.
prepare();
// Init the greaman client.
$Client= new GearmanClient;
$Client->addServer();
// Empty the log file.
file_put_contents($sLogFilePath, '');
// Start some worker processes.
$aPids = array();
for ($i = 0; $i < 100; $i++)
$aPids[] = exec('php ' . __FILE__ . ' worker > /dev/null 2>&1 & echo $!');
// Start some jobs. Also try doHigh(), doBackground() and
// doBackgroundHigh();
for ($i = 0; $i < 50; $i++)
$Client->doNormal('test', $i);
// Wait a second (when running jobs in background).
// sleep(1);
// Prepare the log file entries.
$aJobs = array();
$aLines = file($sLogFilePath);
foreach ($aLines as $sLine) {
list($sTime, $sPid, $sHandle, $sWorkload) = $aAttributes = explode("\t", $sLine);
$sWorkload = trim($sWorkload);
if (!isset($aJobs[$sWorkload]))
$aJobs[$sWorkload] = array();
$aJobs[$sWorkload][] = $aAttributes;
}
// Remove all jobs that have been passed to only one worker as expected.
foreach ($aJobs as $sWorkload => $aJob) {
if (count($aJob) === 1)
unset($aJobs[$sWorkload]);
}
echo "\n\n";
if (empty($aJobs))
echo "No job has been passed to more than one worker.";
else {
echo "Those jobs has been passed more than one times to a worker:\n";
foreach ($aJobs as $sWorload => $aJob) {
echo "\nJob #" . $sWorload . ":\n";
foreach ($aJob as $aAttributes)
echo " $aAttributes[2] (Worker PID: $aAttributes[1])\n";
}
}
echo "\n\n";
// Kill all started workers.
foreach ($aPids as $sPid)
exec('kill ' . $sPid . ' > /dev/null 2>&1');
break;
case 'worker-standard':
$Worker = new GearmanWorker;
$Worker->addServer();
$Worker->addFunction('test', 'logJob');
$bShutdown = false;
while ($Worker->work())
if ($bShutdown)
continue;
break;
case 'worker-exception':
try {
$Worker = new GearmanWorker;
$Worker->addServer();
$Worker->addFunction('test', 'logJob');
$bShutdown = false;
while ($Worker->work())
if ($bShutdown)
throw new \Exception;
} catch (\Exception $E) {
}
break;
case 'worker-exception-sleep':
try {
$Worker = new GearmanWorker;
$Worker->addServer();
$Worker->addFunction('test', 'logJob');
$bShutdown = false;
while ($Worker->work())
{
if ($bShutdown) {
sleep(1);
throw new \Exception;
}
}
} catch (\Exception $E) {
}
break;
}
function logJob(\GearmanJob $Job)
{
global $bShutdown, $sLogFilePath;
$sLine = microtime(true) . "\t" . getmypid() . "\t" . $Job->handle() . "\t" . $Job->workload() . "\n";
file_put_contents($sLogFilePath, $sLine, FILE_APPEND);
$bShutdown = true;
}
function prepare()
{
$rGearman = fsockopen('127.0.0.1', '4730', $iErrno, $sErrstr, 3);
$aBuffer = array();
fputs ($rGearman, 'STATUS' . PHP_EOL);
stream_set_timeout($rGearman, 1);
while (!feof($rGearman))
if ('.' . PHP_EOL !== $sLine = fgets($rGearman, 128))
$aBuffer[] = $sLine;
else
break;
fclose($rGearman);
$bJobsInQueue = false;
$bWorkersRunning = false;
foreach ($aBuffer as $sFunctionLine) {
list($sFunctionName, $iQueuedJobs, $iRunningJobs, $iWorkers) = explode("\t", $sFunctionLine);
if ('test' === $sFunctionName) {
if (0 != $iQueuedJobs)
$bJobsInQueue = true;
if (0 != $iWorkers)
$bWorkersRunning = true;;
}
}
// Exit if there are workers running.
if ($bWorkersRunning)
die("There are some Gearman workers running that have registered a 'test' function. Please stop these workers and run again.\n\n");
// If there are test jobs in the queue start a worker that eat up the jobs.
if ($bJobsInQueue) {
$sPid = exec('gearman -n -w -f test > /dev/null 2>&1 & echo $!');
sleep(1);
exec ("kill $sPid > /dev/null 2>&1");
// Repeat this method to make sure all jobs are removed.
prepare();
}
}
When you run this code on the command line it should output "No job
has been passed to more than one worker." but insted it alway outputs a list of some jobs that have been passed to more than one worker. The error doesn't appear if you set $sWorker = 'standard'; or 'exception-sleep'.
It would help me a lot if you could run the code and tell me if you are able to reproduce the error of if I have a bug in the code.
Had exactly the same issue with Gearman 0.24, PECL lib 1.0.2. Was able to reproduce the error with your script every time.
An older version of Gearman (0.14 I think) used to work fine.
Upgrading Gearman to 0.33 fixed the issue.
I have gone through all the similar questions and nothing fits the bill.
I am running a big script, which ran on chron on an old server but failed on the new so I am working on and testing in browser.
I have two functions, one pulls properties from the database, and then runs them through another which converts the price into 4 currencies, and if the value is different updates the row. The functions are as follows:
<?php
function convert_price($fore_currency, $aft_currency, $amount)
{
echo "going into convert<br/>";
$url = "http://www.currency.me.uk/remote/ER-ERC-AJAX.php?ConvertFrom=" . $fore_currency .
"&ConvertTo=" . $aft_currency . "&amount=" . $amount;
if (!is_int((int)file_get_contents($url))) {
//echo "Failed on convert<br/>";
return false;
} else {
//echo "Conversion done";
return (int)file_get_contents($url);
}
}
function run_conversion($refno = '', $output = false)
{
global $wpdb;
$currencies = array("GBP", "EUR", "TRY", "USD");
$q = "SELECT * FROM Properties ";
$q .= (!empty($refno)) ? " WHERE Refno='" . $refno . "'" : "";
$rows = $wpdb->get_results($wpdb->prepare($q), ARRAY_A);
$currencies = array("USD", "GBP", "TRY", "EUR");
//$wpdb->show_errors();
echo "in Run Conversion " . "<br/>";
foreach ($rows as $row) {
echo "In ROw <br/>";
foreach ($currencies as $currency) {
if ($currency != $row['Currency'] && $row['Price'] != 0) {
$currfield = $currency . "_Price";
$newprice = convert_price($row['Currency'], $currency, $row['Price']);
echo "Old Price Was " . $row['Price'] . " New Price Is " . $newprice . "<br/>";
if ($newprice) {
if ($row[$currfield] != $newprice) {
$newq = "UPDATE Properties SET DateUpdated = '" . date("Y-m-d h:i:s") . "', " .
$currfield . "=" . $newprice . " WHERE Refno='" . $row['Refno'] . "'";
$newr = $wpdb->query($newq);
if ($output) {
echo $newq . " executed <br/>";
} else {
echo "query failed " . $wpdb->print_error();
}
} else {
if ($output) {
echo "No need to update " . $row['Refno'] . " with " . $newprice . "<br/>";
}
}
} else {
echo "Currency conversion failed";
}
}
}
}
}
?>
I then run the process from a seperate file for the sake of chron like so:
require($_SERVER['DOCUMENT_ROOT'] . "/functions.php"); // page containing functions
run_conversion('',true);
If I limit the mysql query to 100 properties it runs fine and I get all the output in a nice stream. But when I try to run it in full the script completes (I can see from rows updated in db) but no output. I have tried upping the memory allowance but no joy. Any ideas gratefully received. Also, I get a 500 error when changing from Apache Module to CGI. Any ideas on that also well received as ideally I would like to turn site onto fastCGI.
You're running out of memory most likely. Probably in your DB layer. Your best bet is to unset your iterative values at the end of each loop:
foreach ( $rows as $row ) {
...
foreach ( $currencies as $currency ) {
...
unset( $currency, $newr, $currfield, $newprice );
}
unset( $row );
}
This will definitely help.
As another answer suggests you may be running out of memory, but if you make an HTTP call everytime the loop is running, and you have 100+ currencies, this may also cause the problem. Try limit it to 5 currencies for example.
If this is indeed the problem I would split the process, so a chunk of currencies have their respective value updated per cron job, instead of everything.
I have zend database profiler set up and turned on in my development site. I can see all the queries except for the DESCRIBE queries, which I know it should be running each time I ask for a new table object. I'm using something like this to look at the queries:
$db = Zend_Registry::get('db');
$profiler = new Zend_Db_Profiler();
$profiler->setEnabled(true);
$db->setProfiler($profiler);
$i = 1;
$output = 'PROFILE FOR: '.$_SERVER['REQUEST_URI'] . "\n";
foreach ($profiler->getQueryProfiles() as $query) {
$output .= "Query ".$i++.": ".$query->getQuery(). "\n";
}
$output .= 'Average query length: ' . $totalTime / $queryCount .
' seconds' . "\n";
$output .= 'Queries per second: ' . $queryCount / $totalTime . "\n";
$output .= 'Longest query length: ' . $longestTime . "\n";
$output .= "Longest query: \n" . $longestQuery . "\n\n";
file_put_contents('/tmp/zend_profiler.log', $output, FILE_APPEND);
}
Not sure why I can't see the describe queries. Has anyone else run into this issue?
Maybe you have activated the metadata cache for you database connection (which usually is a very good thing especially in production). In that case the DESCRIBE queries won't be executed until the cache expires.
Verify it by executing this line somewhere in application after the bootstrap
if (null != Zend_Db_Table_Abstract::getDefaultMetadataCache()) {
echo "Cache is active, describe queries not run";
} else {
echo "Cache is not active";
}
I have a PHP page I need to limit execution access of
to only clients inside our firewall.
How would I write a php-script that can look up the clients
ip-address and match it to a ip-range (for instance 10...* or 200.10.10.*).
You can use ip2long to convert dotted quads to long values, then just perform some arithmetic to check a given network/mask combination:
$network=ip2long("200.10.10.0");
$mask=ip2long("255.255.255.0");
$remote=ip2long($_SERVER['REMOTE_ADDR']);
if (($remote & $mask) == $network)
{
//match!
}
else
{
//does not match!
}
Well, assuming you're using Apache, there's a module called mod_authz_host that you can use.
Together with the file directive, you could limit access to a given php script for a range of ip addresses.
Here is the link to the documentation on that module:
http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html
Here's a quick example (assuming your php file is called admin.php):
<file admin.php>
Order Deny,Allow
Deny from all
Allow from 200.10.10
</file>
The added benefit to the other solution suggested here is that you can control the security aspects from outside your application logic - a more flexible approach that does not impose any limitations on your PHP code.
PHP Limit/Block Website requests for Spiders/Bots/Clients etc.
Here i have written a PHP function which can Block unwanted Requests to reduce your Website-Traffic. God for Spiders, Bots and annoying Clients.
DEMO:
http://szczepan.info/9-webdesign/php/1-php-limit-block-website-requests-for-spiders-bots-clients-etc.html
/* Function which can Block unwanted Requests
* #return boolean/array status
*/
function requestBlocker()
{
/*
Version 1.0 11 Jan 2013
Author: Szczepan K
http://www.szczepan.info
me[#] szczepan [dot] info
###Description###
A PHP function which can Block unwanted Requests to reduce your Website-Traffic.
God for Spiders, Bots and annoying Clients.
*/
$dir = 'requestBlocker/'; ## Create & set directory writeable!!!!
$rules = array(
#You can add multiple Rules in a array like this one here
#Notice that large "sec definitions" (like 60*60*60) will blow up your client File
array(
//if >5 requests in 5 Seconds then Block client 15 Seconds
'requests' => 5, //5 requests
'sek' => 5, //5 requests in 5 Seconds
'blockTime' => 15 // Block client 15 Seconds
),
array(
//if >10 requests in 30 Seconds then Block client 20 Seconds
'requests' => 10, //10 requests
'sek' => 30, //10 requests in 30 Seconds
'blockTime' => 20 // Block client 20 Seconds
),
array(
//if >200 requests in 1 Hour then Block client 10 Minutes
'requests' => 200, //200 requests
'sek' => 60 * 60, //200 requests in 1 Hour
'blockTime' => 60 * 10 // Block client 10 Minutes
)
);
$time = time();
$blockIt = array();
$user = array();
#Set Unique Name for each Client-File
$user[] = isset($_SERVER['REMOTE_ADDR']) ? $_SERVER['REMOTE_ADDR'] : 'IP_unknown';
$user[] = isset($_SERVER['HTTP_USER_AGENT']) ? $_SERVER['HTTP_USER_AGENT'] : '';
$user[] = strtolower(gethostbyaddr($user[0]));
# Notice that i use files because bots does not accept Sessions
$botFile = $dir . substr($user[0], 0, 8) . '_' . substr(md5(join('', $user)), 0, 5) . '.txt';
if (file_exists($botFile)) {
$file = file_get_contents($botFile);
$client = unserialize($file);
} else {
$client = array();
$client['time'][$time] = 0;
}
# Set/Unset Blocktime for blocked Clients
if (isset($client['block'])) {
foreach ($client['block'] as $ruleNr => $timestampPast) {
$left = $time - $timestampPast;
if (($left) > $rules[$ruleNr]['blockTime']) {
unset($client['block'][$ruleNr]);
continue;
}
$blockIt[] = 'Block active for Rule: ' . $ruleNr . ' - unlock in ' . ($left - $rules[$ruleNr]['blockTime']) . ' Sec.';
}
if (!empty($blockIt)) {
return $blockIt;
}
}
# log/count each access
if (!isset($client['time'][$time])) {
$client['time'][$time] = 1;
} else {
$client['time'][$time]++;
}
#check the Rules for Client
$min = array(
0
);
foreach ($rules as $ruleNr => $v) {
$i = 0;
$tr = false;
$sum[$ruleNr] = '';
$requests = $v['requests'];
$sek = $v['sek'];
foreach ($client['time'] as $timestampPast => $count) {
if (($time - $timestampPast) < $sek) {
$sum[$ruleNr] += $count;
if ($tr == false) {
#register non-use Timestamps for File
$min[] = $i;
unset($min[0]);
$tr = true;
}
}
$i++;
}
if ($sum[$ruleNr] > $requests) {
$blockIt[] = 'Limit : ' . $ruleNr . '=' . $requests . ' requests in ' . $sek . ' seconds!';
$client['block'][$ruleNr] = $time;
}
}
$min = min($min) - 1;
#drop non-use Timestamps in File
foreach ($client['time'] as $k => $v) {
if (!($min <= $i)) {
unset($client['time'][$k]);
}
}
$file = file_put_contents($botFile, serialize($client));
return $blockIt;
}
if ($t = requestBlocker()) {
echo 'dont pass here!';
print_R($t);
} else {
echo "go on!";
}