PHP ErrorException creation takes a long time - php

After profiling a PHP script, I found that creating an exception appears to take a huge amount of time:
%Time Real User System Calls secs/call Name
100.0 0.00 3448.05 0.00 8.89 0.00 1.81 1 0.0000 3448.0543 main
64.7 2230.28 2230.28 0.02 0.02 0.00 0.00 892 2.5003 2.5003 sleep
35.1 0.01 1211.84 0.00 8.16 0.00 1.49 1001 0.0000 1.2106 do_process
34.7 986.81 1197.34 2.59 2.60 1.18 1.18 1330 0.7420 0.9003 file_get_contents
6.1 0.00 210.53 0.00 0.01 0.00 0.01 28 0.0000 7.5191 __lambda_func
6.1 210.53 210.53 0.01 0.01 0.01 0.01 28 7.5191 7.5191 ErrorException->__construct
0.4 0.00 13.47 0.01 5.21 0.00 0.18 206 0.0000 0.0654 do_check
0.4 13.15 13.15 5.08 5.08 0.15 0.15 402 0.0327 0.0327 preg_replace
So that's 7.5 seconds of real (not CPU) time per ErrorException->__construct. I'm looking for ideas on how to improve this!
Relevant code:
set_error_handler(
create_function(
'$severity, $message, $file, $line',
'throw new ErrorException($message, $severity, $severity, $file, $line);'
)
);
$opts = array(
'http' => array(
'method' => 'GET',
'timeout' => 60
)
);
$ctx = stream_context_create($opts);
try {
$this->data = file_get_contents($url, false, $ctx);
} catch (Exception $e) {
# The set_error_handler call ensures that we arrive here on any type
# of error.
$this->data = '';
if(preg_match('/HTTP\/1.[0|1] 4[0-9][0-9] /', $e->getMessage()) == 1) {
return 404;
} else if(strpos($e->getMessage(), "php_network_getaddresses: getaddrinfo failed") !== false) {
return 1000;
} else {
$this->message = $e->getMessage();
return -1;
}
}
Even if there's no obvious answer, I'd like to understand what factors could influence the real time to be so large. My understanding is that the constructor doesn't include time for the file_get_contents call itself or the catch clause, so can't really think of a good reason (other than trying to allocate a huge amount of memory) why this should be slow.

You are using an oldschool lambda function created with a call to create_function(). Is there a reason for this? Because the way that function operates is slow. You pass it a STRING with the code of the function, but that string has to be evaluated during runtime. There is no way the opcode cache could cache that code for you.

After quite a bit more testing, the problem appears to be with how APD profiling measures the elapsed time. If I put any other statement inside the error handler before throw, the profiler counts all the elapsed time against that, as mentioned in my comment.
Therefore, I'm convinced that the high elapsed time is because the profiler actually includes elapsed time from file_get_contents.
I tried using XHProf instead, and this seems to give more correct results. At least, it shows the ErrorException->__construct as taking only minimal elapsed time.

Related

Most optimized way to create associative array

What is the best (fastest, most readable, etc.) way to create associative array?
Last time I saw that my co-worker, create small assoc array by:
$array['is_enabled'] = false;
In code review i pass my opinion that more readable form is:
$array = [
'is_enabled' => false
];
But "readable" is subjective thing, so it's not the best argument.
The first approach 3v4l link:
$subscription['is_enabled'] = false;
You can see in link that's generated 3 operations
number of ops: 3
compiled vars: !0 = $subscription
line #* E I O op fetch ext return operands
-------------------------------------------------------------------------------------
3 0 E > ASSIGN_DIM !0, 'is_active'
1 OP_DATA
2 > RETURN 1
Avg. memory usage is 25.15 MiB and runs in 0.013s (user time PHP 7.1.x).
The second approach 3v4l link:
$subscription = ['is_active' => false];
Now it's only 2 operations. Assign and return.
number of ops: 2
compiled vars: !0 = $subscription
line #* E I O op fetch ext return operands
-------------------------------------------------------------------------------------
3 0 E > ASSIGN !0,
1 > RETURN 1
Avg. memory usage is 24.40 MiB and runs in 0.010s (user time PHP 7.1.x).
I'm not sure what operation is op_data that is missing in second approach. But still it's one operation less, and in result we gain almost 0,8MiB less memory usage and ~23% faster execution time.
So it's looks like $array= ['key' => false];
It's not only more readable but also quite simpler for parser.
With this notation we skip one operation from three, and that's give us additional free memory and time.

Rate limiting yourself from overloading external API's

I found a lot of information and script examples around that showed how to rate limit the users of an API but I wasn't able to find any examples of how to rate limit your own requests of an API when these limits are imposed.
I've always rate limited my scripts with code such as sleep or usleep commands but it feels like an inefficient way of doing things, especially when the API endpoints have pretty high rate limits and hammering API's until you hit the limits is also inefficient.
For example, Google's API limits vary based on the API you are using and can increase/decrease, in this case a fixed rate limit hard coded into the code just seems like primitive guess work!
Have I missed something pretty obvious? Or is this just not as common as I expect it to be?
Okay, for giggles I've thrown together a limiter class that will allow you to specify the limit per second, minute and hour. I can't resist having a good reason to use a circular queue!
If you have multiple processes doing the consumption, whether simultaneous or not, you'll have to devise a way to store and/or share the usage history on your own.
// LIMITER.PHP
class Limiter
{
private $queue = array();
private $size;
private $next;
private $perSecond;
private $perMinute;
private $perHour;
// Set any constructor parameter to non-zero to allow adherence to the
// limit represented. The largest value present will be the size of a
// circular queue used to track usage.
// -------------------------------------------------------------------
function __construct($perSecond=0,$perMinute=0,$perHour=0)
{
$this->size = max($perSecond,$perMinute,$perHour);
$this->next = 0;
$this->perSecond = $perSecond;
$this->perMinute = $perMinute;
$this->perHour = $perHour;
for($i=0; $i < $this->size; $i++)
$this->queue[$i] = 0;
}
// See if a use would violate any of the limits specified. We return true
// if a limit has been hit.
// ----------------------------------------------------------------------
public function limitHit($verbose=0)
{
$inSecond = 0;
$inMinute = 0;
$inHour = 0;
$doneSecond = 0;
$doneMinute = 0;
$doneHour = 0;
$now = microtime(true);
if ( $verbose )
echo "Checking if limitHit at $now<br>\n";
for ($offset=1; $offset <= $this->size; $offset++)
{
$spot = $this->next - $offset;
if ( $spot < 0 )
$spot = $this->size - $offset + $this->next;
if ( $verbose )
echo "... next $this->next size $this->size offset $offset spot $spot utime " . $this->queue[$spot] . "<br>\n";
// Count and track within second
// -----------------------------
if ( $this->perSecond && !$doneSecond && $this->queue[$spot] >= microtime(true) - 1.0 )
$inSecond++;
else
$doneSecond = 1;
// Count and track within minute
// -----------------------------
if ( $this->perMinute && !$doneMinute && $this->queue[$spot] >= microtime(true) - 60.0 )
$inMinute++;
else
$doneMinute = 1;
// Count and track within hour
// ---------------------------
if ( $this->perHour && !$doneHour && $this->queue[$spot] >= microtime(true) - 3600.0 )
$inHour++;
else
$doneHour = 1;
if ( $doneSecond && $doneMinute && $doneHour )
break;
}
if ( $verbose )
echo "... inSecond $inSecond inMinute $inMinute inHour $inHour<br>\n";
if ( $inSecond && $inSecond >= $this->perSecond )
{
if ( $verbose )
echo "... limit perSecond hit<br>\n";
return TRUE;
}
if ( $inMinute && $inMinute >= $this->perMinute )
{
if ( $verbose )
echo "... limit perMinute hit<br>\n";
return TRUE;
}
if ( $inHour && $inHour >= $this->perHour )
{
if ( $verbose )
echo "... limit perHour hit<br>\n";
return TRUE;
}
return FALSE;
}
// When an API is called the using program should voluntarily track usage
// via the use function.
// ----------------------------------------------------------------------
public function usage()
{
$this->queue[$this->next++] = microtime(true);
if ( $this->next >= $this->size )
$this->next = 0;
}
}
// ##############################
// ### Test the limiter class ###
// ##############################
$psec = 2;
$pmin = 4;
$phr = 0;
echo "Creating limiter with limits of $psec/sec and $pmin/min and $phr/hr<br><br>\n";
$monitorA = new Limiter($psec,$pmin,$phr);
for ($i=0; $i<15; $i++)
{
if ( !$monitorA->limitHit(1) )
{
echo "<br>\n";
echo "API call A here (utime " . microtime(true) . ")<br>\n";
echo "Voluntarily registering usage<br>\n";
$monitorA->usage();
usleep(250000);
}
else
{
echo "<br>\n";
usleep(500000);
}
}
In order to demonstrate it in action I've put in some "verbose mode" statements in the limit checking function. Here is some sample output.
Creating limiter with limits of 2/sec and 4/min and 0/hr
Checking if limitHit at 1436267440.9957
... next 0 size 4 offset 1 spot 3 utime 0
... inSecond 0 inMinute 0 inHour 0
API call A here (utime 1436267440.9957)
Voluntarily registering usage
Checking if limitHit at 1436267441.2497
... next 1 size 4 offset 1 spot 0 utime 1436267440.9957
... next 1 size 4 offset 2 spot 3 utime 0
... inSecond 1 inMinute 1 inHour 0
API call A here (utime 1436267441.2497)
Voluntarily registering usage
Checking if limitHit at 1436267441.5007
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957
... next 2 size 4 offset 3 spot 3 utime 0
... inSecond 2 inMinute 2 inHour 0
... limit perSecond hit
Checking if limitHit at 1436267442.0007
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957
... next 2 size 4 offset 3 spot 3 utime 0
... inSecond 1 inMinute 2 inHour 0
API call A here (utime 1436267442.0007)
Voluntarily registering usage
Checking if limitHit at 1436267442.2507
... next 3 size 4 offset 1 spot 2 utime 1436267442.0007
... next 3 size 4 offset 2 spot 1 utime 1436267441.2497
... next 3 size 4 offset 3 spot 0 utime 1436267440.9957
... next 3 size 4 offset 4 spot 3 utime 0
... inSecond 1 inMinute 3 inHour 0
API call A here (utime 1436267442.2507)
Voluntarily registering usage
Checking if limitHit at 1436267442.5007
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 2 inMinute 4 inHour 0
... limit perSecond hit
Checking if limitHit at 1436267443.0007
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 2 inMinute 4 inHour 0
... limit perSecond hit
Checking if limitHit at 1436267443.5027
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 0 inMinute 4 inHour 0
... limit perMinute hit
Checking if limitHit at 1436267444.0027
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 0 inMinute 4 inHour 0
... limit perMinute hit
Well, first things first - you should call any external API's only when you actually need to - the providers will thank you dearly.
There is two ways I usually "impose" a limit on my own API usage - if possible, cache the result for N amount of time, usually a lot less than hard limit of the API itself. This, however, works only in very specific cases.
The second is persistent/semi-persistent counters, where you store a counter in some sort of memory backend along with time when the limiting period begins. Every time before calling API check the storage and see whether the current time minus interval begin and number of requests you have already made is less than imposed by API. If it is, you can make a request - if the interval is larger, you can reset the limit, and if your next request will exceed the limit, and you are still in the previous interval, you can show a pretty error. On each external request then update the interval time if it's exceeded and increment the counter.
Wrap your API calls with Jobs and push them to separate queue:
ApiJob::dispatch()->onQueue('api');
Use queue rate limiting with Redis or with mxl/laravel-queue-rate-limit package (I'm the author). See also SO answer about its usage.
If using mxl/laravel-queue-rate-limit then after its setup run queue worker:
$ php artisan queue:work --queue api
I think we can not answer your question in a few sentences . It takes a true reflection of architecture linked to your application . For me to make an API rate limit for repeat I use caches that store values ​​and utilization of my API. I have to date found no code ready .

Best Approach to Getting Adjustment from Raw Number

I would like advice on the best/most efficient way to get an adjusted score for a team based on a raw score. Let me explain.
The teams are contract bridge teams and the raw score for the winner is from 0 (tie) to any number, but greater than 100 would be rare. The raw score is called IMPS and the adjusted score is called VPs, but that is just for clarity.
The adjustment table is:
IMPs VPs
1 thru 2 16 to 14
3 thru 4 17 to 13
5 thru 7 18 to 12
8 thru 11 19 to 11
12 thru 15 20 to 10
16 thru 20 21 to 9
21 thru 26 22 to 8
27 thru 33 23 to 7
34 thru 41 24 to 6
42 thru 50 25 to 5
51 thru 60 26 to 4
61 thru 71 27 to 3
72 thru 83 8 to 2
84 thru 95 29 to 1
96+ 30 to 0
Here is my code. It works fine:
PHP
$teamArawScore = 20; //Actual result will come from form input
if ($teamArawScore >95 )
{
$teamAadjScore = 30;
$teamBadjScore = 0;
}
else
{
$adjustmentArray = array
(15,
16,16,
17,17,
18,18,18,
19,19,19,19,
20,20,20,20,
21,21,21,21,21,
22,22,22,22,22,22,
23,23,23,23,23,23,23,
24,24,24,24,24,24,24,24,
25,25,25,25,25,25,25,25,25,
26,26,26,26,26,26,26,26,26,26,
27,27,27,27,27,27,27,27,27,27,27,
28,28,29,29,28,28,28,28,28,28,28,28,
29,29,29,29,29,29,29,29,29,29,29,29);
$teamAadjScore = $adjustmentArray[$teamArawScore];
$teamBadjScore = 30 - $teamAadjScore;
}
echo "TeamA won by $teamArawScore so it won $teamAadjScore VPs and TeamB won $teamBadjScore VPs.";
My approach just seems inefficient. Since the array is small, I doubt there is any performance issues, but I would like to do the conversion as efficiently as possible.
What do you suggest?
Since there's no consistency in the table you described you will always be populating a lookup array containing the actual logic. You could also structure this differently, with a shorter array containing the sequential amount of points leading to a given score etc., but in the end those would all result in having to loop over the array to see where you end up - swapping memory for CPU cycles.
Effectively, you need a lookup table anyway because there appears to be no algorithm that can reliably map the contents of the lookup table, and your implementation is now O(1) in big-O notation. As such it can by definition not be made more efficient.
For reference on lookup tables (emphasis added in quote):
In computer science, a lookup table is an array that replaces runtime
computation with a simpler array indexing operation. The savings in
terms of processing time can be significant, since retrieving a value
from memory is often faster than undergoing an 'expensive' computation
or input/output operation. The tables may be precalculated and
stored in static program storage, calculated (or "pre-fetched") as
part of a program's initialization phase (memoization), or even stored
in hardware in application-specific platforms.
There's nothing 'bad practice' about using them. Back in the days when CPUs didn't have floating point units on board we'd have entire sine and sqrt tables embedded in programs to circumvent computationally expensive calculations at runtime.
I got a little something for you.Even though I'm not sure it would actualy optimize the execution speed.
Looping through an array, knowing the base value being 16 if we have a positive score (higher than one) from the A team.
This is not optimized at all (mainly because of the $a == 0 condition), but here it is:
<?php
$teamArawScore = 30;
$a = $teamArawScore;
$teamAfinalScore = 16;
$scoreArray = array(3,5,8,12,16,21,27,34,42,51,61,72,84,10000000000);
$count = 0;
foreach($scoreArray as $elem)
{
if($a < $elem)
{
$teamAfinalScore += $count;
break;
}
$count++;
}
if($a ==0)
{
$teamAfinalScore = 15;
$teamBfinalScore = 30 - $teamAfinalScore;
}
echo "Team A: ".$teamAfinalScore. "<br />Team B:".$teamBfinalScore;
?>
<?php
$teamArawScore = 1000; //Actual result will come from form input
if ($teamArawScore >95 )
{
$teamAadjScore = 30;
$teamBadjScore = 0;
}
else
{
$adjustmentArray = array('1'=>16,'3'=>17,'5'=>18,'8'=>19,'12'=>20,'16'=>21,'21'=>22,'27'=>23,'34'=>24,'42'=>50,'51'=>60,'61'=>27,'72'=>83,'84'=>95);
$base_score=array(1,3,5,8,12,16,21,27,34,42,51,61,72,84);
$count=count($base_score);
$adjustment_value=$adjustmentArray['1'];
for($i=1; $i<$count-1; $i++){
if($teamArawScore < $base_score[$i+1]){
$adjustment_value=$adjustmentArray[$base_score[$i]];
break;
}
else{
$adjustment_value=$adjustmentArray[$base_score[$i]]; // for values greater than 84
}
}
$teamAadjScore = $adjustment_value;
$teamBadjScore = 30 - $teamAadjScore;
}
echo "TeamA won by $teamArawScore so it won $teamAadjScore VPs and TeamB won $teamBadjScore VPs." ;

Working with percentages in PHP [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 11 years ago.
I'm trying to create a simple split tester in PHP but I'm having some trouble getting the percentages all figured out.
So, I have a settings page where I want to put a percentage, something like 20, so that 20% of the traffic to my page will be redirected.
This will obviously need to work with rand, but I'm not exactly sure how.
This if statement returns 20% true:
if (rand(1,5) % 5 == 0)
if (rand(1, 100) <= 20){
// show page 1
} else {
// show page 2
}
Some testing and comparison:
Results for round 0
Method1 redirect rate: 0.200302%
Method2 redirect rate: 0.200318%
Results for round 1
Method1 redirect rate: 0.199277%
Method2 redirect rate: 0.199479%
Results for round 2
Method1 redirect rate: 0.200262%
Method2 redirect rate: 0.19995%
Results for round 3
Method1 redirect rate: 0.200254%
Method2 redirect rate: 0.200315%
Results for round 4
Method1 redirect rate: 0.199943%
Method2 redirect rate: 0.19977%
Results for round 5
Method1 redirect rate: 0.20006%
Method2 redirect rate: 0.20024%
Summary:
Method1 deviation: -9.7999999999931E-5
Method2 deviation: -7.1999999999905E-5
and code for that testing:
<?php
function method1(){
return (rand(1,5) % 5 == 0)?true:false;
}
function method2(){
return (rand(1,100) <= 20)?true:false;
}
$iterations = 1000000;
for ($j = 0; $j <= 5; $j++){
$m1 = 0;
$m2 = 0;
for ($i = 0; $i < $iterations; $i++){
if (method1()){
$m1++;
}
if (method2()){
$m2++;
}
}
$dx1 = $m1/$iterations;
$dx2 = $m2/$iterations;
$dev1 += 0.2 - $dx1;
$dev2 += 0.2 - $dx2;
echo "Results for round $j\n";
echo "Method1 redirect rate: " . $dx1 . "%\n";
echo "Method2 redirect rate: " . $dx2 . "%\n";
}
echo "Summary:\n";
echo "Method1 deviation: $dev1\n";
echo "Method2 deviation: $dev2\n";
Also < would take less cpu power than % :) First in timing is the % and then <
$ /usr/bin/time -l php -q test.php
5.25 real 5.19 user 0.01 sys
8224768 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
2090 page reclaims
0 page faults
0 swaps
17 block input operations
4 block output operations
0 messages sent
0 messages received
0 signals received
17 voluntary context switches
592 involuntary context switches
$ /usr/bin/time -l php -q test.php
4.75 real 4.73 user 0.01 sys
8216576 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
2088 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
0 voluntary context switches
100 involuntary context switches

Best way to time a PHP script

What is the best way to see how long it takes for your PHP script to run?
I was thinking something like this:
$start_time = time(); //this at the beginning
$end_time = time(); //this at the end
echo = $end_time-$start_time;
But how can I make it into something that is readable to me and make sense to me?
If you want any further granularity to your timing than seconds, you'll need to use microtime() (Return current Unix timestamp with microseconds)
<?php
$time_start = microtime(true);
// Sleep for a while
usleep(100);
$time_end = microtime(true);
$time = $time_end - $time_start;
echo "Did nothing in $time seconds\n";
?>
** Below was added later **
As far as formatting this result further:
Well, depending on what you're doing, you typically don't have scripts going past a minute. You definitely shouldn't have anything exceeding an hour. (If you do, you need to ask yourself what you are doing with your life)
With that in mind, all you need is simple calculations:
$tmp = floor($time);
$minutes = $tmp / 60;
$seconds = ($tmp % 60) + ($time - $tmp);
$output = 'Script took ';
if ($minutes > 0) $output .= $minutes . ' minutes and ';
$output .= $seconds . ' seconds to complete.';
echo $output;
(This isn't tested, and it could potentially be optimized, but should start you in the right direction)
I would use microtime(TRUE) instead. That'll give you better results with microsecond resolution. There's also the PECL APD extension which profiles scripts.
Expanding a bit on APD, assuming one has APD installed, you just need to add the line
apd_set_pprof_trace();
to the top (or whenever you want to start tracing) of your script. It generates a profiling file with pprofp which generates very readable output
Real User System secs/ cumm
%Time (excl/cumm) (excl/cumm) (excl/cumm) Calls call s/call Memory Usage Name
--------------------------------------------------------------------------------------
100.0 0.00 0.00 0.00 0.00 0.00 0.00 1 0.0000 0.0009 0 main
56.9 0.00 0.00 0.00 0.00 0.00 0.00 1 0.0005 0.0005 0 apd_set_pprof_trace
28.0 0.00 0.00 0.00 0.00 0.00 0.00 10 0.0000 0.0000 0 preg_replace
The example output is from the PHP manual.
I personally find APD extremely useful and use it quite frequently in heavily-hit scripts.
Knowing how long something takes to run is one thing, finding out where (in your php operations) it slows down and why is another question.
If you're serious about find an answer to the latter question then install xdebug, and you get the bonus of not having to call the likes of microtime() which itself slows down your script.
http://xdebug.org/docs/profiler
http://xdebug.org/docs/
Is this a web-accessible script (i.e. http://www.yoursite.com/BLA.php), or a script that you're running through the command line? If it's a command line script, you can use the time command:
time php FILE.php
Otherwise, microtime is probably your best bet.
I find this class to be useful to time the scripts, microtime turn out to be friend in that:
class timer
{
private $start_time = NULL;
private $end_time = NULL;
private function getmicrotime()
{
list($usec, $sec) = explode(" ", microtime());
return ((float)$usec + (float)$sec);
}
function start()
{
$this->start_time = $this->getmicrotime();
}
function stop()
{
$this->end_time = $this->getmicrotime();
}
function result()
{
if (is_null($this->start_time))
{
exit('Timer: start method not called !');
return false;
}
else if (is_null($this->end_time))
{
exit('Timer: stop method not called !');
return false;
}
return round(($this->end_time - $this->start_time), 4);
}
# an alias of result function
function time()
{
$this->result();
}
}

Categories