I'm making a blocking algorithm, and I just realised that adding a timeout to such algorithm is not so easy if it should remain precise.
Adding timeout means, that the blocking algorithm should abort after X ms if not earlier. Now I seem to have two options:
Iterating time (has mistake, but is fast)
Check blocking condition
Iterate time_elapsed by 1 (which means 1e-6 sec with use of usleep)
Compare time_elapsed with timeout. (here is the problem I will talk about)
usleep(1)
Getting system time every iteration (slow, but precise)
I know how to do this, please do not post any answers about that.
Compating timeout with time_elapsed
And here is what bothers me. The timeout will be in milliseconds (10e-3) while usleep sleeps for 10e-6 seconds. So my time_elapsed will be 1000 times more precise than timeout. I want to truncate last three digits of time_elapsed (operation equal to floor($time_elapsed/1000) without dividing it. Division algorithm is too slow.
Summary
I want to make my variable 1000 times smaller without dividing it by 1000. I want just get rid of the data. In binary I'd use bit-shift operator, but have no idea how to apply it on decimal system.
Code sample:
Sometimes, when people on SO cannot answer the theoretical question, they really hunger for the code. Here it is:
floor($time_elapsed/1000);
I want to replace this code with something much faster. Please note that though the question itself is full of timeouts, the question title is only about truncating that data. Other users may find the solution useful for other purposes than timing.
Maybe this will help Php number format. Though this does cause rounding, if that is unacceptable then I don't think its possible because PHP is loosely typed that you cant define numbers with a particular level of precision.
try this:
(int)($time_elapsed*0.001)
this should be a lot faster
Related
I already read the documentation, but when testing, I'm still not able to understand well the difference between them.
For example, with this simple file:
<?php
class StackOverflowBench
{
public function benchNothing()
{
}
}
When I set 1000 revolutions, and only one iteration, here is my result:
subject
set
revs
its
mem_peak
best
mean
mode
worst
stdev
rstdev
diff
benchNothing
0
10000
1
2,032,328b
10.052μs
10.052μs
10.052μs
10.052μs
0.000μs
0.00%
1.00x
the best, mean, mode and worst are always the same, which means they are based on the only iteration I made.
When I run it with 10 revolutions and still 1 iteration, I have this:
subject
set
revs
its
mem_peak
best
mean
mode
worst
stdev
rstdev
diff
benchNothing
0
10
1
2,032,328b
10.200μs
10.200μs
10.200μs
10.200μs
0.000μs
0.00%
1.00x
which seems to mean the times calculated are not a sum of all the revolutions, but something like an average for each iteration.
If I wanted to measure the best and worst execution time of each time the method is executed, I'd try 1000 iterations and only 1 revolution each, but it takes waay to much time. I launched it with 100 iterations of 1 revolution, here's the result :
subject
set
revs
its
mem_peak
best
mean
mode
worst
stdev
rstdev
diff
benchNothing
0
1
100
2,032,328b
20.000μs
25.920μs
25.196μs
79.000μs
5.567μs
21.48%
1.00x
This time, the time seems to be at least twice as long, and I'm wondering what I didn't understand well. I may be using these informations badly (I know my last example is a wrong one).
Is it necessary to measure the best and worst of each revolution, like I want to do ?
What are the interests of iterations ?
Revolution vs iteration
Let's take your example class:
class StackOverflowBench
{
public function benchNothing()
{
}
}
If you have 100 revolutions and 3 iterations, this is the pseudo code that will be run:
// Iterations
for($i = 0; $i < 3; $i++){
// Reset memory stats code here...
// Start timer for iteration...
// Create instance
$obj = new StackOverflowBench();
// Revolutions
for($j = 0; $j < 100; $j++){
$obj->benchNothing();
}
// Stop timer...
// Call `memory_get_usage` to get memory stats
}
What does the report mean?
Almost all of the calculated stats (mem_peak, best, mean, mode, worst, stdev and rstdev) in the output are based on individual iterations and are documented here.
The diff stat is the weird one and document here and mentioned elsewhere as:
the percentage difference from the lowest measurement
When you run a test, you can specify what column to report the difference on. So if you diff_column on run time, if iteration #1 takes 10 seconds and #2 takes 20 seconds, the diff for #1 would be 1.00 (since it is the lowest) and #2 would be 2.00 since it took twice as long. (Actually, I'm not 100% sure that is the exact usage of that column
Measuring revolution vs iteration
Some code needs to be run thousands or millions of times in a task/request/action/etc. which is why revolutions exist. If I run a simple but critical block of code just once, a report might tell me it takes 0.000 seconds which isn't helpful. That's why some blocks of code need to have their revolution count kicked up to get a rough idea, based on possible real-world usage, how they perform under load. Array sorting algorithms are great examples of a tightly-coupled call that will happen a lot in a single request.
Other code might only do a single thing, such as making an API or database request, and for those blocks of code we need to know how much system resources will they take up as a whole. So if I make a database call and consume 2MB of, and I'm expecting to have 1,000 concurrent users, those calls could take up 2GB of memory. (I'm simplifying but you should get the gist.)
If you look at my pseudo code above, you'll see that setting up each iteration is more expensive than each revolution. The revolution basically just invokes a method, but the iteration calculates memory and does instantiation-related work.
So, to your second-to-last question:
Is it necessary to measure the best and worst of each revolution, like I want to do?
Probably not, although there are tools out there that will tell you. You could for instance, find out how much memory was used before a method and after to determine if your code is sub-optimal, but you can also do that with PHPBench by making a 1 iteration, 1 revolution run and looking for methods with high memory.
I'd further say that if you have code that has great variance per revolution, it is almost 100% related to IO factors and not code, or it is related to the test dataset, and most probably size.
You should hopefully know all of your IO-related paths, so benchmarking the various problems related to those paths really isn't a factor of this tool.
For dataset-related problems, however, that is interesting and is a case where you'd want to know each run potentially. There, too, however, the measurements are there to know either how to fix/change your code, or to know that your code runs with a certain time complexity.
I have an issue with a very slow API call and want to find out, what it caused by, using Xhprof: the default GUI and the callgraph. How should this data be analyzed?
What is the approach to find the places in the code, that should be optimized, and especially the most expensive bottlenecks?
Of all those columns, focus on the one called "IWall%", column 5.
Notice that send, doRequest, read, and fgets each have 72% inclusive wall-clock time.
What that means is if you took 100 stack samples, each of those routines would find itself on 72 of them, give or take, and I suspect they would appear together.
(Your graph should show that too.)
So since the whole thing takes 23 seconds, that means about 17 seconds are spent simply reading.
The only way you can reduce that 17 seconds is if you can find that some of the reading is unnecessary. Can you?
What about the remaining 28% (6 seconds)?
First, is it worth it?
Even if you could reduce that to zero (17 seconds total, which you can't), the speedup factor would 1/(1-0.28) = 1.39, or 39%.
If you could reduce it by half (20 seconds total), it would be 1/(1-0.14) = 1.16, or 16%.
20 seconds versus 23, it's up to you to decide if it's worth the trouble.
If you decide it is, I recommend the random pausing method, because it doesn't flood you with noise.
It gets right to the heart of the matter, not only telling you which routines, but which lines of code, and why they are being executed.
(The why is most important, because you can't replace it if it's absolutely necessary.
With profilers, you tend to assume it is necessary, because you have no way to tell otherwise.)
Since you are looking for something taking about 14% of the time, you're going to have to examine 2/0.14 = 14 samples, on average, to see it twice, and that will tell you what it is.
Keep in mind that about 14 * 0.72 = 10 of those samples will land in fgets (and all its callers), so you can either ignore those or use them to make sure all that I/O is really necessary.
(For example, is it just possible that you're reading things twice, for some obscure reason like it was easier to do that way? I've seen that.)
I've been playing around with the ApacheBench (ab) command line utility but I'm not sure if it's the right tool for what I'm trying to do.
I was running 10 concurrent and 10 requests doing an INSERT into a MySQL table and looking at round(microtime(1) * 10000) to see if I was getting the same values.
I would like to know if using delayed execution usleep() is enough to prevent two simultaneous requests getting the same value on the same box, or is time_sleep_until better.
I know I could set more entropy with uniqid().
So I'm just wondering if using delayed execution will be enough to prevent two exact time stamps.
So you want to know if round(microtime(1) * 10000) can give duplicate values? Well of course it can, I don't think you need to test for this.
uniqid is also not guaranteed to be unique, even with more entropy.
If uniqueness is really important you could always use MySQL with a UNIQUE index to enforce uniqueness.
I was just reviewing the answers to different questions to learn more. I saw an answer which says that it is bad practice in php to write
for($i=0;$i<count($array);$i++)
It says that calling the count function in the loop reduces the speed of the code. The discussion in the comments on this question was not clear. I want to know why it is not good practice. What should be the alternative way of doing this?
You should do this instead:
$count = count($array);
for($i=0;$i<$count;$i++)...
The reason for doing this is because if you put the count($array) inside the for loop then the count function would have to be called for every iteration which slows down speed.
However, if you put the count into a variable, it is a static number that won't have to be recalculated every time.
For every iteration, PHP is checking that part of the loop (the condition) to see if it should keep looping, and every time it checks it, it is calculating the length of the array.
An easy way to cache that value is...
for($i=0,$count=count($array);$i<$count;$i++) { ... }
It is probably not necessary in small loops, but could make a huge difference when iterating over thousands of items, and dependent on what function call is in the condition (and how it determines its return value).
This is also why you should use foreach() { ... } if you can, it uses an iterator on a copy of the array to loop over the set and you don't have to worry about caching the condition of the loop.
I heard of a database in a doctor's surgery that made exactly this mistake with a piece of software. It was tested with about 100 records, all worked fine. Within a few months, it was dealing with millions of records and was totally unusable, taking minutes to load results. The code was replaced as per the answers above, and it worked perfectly.
To think about it another way, a fairly powerful dedicated server that's not doing much else will take about 1 nanosecond to do count($array). If you had 100 for loops, each counting 1,000 rows then that's only 0.0001 of a second.
However, that's 100,000 calculations for EVERY page load. Scale that up to 1,000,000 users (and who doesn't want to have 1 million users?)... doing a 10 page loads and now you have 1,000,000,000,000 (1 trillion) calculations. That's going to put a lot of load on the server. It's a 1,000 seconds (about 16.5 minutes) that your processor spends running that code.
Now increase the time it takes the machine to process the code, the number of items in the arrays, and the number of for loops in the code... you're talking of literally many trillions of processes and many hours of processing time that can be avoided by just storing the result in a variable first.
It is not good practice because as written, count($array) will be called each time through the loop. Assuming you won't be changing the size of the array within the loop (which itself would be a horrible idea), this function will always return the same value, and calling it repeatedly is redundant.
For short loops, the difference probably won't be noticeable, but it's still best to call the function once and use the computed value in the loop.
I am doing some large timestamp-list iterations: Putting them in tables with date-ranges, and grouping them by ranges.
In order to do that, I found strtotime() a very helpfull function, but I am worried about its performance.
For example, a function that loops over a list of weeks (say, week 49 to 05) and has to decide the beginning of the week and the timestamp at the end of that week. A usefull way to do that, would be:
foreach ($this->weeks($first, $amount) as $starts_at) {
$ends_at = strtotime('+1 week', $starts_at);
$groups[$week_key] = $this->slice($timestamps, $starts_at, $ends_at);
}
//$this->weeks returns a list of timestamps at which each week starts.
//$this->slice is a simple helper that returns only the timestamps within a range, from a list of timestamps.
Instead of strtotime(), I could, potentially, find out the amount of seconds between begin and end of the week, 99% of the times that would be 24 * 60 * 60 * 7. But in these rare cases where there is a DST-switch, that 24 should either be 23 or 25. Code to sort that out, will probably be a lot slower then strtotime(), not?
I use the same patterns for ranges of years, months (months, being very inconsistent!), days and hours. Only with hours would I suspect simply adding 3600 to the timestamp is faster.
Any other gotcha's? Are there ways (that do not depend on PHP5.3!) that offer better routes for consistent, DST and leap-year safe dateranges?
Why are you worried about its performance? Do you have evidence that it's slowing down your system? If not, don't try to over-complicate the solution for unnecessary reasons. Remember that premature optimization is the root of all evil. Write readable code that makes sense, and only optimize if you KNOW it's going to be an issue...
But something else to consider is that it's also compiled C code, so it should be quite efficient for what it does. You MIGHT be able to build a sub-set of the code in PHP land and make it faster, but it's going to be a difficult job (due to all the overhead involved in PHP code).
Like I said before, use it until you prove it's a problem, then fix that problem. Don't forget re-writing it for you needs isn't free either. It takes time and introduces bugs. Is it worth it if the gain is minimal (meaning it wasn't a performance problem to begin with). So don't bother trying to micro-optimize unless you KNOW it's a problem...
I know this probably isn't the answer you're looking for, but your best bet is profiling it with a real use case in mind.
My instinct is that, as you think, strtotime will be slower. But even if it's, say, 3 times slower, this is only meaningful in context. Maybe your routine, with real data, takes 60 ms using strtotime, so in most cases, you'd be just saving 40 ms (I totally made up these numbers, but you get the idea). So, you might find out that optimising this wouldn't really pay off (considering you're opening your code to more potential bugs and you'll have to invest more time to get it right).
By the way, if you have good profiling tools, awesome, but even if you don't comparing timestamps should give you a rough idea.
To respond to the question, finaly :
Based on many benchmarks like this one: https://en.code-bude.net/2013/12/19/benchmark-strtotime-vs-datetime-vs-gettimestamp-in-php/
We can see that strtotime() is more effective that we can think.
So yes, to convert a string to a timestamp, the strtotime function has pretty good performance.
Very interesting question. I'd say that the only way you can really figure this out, is to set up your own performance test. Observe the value of microtime() at the beginning and end of the script, to determine performance. Run a ridiculous number of values through a loop with one method, then the other method. Compare times.