I was wondering what happens if multiple scripts are sharing same file. I uploaded the test on remote server, where they use HDD to store data. There were 7 tests total, but the family of 6 are compatible.
I have 7 files of different size which I uploaded to server and the test. It is loop which reads and writes data from the files.
There is 50 microseconds delay in the loop. The loop repeats 50x.
I measure the time needed to perform every circle.
The differences in the tests (T):
Using file_get_contents/file_put_contents
T2 - SOURCE <> TARGET - reads data from original file, writes data do different (new) file
T3 - SOURCE = TARGET - 1. copies data from original file to target; 2. reads source data -> writes data; 3. the point 3 is repeated: i.e I read the data which I have written. This test uses same file to write data.
T4 - SOURCE = TARGET - I repeated the same test as in T3 getting shorted times.
Using fopen, flock, fread, flock, fclose, fopen, flock, fopen, fwrite, fflush, fclock, fclose ... This is complicated code, but here I have tested the fflush. I also use clearstatcache, stat and touch and clearstatcache, filesize. To check validity. The tests T5 - T7 were less reliable than T2-T4 because sometimes the write operation failed. I tested the file size and when it was not correct, I copied (restored) the file back from original file.
T5: (fflush) SOURCE = TARGET
T6: (fflush) SOURCE <> TARGET
T7: (fflush) SOURCE <> TARGET + I have removed the 50 microseconds delay from the loop (It seems like the validity/reliability is worse when there is a delay).
I made 4 requests from 4 different browsers - so every test have 4 sets of data (7*50*4 values total).
Now I have collected all data, created tables and diagrams. This is one diagram of many, showing minimal and maximal values of avarage value.
T4 yellow color and T3 green provides very small times so they are suspicious. For example T4 avarage times are these: 0,001
0.001 0.002 0.003 0.002 0.004 0.003 0.004 0.001 0.004 0.001 0.004 0.001 0.004
And T3 times:
0.002 0.003 0.001 0.001 0.003 0.003 0.006 0.007 0.002 0.003 0.004 0.004 0.019 0.019
The values of T2 seems normal, but this can be explained by the fact, that that was read from different file than was written to.
T5-T7 just show normal times as expected - the bigger the file the bigger the time needed to process. Fairly slow as expected from HDD and 4 scripts running at the same time.
So my question here is:
Does the results of T3-T4 mean, that the file_read_contents and file_put_contents are not reliable for this type of job? To me it looks like they simply do not read the data from file but they are copied from buffer, which means, that old data are saved, not the current data been changed by concurent script. I would welcome more information. I spent a lot of time searching for answers but did not found clear answer. I did this tests because I need proofs. You man want to use my scripts but I am not sure if can I paste here the 6 scripts? Now I will add just the fflush test number 7 which is most useful.
<?PHP
clearstatcache();
$_DEBUG_ = false;
echo "Lock and flush tester.".time()."<br>";
die;
while ( time()<1570787996 )
{
usleep(500);
}
function test($n, $p, $_DEBUG_){
$sname = "$n"; // source
$tname = "$n.txt";// target
echo "<h4>$n at ".time()."</h4>";
for ($i = 0; $i<50; $i++ ){
$start = microtime(true);
clearstatcache(); // needed for filesize and touch
$st = stat("$sname");
$original_size = $st['size'];
if ( $_DEBUG_ )
echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
$fsize = filesize($sname);
if ( $original_size <> $fsize )
die("; fsize total FAILTURE; ");
if ($fsize === 0)
echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";
else
{
// READ OPERATION AND LOCK FOR SHARE
$locked = false;
for ($c = 0; !$locked; $c++):
if ( $c > 400)
break;
$fp = fopen($sname, "r");
$locked = flock($fp, LOCK_SH);
if ($locked)
break;
else
{
echo "failed to get LOCK_SH;<br>";
usleep(5000);
}
endfor;
$s = fread($fp, $fsize );
$success = flock($fp, LOCK_UN);
if ( $success === false )
die("; r flock release failed; ");
$success = fclose($fp);
if ( $success === false )
die("; fclose failed; ");
// 10 - data loaded , $p - browser
if ( $success )
{
$result = touch("$sname",strlen($s),$p);
if ( $_DEBUG_ )
echo "; TOUCH: $result;";
}
else
die("fclose FAIL.");
if ( strlen($s)<60 )
echo "*$s LENGTH:".strlen($s)."<br>";
}
clearstatcache();
$st = stat("$tname");
if ( $_DEBUG_ )
echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";
// WRITE OPERATION WITH LOC_EX
$fp = fopen($tname, "w");
$locked = false;
$locked = flock($fp, LOCK_EX);
if ( $locked ) { // acquire an exclusive lock
$success = fwrite($fp, $s);
if ( $success === false)
echo "; w FAILED;";
else
if ( $_DEBUG_ )
echo " $success B written; ";
$success = fflush($fp);// flush output before releasing the lock
if ( $success === false )
echo "; flush FAILED; ";
$success = flock($fp, LOCK_UN); // release the lock
if ( $success === false )
echo "; release FAILED; ";
$success = fclose($fp);
if ( $success === false )
echo "; fclose FAILED; ";
clearstatcache(); // needed for filesize and touch
$fsize = filesize($tname);
if ($original_size>$fsize)
{
echo "; <b>WRITE FAILED, restoring</b>;";
$original_fname = "$n";
$result = copy($original_fname, $tname);
if ($result == false )
die(" <b>TOTAL FAILTURE: copy failed.</b>");
else
echo " <b>RESTORED</b>;";
}
else
{
if ($fsize === 0)
echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";
if ( $success )
touch("$tname",$fsize,$p);
}
} else {
echo "Couldn't get the lock!";
}
$time_elapsed_secs = microtime(true) - $start;
if ( $time_elapsed_secs === 0 )
echo " FAILED ";
echo "time: $time_elapsed_secs s<br>";
}
}
switch ( $_SERVER['HTTP_USER_AGENT'] ):
// FF 1:
case "Mozilla/5.0 (Windows NT 5.1; rv:49.0) Gecko/20100101 Firefox/49.0":
$p = 1; break;
// Chrome:
case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36":
$p = 2; break;
// OPERA:
case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36 OPR/36.0.2130.80":
$p = 3; break;
endswitch;
copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");
test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);
die;
echo "php: " . phpversion();
?>
<?PHP echo "php: " . phpinfo();
?>
You may want to enable $DEBUG option to monitor each process. Note: The touch maybe do not work correctly always.
Note: This is not a request for test, this is just request for review.
Also: Please do not be confused by the yellow color curve. There are two yellow colors. The T4 yellow is almost no visible on the diagram because it has very low values.
I don't know what you're trying to do, but I'm afraid you've gone the wrong way. If you are concerned about a collision, you should use a database that takes care of such problems and offers you luxury access methods. PHP comes with 5 different databases that you can choose from.
Notice, there is not a collision between these two functions, both are atomic and reliable. The problem is if you read, modify, and save the file. These three actions are not in one transaction and therefore you may lose data when you overlap. If you need such a use case, use the database.
Buffering is a basic file system feature that every programmer should know. This applies to all programming languages, not just PHP.
Realize that you are actually trying to create a database engine, that is, inventing the wheel. Many databases look like a plain text file, but the engine above them is ready and tested. Why don't you use any of the five?
I would like to add one more test. This one was made using a "directory lock". Instead of using flock, this creates directory. If the directory does not exists it attempts to create one and continues to read and write data. Notice: this is not perfect solution. The loop has 50 cicles. No delay. But the function atomicFuse has delay. I post this not as real solution, but just as a test and the result of the test for comparation.
/*
n is file size in kB
c is counter for optimalization
first call must have c = 0;
*/
function atomicFuse($n, $c, $disableDelay = false){
$start = false;
if ( !file_exists("$n.t") )
$start = mkdir("$n.t");
if ( !$disableDelay ){
if ( $start == false )
{
$n = $n*30;
switch($c): // Delay example increase:
case 0: break; // 0,01569 total
case 1: break; // 0,03138 total
case 2: $n = $n*2; break; // 0,06276 total
case 3: $n = $n*4; break; // 0,12552 total
// case 4: You need at least *6 or *8 to get out of problems with extrem times
case 4: $n = $n*8; break; // 0,25104 t.(upper limit)
// In case of heavy traffic:
case 5: $n = $n*8; break; // 0,36087 total extrem
case 6: $n = $n*10; break; // 0,51777 total extrem
case 7: $n = $n*20; break; // 1,03554 total extrem
default: $n = $n*8; break;
endswitch;
usleep($n);
echo ($n)."<br>";
}
}
return $start;
}
Implementation of the atomicFuse:
for ($i = 0; $i<50; $i++ ){
$start_time = microtime(true);
{
$start = atomicFuse($n,0);
if (!$start) $start = atomicFuse($n,1);
if (!$start) $start = atomicFuse($n,2);
if (!$start) $start = atomicFuse($n,3);
if (!$start) $start = atomicFuse($n,4);
if (!$start) $start = atomicFuse($n,5);
if (!$start) $start = atomicFuse($n,6);
if (!$start) $start = atomicFuse($n,7);
if (!$start) $start = atomicFuse($n, false);
if (!$start) echo "<b>Atomicity failed.</b> ";
if ( $start )
{
// do some action
$success = rmdir("$n.t"); // remove atomic fuse
}
}
}
The T8 results min, max of average:
0.006 0.083 0.018 0.156 0.072 0.182 0.100 0.255 0.168 0.276 0.224 0.383 0.224 0.406
Important notice: This test is very specific. It has some atomic failtures, so in the begin of some section are big delays.
So every request made by specific browser on my PC lead to these errors:
request from Chrome: 6 failed (4x 523kB and 2x 948kB)
request from FF1: 5 failed (first 5 files 523kB)
request from Opery: 0 failed (100% OK)
request from FF2: 0 failed (100% OK)
I will add yet one more diagram, without the values where test failed. That will be completely different.
Another diagram with T8b, I have removed the very high numbers from the begin of function start. This changes average just very slightly.
I have this code below.
<?php
require_once 'con.php';
$start_time_input = strtotime('08:00:00');
$finish_time_input = strtotime('17:00:00');
$total_time = 0;
$query = mysqli_query($con, "SELECT `start_break`, `finish_break` FROM `break_time` WHERE `start_break` AND `finish_break` BETWEEN '".$start_time_input."' AND '".$finish_time_input."' " );
while($tampil = mysqli_fetch_array($query)){
$start_time_db = strtotime($tampil['start_break']);
$finish_time_db = strtotime($tampil['finish_break']);
if ($start_time_input <= $start_time_db AND $finish_time_input >= $finish_time_db) {
$total_time = (($finish_time_input - $start_time_input) - ($finish_time_db - $start_time_db)) / 3600;
} else {
$total_time = ($finish_time_input - $start_time_input) / 3600;
}
echo $total_time;
}
?>
I try to execute the php file, the page shows me nothing, mysql query seems incorrect. But when I try to run this query in phpmyadmin SELECT start_break, finish_break FROM break_time WHERE start_break AND finish_break BETWEEN '08:00:00' AND '17:00:00' the query gave me the result as expected.
Anyone can help me with this? I've tried to do some research, but as far as I get that I just need to add strtotime to my variable, I did that already and nothing happens.
Any help will be much appreciated.
Here is your current query:
SELECT start_break,
finish_break
FROM break_time
WHERE start_break AND finish_break BETWEEN '08:00:00' AND '17:00:00'
The WHERE clause is saying where start_break evaluates to true, and finish_break is between 8am and 5pm. This probably isn't what you want, because start_break will always evaluate to true. From the MySQL documentation:
MySQL evaluates any nonzero, non-NULL value to TRUE
Besides the logical problem in the WHERE clause, you were also trying to compare the start and finish columns directly against a time-only string. This won't work unless these columns are also time, which I doubt.
Here is the query which you probably logically intended:
SELECT start_break,
finish_break
FROM break_time
WHERE DATE_FORMAT(start_break, '%H:%i:%s') BETWEEN '08:00:00' AND '17:00:00' AND -- both the start break and
DATE_FORMAT(finish_break, '%H:%i:%s') BETWEEN '08:00:00' AND '17:00:00' -- finish break are within range
This assumes that the start_break and end_break columns are datetime and you only want to compare the time of day.
I tested today PDO performance like this :
<?php
$start = microtime(true);
new PDO("mysql:host=localhost;dbname=mw", 'root', '');
$stop = microtime(true);
echo $stop - $start;
And the result was pretty surprising (running locally on my Windows 8.1 Laptop)
ELLAPSED : 1.0117259025574
During a the script execution, I cache the PDO object in a static variable so I don't have to create a new one for another query.
But this caching method only works during the script execution.
My script runs in 1.25 seconds of which 1.01 are used to create the PDO object.
Is there a way to cache the PDO object for the whole session or for multiple users ?
Am I missing something ?
I tested same code on my server and connecting with db Via PDO .
$start time is 1414590258.7735
$end time is 1414590258.7736
And $end time- $start time is 0.0001530647277832
My code is
enter code here
$start = microtime(true);
new PDO("mysql:host=$mysqlnd_appname1;dbname=$db1",$user1,$pass1);
$stop = microtime(true);
echo($start);
echo "<br />";
echo $stop;
echo "<br />";
echo $stop - $start;
So its PDO connection performance is good , its also depend on your sever and how my sql server respond to your php server.
for more details visit this http://archive.jnrbsn.com/2010/06/mysqli-vs-pdo-benchmarks
If connection overhead is an issue, you can do this:
$dbh = new PDO("mysql:host=localhost;dbname=mw", 'root', '', array(
PDO::ATTR_PERSISTENT => true
));
http://php.net/manual/en/pdo.connections.php
Server Information:
CentOS 6.5
12GB RAM
Intel(R) Xeon(R) CPU E5-2430 # 6 CPU x 2.20GHz
PHP CLI 5.5.7
I am currently trying to use Perl to fire off 1000 PHP CLI processes in parallel. This however takes 9.9 seconds vs 2.3 seconds for the equivalent Perl script. When I test using the Perl script /opt/test.pl, all 1000 processes are launched in parallel (ps -eLf | grep -ic 'test.pl'). When I test using /opt/testphp.php, using ps -eLf | grep -ic 'testphp.php', I see a count of 250, then it rises to 580 and then it drops to 0 (the script is executed 1000 times, just not in parallel).
Is there a limitation preventing a high number of PHP CLI processes from being launched in parallel?
Has anyone experienced this issue?
Please let me know if I have left out anything that would help to identify the issue.
Thanks
Perl launcher script:
use Time::HiRes qw/ time sleep /;
my $command = '';
my $start = time;
my $filename = '/tmp/report.txt';
# open(my $fh, '>', $filename) or die "Could not open file '$filename' $!";
for $i(1 .. 1000) {
# $command = $command . "(perl /opt/test.pl &);"; // takes 2.3 seconds
$command = $command . "(php -q /opt/testphp.php &);"; // takes 9.9 seconds
}
system($command);
my $end = time;
print 'Total time taken: ', ( $end - $start ) , "\n";
PHP file (testphp.php):
sleep(5);
$time = microtime(true);
file_put_contents('/tmp/report_20140804_php.log', "This is the record: $time\n", FILE_APPEND);
Perl file (test.pl):
#! /usr/bin/perl
use Time::HiRes qw/ time sleep /;
sleep(5);
my $command = '';
my $start = time;
my $filename = '/tmp/report_20140804.log';
open(my $fh, '>>', $filename) or die "Could not open file '$filename' $!";
print $fh "Successfully saved entry $start\n";
close $fh;
What is the best way to see how long it takes for your PHP script to run?
I was thinking something like this:
$start_time = time(); //this at the beginning
$end_time = time(); //this at the end
echo = $end_time-$start_time;
But how can I make it into something that is readable to me and make sense to me?
If you want any further granularity to your timing than seconds, you'll need to use microtime() (Return current Unix timestamp with microseconds)
<?php
$time_start = microtime(true);
// Sleep for a while
usleep(100);
$time_end = microtime(true);
$time = $time_end - $time_start;
echo "Did nothing in $time seconds\n";
?>
** Below was added later **
As far as formatting this result further:
Well, depending on what you're doing, you typically don't have scripts going past a minute. You definitely shouldn't have anything exceeding an hour. (If you do, you need to ask yourself what you are doing with your life)
With that in mind, all you need is simple calculations:
$tmp = floor($time);
$minutes = $tmp / 60;
$seconds = ($tmp % 60) + ($time - $tmp);
$output = 'Script took ';
if ($minutes > 0) $output .= $minutes . ' minutes and ';
$output .= $seconds . ' seconds to complete.';
echo $output;
(This isn't tested, and it could potentially be optimized, but should start you in the right direction)
I would use microtime(TRUE) instead. That'll give you better results with microsecond resolution. There's also the PECL APD extension which profiles scripts.
Expanding a bit on APD, assuming one has APD installed, you just need to add the line
apd_set_pprof_trace();
to the top (or whenever you want to start tracing) of your script. It generates a profiling file with pprofp which generates very readable output
Real User System secs/ cumm
%Time (excl/cumm) (excl/cumm) (excl/cumm) Calls call s/call Memory Usage Name
--------------------------------------------------------------------------------------
100.0 0.00 0.00 0.00 0.00 0.00 0.00 1 0.0000 0.0009 0 main
56.9 0.00 0.00 0.00 0.00 0.00 0.00 1 0.0005 0.0005 0 apd_set_pprof_trace
28.0 0.00 0.00 0.00 0.00 0.00 0.00 10 0.0000 0.0000 0 preg_replace
The example output is from the PHP manual.
I personally find APD extremely useful and use it quite frequently in heavily-hit scripts.
Knowing how long something takes to run is one thing, finding out where (in your php operations) it slows down and why is another question.
If you're serious about find an answer to the latter question then install xdebug, and you get the bonus of not having to call the likes of microtime() which itself slows down your script.
http://xdebug.org/docs/profiler
http://xdebug.org/docs/
Is this a web-accessible script (i.e. http://www.yoursite.com/BLA.php), or a script that you're running through the command line? If it's a command line script, you can use the time command:
time php FILE.php
Otherwise, microtime is probably your best bet.
I find this class to be useful to time the scripts, microtime turn out to be friend in that:
class timer
{
private $start_time = NULL;
private $end_time = NULL;
private function getmicrotime()
{
list($usec, $sec) = explode(" ", microtime());
return ((float)$usec + (float)$sec);
}
function start()
{
$this->start_time = $this->getmicrotime();
}
function stop()
{
$this->end_time = $this->getmicrotime();
}
function result()
{
if (is_null($this->start_time))
{
exit('Timer: start method not called !');
return false;
}
else if (is_null($this->end_time))
{
exit('Timer: stop method not called !');
return false;
}
return round(($this->end_time - $this->start_time), 4);
}
# an alias of result function
function time()
{
$this->result();
}
}