In a Linux server Nginx as web server,
Response to http request from Android application to PHP code with Nginx is too slow
The point is the response from the server is as following:
echo "{mResponse}"
When mResponse is small (you say 500 bytes), there is no problem and the response is too quick
When mResponse is big (you say 200K bytes), there is a problem and the response is too slow
I doubted on echo and searched a lot and found out the echo may be too slow
There were some solutions like chuncking the big data into smaller chuncks (4096) and then echoing
Or ob_start and ...
I tested them and finally found out the problem is somewhere else because when I used the following code, the time of echo is ok:
$time_start = microtime(true);
$this->echobig("###{$mCommand}70{$mUserID}{$mResponse}{$mCheckSum}###");
echo "\nThe first script took: " . ( microtime(true) - $time_start) . " sec";
$time_start = microtime(true);
ob_start();
$this->echobig("###{$mCommand}70{$mUserID}{$mResponse}{$mCheckSum}###");
ob_end_flush();
echo "\nThe Second script took: " . ( microtime(true) - $time_start) . " sec";
public function echobig($string)
{
$splitString = str_split($string, 4096);
foreach($splitString as $chunk)
{
echo $chunk;
}
}
On both above codes (The first and second scripts) the time was near
0.0005 sec
which is ok
But the Android application is receiving the response in 13 seconds
As I said when the response is small, the Android application quickly receives the response (in 2 seconds)
Now I doubt on Nginx settings or PHP settings (may be buffer limits somewhere)
I don't know which parameter is problematic
Sorry, It has nothing to do with Nginx or PHP
The problem is the following code to calculate the check sum of the mentioned big response:
$cs = 0;
$content_length = strlen($string);
for ($i=0; $i < $content_length;$i++)
{
$K = mb_substr($string,$i, 1, 'UTF-8');
$A = mb_convert_encoding(mb_substr($string,$i, 1, 'UTF-8'),"UTF-8");
$B = $this->mb_ord_WorkAround($A);
$cs = $cs + $B;
}
Trying to create a old school terminal text effect (one character at a time with a small delay) in PHP - without javascript if possible.
All text written to the screen should go through this function.
I was thinking something like a buffer you can dynamically append text to make sure it would finish one line, before starting on the next.
Not sure how to preceed or if it's even possible without using Javascript.
Inefficient, but to achieve the goal you set (without javascript), you could use PHP's output buffering to achieve a small delay between characters output:
<?php
ob_start();
$buffer = str_repeat(" ", 4096); // fill the buffer
$string = 'Hello World';
$len = strlen($string);
$sleep = 0.5; // sleep half a second between output chars
for($i=0; $i < $len; $i++) {
echo $buffer . $string[$i];
ob_flush();
flush();
usleep($sleep * 1000000);
}
I need to login to a production server retrieve a file and update my data base with the data in this file. Since this is a production database, I don't want to get the whole file every 5 minutes since the file may be huge and this may impact the server. I need to get the last 30 lines of this file every 5 minutes interval and have as little impact as possible.
The following is my current code, I would appreciate any insight to how best accomplish this:
<?php
$user="id";
$pass="passed";
$c = curl_init("sftp://$user:$pass#server1.example.net/opt/vmstat_server1");
curl_setopt($c, CURLOPT_PROTOCOLS, CURLPROTO_SFTP);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($c);
curl_close($c);
$data = explode("\n", $data);
?>
Marc B is wrong. SFTP is perfectly capable of partial file transfers. Here's an example of how to do what you want with phpseclib, a pure PHP SFTP implementation:
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
$size = $sftp->size('filename.remote');
// outputs the last ten bytes of filename.remote
echo $sftp->get('filename.remote', false, $size - 10);
?>
In fact I'd recommend an approach like this anyway since some SFTP servers don't let you run commands via the system shell. Plus, SFTP can work on Windows SFTP servers whereas tail is unlikely to do so even if you do have shell access. ie. overall, it's a lot more portable a solution.
If you want to get the last x lines of a file you could loop repeatedly, reading however many bytes each time, until you encounter 10x new line characters. ie. get the last 10 bytes, then the next to last 10 bytes, then the ten bytes before those ten bytes, etc.
An answer by #Sammitch to a duplicate question Get last 15 lines from a large file in SFTP with phpseclib:
The following should result in a blob of text with at least 15 lines from the end of the file that you can then process further with your existing logic. You may want to tweak some of the logic depending on if your file ends with a trailing newline, etc.
$filename = './file.txt'
$filesize = $sftp->size($filename);
$buffersize = 4096;
$offset = $filesize; // start at the end
$result = '';
$lines = 0;
while( $offset > 0 && $lines < 15 ) {
// work backwards
if( $offset < $buffersize ) {
$offset = 0;
} else {
$offset -= $buffer_size;
}
$buffer = $sftp->get($filename, false, $offset, $buffer_size));
// count the number of newlines as we go
$lines += substr_count($buffer, "\n");
$result = $buffer . $result;
}
SFTP is not capable of partial file transfers. You might have better luck using a fullblowin SSH connection and use a remote 'tail' operation to get the last lines of the file, e.g.
$lines = shell_exec("ssh user#remote.host 'tail -30 the_file'");
Of course, you might want to have something a little more robust that can handle things like net.glitches that prevent ssh from getting through, but as a basic starting point, this should do the trick.
As suggested in a reply to a previous question (PHP External Oauth : how to displaying a waiting message while waiting for callback (not using AJAX) ), I am using transfer encoding: chunked to display a waiting message while some tasks are performed. My first attempts failed, and I found a solution in this question “Transfer-Encoding: chunked” header in PHP. There is a "padding" of 1024 blank space. Without this padding it doesn't work. I have googled around but I can't find what this padding is for. Here is the sample code (from the related question).
<?php
header('Content-Encoding', 'chunked');
header('Transfer-Encoding', 'chunked');
header('Content-Type', 'text/html');
header('Connection', 'keep-alive');
ob_flush();
flush();
$p = ""; //padding
for ($i=0; $i < 1024; $i++) {
$p .= " ";
};
echo $p;
ob_flush();
flush();
for ($i = 0; $i < 10000; $i++) {
echo "string";
ob_flush();
flush();
sleep(2);
}
?>
Does anybody have an explanation why it works with and doesn't work without the "padding" ?
The padding is for filling the server buffer, as I understand.
Without it the server will wait until PHP will fill it and after this will flush it - even if in PHP code you do flush().
Related:
PHP Output buffer flush issue on Apache/Linux (May 2010; on Server Fault)
I have no idea what this padding is supposed to do, and actually it shouldn't work (someone may enlighten me if I'm wrong on this). The idea with chunked encoding is that you send your data in chunks. Each chunk consists of a line containing the length of a chunk, followed by a newline and then the data of the chunk. A response can contain as many chunks as you want to have. So basically the response of 3 chunks containing "Hello" would look like this:
5 <--- this is the length of the chunk, that is "Hello" == 5 chars
Hello <--- This is a the actual data
<-- an empty line is between the chunks
5
Hello
5
Hello
<-- send two empty lines to end the transmission
So I'd rewrite this to something like:
<?php
header('Content-Encoding', 'chunked');
header('Transfer-Encoding', 'chunked');
header('Content-Type', 'text/html');
header('Connection', 'keep-alive');
ob_flush();
flush();
for ($i = 0; $i < 10000; $i++) {
$string = "string";
echo strlen($string)."\r\n"; // this is the length
echo $string."\r\n"; // this is the date
echo "\r\n"; // newline between chunks
ob_flush(); // rinse and repeat
flush();
sleep(2);
}
echo "\r\n"; // send final empty line
ob_flush();
flush();
?>
The code above will not work at all circumstances (e.g. with strings containing newlines or non-ascii encodings), so you will have to adapt it to your use case.
I'm trying to get a "live" progress indicator working on my php CLI app. Rather than outputting as
1Done
2Done
3Done
I would rather it cleared and just showed the latest result. system("command \C CLS") doesnt work. Nor does ob_flush(), flush() or anything else that I've found.
I'm running windows 7 64 bit ultimate, I noticed the command line outputs in real time, which was unexpected. Everyone warned me that out wouldn't... but it does... a 64 bit perk?
Cheers for the help!
I want to avoid echoing 24 new lines if I can.
Try outputting a line of text and terminating it with "\r" instead of "\n".
The "\n" character is a line-feed which goes to the next line, but "\r" is just a return that sends the cursor back to position 0 on the same line.
So you can:
echo "1Done\r";
echo "2Done\r";
echo "3Done\r";
etc.
Make sure to output some spaces before the "\r" to clear the previous contents of the line.
[Edit] Optional: Interested in some history & background? Wikipedia has good articles on "\n" (line feed) and "\r" (carriage return)
I came across this while searching for a multi line solution to this problem. This is what I eventually came up with. You can use Ansi Escape commands. http://www.inwap.com/pdp10/ansicode.txt
<?php
function replaceOut($str)
{
$numNewLines = substr_count($str, "\n");
echo chr(27) . "[0G"; // Set cursor to first column
echo $str;
echo chr(27) . "[" . $numNewLines ."A"; // Set cursor up x lines
}
while (true) {
replaceOut("First Ln\nTime: " . time() . "\nThird Ln");
sleep(1);
}
?>
I recently wrote a function that will also keep track of the number of lines it last output, so you can feed it arbitrary string lengths, with newlines, and it will replace the last output with the current one.
With an array of strings:
$lines = array(
'This is a pretty short line',
'This line is slightly longer because it has more characters (i suck at lorem)',
'This line is really long, but I an not going to type, I am just going to hit the keyboard... LJK gkjg gyu g uyguyg G jk GJHG jh gljg ljgLJg lgJLG ljgjlgLK Gljgljgljg lgLKJgkglkg lHGL KgglhG jh',
"This line has newline characters\nAnd because of that\nWill span multiple lines without being too long",
"one\nmore\nwith\nnewlines",
'This line is really long, but I an not going to type, I am just going to hit the keyboard... LJK gkjg gyu g uyguyg G jk GJHG jh gljg ljgLJg lgJLG ljgjlgLK Gljgljgljg lgLKJgkglkg lHGL KgglhG jh',
"This line has newline characters\nAnd because of that\nWill span multiple lines without being too long",
'This is a pretty short line',
);
One can use the following function:
function replaceable_echo($message, $force_clear_lines = NULL) {
static $last_lines = 0;
if(!is_null($force_clear_lines)) {
$last_lines = $force_clear_lines;
}
$term_width = exec('tput cols', $toss, $status);
if($status) {
$term_width = 64; // Arbitrary fall-back term width.
}
$line_count = 0;
foreach(explode("\n", $message) as $line) {
$line_count += count(str_split($line, $term_width));
}
// Erasure MAGIC: Clear as many lines as the last output had.
for($i = 0; $i < $last_lines; $i++) {
// Return to the beginning of the line
echo "\r";
// Erase to the end of the line
echo "\033[K";
// Move cursor Up a line
echo "\033[1A";
// Return to the beginning of the line
echo "\r";
// Erase to the end of the line
echo "\033[K";
// Return to the beginning of the line
echo "\r";
// Can be consolodated into
// echo "\r\033[K\033[1A\r\033[K\r";
}
$last_lines = $line_count;
echo $message."\n";
}
In a loop:
foreach($lines as $line) {
replaceable_echo($line);
sleep(1);
}
And all lines replace each other.
The name of the function could use some work, just whipped it up, but the idea is sound. Feed it an (int) as the second param and it will replace that many lines above instead. This would be useful if you were printing after other output, and you didn't want to replace the wrong number of lines (or any, give it 0).
Dunno, seemed like a good solution to me.
I make sure to echo the ending newline so that it allows the user to still use echo/print_r without killing the line (use the override to not delete such outputs), and the command prompt will come back in the correct place.
i know the question isn't strictly about how to clear a SINGLE LINE in PHP, but this is the top google result for "clear line cli php", so here is how to clear a single line:
function clearLine()
{
echo "\033[2K\r";
}
function clearTerminal () {
DIRECTORY_SEPARATOR === '\\' ? popen('cls', 'w') : exec('clear');
}
Tested on Win 7 PHP 7. Solution for Linux should work, according to other users reports.
something like this :
for ($i = 0; $i <= 100; $i++) {
echo "Loading... {$i}%\r";
usleep(10000);
}
Use this command for clear cli:
echo chr(27).chr(91).'H'.chr(27).chr(91).'J'; //^[H^[J
Console functions are platform dependent and as such PHP has no built-in functions to deal with this. system and other similar functions won't work in this case because PHP captures the output of these programs and prints/returns them. What PHP prints goes to standard output and not directly to the console, so "printing" the output of cls won't work.
<?php
error_reporting(E_ERROR | E_WARNING | E_PARSE);
function bufferout($newline, $buffer=null){
$count = strlen(rtrim($buffer));
$buffer = $newline;
if(($whilespace = $count-strlen($buffer))>=1){
$buffer .= str_repeat(" ", $whilespace);
}
return $buffer."\r";
};
$start = "abcdefghijklmnopqrstuvwxyz0123456789";
$i = strlen($start);
while ($i >= 0){
$new = substr($start, 0, $i);
if($old){
echo $old = bufferout($new, $old);
}else{
echo $old = bufferout($new);
}
sleep(1);
$i--;
}
?>
A simple implementation of #dkamins answer. It works well. It's a bit- hack-ish. But does the job. Wont work across multiple lines.
function (int $count = 1) {
foreach (range(1,$count) as $value){
echo "\r\x1b[K"; // remove this line
echo "\033[1A\033[K"; // cursor back
}
}
See the full example here
Unfortunately, PHP 8.0.2 does not has a function to do it. However, if you just want to clear console try this: print("\033[2J\033[;H"); or use : proc_open('cls', 'w');
It works in php 8.0.2 and windows 10. It is the same that system('cls') using c language programing.
Tried some of solutions from answers:
<?php
...
$messages = [
'11111',
'2222',
'333',
'44',
'5',
];
$endlines = [
"\r",
"\033[2K\r",
"\r\033[K\033[1A\r\033[K\r",
chr(27).chr(91).'H'.chr(27).chr(91).'J',
];
foreach ($endlines as $i=>$end) {
foreach ($messages as $msg) {
output()->write("$i. ");
output()->write($msg);
sleep(1);
output()->write($end);
}
}
And \033[2K\r seems like works correct.