I've written and compiled a command line program (abc.exe) that runs for about 10 mins, makes some calculations and outputs 1 line of results per 10 secs, like this:
C:\>abc.exe
Result 1
Result 2
Result 3
.
.
.
The following php script should theoretically show these results in real time, as they are produced by the program.
<?php
$a = popen('abc.exe', 'r');
while($b = fgets($a)) {
echo $b."<br>\n";
ob_flush();flush();
}
pclose($a);
?>
However, I just get all the results at the end of the execution of the program. Why could this be happening? Please note that if I replace abc.exe with ping 8.8.8.8 or tracert 8.8.8.8, the script works like a charm. Please, help me, I've tried everything that is suggested in similar questions here, but nothing seems to work!
Related
I'm having some trouble using the PHP exec() function. Whenever the script I'm attempting to run is short, exec() works just fine. But if the script takes any more than a second or so, it fails. Note, I've attempted run the long script manually on the command line, and it works just fine. It seems as though the PHP interpreter is killing my external script if it takes any longer than a second or so to run. Any thoughts or suggestions? Here is my code:
<?php
$fileName = "foobar.docx";
$argVar = $fileName;
exec("python3 /var/www/html/jan8/alexandrina.py /var/www/html/jan8/$argVar");
echo "$output";
?>
And here is my script:
#!/usr/bin/env python3
import docx
import sys
docxFile = "".join(sys.argv[1:])
# The Three Lines Below Create a New Variable "htmlFile"
# "htmlFile" is the same as "docxFile", except ".docx" is cut off
# and replaced with ".html"
myNumber = len(docxFile) - 5
htmlFile = docxFile[0:myNumber]
htmlFile = htmlFile + '.html'
def generateHTML(filename):
doc = docx.Document(filename)
fullText = []
for para in doc.paragraphs:
fullText.append('<p>')
fullText.append(para.text)
fullText.append('</p>')
fullText.append('\n')
return '\n'.join(fullText)
file = open(htmlFile, "w")
file.write(generateHTML(docxFile))
file.close()
print("end python script")
Additional notes: I've increased the max execution time limits in php.ini, but I don't think that should matter, as the "long script" should only take a few seconds to run. Also, when I refer to the "short script", and the "long script", I'm actually referring to the same script. The difference between the "long script" and the "short script" is just the time to execute as it may vary depending on the size of the file I'm asking the script to process. Anyway... any suggestions would really be appreciated!
Ordinarily, php exec function should block until the command you run has completed. I.e., the PHP script will halt, waiting for the command to finish until continuing with the rest of your script. I was half thinking that your server was experiencing a max_execution_time timeout, but you've clearly stated that even just a couple of seconds is too long and even these fairly short scripts are having trouble.
A couple of solutions occur to me. The simplest one is to alter the python command so that a) any output is routed to a file or output stream and b) the process is run in the background. According to the docs on exec:
If a program is started with this function, in order for it to continue running in the background, the output of the program must be redirected to a file or another output stream. Failing to do so will cause PHP to hang until the execution of the program ends.
I also would like you to make use of the two additional optional parameters for the exec function.
$fileName = "foobar.docx";
$argVar = $fileName;
$cmd = "python3 /var/www/html/jan8/alexandrina.py /var/www/html/jan8/$argVar";
// modify your command to toss output, background the process, and output the process id
$cmd_modified = $cmd . " >/dev/null & echo \$!";
$cmd_output = NULL; // this will be an array of output
$cmd_return_value = NULL; // this will be the return value of the script
exec($cmd_modified, $cmd_output, $cmd_return_value);
echo "exec has completed</br>";
echo "output:<br>" . print_r($cmd_output, TRUE) . "<br>";
echo "return value: " . print_r($cmd_return_value, TRUE);
This may help or may not. If it does not, we still might be able to solve the problem using posix commands.
EDIT: according to crispytx, the long scripts are resulting in a $cmd_return_val of 1 which means an error is happening. Try changing this one line:
$cmd_modified = $cmd . " >/dev/null & echo \$!";
to this
$cmd_modified = $cmd . " & echo \$!";
And let us know what the output of $cmd_output is -- it should at the very least have the process id of the newly spawned process.
Thanks for all the help S. Imp. I had a little trouble debugging using your suggestions because I happened to be using AJAX to call the script. However, I wrote simpler script using your suggestions to try and debug the problem and this is what I found:
Array ( [0] => Traceback (most recent call last): [1] => File "/var/www/html/jan8/alexandrina.py", line 28, in [2] => file.write(generateHTML(docxFile)) [3] => UnicodeEncodeError: 'ascii' codec can't encode character '\u2026' in position 25: ordinal not in range(128) )
So it looks like the problem has to do with ascii encoding! Even though the larger file was just a docx file with the same text as the shorter docx file repeated over and over again for 300 pages. It seems that if a docx file exceeds 1 pages, ascii characters are inserted that aren't present in single page docx files. I have no idea if this post will ever end up helping anyone, but who knows!
[SOLVED]
I hope somebody can help me. :)
I am calling a python script from PHP, with the help from phpseclib/ssh2 i ssh into my server and it works fine.
My problem is that if i use "time.sleep(5)" in the loop of my python script i dont get a result back, but if i remove time.sleep(5) and time.sleep(3) it works.
Anybody have an idea why this happen?
If i try the python script i my console everything is picture perfect.!
items = [
'1',
'2',
'3'
]
itemArray = {}
def checker():
for item in items:
time.sleep(5) # If added not working, if removed working, result gets send back
position = 1 # keeps track of the ranking position
for start in range(int(deep)):
time.sleep(3)
results = 'something'
for div in results:
try:
if div.find('i', href=True)['href'].find(something) != -1:
exit_conditon = True
break
else:
position += 1
except:
print "Unexpected error:", sys.exc_info()[0]
raise
if 'exit_conditon' in locals():
if exit_conditon is True:
exit_conditon = False
itemArray.update({value: 1})
break
sys.exit(itemArray)
checker()
Please help.
Update: if i have 3 rows in the items array i need to remove the second time.sleep(5) to get it working, if i have 2 items in my array i only need to remove the first time.sleep(5).
It depends how long your script take time to execute. If your script (php + python) take longer than 30 second with the default config, them php kill it.
Just add set_time_limit(120) at the beginning of your php code
See http://php.net/manual/en/function.set-time-limit.php
And If you are executing the php and python script on the same server you should use exec or shell_exec. It will be faster this way. See php shell_exec() vs exec()
The script is now working as i should do :) Thanks to "neubert" for the answer.
If i set the $ssh->setTimeout() to unlimited, whould i still get a respons if the script halts at some point?
I have a bit of backend code that is executed like so:
python filepath $inputvariable
This code prints out some data. As you can see in the screenshot below, when I run this code through terminal it works flawlessly, outputting the expected value:
CHI 110^*^Integrated Chinese Level 1 Part 1^*^https://www.amazon.com/Integrated-Chinese-Simplified-Characters-Textbook/dp/0887276385/ref=sr_1_1?ie=UTF8&qid=1466983577&sr=8-1&keywords=integrated+chinese^*^47.49
But I run into issues when I try to run the same code through php:
echo exec ("python /Users/USERNAME/Desktop/Exeter_Bookstore_Project/localserver/cont/Scripts/Python/serverside.py $classToSend");
This code returns a null value. At first I assumed that I wasn't passing variables through correctly, but echo $classToSend; yielded the correct variable. Then I tried having the php execute a hello world python script, but this also worked proving that the issue wasn't in my python interpreter. Then I thought that maybe the python script wasn't forwarding data quickly enough, but the helloworld.py still worked even with a time delay of 3 seconds.
Does anyone have any idea of what I might have done wrong, or do you need more information. Any help will be greatly appreciated.
I am running VW in daemon mode. As a standalone executable, it runs perfectly fine. In daemon mode, I see something about predictions and options initially but not the end result. Not sure what exactly is going on.
This is how I call VW6
/bin64/vw --daemon --num_children 2 -t -i ~/modelbow.vw6 --min_prediction 0 --max_prediction 1 -p stdout 2>&1
I check vw6 is running fine. I send data using simple php script (removed debug lines for brevity):
$fp = fsockopen("localhost",26542, $errno, $errstr, 3);
$fp_dat = fopen("/tmp/ml.dat", "r");
$mldata = explode("\n", file_get_contents("/tmp/ml.dat"));
$mlstr = implode($mldata);
fwrite($fp, $mlstr);
$result = trim(fgets($fp, 1024));
print $result;
Print $result above prints nothing. The only thing I see in stdout is
num sources = 1
Num weight bits = 28
learning rate = 10
initial_t = 1
power_t = 0.5
decay_learning_rate = 1
predictions = stdout
only testing
average since example example current current current
loss last counter weight label predict features
While in a standalone executable mode, if I run with the same model same dat file just without the -daemon option, it happily gives a result at the end
...
...
predictions = stdout
only testing
average since example example current current current
loss last counter weight label predict features
1.000000 ba66dfc7a135e2728d08010b40586b90
Any idea what could be going wrong here with the daemon mode? I tried using -p /tmp/ option as well...ran the daemon mode with sudo but nothing helped. Is there a debug dump option or verbose option or something else to know what exactly is going on?
thanks
The reason it is not working is not in vw but in the PHP client code.
explode on "\n", strips the newlines out.
implode without a glue-string parameter results in glue-string defaulting to the empty string.
Result: newlines are stripped out.
All examples are merged into one big (and incomplete, since there's no newline at the end) example.
vw needs newlines to separate examples, without them it will be waiting forever for the 1st example to complete.
So I think you need to change the implode line of code to:
$mlstr = implode("\n", $mldata);
for it to work.
You will also need an additional ending newline to get the last line through.
I am using php, I see while() loop in php. I want to ask something can we use
while() loop as mysql_unbuffered_query().
This code make good understanding.
<?php
echo 'While Loop is going to start';
$i = 0;
while($i <= 1000){
echo 'Now number is '.$i;
$i++;
}
echo 'Continue to running script';
?>
What happens when we run this code first of all this code read while loop is going to start
then read the while loop and create the statement to run Now number is 0.....1000
then read Continue to running script And when the code is finished it print out whole data.But I did not want this.
What I want.
First script read the while Loop is going to start and print out then read while loop and
print out Now number is 0 then go Continue to running script and print out But the
back end of script while loop still working, My mean both while loop and continue script working at once.
This give you more understanding.
OUTPUT(First time when script running is start)
1 - While Loop is going to start
2 - Now number is 0
3 - Continue to running script
OUTPUT(Script running is continue)
1 - While Loop is going to start
2 - Now number is 0
3 - Now number is 1
. - ...............
1000 - Now number is 1000
1001 - Continue to running script
Might be this is impossible.If yes how I can do that.
But maybe this is possible like mysql_unbuffered_query().
Like when while loop complete one cycle it print out the number and then other one and so on to complete.
OUTPUT(First time when while loop complete one cycle)
1 - While Loop is going to start
2 - Now number is 0
OUTPUT(while loop complete second cycle)
1 - While Loop is going to start
2 - Now number is 0
3 - Now number is 1
If this is possible, Please guide me how can I do that.
Thanks..............
If you are running this script from the command line, output buffering is always off, and implicit flush is always on (according to http://us3.php.net/manual/en/outcontrol.configuration.php ). Because of this, I suspect you are running this PHP within a web server.
There are several PHP INI values which affect buffering, so you may wish to turn off these features:
<?php
ini_set('zlib.output_compression', 'off');
ini_set('implicit_flush', 'on');
ini_set('output_buffering', 'off');
echo "While Loop is going to start.\n";
$i = 0;
while($i <= 10){
echo "Now number is ".$i."\n<br>\n";
$i++;
sleep(1);
}
echo "\n<br>Finished running script\n\n";
?>
Even after all that, it's possible that PHP is behaving exactly as you want, but your web server is buffering the output. This thread may be of interest: php flush not working
If you are using apache, you can add a line to an .htaccess file to turn off gzip compression (which automatically buffers until there's enough data to compress and send)
I found this page (http://www.bluehostforum.com/showthread.php?18996-Turning-off-Gzip) which describes the process, but it basically says to add
SetEnv no-gzip dont-vary
to your .htaccess file.
Can you confirm that the script I posted above works as you'd expect in your command line environment or not?
Also, if you are working within a web server, please post which web server you're using (IIS/apache/etc)
You need threads or processes. This is a good thread: Does PHP have threading?
My favorite answer there is https://stackoverflow.com/a/14201579/650405
Forcing output to the browser works with the following calls in this order
ob_flush();
flush();
It, however, will not background any processing going on.