PHP hangs when call Python script that use multiprocess/multithreading - php

I've write a php code to call a Python script much like this:
<?php
system("tmn", $return_value);
echo $return_value;
?>
Below is the Python script.
#!/usr/bin/env python
import os
from subprocess import Popen
devnull = open(os.devnull, 'wb')
p = [] # ip -> process
for n in range(1, 20): # start ping processes
ip = "172.28.83.%d" % n
p.append((ip, Popen(['ping', '-c', '1', '-w', '1', ip], stdout=devnull)))
#NOTE: you could set stderr=subprocess.STDOUT to ignore stderr also
while p:
for i, (ip, proc) in enumerate(p[:]):
if proc.poll() is not None: # ping finished
p.remove((ip, proc)) # this makes it O(n**2)
if proc.returncode == 0:
print('%s active' % ip)
elif proc.returncode == 2:
print('%s no response' % ip)
else:
print('%s error' % ip)
devnull.close()
But when I load the php page using my broswer, the page will loading forever, it seems that PHP is stuck at the system or exec call.
I tried using different Python script, but as long as the script is parallel(using either Multiproccessing or Multithreading), this problem will definitely happen.
The weirdest thing is that this issue only happens on one of my linux server(CentOS 6.5).
$php -v
PHP 5.5.7 (cli) (built: Jan 3 2014 11:19:10)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2013 Zend Technologies
python --version
Python 2.7.6
I've squeeze my head all day for this. It would be a huge help if you give any suggestion.

This is probably widely off mark, but the rule of thumb solution for solving "weird problems which only happen on centos" is "have you tried disabling selinux?"
Maybe you should try disabling it (http://www.cyberciti.biz/faq/howto-turn-off-selinux/), rebooting and trying your code again. If it works, you will either learn to keep selinux disabled on all your systems or you will have an excellent adventure in trying to understand how selinux works, in which case, good luck and bring a lot of aspirin.

Related

Real-time SSH output with Net_SSH (phpseclib)

I use Net_SSH (phpseclib) to execute SSH commands on an external server. I simply cannot figure out how to have real-time output from the command. I know how to make it run in the background so it's not dependant on the Apache process, but it's unclear how I'd go about showing the external output in real-time instead of having to wait for the command to finish.
My current code is as simple as $ssh->exec('command').
The PHP version used is:
[admin# ~]$ php -v
PHP 7.1.9 (cli) (built: Sep 10 2017 11:31:06) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
I managed to get it working with libssh2 and output buffering, see example below:
$session = ssh2_connect("server.local", 22, array('hostkey'=> 'ssh-rsa' )) or die("Couldn't connect to the SSH Server.");
ssh2_auth_pubkey_file($session, "root", "/path/to/public/key.pub", "/path/to/private/key") or die("couldn't authenticate to server"); // Authenticating to the server with a ssh-key for security purposes
while (ob_end_flush()); // end all output buffers if any
$proc = ssh2_exec($session, "ping -c 40 google.nl");
echo '<pre class="scroll">';
echo "[root#server ~]# ping -c 5 google.nl\n"; // Command you will execute
while (!feof($proc))
{
echo fread($proc, 4096); // Read the output from the command
# flush(); // Flushes the whole php buffer so we can output new data
}
echo "\nDone";
echo '</pre>';
Don't forget that you need php 5.6 or lower for ssh2, you can replace the command in the variable $proc by $ssh->exec('command') as you use it.
I was able to get it working using this:
$ssh->exec('ping 127.0.0.1', function($output) {
echo $output;
});
To eliminate the variability in how your system is configured versus mine I'll use Vagrant to establish a common configuration. To that end, here's my Vagrantfile:
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/trusty64"
end
My full phpseclib code (using 1.0.7):
<?php
include('Net/SSH2.php');
$ssh = new Net_SSH2('127.0.0.1', 2222);
$ssh->login('vagrant', 'vagrant');
$ssh->exec('ping 127.0.0.1', function($output) {
echo $output;
});
A youtube video of the output:
https://youtu.be/j9-q3024eEk
If it's not working then several possibilities exist.
Maybe the "command" you're running simply doesn't dump output real time. Or maybe it requires a PTY or something. It's hard to comment since you haven't said what the command you're trying to run is. As my post demonstrates there are commands that my solution does work with.
Maybe it works with Vagrant but not with your system. Maybe your system has been configured in some funky way or something. In this scenario I guess what'd help is if you provided the SSH logs. You can get them by doing define('NET_SSH2_LOGGING', 2); and then echo $ssh->getLog();. Post the results in pastebin.com and then post the link.
edit: if you're running this in a webserver vs in the CLI you may encounter issues with how the webserver is setup - issues that go past phpseclib. For example, does this output real time or does it lock up?:
while (true) {
echo "test\n";
sleep(1);
}
flush() / ob_flush() might help but ultimately this would depend on the webserver you're using (Apache, nginx, etc), the SAPI you're using (CGI, Apache module, etc), etc.
I would consider this to be a "funky configuration".

Can't execute a php script without using "php" command before

I need to use a php script without "php" command.
For example:
$ ./test.php
Permissions are sets to 755.
This is the script
#!/usr/bin/php -q
<?php
echo "hello world";
?>
/usr/bin/php -v (so path exists)
returns
PHP 7.0.15-1+deb.sury.org~xenial+1 (cli) (built: Jan 20 2017 08:53:13) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.15-1+deb.sury.org~xenial+1, Copyright (c) 1999-2017, by Zend Technologies
This is the error I'll get everytime:
Exception: Zend Extension ./test.php does not exist
Also calling script with fullpath I'll get same error.
Calling this it works properly
$ php ./test.php
Any idea?
NOTE: The author found the solution and put it up in the comments but never posted an actual answer, so this answer is just clarifying what the author already said above so as to make the answer more obvious.
I was also getting the Exception: Zend Extension does not exist when I was trying to pipe an email via cpanel forwarder into a php script.
I opened the file in my editor (Komodo Edit on Windows) and went to EDIT > CURRENT FILE PREFERENCES and noticed that LINE ENCODINGS was set to DOS/Windows (\r\n)
I changed the LINE ENCODING to UNIX (\n) and saved it and re-uploded it and the error went away and all is good now.
Obviously the steps will vary depending on what editor you use, but the solution is to make sure your Line Encodings are UNIX and not DOS/Windows.
Just run dos2unix on the file
# ./database.php
Exception: Zend Extension ./database.php does not exist
# apt install dos2unix
# dos2unix database.php
dos2unix: converting file database.php to Unix format...
# ./database.php
Yeah!!! It work's!!!!

Executing gulp in php returns error but works in shell

I have a wierd problem with executing gulp from php script. Tests:
browser: http://server.com/pull.php <- does not work
shell: php pull.php <- works
shell: gulp <- works
Here is my setup:
Ubuntu 12.04.5 LTS
Node v5.6.0
PHP 5.5.30-1+deb.sury.org~precise+1 (cli) (built: Oct 4 2015 16:14:34)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies
Here are packages in package.json:
"gulp": "^3.9.1",
"gulp-concat": "^2.6.0",
"gulp-less": "^3.0.5",
"gulp-rename": "^1.2.2",
"gulp-uglify": "^1.5.2",
"gulp-uglifycss": "^1.0.6",
"gulp-watch": "^4.3.5"
It works perfectly when i run this from shell but returns error when run in php as webserver:
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
TypeError: Object #<Object> has no method 'existsSync'
at /usr/local/lib/node_modules/gulp/node_modules/liftoff/lib/find_config.js:21:10
at Liftoff.buildEnvironment (/usr/local/lib/node_modules/gulp/node_modules/liftoff/index.js:67:20)
at Liftoff.<anonymous> (/usr/local/lib/node_modules/gulp/node_modules/liftoff/index.js:192:32)
at /usr/local/lib/node_modules/gulp/node_modules/liftoff/node_modules/flagged-respawn/index.js:17:3
at Liftoff.<anonymous> (/usr/local/lib/node_modules/gulp/node_modules/liftoff/index.js:185:9)
at /usr/local/lib/node_modules/gulp/node_modules/liftoff/index.js:159:9
at /usr/local/lib/node_modules/gulp/node_modules/v8flags/index.js:99:14
at Array.0 (/usr/local/lib/node_modules/gulp/node_modules/v8flags/index.js:38:7)
at EventEmitter._tickCallback (node.js:192:41)
Command to run it is simple:passthru('gulp 2>&1');. I hoestly don't know what is wrong, owner of the files, web server (nginx runs as) user and git owner is the same user named git. Tried various different things as making bash script to run it from php but no luck. I am guessing i am missing some system variables?
Thanks in advance, let me know if you need more info.
Turns out node couldn't play nicely with gulp. My solution:
passthru('node ./node_modules/gulp/bin/gulp.js 2>&1');
I can't explain why, just tried few random ideas and that one worked. One thing bothers me though, why can't i get full info on git pull? I am only getting:
Updating dcd958f..db05960

Troubleshoot Codeigniter CLI stopped

I have been using the CLI interface to send out cron jobs from my codeigniter page. It worked fine until I updated Wordpress yesterday. I do not know how this effected Codeigniter but that is when the trouble started. I also installed cURL at about the same time. I am not sure if that could have made a difference.
SYMPTOMS:
None of my codeigniter CLI scripts work. I have two scripts that send out email reminders, and another that synchronizes my database and none function.
ERRORS:
I had some errors come up when I tried to run my scripts such as:
Use of undefined constant __DIR__ - assumed '__DIR__'
This was never a problem before. But for now I change that to
dirname(__FILE__)
and that seemed to help. At least that error stopped.
Next another error notice appeared regarding code in my scripts that I was not getting before: "Can't use method return value in write context in . . ."
This error was in reference to this line of code:
if (!empty($this->get_available_hours($date, $provider_id))) {
I modified this to
$availabehours=$this->get_available_hours($date, $provider_id);
if (!empty($availabehours)) {
And the error stopped. But the script usually sends out email regarding availability and no email is sent.
Now I have no errors. I run the scripts and I get no results. If I purposefully mess with the code and do things wrong, I get the appropriate error messages. So, at some level it is reading the file.
I tried just running a simple "hello world file" as discribed here
https://ellislab.com/codeigniter/user-guide/general/cli.html
And nothing was returned.
I tried a simple email script that would send out an email without accessing my database and it did not send anything to me.
It appeares to me like something has caused my code to be interpreted in an older version of php. So I looked at the version currently running:
When logged into the terminal in PuTTY I get:
PHP 5.2.17 (cli) (built: Feb 23 2012 10:42:34)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies
Out of date ...
But when I look in the terminal within WinSCP I get:
PHP 5.5.28 (cli) (built: Sep 4 2015 12:07:49)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies
This looks up to date.
Running this works: php -r 'echo "Hello World!\n";'
MY QUESTIONS:
1) What tests can I run to find out what is blocking things with my CLI?
Any tips would be appreciated.
2) Why am I seeing two different versions of PHP depending on the terminal I am running and could this be the cause of my problem?

Execute "less" from command-line PHP w/ Scrolling

I want to execute less and similar programs from the command-line in PHP.
I have tried the usual suspects (exec, shell_exec, passthru, etc), and while many of them can dump the file to the screen, the process is terminated before I can make use of it. If I wanted cat, I'd use it.
How do I execute a program in this fashion?
You could use proc_open to feed input to and get output back from a process via pipes. However, it doesn't seem like less allows for user interaction via pipes as it basically degrades to a cat command. Here's my first (failed) approach:
<?php
$dspec = array(
0 = array('pipe', 'r'), // pipe to child process's stdin
1 = array('pipe', 'w'), // pipe from child process's stdout
2 = array('file', 'error_log', 'a'), // stderr dumped to file
);
// run the external command
$proc = proc_open('less name_of_file_here', $dspec, $pipes, null, null);
if (is_resource($proc)) {
while (($cmd = readline('')) != 'q') {
// if the external command expects input, it will get it from us here
fwrite($pipes[0], $cmd);
fflush($pipes[0]);
// we can get the response from the external command here
echo fread($pipes[1], 1024);
}
fclose($pipes[0]);
fclose($pipes[1]);
echo proc_close($proc);
I guess for some commands this approach might actually work - and there are some examples in the php manpage for proc_open that might be helpful to look over - but for less, you get the whole file back and no possibility for interaction, maybe for reasons mentioned by Viper_Sb's answer.
...But it seems easy enough to simulate less if that's all you need. For example, you could read the output of the command into an array of lines and feed it in bite-sized chunks:
<?php
$pid = popen('cat name_of_file_here', 'r');
$buf = array();
while ($s = fgets($pid, 1024))
$buf[] = $s;
pclose($pid);
for ($i = 0; $i < count($buf)/25 && readline('more') != 'q'; $i++) {
for ($j = 0; $j < 25; $j++) {
echo array_shift($buf);
}
}
I don't believe this is possible. PHP is not a VM/shell environment, the commands it has to access other programs all return control to it, and normally there is no interaction while PHP is running.
One last thing, try with the backtick operators, if that doesn't work then I'm pretty sure you can't do this without writing up something yourself that will sleep and allow user input etc... by default no
`nano file.txt`
Adding exec('stty cbreak'); to the PHP script also fixes the issue.
I put the following in a file defined by the auto_prepend_file setting in php.ini
So, I would do something like edit php.ini to the following:
auto_prepend_file = /path/to/prepend.php
Then in, /path/to/prepend.php, I would add the following line:
if (php_sapi_name() == 'cli') exec('stty cbreak');
I'm not exactly sure of the cause. I've read bug reports for PHP. I'm not sure about versions though. I noticed the problem with the following setup:
$ php -v
PHP 5.3.3 (cli) (built: Jul 12 2013 20:35:47)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
However, the following did not show the issue:
# php -v
PHP 5.3.26 (cli) (built: Oct 21 2013 16:50:03)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2013 Zend Technologies
with the ionCube PHP Loader v4.4.1, Copyright (c) 2002-2013, by ionCube Ltd.
It is worth noting that the version without the issue was using cPanel and the other was using the default CentOS 6 install via yum.

Categories