I have followed the famous michael noll tutorial http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ to install hadoop in single node.
Now I want to execute the following command via browser through php
usr/local/hadoop/bin/hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-*streaming*.jar -mapper "/usr/bin/python /var/www/DataMining/AnalysisByYear/AnalysisByYear_mapper.py 2011" -reducer "/usr/bin/python /var/www/DataMining/AnalysisByYear/AnalysisByYear_reducer.py" -input /user/hduser/dataset/final_eval.txt -output /user/hduser/dataset-outputyear
I am using the exec() command of php to execute it but it doesn't show any result.
In the tutorial the hduser is given the ownership of hadoop. However the browser logs in as the user www-data. As a result it cannot execute. I am assuming that is the error. Is it possible to execute the script in any way?
The namenode logs are given below:
2013-11-18 14:54:45,712 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:www-data cause:org.apache.hadoop.security.AccessControlException: Permission denied: user=www-data, access=WRITE, inode="staging":hduser:supergroup:rwxr-xr-x
2013-11-18 14:54:45,712 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 54310, call mkdirs(/app/hadoop/tmp/mapred/staging/www-data/.staging, rwx------) from 127.0.0.1:40876: error: org.apache.hadoop.security.AccessControlException: Permission denied: user=www-data, access=WRITE, inode="staging":hduser:supergroup:rwxr-xr-x
org.apache.hadoop.security.AccessControlException: Permission denied: user=www-data, access=WRITE, inode="staging":hduser:supergroup:rwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:199)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:180)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:128)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5468)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5442)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:2209)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2178)
at org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:857)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
The solution I used is starting my apache server as hduser.
Related
I am trying to run R code from within PHP. I am using RHEL 7.4.
However, because Apache runs as a different user than installed the R packages, it cannot load packages:
My PHP code has this line:
echo shell_exec("(Rscript -e 'library(lme4, lib.loc = \"/var/www/R/lib\")')2>&1");
which produces this error in the browser:
Error: package or namespace load failed for 'lme4' in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/var/www/R/lib/Rcpp/libs/Rcpp.so':
/var/www/R/lib/Rcpp/libs/Rcpp.so: failed to map segment from shared object: Permission denied
Execution halted
The file /var/www/R/lib/Rcpp/libs/Rcpp.so has permissions 775, so I don't think the permissions are actually the problem.
How can I tell Apache / PHP where to look for the shared libraries?
The server is not public-facing, so I would take any solution from properly accessing the shared library, to giving user apache sudo access with NOPASSWD to run as my user (or something similarly reckless, since that didn't actually work...).
Thank you!
This appears due in part to an SELinux setting in Red Hat. I'm not entirely sure whether this is Red Hat-centric, or specific to the configuration at my university (who installed Red Hat in the first place).
Originally:
$ getenforce
Enforcing
Fix:
$ sudo setenforce 0
$ getenforce
Permissive
Persistent fix: modify /etc/selinux/config to have the line
SELINUX=Permissive
Credit to this StackOverflow thread.
I have a PHP instance on IBM Bluemix installed.
Sometimes I get the error message:
[proxy_fcgi:error] [pid 46:tid 140596635207424] (70007)The timeout
specified has expired: [client 159...***:55528] AH01075: Error
dispatching request to : (polling)
I this error occurs, the php page within the instance won’t response any more.
I use a standard php instance from IBM Bluemix.
What can cause this error and can this be solved by changing a apache of php parameter?
Seems that you have to increase TimeOut in apache configuration, or, if you'd like to have a proxy-specific timeout set ProxyTimeout.
In your local source code you have to create a folder called ".bp-config" like this to set httpd configuration parameters
.bp-config/httpd/extra/httpd-php.conf
and re-push your application on bluemix environment
> cf push your_app_name
In this case this link could be useful for you: https://serverfault.com/questions/500467/apache2-proxy-timeout
You should also try with another custom buildpack (heroku php buildpack for eample):
> cf push your_app_name -b https://github.com/heroku/heroku-buildpack-php.git
I'm working on Laravel 5 and I'm using PHP function php -S localhost:8888 folder-name - t to show the web site.
Everything is working fine until I updated to Window 10. Now, I try to run the project in my browser, I get a blank page and this message in my cmd:
[Mon Aug 03 00:17:05 2015] PHP Fatal error: Unknown: Failed opening
required 'public' (include_path='.;C:\php\pear\') in Unknown on line 0
What is going wrong?
It sounds like a permissions issue. I don't have much experiencing working with Laravel on Windows, but I just fixed a similar issue on an Ubuntu box.
The source of trouble for me was that I installed composer as root, so I had to remove ./vendor, change the owner and group of ~/.composer to ubuntu:www-data (ubuntu is my user, www-data is the Nginx user), and rerun composer install. I also made sure the ./storage permissions were recursively set to 775, and change the owner of my entire Laravel project to ubuntu:www-data.
Some stuff will definitely be different if you're on Windows, but hope this helps!
It is a permission error actually. But if you try to run directly from your localhost server, it will work.
On windows 10 PHP Server will not be able to access PHP Pear extension that is why it is showing this error.
If you are running XAMPP. you have to go to your apache localhost with http://localhost/laravel-folder/public.
I am trying to create a helloworld GAE projects to use PHP to connect to a Google Cloud SQL database. I am following the steps found at:
https://developers.google.com/appengine/docs/php/gettingstarted/helloworld
However, whenever I start the dev_appserver.py to test my helloworld.php I eventually get a permission denied error. I am running Cygwin (mintty) on a Win8 (64) machine. I have run chmod -R 777 * on a number of related directories like:
/usr/bin/
c:/Development/appengine-php-sdk-1.8.0/
c:/Development/php/
c:/Development/Php Scripts/helloworld/
Error:
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 504, in run
self.__target(*self.__args, **self.__kwargs)
File "/cygdrive/c/Development/appengine-php-sdk-1.8.0/google_appengine/google/appengine/tools/devappserver2/server.py", line 1232, in _loop_adjusting_instances
self._adjust_instances()
File "/cygdrive/c/Development/appengine-php-sdk-1.8.0/google_appengine/google/appengine/tools/devappserver2/server.py", line 1209, in _adjust_instances
self._add_instance(permit_warmup=True)
File "/cygdrive/c/Development/appengine-php-sdk-1.8.0/google_appengine/google/appengine/tools/devappserver2/server.py", line 1087, in _add_instance
expect_ready_request=perform_warmup)
File "/cygdrive/c/Development/appengine-php-sdk-1.8.0/google_appengine/google/appengine/tools/devappserver2/php_runtime.py", line 195, in new_instance
self._check_environment(php_executable_path)
File "/cygdrive/c/Development/appengine-php-sdk-1.8.0/google_appengine/google/appengine/tools/devappserver2/php_runtime.py", line 138, in _check_environment
env={})
File "/cygdrive/c/Development/appengine-php-sdk-1.8.0/google_appengine/google/appengine/tools/devappserver2/safe_subprocess.py", line 61, in start_process
stdin=subprocess.PIPE)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1248, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied
How can I determine what file the permission issue is with?
EDIT
There is no traceback from this new error (now that I am calling the actual exe):
WARNING 2013-06-04 02:21:13,243 api_server.py:329] Could not initialize images API; you are likely missing the Python "PIL" module.
INFO 2013-06-04 02:21:13,252 api_server.py:153] Starting API server at: http://localhost:51615
INFO 2013-06-04 02:21:13,261 dispatcher.py:164] Starting server "default" running at: http://localhost:8080
INFO 2013-06-04 02:21:13,324 admin_server.py:117] Starting admin server at: http://localhost:8000
ERROR 2013-06-04 02:21:13,553 php_runtime.py:199] The PHP runtime is not available because: No input file specified.
Then when I go to localhost:8080 I get this:
`The PHP interpreter specified with the --php_executable_path flag ("/cygdrive/c/Development/php/php-cgi.exe") is not compatible with the App Engine PHP development environment.
No input file specified.`
The problem is in your command line:
appengine-php-sdk-1.8.0/google_appengine/dev_appserver.py --php_executable_path=/cygdrive/c/Development/php/ /cygdrive/c/Development/PhpScripts/helloworld/
Your php_executable_path is not an executable, but a directory. So, GAE tries to run /cygdrive/c/Development/php/ as if it were a program, and gets an EACCES ("permission denied"), exactly as it should.
The fix is to pass /cygdrive/c/Development/php/php.exe (or whatever it's called) instead.
I believe your new problem is exactly what I mentioned in an early comment, and in the earlier version of this answer: You're mixing cygwin and native code in ways that don't work.
In particular, it looks like you're running a native Windows version of GAE in a cygwin Python interpreter, and then pointing it at a native Windows php-cgi interpreter. It's hard to guess exactly which one of these multiple mismatches is to blame, but it's quite likely that one of them is. So, either:
Use a native Windows Python, or
Use a cygwin/POSIX GAE and a cygwin PHP interpreter.
I need to get Dropbox's status on linux.
This is done by interacting with Dropbox using a unix socket file as IPC.
Currently, a Python script exists to do this.
I've got this code so far:
echo 'usr='. get_current_user().'<br/>';
$address='/root/.dropbox/iface_socket';
$socket=socket_create(AF_UNIX,SOCK_STREAM,0);
if(!socket_connect($socket,$address))
die('socket_connect '.socket_last_error().': '.socket_strerror(socket_last_error()));
The above works in that it does know what I want to do, but it fails with this error/output:
usr=root
socket_bind 13: Permission denied
It is interesting to note that both PHP and Dropboxd are running under the same user.
Note: I tried using PHP's fsockopen, but failed (something to do with "bad protocol"). Tried it again and this time round it works....till I get the same error below...
Edit: Again, I know running as root is bad, spare it, ok? :)
Edit 2: As I said earlier, PHP, Apache, Dropbox and this socket file are all under user "root", group "root".
However, if I run the socket under stat, I get the following:
[root#cov .dropbox]# stat iface_socket
File: `iface_socket'
Size: 0 Blocks: 0 IO Block: 4096 socket
Device: 7dh/125d Inode: 255754311 Links: 1
Access: (0600/srw-------) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-03-06 17:10:08.000000000 -0600
Modify: 2011-03-06 17:10:08.000000000 -0600
Change: 2011-03-06 17:10:08.000000000 -0600
Couldn't it be that all those -0600 is what is causing this issue? Note that if I chmod 0777 iface_socket, only the first line, (Access: (0600/srw-------)), changes; but not the other 3 underneath.
Edit 3: I was wondering, perhaps this topic would be better moved at unix/unix-like? At this point in time, it's not clear who's at fault in this issue.
Edit 4: Just ran the PHP script through strace like this:
strace php -nef /var/www/html/index.php
The relevant lines from output:
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
connect(3, {sa_family=AF_FILE, path="/root/.dropbox/iface_socket"...}, 29) = 0
fcntl(3, F_SETFL, O_RDWR) = 0
close(3) = 0
In the rare case that Apache is running under the same user as Dropbox, I would just use the python command-line interface (Debian) /usr/bin/dropbox as you normally would from a terminal.
root#DevServer1:~# dropbox help
Dropbox command-line interface
commands:
Note: use dropbox help <command> to view usage for a specific command.
status get current status of the dropboxd
help provide help
puburl get public url of a file in your dropbox
stop stop dropboxd
running return whether dropbox is running
start start dropboxd
filestatus get current sync status of one or more files
ls list directory contents with current sync status
autostart automatically start dropbox at login
exclude ignores/excludes a directory from syncing
The fronted script can only effectively be used by the user that dropbox is running under. Everybody else should get a "Dropbox isn't running!" output. In your case you should be able to manipulate dropbox how you see fit from within PHP. Personally I run Dropbox as a restricted user other then my superuser. Using groups, you can safely link in folders at will and file permissions will be enforced.
<?php
$output = shell_exec('dropbox status');
echo "<pre>$output</pre>";
Dropbox isn't running!
A viable alternative.
try this instead:
<?php
$output = shell_exec("ps aux | grep '[d]ropbox'");
echo "<pre>$output</pre>";