I was trying the Perfect Workflow, with Git, GitHub, and SSH, and i have everything set up, except running the command git pull from php.
When i run exec('git pull') i get:
Could not create directory '/.ssh'. Host key verification failed.
fatal: The remote end hung up unexpectedly
If i run it in the terminal (as root) it works just fine, but i need this hook to work from the Post-Receive URL (Github).
If i do exec('whoami') i get apache.
It's a (dv) from mediatemple with CentOS.
If you want apache (the user) to be able to pull from git, you'll have to create an ssh key for apache, then add that to the read only keys on github.
The flow is something like this (tweak to your needs)
usermod -s /bin/bash apache
su apache
cd ~
ssh-keygen # work through the keygen dance (added a dash)
Upload (tilde here refers to apache's homedir) ~/.ssh/id_rsa.pub to github and give apache access to whichever repos it needs to pull from.
Then you can test on the server by again su'ing to apache and running the git pull
su apache
cd ~/working-copy
git clone my-project
Once that's working you should be able to run a git pull through PHP.
Related
I got a working php-fpm docker container acting as the php backend to a nginx frontend. What I mean by working, is that it renders phpinfo output in the browser as expected.
My php-fpm container was produced by php-fpm-7.4 prod of the devilbox docker repo. It has OCI8 enable.
The issue: I keep getting ORA-28547 when trying oci_connect
What I have done:
1--add /usr/lib/oracle/client64/lib to a file inside ld.so.conf.d and run ldconfig -v
2--restart docker container.
3-- Now phpinfo shows ORACLE_HOME=/usr/lib/oracle/client64/lib
4--Add tnsnames.ora to /usr/lib/oracle/client6/lib/network/admin (there is a README.md file inside that folder that even tells you to do that)
5--Restart docker container again.
6-oci_connect still fails with the same error.
What I am missing?
Thank you very much for any pointers, I think I have browsed to the end of the internet and back without finding a solution yet.
----SOLUTION: reinstall instantclient, relink libraries (ldconfig) to use new instantclient libraries. Create modified dockerfile to do it when container is created.
I modified the Dockerfile file of the php-fpm to add new instant client files and not the one that were provided by the original file. I was not able to make it work with them. I have tried a few times rebuilding the image (docker-compose up --build) and this is the file that does the trick:
FROM devilbox/php-fpm:7.4-work
#instantclient.conf content: /opt/instantclient
RUN echo "/opt/instantclient" >/etc/ld.so.conf.d/instantclient.conf
WORKDIR /opt
RUN wget https://download.oracle.com/otn_software/linux/instantclient/19800/instantclient-sdk-linux.x64-19.8.0.0.0dbru.zip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/19800/instantclient-sqlplus-linux.x64-19.8.0.0.0dbru.zip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/19800/instantclient-basic-linux.x64-19.8.0.0.0dbru.zip
RUN unzip instantclient-sdk-linux.x64-19.8.0.0.0dbru.zip
RUN unzip instantclient-sqlplus-linux.x64-19.8.0.0.0dbru.zip
RUN unzip instantclient-basic-linux.x64-19.8.0.0.0dbru.zip
RUN mv instantclient_19_8 instantclient
ADD tnsnames.ora /opt/instantclient/network/admin
RUN ldconfig -v
CMD ["php-fpm"]
expose 9000
# Insert following to .bash_profile or .profile of the User starting the php-fpm
export ORACLE_HOME=/usr/lib/oracle/client64
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib
export PATH=$PATH:$ORACLE_HOME/bin
export TNS_ADMIN=$ORACLE_HOME/network/admin
# Test to Ping Remote Db to be connected by PHP
tnsping <tns-name of remote DB - i.e. db12c.world>
# restart here the php Engine
Can you please check
https://github.com/caffeinalab/php-fpm-oci8/blob/master/Dockerfile
which seems to create a p-fpm-oci8 docker image
the "wget" for
wget -qO- https://raw.githubusercontent.com/caffeinalab/php-fpm-oci8/master/oracle/instantclient-basic-linux.x64-12.2.0.1.0.zip | bsdtar -xvf- -C /usr/local &&
wget -qO- https://raw.githubusercontent.com/caffeinalab/php-fpm-oci8/master/oracle/instantclient-sdk-linux.x64-12.2.0.1.0.zip | bsdtar -xvf- -C /usr/local &&
wget -qO- https://raw.githubusercontent.com/caffeinalab/php-fpm-oci8/master/oracle/instantclient-sqlplus-linux.x64-12.2.0.1.0.zip | bsdtar -xvf- -C /usr/local && \
can be dropped when you place downloaded instant client files into local host dir
/usr/local
and extract them - resulting in
/usr/local/instantcient_12_2
or 18, 19c equivalents
the 4 "ln" commands have to be adjusted to reflect the local host instantclient dir
the tnsnames.ora for instantclient is available from host by VOLUME command
-------------FINAL SOLUTION------------(it was not network related, I had done a couple of changes to the files, and also tried a different database, all at the same time, so it made me think that it was the different database what fixed the issue)
After many trial and errors, I came up with a Dockerfile that creates the correct configuration of files and connects without any issues to the database:
--Dockerfile: (to build php-fpm 7.4 using devilbox image)
Final solution:
I modified the Dockerfile file of the php-fpm to add new instant client files and not the one that were provided by the original file. I was not able to make it work with them. I have tried a few times rebuilding the image (docker-compose up --build) and this is the file that does the trick:
FROM devilbox/php-fpm:7.4-work
ADD instantclient.conf /etc/ld.so.conf.d/
WORKDIR /opt
RUN wget https://download.oracle.com/otn_software/linux/instantclient/19800/instantclient-sdk-linux.x64-19.8.0.0.0dbru.zip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/19800/instantclient-sqlplus-linux.x64-19.8.0.0.0dbru.zip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/19800/instantclient-basic-linux.x64-19.8.0.0.0dbru.zip
RUN unzip instantclient-sdk-linux.x64-19.8.0.0.0dbru.zip
RUN unzip instantclient-sqlplus-linux.x64-19.8.0.0.0dbru.zip
RUN unzip instantclient-basic-linux.x64-19.8.0.0.0dbru.zip
RUN mv instantclient_19_8 instantclient
ADD tnsnames.ora /opt/instantclient/network/admin
RUN ldconfig -v
CMD ["php-fpm"]
expose 9000
That's why I have suggested to use tnsping - unfortunaly it is not included in any of the instant client files which is a pity - so you have to pick it up from regular client with matching OS, bitsize and Oracle release. As workaround you could place SQL*Plus package files into container and try to connect with a foo user like
sqlplus foo/foo#\<ip>:\<port>/\<dbname>
which should generate an error - if
user/password not matching - ORA-1017 i.e. DB & listener running
listener running - ORA-1034 i.e. DB down
listener down (no return, or TNS-Errors)
I got it!. It was a firewall issue. I launched a tcpdump capture
session and there was nothing wrong with php-fpm, oci8 and
instantclient libraries. The traffic was initiated but there was no
response from the database. I made it work against a different
database where this box has no firewall issues.
I now will try rebuilding the docker image so I can see what I have to
manually add if any.
That was incorrect (the firewall as the origin of the problem). Rebuilding the docker file showed me where I had it wrong. See original question for solution.
What I am trying to do
I have a git repository on bitbucket. After pushing to the repository from my local machine I want to automatically pull the master branch to my webspace.
What I have done so far
I connected to my server using ssh, created the ssh key and registered the public key on github.
I created a .sh script which pulls the master branch using ssh - so far so god - everything works when I run the script from the command line/putty
What is the problem
I want to trigger the .sh script with a webhook on bitbucket (I can give an url). For that purpose I created a .php file in my webspace:
<?php
$output = shell_exec('./deploy.sh 2>&1');
echo $output;
my .sh script looks like this:
#!/bin/bash
git pull git#bitbucket.org:dualmeta/test.git master
As already said, running the .sh script with putty works perfectly fine. However if I enter the url to the .php file in my browser it gives me an error:
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I already did some reserach and found that many people have the exact same problem. However in my case i do not have root/sudo access because it is a rented webspace and not my own vServer.
Is there any chance getting this to work?
You must add access www-data or apache user to your git directory.
chown -R apache:apache git_directory
or
chown -R www-data:www-data git_directory
or
chmod o+rw -R git_directory
Use this too :
git config credential.helper store
I have set up ssh keys properly and added them to my github account . Whenever I ssh into the server and run git pull , everything runs normally and it pulls changes from the repository . However I have a deploy script that runs git pull via shell_exec() but it returns this error;
origin git#github.com:sayopaul/autodeploy-tutorial.git (fetch)
origin git#github.com:sayopaul/autodeploy-tutorial.git (push)
ERROR: Repository not found.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
PHP (the webserver) likely doesn't run as the same user you use when you SSH into the server. Thus, it doesn't have access/permission / doesn't use the correct SSH keys to authenticate vs GitHub.
I can think of 2 easy solutions:
Utilize sudo:
Add this rule in the sudo-conf (sudo visudo) to allow the user www-data to run (only) /usr/bin/git as yourotheruser:
www-data ALL=(yourotheruser) NOPASSWD: /usr/bin/git
Now you can invoke git using:
sudo -u yourotheruser git pull
Security advise: To limit the potential damage done if someone manages to execute arbitrary code through www-data:
Create a script owned by yourotheruser (and not writeable by others), e.g. /home/yourotheruser/deploy.sh with the contents:
cd /path/to/repo
git pull
And allow the sudo access only to this script. This way, no other git action than pull in the intended directory can be performed.
Change the user PHP itself is executed with:
Use php-fpm
Use the ITK MPM
I have create a webhook in my github repository which post on the hook url on my live server to run pull command for update my repo files on the server.
The problem is the hook file which i have created is in the /var/www/site/web/hookfile.php (the post request is going there. i am getting the body response also)
and my repo files are in /var/www/git-repo/
its not updating the git-repo when i push anything to my github repository.
I run this command using terminal and its working.
cd /var/www/git-repo && git pull
But through my php file its not working
shell_exec('cd /var/www/git-repo && git pull')
shell_exec() fail silently because only report STDOUT and not STDERR.
Try with:
echo shell_exec("cd /var/www/git-repo && /full/path/to/bin/git pull 2>&1");
Normally is a permission error, and could be fixed adding permission to the user that execute php (apache?)
chown -R www-agent:www-agent repository/
But could be also a connection error to the remote repository (authentication, ssh-keys, ...).
First of all in your php file run a test against your server instance to get any error messages output on screen because the exec() family of functions simply fail silently and only report STDOUT and not STDERR:
echo shell_exec("cd /website/root/htdocs && git checkout . && git status 2>&1");
In my case this threw an error that it could not find git command due to lack of binary path defined for apache user. Therefore, a full path needs to be provided to git's binary. It can be obtained by finding it manually or running in shell:
'which git'
It returned (further called YOU_FULL_GIT_BINARY_PATH_HERE):
/usr/local/git/bin/git
A full path with git command e.g. '/usr/local/git/bin/git status' now runs git commands nicely.
Another thing is to ensure your web server user has enough permissions to read/write to your repo folder/files. I have set mine to be owned by the apache user (Centos 6.8; other releases might be www:www or www-data:www-data etc.):
chown -R apache:apache YOUR_WEB_OR_REPO_FOLDER
In order to ensure any newly added files inherit correct permissions run:
chmod -R g+s YOUR_WEB_OR_REPO_FOLDER
The above should get your script to run commands now. Though it doesn't overcome git password prompt to use 'git pull' command for a git user set in YOUR_WEB_OR_REPO_FOLDER/.git/config file. Running below command inside repo:
git config credential.helper store
command will prompt for password and let you store it locally. Please note your stored password will be unencrypted and protected only by file system e.g. in /root/.git-credentials. This will allow to run 'git pull' without prompting for password.
It's not ideal for my fully automated continuous integration environment deploying test VPS on demand as it requires to manually enter git user (defined in repo's .git/config git) password at least once.
Since my environment should always run on code from remote's origin/master copy I am also running
/YOU_FULL_GIT_BINARY_PATH_HERE/git checkout .
before invoking 'git pull' to ensure any local changes are lost forever alternatively do a hard reset instead using:
/YOU_FULL_GIT_BINARY_PATH_HERE/git fetch origin
/YOU_FULL_GIT_BINARY_PATH_HERE/git reset --hard origin/master
I'm trying to use a post-receive hook to update a remote server. I'm using ssh and everything works great (running git pull does indeed pull). I've started with this deploy.php and the other commands, like git status, do run and output as expected, but the pull doesn't appear to do anything at all.
I've chowned the hell out of everything (the git repo, known_hosts, etc...), so my www-data user should be able to access just about anything on the sever. Is there any way I can at least find some error logging for what's going wrong?
edit: I don't see any change in the output after changing git pull to git pull --verbose as suggested by adder. The output is still:
$ whoami
www-data
$ git pull --verbose
$ git status
# On branch master
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# deploy.php
nothing added to commit but untracked files present (use "git add" to track)
Solution: the apache user needs to be setup to ssh to github. I was connecting as root then getting confused when my post-receive hook target couldn't make the same update. I was doing it as root, the script was running as www-data.
git help pull
--verbose
Pass --verbose to git-fetch and git-merge.