If I make small changes in let's say five different files and commit these to Subversion, how can I checkout just exactly these files in original folder structure to upload and overwrite on FTP Server later on?
If I try to check out HEAD non recursive I just get index.php and some different files which have nothing in common with last changes.
Attached solution:
I made a small script - not very elegant but it works ...
#!/bin/bash
URL='http://svn ...';
TARGET='./ftp';
read -p "Please enter start revision: " VERSION1
read -p "Please enter ending revision: " VERSION2
read -p "Remove old? [yes] " remove
# getting changes
svn diff $URL --summarize -r$VERSION1:$VERSION2 > changes
# checkout complete revision
svn checkout $URL -r HEAD
# remove previous
if [ $remove = 'yes'] ; then
if [ -d $TARGET ] ; then
rm -r $TARGET
else
mkdir $TARGET
fi
fi
for entry in `cat changes`; do
e=${entry#*$URL/}
item=$TARGET'/'$e;
DIR=${item%/*}
# create directory
if [ -d $DIR ] ; then
echo '';
else
mkdir -p $DIR
fi
cp './trunk/'$e $DIR
echo $e
done
echo "Done ..."
read any
It checks out the complete trunk and extracting the files which were changed in range of revisions given.
In the root of existing working copy
svn diff --summarize -rN:M
and work with second column
If you have full control of your server, you can svn checkout the particular repo path on the target server, and then run an svn update to acquire the latest files as needed.
Related
I'm trying to make a script where I can Git pull on my ubuntu server after push to Bitbucket repository. I've setup ssh keys to Bitbucket and it works to do git pull command on the repository but it doesn't work when I try it from php exec.
I've tried chmod commands like /.ssh/bitbucket_rsa like 775 and 777 and chown -R www-data:www-data/.ssh without any luck.
Response:
array (
0 => 'Host key verification failed.',
1 => 'fatal: Could not read from remote repository.',
2 => '',
3 => 'Please make sure you have the correct access rights',
4 => 'and the repository exists.',
)
Code:
public function gitPull() {
try {
exec("cd " . env("REPO_PATH") . " && git pull 2>&1", $output);
Log::info($output);
} catch (\Exception $e) {
Log::error($e);
}
http_response_code(200);
}
I guess you are stuck with the fact that the user www-data can not establish the SSH connection to the git server. I think the simplest was is to create a home directory for the www-data user and create a .ssh directory with the proper permissions, a config file and the key file in there. You could always test the setup as root with
# su - www-data
$ cd <to your repository>
$ git pull
Google for "SSH connections without password" to set it up correctly. And also be aware that SSH refuses to use a key file if the permissions are to loose.
Host key verification failed.
means that ssh could not verify the host key, most likely because there's no known_hosts file in www-data's home/.ssh directory that contains the expected host key for your repo's server.
There's at least two ways to fix that:
Use ssh-keyscan as described over on Serverfault.se:
ssh-keyscan -H [hostname] >> /path/to/www-data's_home_directory/.ssh/known_hosts
You only need to do that once (unless the key changes), but you should check that the key is indeed correct after you run ssh-keyscan.
Set the GIT_SSH_COMMAND environment variable before running git. You can use this to have ssh use a different known_hosts file:
export GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/path/to/known_hosts"
Note that the above assumes shell syntax (e.g. Bash), you may need to adjust for PHP, particularly the export GIT_SSH_COMMAND= part.
I stack with the same problem working with github:
ssh-keyscan -t rsa github.com | tee github-key-temp | ssh-keygen -lf -
cat github-key-temp >> ~/.ssh/known_hosts
cat github-key-temp >> /etc/ssh/ssh_known_hosts
But that is not all, with next command you can check what is goes wrong (run it throught exec or shell_exec (save out put to some log):
ssh -vT git#github.com 2>&1
So, with help of privious command, i understand that in my case: cron run's command via php script, but duaring ssh connection it could not find my keysfile (i have custom name for that file):
cd /etc/ssh/ssh_config.d/
sudo touch <some_name>.conf
sudo echo 'IdentityFile ~/.shh/<custom_key_file_name>' > <some_name>.conf
Or try to add full path to location of your keyfile (~/ = current user home dir).
You can check cron user by runing, this can helps to:
shell_exec('whoami');
P.S. I have no idea if this solution is enough secure. but i think fine.
I have a file upload form and after the file uploads I want to push the files up to GitHub by running:
git add .
git commit -m "some message"
git push origin master
How do I go about this? I've seen examples of using exec() but that makes me nervous.
shell_exec('cd /var/www/vhost/xxx.com/httpdocs/clients/portal/upoads/54 && /usr/bin/git add -A');
shell_exec('cd /var/www/vhost/xxx.com/httpdocs/clients/portal/upoads/54 && /usr/bin/git commit -m "something 1"');
shell_exec('cd /var/www/vhost/xxx.com/httpdocs/clients/portal/upoads/54 && /usr/bin/git push origin master');
Those commands don't error but don't work either. Do I need to grant access to the apache user to use the ssh key?
i guess is permission problems, you can use exec() , and get the error info by $output
exec($your_command.' 2>&1', $output, $return_var);
var_dump($output);
Do I need to grant access to the apache user to use the ssh key?
Yes.
This means you have to copy the key somewhere that the apache user can read it. SSH won't work unless the key file is readable by the user only (i.e. 0600 permissions on the key file).
Copy the key like:
mkdir -p --mode=0700 ~apache/.ssh
cp /my/id_rsa ~apache/.ssh/id_rsa
chown -R apache:apache ~apache/.ssh/id_rsa
chmod 0600 ~apache/.ssh/id_rsa
Also, you don't need to cd every time you want to run the command. Use GIT_DIR:
putenv('GIT_DIR=/path/to/git/repo')
shell_exec('git commit ...')
I solved it. I ran all of this as root user.
Inside my PHP script I ran
exec("whoami");
to get the user that is running that script. Then I ran
cat /etc/passwd
to get the home directory for that user (/var/www/vhost/mydomain.com)
I noticed that on my web server (Centos 7) that all my web files were chown'd as opcode:psacln so I created a .ssh directory inside opcode's home folder:
mkdir -p --mode=0700 /var/www/vhost/mydomain.com/.ssh
cd (back to root)
cp .ssh/id_rsa /var/www/vhost/mydomain.com/.ssh/id_rsa
chown -R opcode:psacln /var/www/vhost/mydomain.com/.ssh/id_rsa
chmod 0600 /var/www/vhost/mydomain.com/.ssh/id_rsa
The thing I was missing was that I had to also move my known_hosts file over, since the script I was using wasn't adding to it.
cp .ssh/known_hosts /var/www/vhost/mydomain.com/.ssh/known_hosts
chmod 0600 /var/www/vhost/mydomain.com/.ssh/known_hosts
Of course, I had to login to my server at the command line and do an initial commit to the repo in order to get it added to my known_hosts file, before I copied it over. Hope this helps someone.
I have a Gitlab server (Ubuntu 14.04) where I am trying use it as both a host for my repositories as well as a testing server for my PHP projects. Ideally, I would like to have Gitlab/Git export the "release" branch to /var/www/git/<project-name> when that branch is updated.
My Question: How can I export a specific branch in Gitlab, to a specific directory on the localhost, when the branch is updated?
I am aware that there are webhooks available in Gitlab, but it seems unnecessary and wasteful to have the server POST to itself for a local operation.
I suppose you are running the community edition of gitlab.
Then, only the server administrator can configure hook scripts by copying the required scripts into the affected repositories.
gitlab itself is using the $GIT_DIR/hooks directory for its own scripts already. Fortunately they forward control to any hook script in the gitlab specific $GIT_DIR/custom_hooks directory. See also this question about how to run multiple hooks with the same type on gitlab.
The script itself could look like this:
#!/bin/bash
#
# Hook script to export current state of repo to a release area
#
# Always hardcode release area - if configured in the repo this might incur data loss
# or security issues
echo "Git hook: $0 running"
. $(dirname $0)/functions
git=git
release_root=/gitlab/release
# The above release directory must be accessible from the gitlab server
# and any client machines that want to access the exports. Please configure.
if [ $(git rev-parse --is-bare-repository) = true ]; then
group_name=$(basename $(dirname "$PWD"))
repo_name=$(basename "$PWD")
else
cd $(git rev-parse --show-toplevel)
group_name=$(basename $(readlink -nf "$PWD"/../..))
repo_name=$(basename $(readlink -nf "$PWD"/..))
fi
function do_release {
ref=$1
branch=$2
# Decide on name for release
release_date=$(git show -s --format=format:%ci $ref -- | cut -d' ' -f1-2 | tr -d -- -: | tr ' ' -)
if [[ ! "$release_date" =~ [0-9]{8}-[0-9]{6} ]]; then
echo "Could not determine release date for ref '$ref': '$release_date'"
exit 1
fi
dest_root="$release_root/$group_name/$repo_name"
dated_dir="dated/$release_date"
export_dir="$dest_root/$dated_dir"
# Protect against multiple releases in the same second
if [[ -e "$export_dir" ]]; then
export_dir="$export_dir-02"
dated_dir="$dated_dir-02"
while [[ -e "$export_dir" ]]; do
export_dir=$(echo $export_dir | perl -pe 'chomp; print ++$_')
dated_dir=$(echo $dated_dir | perl -pe 'chomp; print ++$_')
done
fi
# Create release area
if ! mkdir -pv "$export_dir"; then
echo 'Failed to create export directory: ' "$export_dir"
exit 1
fi
# Release
if ! git archive $branch | tar -x -C "$export_dir"; then
echo 'Failed to export!'
exit 1
fi
chmod a-w -R "$export_dir" # Not even me should change this dir after release
echo "Exported $branch to $export_dir"
( cd "$dest_root" && rm -f latest && ln -s "$dated_dir" latest )
echo "Adjusted $dest_root/latest pointer"
}
process_ref() {
oldrev=$(git rev-parse $1)
newrev=$(git rev-parse $2)
refname="$3"
set_change_type
set_rev_types
set_describe_tags
echo " Ref: $refname","$rev_type"
case "$refname","$rev_type" in
refs/heads/*,commit)
# branch
refname_type="branch"
function="branch"
short_refname=${refname##refs/heads/}
if [[ $short_refname == release ]]; then
echo " Push accepted. Releasing export for $group_name/$repo_name $short_refname"
do_release "$refname" "$short_refname"
else
echo " Push accepted. No releases done for $group_name/$repo_name $short_refname"
fi
;;
refs/tags/*,tag)
# annotated tag
refname_type="annotated tag"
function="atag"
short_refname=${refname##refs/tags/}
;;
esac
}
while read REF; do process_ref $REF; done
exit 0
The script was started based on this post-receive.send_email script which is already quoted on SO multiple times.
Configure a release area in the variable hardcoded in the script, or e.g. add a mechanism to read a config file in the repo. Maybe you want to give users control over this area. Depends on your security circumstances.
The release area must be accessible by the git#gitlab user, and of course by any client expecting the export.
The branch to export is hardcoded in the script.
The release area will be populated like this:
$release_root/$group_name/$repo_name/dated/$release_date
Plus a symbolic link latest pointing to the latest $release_date. The idea is that this is extensible to later be able to also export tags. If you expect to export different branches, a $branch should be included as a path component, too.
Access control of the gitlab server is not passed down to the directory structure. Currently I do this manually, and that is why I do not auto-populate all new repositories with this hook. I'd rather configure manually, and then adjust unix group permissions (and/or ACLs) on the $release_root/$groupname paths accordingly. This needs to be done only once per group and works because no one else is allowed to create new groups on my gitlab instance. This is very different from the default.
Anything else we can do for you? ;-)
I have very annoying problem here that I am completely lost on.
Am just trying to run a bash script from a php page.
The bash script is a long one.... so I created a caller.sh which calls the ./mainScript.sh to run in the background in the following:
nohup /bin/bash /home/test/customcoincode/CoinCreationBashFile.sh $coinName $coinNameAbreviation $blockReward $blockSpacing $targetTimespan $totalCoins $seedNode $nameSeedNode $headline >> /tmp/BASH2log.txt 2>&1 &
in reading my log file it seems some variables are not being passed in...
and at the following lines of code:
echo "Creating New Coin - Downloading code base repo"
echo "$localFolder/$coinName"
mkdir -p "$localFolder/$coinName";
cd "$localFolder/$coinName"
git clone "$baseRepository" "$localFolder/$coinName"
echo "Made it here 1"
i get outputs of:
Creating New Coin - Downloading code base repo
/home/test/Foocoin
cloning into '/home/test/Foocoin'
could not create directory '/var/www/.ssh'
host key verification failed
blah blah ....
Why is it looking in the /var/www/ directory?? works fine if I run the script from terminal?
many thanks
So to pack up my comments in an answer:
The shell script is now run as apache, as git uses ssh, corresponding config files are needed. Which were created in /var/www; apaches home directory. Apache did not have write permissions in /var/www thus could not create these files.
To resolve, create the /var/www/.ssh directory yourself and give www-data (or whatever user apache runs under in your system) write access to that folder.
Next, github requires you to authorize ssh keys. It is safer to create a new one for apache in the newly created /var/www/.ssh directory and add this key to your github keychain.
Just a Question Regarding unix and PHP today.
What I am doing on my PHP is using the Unix system to untar a tarred file.
exec("tar -xzf foo.tar.gz");
Generally everything works fine until I run into this particular foo.tar.gz, which has a file system as follows:
Applications/
Library/
Systems/
After running the tar command, it seems that the file permissions get changed to 644 (instead of 755).
This causes Permission denied (errno 13) and therefore disabling most of my code. (I'm guessing from lack of privileges)
Any way I can stop this tar command completely ruining my permissions?
Thanks.
Oh and this seems to only happen when I have a foo.tar.gz file that Has this particular file system. Anything else and I'm good.
If you want to keep the permissions on files then you have to add the -p (or --preserve-permissions or --same-permissions) switch when extracting the tarball. From the tar man pages :
--preserve-permissions
--same-permissions
-p
When `tar' is extracting an archive, it normally subtracts the
users' umask from the permissions specified in the archive and
uses that number as the permissions to create the destination
file. Specifying this option instructs `tar' that it should use
the permissions directly from the archive.
So PHP code should be :
exec("tar -xzfp foo.tar.gz");
Edit: --delay-directory-restore solved the problem below about being unable to untar a file. The permissions of pwd are still altered, so the problem of the original poster might not be solved.
Not really an answer, but a way to reproduce the error.
First create some files and directories. Remove write access to the directories:
mkdir hello
mkdir hello/world
echo "bar" > hello/world/foo.txt
chmod -w hello/world
chmod -w hello
Next, create the tar file from within the directory, preserving permissions.
cd hello
tar -cpf ../hw.tar --no-recursion ./ world world/foo.txt
cd ..
Listing the archive:
tar -tvf hw.tar
# dr-xr-xr-x ./
# dr-xr-xr-x world/
# -rw-r--r-- world/foo.txt
So far, I've been unable to untar the archive as a normal user due to the "Permission denied"-error. The archive can't be untarred naively. The permissions of the local directory change as well.
mkdir untar
cd untar
ls -ld .
# drwxr-xr-x ./
tar -xvf ../hw.tar
# ./
# world/
# tar: world: Cannot mkdir: Permission denied
# world/foo.txt
# tar: world/foo.txt: Cannot open: No such file or directory
# tar: Exiting with failure status due to previous errors
ls -ld .
# dr-xr-xr-x ./
Experimenting with umask and/or -p did not help. However, adding --delay-directory-restore does help untarring:
tar -xv --delay-directory-restore -f ../hw.tar
# ./
# world/
# world/foo.txt
ls -ld .
# dr-xr-xr-x ./
chmod +w .
It is also possible to untar the file as root. What suprised me most is that tar apparently can change the permissions of pwd, which is still unsolved.
By the way, I originally got into this problem by creating a tarball for / with
tar -cvpzf backup.tar.gz --exclude=/backup.tar.gz --one-file-system /
as root (pwd=/) and untarring it as a normal user to create a linux container.