Running Hugo from a PHP exec() function - php

I'm attempting to setup a continuous deployment script for Hugo on a LAMP stack server, and the last snag I'm running into is getting Hugo to run.
The Background
I have Hugo and PHP installed on a VPS, and for my app, two directories:
public, where my actual site lives.
hugo, where my entire Hugo setup gets pulled into from GitHub.
I'm using a GitHub webhook to ping my deploy.php script when I merge into the master branch. Here's the relevant part of that script:
// 1. Pull the latest build from GitHub
exec('cd /path/to/hugo && git fetch --all && git reset --hard origin/master');
// 2. Run Hugo to compile any new posts and pages into HTML
exec('cd /path/to/hugo && hugo');
// 3. Copy the files in the /public directory from Hugo into the /public app directory
exec('cd /path/to/hugo && cp -r /path/to/hugo/public/. public');
What's happening
Items 1 and 3 run. Item 2 does not.
More clearly stated: the latest build pulls from GitHub, and any files I've built ahead of time are copied from the Hugo build into the /public directory.
I cannot get hugo to run via exec(), though, and I'm not entirely sure why.
If I SSH into the server in a terminal window, cd into the /path/to/hugo directory, and run hugo there, everything compiles as expected, which eliminates things like, "Maybe Hugo is installed wrong" or "Maybe Hugo isn't accessible in that directory".
So... what am I missing? Thanks in advance!

Check if this is a path issue, meaning by default, a bash session won't execute a program just because you are in the right folder: you have to specify its path (in your case: the current folder)
That means: replace && hugo with && ./hugo
Or, as the OP Chris Ferdinandi did (comments), you can use an absolute path:
I ssh-ed into the server and ran which hugo to get the appropriate path, and replaced && hugo with that.

Related

bash script creating virtual filesystem and mounting it on a specific directory

here is my problem: i would like to create a directory and limit it size using this method.
The thing is when i try it via cli its working perfectly (the file system is mounted on the newly created directory with its size limit), however when i put it in a bash script the directory is created but not with its size limit.
Here is my script.sh:
# !/bin/bash
# Set default parameters
limited_directory_name=$1
size=$2
# Setting path
parent_path="/path/to/directory/parent/"
directory_path="$parent_path$limited_directory_name"
# Creating directory/mountpoint
mkdir "$directory_path"
#Creating a file full of /dev/zero
limited_size_file="$directory_path.ext4"
touch "$limited_size_file"
dd if=/dev/zero of="$limited_size_file" bs="$size" count=1
#Formating the file
sudo mkfs.ext4 "$limited_size_file"
#Mount the disk
sudo mount -o loop,rw,usrquota,grpquota "$limited_size_file" "$directory_path"
I believe (pretty sure actually), that the problem is in these two last lines
sudo mkfs.ext4 "$limited_size_file"
or/and
sudo mount -o loop,rw,usrquota,grpquota "$limited_size_file" "$directory_path"
because as i said, the file and directory are created but just not with the size limit.
Also when i try to delete the directory ($directory_path/) after executing those command via cli i got : rm: cannot delete '$directory_path/': Device or resource busy, that i dont get when trying to delete it after executing the script. So i guess that the file system is not mounted when executing the script, and the problem is probably in the last two lines. I dont know if its has something to do with the way of using sudo inside a script or just something with mounting a file system inside a bash script.
I just wanna say that i am fairly new to bash scripting and i am sorry if my mistake is something like an obvious (noob) error. You can also say if i can improve my question in any way and i apologize if it's not clear enough.
And one last thing, i have tried different syntax for the last two line like:
sudo $(mkfs.ext4 "$limited_size_file")
or
sudo `mkfs.ext4 "$limited_size_file"`
or just
mkfs.ext4 "$limited_size_file" without sudo.
But nothing seems to work. I am using debian 10 btw and im calling the script like this in a PHP page (if it can help):
exec("myscript.sh $dname $dsize");

Simulate `cd` command with PHP

I am making a CLI-tool for work using the symfony/console composer package. (Much like laravel/installer)
The goal of this tool is to improve the daily workflow for my coworkers.
My [COMPANY] install command installs all active repositories in the current working directory. I save this directory in a configuration file on the home directory.
I now want to add a [COMPANY] cd command which should simulate an actual cd command by changing the current directory of my terminal to the install directory. So far, I have tried the following:
What I already tried
protected function handle(): void
{
$config = new Config;
$path = $config->get('directory');
if (is_null($path)) {
$this->error("It seems like you didn't install the [COMPANY] projects. Take a look at the `[COMPANY] install` command.");
exit;
}
// These options do not work because they are executed in an isolated sub-process.
chdir($path);
exec("cd $path");
shell_exec("cd $path");
$this->info("Changed working directory to $path");
}
The chdir() method only changes the working directory of the current php script. While exec() starts a completely isolated process.
Desired behaviour
~ cd ~/Development/Company
~/Development/Company company install
~/Development/Company cd ~
~ company cd
~/Development/Company
My question: Is this kind of behavior even possible with PHP? And if so, how can I achieve this.
Thanks in advance.
No, you cannot change the working directory of a terminal by running a script in it.
cd is a command built into your shell, not an external command in e.g. /bin.

git does not recognize renamed file with only a change in case [duplicate]

I have changed a few files name by de-capitalize the first letter, as in Name.jpg to name.jpg. Git does not recognize this changes and I had to delete the files and upload them again. Is there a way that Git can be case-sensitive when checking for changes in file names? I have not made any changes to the file itself.
As long as you're just renaming a file, and not a folder, you can just use git mv:
git mv -f yOuRfIlEnAmE yourfilename
(As of a change in Git 2.0.1, the -f flag in the incantation above is superfluous, but it was needed in older Git versions.)
Git has a configuration setting that tells it whether to expect a case-sensitive or insensitive file system: core.ignorecase. To tell Git to be case-senstive, simply set this setting to false. (Be careful if you have already pushed the files, then you should first move them given the other answers).
git config core.ignorecase false
Note that setting this option to false on a case-insensitive file system is generally a bad idea. Doing so will lead to weird errors. For example, renaming a file in a way that only changes letter case will cause git to report spurious conflicts or create duplicate files(from Mark Amery's comment).
Documentation
From the git config documentation:
core.ignorecase
If true, this option enables various workarounds to enable git to work better on filesystems that are not case sensitive, like FAT. For example, if a directory listing finds makefile when git expects Makefile, git will assume it is really the same file, and continue to remember it as Makefile.
The default is false, except git-clone(1) or git-init(1) will probe and set core.ignorecase true if appropriate when the repository is created.
Case-insensitive file-systems
The two most popular operating systems that have case-insensitive file systems that I know of are
Windows
OS X
Using SourceTree I was able to do this all from the UI
Rename FILE.ext to whatever.ext
Stage that file
Now rename whatever.ext to file.ext
Stage that file again
It's a bit tedious, but if you only need to do it to a few files it's pretty quick
Fix git filename case on whole repo:
git rm -r --cached .
git add --all .
git status ##Review that **only** changes staged are renames
## Commit your changes after reviewing:
git commit -a -m "Fixing file name casing"
git push origin master
Explanation from #Uriahs Victor comment:
What this command actually does is deletes the cached version of the
file/folder names that git thought still existed. So it will clear its
cache but leave everything in the current folder (where you've made
your changes locally) but it will see that those other wrong case
folders/files do not exist anymore so will show them as deleted in git
status. Then you can push up to github and it will remove the
folders/files with wrong cases. This answer has a graphic depicting
what the command means.
This is what I did on OS X:
git mv File file.tmp
git mv file.tmp file
Two steps because otherwise I got a “file exists” error. Perhaps it can be done in one step by adding --cached or such.
Sometimes it is useful to temporarily change Git's case sensitivity.
Method #1 - Change case sensitivity for a single command:
git -c core.ignorecase=true checkout mybranch to turn off case-sensitivity for a single checkout command. Or more generally: git -c core.ignorecase= <<true or false>> <<command>>. (Credit to VonC for suggesting this in the comments.)
Method #2 - Change case sensitivity for multiple commands:
To change the setting for longer (e.g. if multiple commands need to be run before changing it back):
git config core.ignorecase (this returns the current setting, e.g. false).
git config core.ignorecase <<true or false>> - set the desired new setting.
...Run multiple other commands...
git config core.ignorecase <<false or true>> - set config value back to its previous setting.
We can use git mv command. Example below , if we renamed file abcDEF.js to abcdef.js then we can run the following command from terminal
git mv -f .\abcDEF.js .\abcdef.js
Under OSX, to avoid this issue and avoid other problems with developing on a case-insensitive filesystem, you can use Disk Utility to create a case sensitive virtual drive / disk image.
Run disk utility, create new disk image, and use the following settings (or change as you like, but keep it case sensitive):
Make sure to tell git it is now on a case sensitive FS:
git config core.ignorecase false
Similar to #Sijmen's answer, this is what worked for me on OSX when renaming a directory (inspired by this answer from another post):
git mv CSS CSS2
git mv CSS2 css
Simply doing git mv CSS css gave the invalid argument error: fatal: renaming '/static/CSS' failed: Invalid argument perhaps because OSX's file system is case insensitive
p.s BTW if you are using Django, collectstatic also wouldn't recognize the case difference and you'd have to do the above, manually, in the static root directory as well
rename file Name.jpg to name1.jpg
commit removed file Name.jpg
rename file name1.jpg to name.jpg
amend added file name.jpg to previous commit
git add name.jpg
git commit --amend
I tried the following solutions from the other answers and they didn't work:
git mv filename
git rm -f filename
If your repository is hosted remotely (GitHub, GitLab, BitBucket), you can rename the file on origin (GitHub.com) and force the file rename in a top-down manner.
The instructions below pertain to GitHub, however the general idea behind them should apply to any remote repository-hosting platform. Keep in mind the type of file you're attempting to rename matters, that is, whether it's a file type that GitHub deems as editable (code, text, etc) or uneditable (image, binary, etc) within the browser.
Visit GitHub.com
Navigate to your repository on GitHub.com and select the branch you're working in
Using the site's file navigation tool, navigate to the file you intend to rename
Does GitHub allow you to edit the file within the browser?
a.) Editable
Click the "Edit this file" icon (it looks like a pencil)
Change the filename in the filename text input
b.) Uneditable
Open the "Download" button in a new tab and save the file to your computer
Rename the downloaded file
In the previous tab on GitHub.com, click the "Delete this file" icon (it looks like a trashcan)
Ensure the "Commit directly to the branchname branch" radio button is selected and click the "Commit changes" button
Within the same directory on GitHub.com, click the "Upload files" button
Upload the renamed file from your computer
Ensure the "Commit directly to the branchname branch" radio button is selected and click the "Commit changes" button
Locally, checkout/fetch/pull the branch
Done
With the following command:
git config --global core.ignorecase false
You can globally config your git system to be case sensitive for file and folder names.
Mac OSX High Sierra 10.13 fixes this somewhat. Just make a virtual APFS partition for your git projects, by default it has no size limit and takes no space.
In Disk Utility, click the + button while the Container disk is selected
Select APFS (Case-Sensitive) under format
Name it Sensitive
Profit
Optional: Make a folder in Sensitive called git and ln -s /Volumes/Sensitive/git /Users/johndoe/git
Your drive will be in /Volumes/Sensitive/
How do I commit case-sensitive only filename changes in Git?
so there are many solutions to this case sensitivity deployment problem with how GitHub handles it.
In my case, I had changed the filename casing convention from uppercase to lowercase.
I do believe that git can track the change but this command
git config core.ignorecase false dictates how git operates behind the scenes
In my case, I ran the command and git suddenly had lots of files to track labeled untracked.
I then hit git add. , then git committed and ran my build on netlify one more time.
Then all errors now displayed could be traced e.g
Module not found: Can't resolve './Components/ProductRightSide' in '/opt/build/repo/components/products and fixed such that git was able to track and implement the changes successfully.
It's quite a workaround and a fingernail away from frustration but trust me this will surely work.
PS: after fixing your issue you may want to run the command
git config core.ignorecase true to restore how git works with case sensitivity.
Also, note git config core.ignorecase false has issues with other filename extensions so you may want to watch out, do it if you know what you’re doing and are sure of it.
Here's a thread on netlify that can help out, possibly
When you've done a lot of file renaming and some of it are just a change of casing, it's hard to remember which is which. manually "git moving" the file can be quite some work. So what I would do during my filename change tasks are:
remove all non-git files and folder to a different folder/repository.
commit current empty git folder (this will show as all files deleted.)
add all the files back into the original git folder/repository.
commit current non-empty git folder.
This will fix all the case issues without trying to figure out which files or folders you renamed.
I've faced this issue several times on MacOS. Git is case sensitive but Mac is only case preserving.
Someone commit a file: Foobar.java and after a few days decides to rename it to FooBar.java. When you pull the latest code it fails with The following untracked working tree files would be overwritten by checkout...
The only reliable way that I've seen that fixes this is:
git rm Foobar.java
Commit it with a message that you cannot miss git commit -m 'TEMP COMMIT!!'
Pull
This will pop up a conflict forcing you to merge the conflict - because your change deleted it, but the other change renamed (hence the problem) it
Accept your change which is the 'deletion'
git rebase --continue
Now drop your workaround git rebase -i HEAD~2 and drop the TEMP COMMIT!!
Confirm that the file is now called FooBar.java
I took #CBarr answer and wrote a Python 3 Script to do it with a list of files:
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import os
import shlex
import subprocess
def run_command(absolute_path, command_name):
print( "Running", command_name, absolute_path )
command = shlex.split( command_name )
command_line_interface = subprocess.Popen(
command, stdout=subprocess.PIPE, cwd=absolute_path )
output = command_line_interface.communicate()[0]
print( output )
if command_line_interface.returncode != 0:
raise RuntimeError( "A process exited with the error '%s'..." % (
command_line_interface.returncode ) )
def main():
FILENAMES_MAPPING = \
[
(r"F:\\SublimeText\\Data", r"README.MD", r"README.md"),
(r"F:\\SublimeText\\Data\\Packages\\Alignment", r"readme.md", r"README.md"),
(r"F:\\SublimeText\\Data\\Packages\\AmxxEditor", r"README.MD", r"README.md"),
]
for absolute_path, oldname, newname in FILENAMES_MAPPING:
run_command( absolute_path, "git mv '%s' '%s1'" % ( oldname, newname ) )
run_command( absolute_path, "git add '%s1'" % ( newname ) )
run_command( absolute_path,
"git commit -m 'Normalized the \'%s\' with case-sensitive name'" % (
newname ) )
run_command( absolute_path, "git mv '%s1' '%s'" % ( newname, newname ) )
run_command( absolute_path, "git add '%s'" % ( newname ) )
run_command( absolute_path, "git commit --amend --no-edit" )
if __name__ == "__main__":
main()
If nothing worked use git rm filename to delete file from disk and add it back.
I made a bash script to lowercase repository file names for me:
function git-lowercase-file {
tmp="tmp-$RANDOM-$1"
git mv -f $1 $tmp
git mv -f $tmp ${1,,}
}
then you can use it like this:
git-lowercase-file Name.jpg
Or simply rename the required file over git repository web UI interface and commit :)
If you're doing more complex change, like directory name casing change,
you can make that change from a Linux machine, because Linux itself (as well as git on Linux) treats files/directories with same names but different casing as completely different files/directories.
So if you're on Windows, you can install Ubuntu using WSL, clone your repo there, open the cloned repo directory using VSCode (use WSL remote extension to access WSL Ubuntu from Windows), then you will be able to make your renames through VSCode and commit/push them using VSCode git integration.

Providing HTTP read-only access to svn checkout on a PHP server?

I have some Subversion repositories (originally created with svnadmin) on a server; there is authenticated SSH read+write access via svn+ssh://. For certain of those SVN repositories, I would like to allow an anonymous read-only access via http://. The problem is I don't have administrative properties on that server, so I cannot really mess with server setups or run svnserve, but I can have PHP scripts. So I was wondering if there is some solution, hopefully in PHP, that would allow me to do that (implement a "bridge" to a subversion repository, that the svn client could check out from)?
I'd like to compare what I want to do with git. If I do a git init in a directory, I get the subfolder .git which contains exactly the same contents of a bare repo. I can clone this bare repo with git clone --bare ... and then upload it to a server - then I can directly clone using git clone http://... and the location of the bare repo (except that at first, git will complain with fatal: ... info/refs not found: did you run git update-server-info on the server?; this means that I should enable the default post-update hook [which] runs git update-server-info to keep the information used by dumb transports (e.g., HTTP) up-to-date.; or run git update-server-info in the bare repo, so info/refs is generated - only then can this bare repo on server be cloned on client via HTTP).
So, I'd consider the svnadmin created repo (with contents dav db format hooks locks README.txt) to be equivalent to the git bare repo (as they both contain the entire history, without the actual files), so I hoped that the svnadmin repo could be setup for read-only HTTP cloning in the same way (that is, just by copying that folder contents on the server). Unfortunately, that is not so - it seems that even with HTTP access, svn actually communicates with a form of WebDAV on the server (Subversion Users: Re: dav directory does not exist; SVN RedBook: What is WebDAV?). So I tried sabre/dav out, but after a succesful plain setup (tested with cadaver DAV command line tool), I can only get svn: OPTIONS of 'http://...': 200 OK (http://...) if I point to a svnadmin repo directory (or to its dav/ subdirectory).
I guess what I want is probably not possible at the time:
Re: SVN or git via WebDav using SabreDAV - Google Groups
The SVN protocol requires a TON of extensions to plain webdav to work. You're basically out of luck here.
... but I wanted to confirm for sure with this question...
Thanks to the answer from #Evert; but unfortunately svnsync doesn't seem to help me here (it fails with "Repository moved permanently"); here is a set of commands that I run in bash on an Apache server directory, with some command responses written prefixed with #:
svn --version
# svn, version 1.6.6 (r40053)
cd /media/www
svnadmin create mytest.svnfs
svn co file:///media/www/mytest.svnfs mytest.svn
cd mytest.svn
echo aaa >> test.txt
svn add test.txt
svn commit -m "init commit"
echo bbb >> test.txt
svn add test.txt
svn commit -m "2nd com mit"
wget -q --no-check-certificate http://localhost/mytest.svn -O - | head --bytes 120
# <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
# <html>
# <head>
# <title>Index of /mytest.svn</title>
cd ..
svnadmin create mytest.mirror
cat > mytest.mirror/hooks/pre-revprop-change <<'EOF'
#!/bin/sh
USER="$3"
if [ "$USER" = "syncuser" ]; then exit 0; fi
echo "Only the syncuser user can change revprops" >&2
exit 1
EOF
chmod +x mytest.mirror/hooks/pre-revprop-change
cat > mytest.mirror/hooks/start-commit <<'EOF'
#!/bin/sh
USER="$2"
if [ "$USER" = "syncuser" ]; then exit 0; fi
echo "Only the syncuser user may commit new revisions" >&2
exit 1
EOF
chmod +x mytest.mirror/hooks/start-commit
ls --ignore="*.tmpl" mytest.mirror/hooks/
# pre-revprop-change start-commit
svnsync initialize file:///media/www/mytest.mirror http://localhost/mytest.svnfs/ --sync-username syncuser --sync-password syncpass
# svnsync: Repository moved permanently to 'http://localhost/mytest.svnfs/'; please relocate
# trying the working copy (even if it shouldn't work):
svnsync initialize file:///media/www/mytest.mirror http://localhost/mytest.svn/ --sync-username syncuser --sync-password syncpass
# svnsync: Repository moved permanently to 'http://localhost/mytest.svn/'; please relocate
I wrote that answer, and it still holds true.
In a nutshell:
The SVN server can speak webdav, Delta-V (a versioning extension for webdav)
The SVN client takes advantage of that server, but also requires svn extensions.
This was true several years ago, so the situation may have changed... but I sincerely doubt it.
However.. for what you want to do, it sounds like you just want to use svnsync.
http://svnbook.red-bean.com/en/1.7/svn.reposadmin.maint.html#svn.reposadmin.maint.tk.svnsync

Cloning git in bash script called from php webpage

I have very annoying problem here that I am completely lost on.
Am just trying to run a bash script from a php page.
The bash script is a long one.... so I created a caller.sh which calls the ./mainScript.sh to run in the background in the following:
nohup /bin/bash /home/test/customcoincode/CoinCreationBashFile.sh $coinName $coinNameAbreviation $blockReward $blockSpacing $targetTimespan $totalCoins $seedNode $nameSeedNode $headline >> /tmp/BASH2log.txt 2>&1 &
in reading my log file it seems some variables are not being passed in...
and at the following lines of code:
echo "Creating New Coin - Downloading code base repo"
echo "$localFolder/$coinName"
mkdir -p "$localFolder/$coinName";
cd "$localFolder/$coinName"
git clone "$baseRepository" "$localFolder/$coinName"
echo "Made it here 1"
i get outputs of:
Creating New Coin - Downloading code base repo
/home/test/Foocoin
cloning into '/home/test/Foocoin'
could not create directory '/var/www/.ssh'
host key verification failed
blah blah ....
Why is it looking in the /var/www/ directory?? works fine if I run the script from terminal?
many thanks
So to pack up my comments in an answer:
The shell script is now run as apache, as git uses ssh, corresponding config files are needed. Which were created in /var/www; apaches home directory. Apache did not have write permissions in /var/www thus could not create these files.
To resolve, create the /var/www/.ssh directory yourself and give www-data (or whatever user apache runs under in your system) write access to that folder.
Next, github requires you to authorize ssh keys. It is safer to create a new one for apache in the newly created /var/www/.ssh directory and add this key to your github keychain.

Categories