I am configuring Jenkins in order to make it build my project.
It checks out source code, installs all required libs etc..
My application configuration is stored in params.php file (Yii 1.x style), it looks the following way:
return array(
'dbHost' => 'dbHostValuePlaceholder',
'dbPasssword' => 'dbPasswordValuePlaceholder',
/* other values go here */
);
Obviously I need to replace dbPasswordValuePlaceholder with password that I set when starting parametrised build.
Current solution I've found is running this as shell command:
sed -i s/dbPasswordValuePlaceholder/$DB_PASSWORD/g $WORKSPACE/protected/config/params.php
where $DB_PASSWORD is corresponding variable.
This solution work perfectly with simple strings, but password may look like a1#2";$&^*&-=+/*-:'/.,,,,,,,,^^^^^ - in this case sed fails because of unescaped characters.
I have searched for different Jenkins plugins, but there are only plugins that work with other formats like .xml etc, and neither works with .php files.
I need to avoid moving my config values out of .php file to some other format.
Is there a way to replace the described tokens in .php file with build parameters values ?
UPDATE
As #Stanjer recommended, I have tried to set param value to:
a1#2";$&^*&-=+/*-:\'/.,,,,,,,,^^^^^
So, in this case, the following code in PHP would be right, valid:
'universalPassword' => 'a1#2";$&^*&-=+/*-:\'/.,,,,,,,,^^^^^',
It does not help, here is output:
[build.int] $ /bin/sh -xe /tmp/hudson642312569695847678.sh
+ sed -i s/defaultUniversalPassword/a1#2";$&^*&-=+/*-:\'/.,,,,,,,,^^^^^/g /var/www/build.int/protected/config/params.php
sed: -e expression #1, char 43: unknown option to `s'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Thanks for the updated question.
In your case you should show that it's already escaped by providing it as $'string'.
So the command will look like:
$ sed s#dbPasswordValuePlaceholder#$'"a1#2";$&^*&-=+/*-:\'/.,,,^^^"'#g test.php
and output will be:
<?php
return array(
'dbHost' => 'dbHostValuePlaceholder',
'dbPassword' => '"a1#2";$dbPasswordValuePlaceholder^*dbPasswordValuePlaceholder-=+/*-:'/.,,,,,,,,^^^^^"',
/* other values go here */
);
?>
If using with variable:
export NEWPASS=$'"a1#2";$&^*&-=+/*-:\'/.,,,^^^"'
sed "s#dbPasswordValuePlaceholder#$NEWPASS#g" test.php
I am not sure about any jenkins plugin which works that way , but I needed to know if its possible to make sed ignore the special characters using this :
sed with special characters
And I suppose you must be using a Password Parameter with your parameterized job, which hides the password.
Related
I just cannot fathom how to get the PHP exec() or shell_exec() functions to treat a '*' character as a wildcard. Is there some way to properly encode / escape this character so it makes it through to the shell?
This is on windows (via CLI shell script if that matters, Terminal or a git-bash yields the same results).
Take the following scenario:
C:\temp\ contains a bunch of png images.
echo exec('ls C:\temp\*');
// output: ls: cannot access 'C:\temp\*': No such file or directory
Permissions is not the problem:
echo exec('ls C:\temp\exmaple.png');
// output: C:\temp\example.png
Therefore the * character is the problem and is being treated as a literal filename rather than a wildcard. The file named * does not exist, so from that point of view, it's not wrong...
It also does not matter if I use double quotes to encase the command:
echo exec("ls C:\temp\*");
// output: ls: cannot access 'C:\temp\*': No such file or directory
I have also tried other things like:
exec(escapeshellcmd('ls C:\temp\*'));
exec('ls C:\temp\\\*');
exec('ls "C:\temp\*"');
exec('ls "C:\temp\"*');
And nothing works...
I'm pretty confused that I cannot find any other posts discussing this but maybe I'm just missing it. At this point I have already worked around the issue by manually programming a glob loop and using the internal copy() function on each file individually, but it's really bugging me that I do not understand how to make the wildcard work via shell command.
EDIT:
Thanks to #0stone0 - The answer provided did not particularly answer my initial question but I had not tried using forward slashes in the path and when I do:
exec('ls C:/temp/*')
It works correctly, and as 0stone0 said, it only returns the last line of the output, which is fine since this was just for proof of concept as I was not actually attempting to parse the output.
Also, on a side note, since posting this question my system had been updated to Win11 22H2 and now for some reason the original test code (with the backslashes) no longer returns the "Cannot access / no file" error message. Instead it just returns an empty string and has no output set to the &$output parameter either. That being said, I'm not sure if the forward slashes would have worked on my system prior to the 22H2 update.
exec() only returns the last output line by default.
The wildcard probably works, but the output is just truncated.
Pass an variable by ref to exec() and log that:
<?php
$output = [];
exec('ls -lta /tmp/*', $output);
var_dump($output);
Without any additional changes, this returns the same as when I run ls -lta /tmp/* in my Bash terminal
That said, glob() is still the preferred way of getting data like this especcially since
You shouldn't parse the output of ls
I have already got lint testing and code standards checking but I've like to go one further and add a hotkey to change all the code to a certain standard.
I have so far got as far as...
:r ! phpcbf --standard=psr2 %
But that only pulls the document in. So how can I make is just act like a filter and replace the entire script?
As described in :help filter, the general format for filtering content through an external program is
:{range}!{filter} [arg]
The expectation is that the filter command reads input on stdin and writes it to stdout.
For your tool, this likely translates to :%!phpcbf --standard=psr2.
Assuming your command can take input from stdin you would do the following:
:%!phpcbf --standard=psr2
Please ready :h filter
I have a CakePHP database.php config file that I'd like to leverage to open a command line mysql client instance.
The goal would be to be able to run a shell script like ./db-login.sh from the project root and have it pull the database details (host, port, database name, username, password) from the PHP config file and pass them to the mysql command line as arguments. This is to avoid having to enter the details every time.
I understand it's possible to create a shell alias that has the values hard-coded: I would like a portable script that could be included with any of my Cake projects. I also would like to keep the task of getting the DB credentials into bash variables separate from launching the mysql client. This would open up the ability to re-use the DB credentials in other shell scripts easily (such as a mysqldump backup script.)
Here's what I have so far:
database.php
Consider this file immutable for the purposes of this question. It must exist exactly as you see it.
<?php
class DATABASE_CONFIG {
public $default = array(
'host' => 'localhost',
'login' => 'cakephpuser',
'password' => 'c4k3 roxx!',
'database' => 'my_cakephp_project',
);
}
db-cred.sh
Acts as middleware to convert the PHP variables into bash (friendly) variables.
#!/usr/bin/env php
<?php
include 'database.php';
$db = new DATABASE_CONFIG();
// Selectively wrap quotes. Has no real effect(?)
$p = (!empty($db->default['password']) ? "\"$db->default['password']\"" : '');
echo <<<EOD
DB_HOST="{$db->default['host']}"
DB_NAME="{$db->default['database']}"
DB_USER="{$db->default['login']}"
DB_PASS={$p}
EOD;
db-login.sh
#!/usr/bin/env bash
# Uses Cake's database.php file to log into mysql on the
# command line with the correct arguments.
# Holy yuck, but nothing else works!
eval $( db-cred.sh )
# Try to set an appropriate password clause.
PATTERN=" |'"
if [ -n "$DB_PASS" ]; then
if [[ $DB_PASS =~ $PATTERN ]]; then
PASS_CLAUSE=" -p'${DB_PASS}'"
else
PASS_CLAUSE=" -p${DB_PASS}"
fi
else
PASS_CLAUSE=""
fi
# Get a rough look at what we're about to run.
echo mysql --host=${DB_HOST} --database=${DB_NAME} --user=${DB_USER}${PASS_CLAUSE}
# Call MySQL.
mysql --host=${DB_HOST} --database=${DB_NAME} --user=${DB_USER}${PASS_CLAUSE}
Things that need help:
The db-login.sh script uses eval to suck in the variables. Yuck.
The MySQL password is leaked on the command line. This is not a dealbreaker, but if there's a clean/portable way to avoid it, I'm open to it.
Spaces and quotes in the mysql password don't get passed to bash properly.
A password like my Pass will cause DB_PASS to be set to my in db-login.sh.
Quoting the string in db-cred.sh produces "my instead.
Likewise, my attempts at quoting PASS_CLAUSE are not effective: Only passwords without spaces/quotes will successfully log into the mysql client.
The db-login.sh script uses eval to suck in the variables. Yuck.
Instead of eval, you can use the source command:
source db-cred.sh
The MySQL password is leaked on the command line. This is not a dealbreaker, but if there's a clean/portable way to avoid it, I'm open to it.
You can just unset the MySQL password after it is used:
unset DB_PASS
However, you will always have this problem if you are supplied the password to MySQL via the command line. You could remove the -pPassword clause and force the user to enter the password when MySQL prompts for it, but from your use-case this does not seem feasible.
Spaces and quotes in the mysql password don't get passed to bash properly.
Likewise, my attempts at quoting PASS_CLAUSE are not effective: Only passwords without spaces/quotes will successfully log into the mysql client.
For 3 and 4, I believe all you need to do is properly quote your variable values.
In PHP:
echo <<<EOD
DB_HOST="{$db->default['host']}"
DB_NAME="{$db->default['database']}"
DB_USER="{$db->default['login']}"
DB_PASS="{$p}"
EOD;
Note the added quotes in DB_PASS="{$p}".
As a reminder, you can also escape the quotes in PHP with addslashes().
Okay, so #nickb's suggestions led me down the right track with enough modifications.
Explanation
Issues arise if you try to pass the MySQL password through multiple bash variables. Here's a stripped down example:
#!/bin/bash
set -x
DB_PASS="pass with space and 'quote"
PASS_CLAUSE=" -p'$DB_PASS'"
# Doesn't work.
mysql $PASS_CLAUSE
# Works.
mysql -p"$DB_PASS"
Note that with the set -x we're able to see the commands that are actually being run. This is what helped me identify the double-escaping. The string in $DB_PASS gets re-escaped when it gets saved into $PASS_CLAUSE. Since it gets passed to MySQL with one level of escaping still in place the first connection fails, but using the original variable succeeds.
Final Scripts
db-cred.sh
#!/usr/bin/php
<?php
include 'database.php';
$db = new DATABASE_CONFIG();
echo <<<EOD
DB_HOST="{$db->default['host']}"
DB_NAME="{$db->default['database']}"
DB_USER="{$db->default['login']}"
DB_PASS="{$db->default['password']}"
EOD;
db-login.sh
#!/bin/bash
# Holy yuck, but nothing else works!
eval $( db-cred.sh )
CMD="mysql --host=${DB_HOST} --database=${DB_NAME} --user=${DB_USER}"
PATTERN=" |'"
if [[ ${DB_PASS} =~ ${PATTERN} ]]; then
${CMD} -p"${DB_PASS}"
elif [ -n "${DB_PASS}" ]; then
${CMD} -p${DB_PASS}
else
${CMD}
fi
Final Thoughts
I haven't specifically tested for it, but there may still be issues with passwords that contain double-quotes.
This could still be improved by somehow removing the need for eval. #Dave's idea about using CSV as a go between has potential, but in my tests it further complicated things by adding another level of escaping/unescaping necessary to get the variables into bash cleanly.
The MySQL password is still used on the command line. Writing a mysql config file, using it with --defaults-extra-file=filename and afterwards deleting it might be a way around that.
db-login.sh now represents a pattern for executing command lines with the database credentials and can be easily modified/extended for other tools like mysqldump.
I have a Ruby script that generates a sed command to replace some PHP code. The command is later executed via SSH.
command = "sed -i \"s*#{find_what}*#{replace_with}*\" #{file} "
The replace_with string will contain multiple lines of PHP code so the following escaping needs to take place:
command.gsub!(/\n/, '\\\n ') # Handle new-line escaping
command.gsub!(/&/, '\\\&') # Handle RegEx variable escaping
command.gsub!(/(\$[a-zA-Z0-9_]+)/) { |s| s.gsub!(/\$/, '\\$') } # Handle bash variable escaping
The command after escaping looks like this:
sed -i "s*require_once('file.php');*\n require_once(\$www_dir . \$path . '/file.php');\n *" /var/www/something.php
When executing this command manually everything works as expected. However if I execute the command via Kernel.system all the PHP variables are stripped out in the replacement string. Something.php ends up looking like this:
require_once( . . '/file.php');
Any ideas would be appreciated.
Thanks.
Update: try having single quotes around the sed command that way no bash variable substitution will be run. I'd try the ruby stuff out like this until it looks just right.
puts `echo #{command}`
If you're using SSH anyway, I'd just cook something up like this to be able to run it locally via ssh, makes it super easy to have total control by keeping it all ruby.
require 'net/sftp'
Net::SFTP.start('host', 'username', :password => 'password') do |sftp|
filedata = sftp.file.open("/path/to/remote", "r").read
filedata.gsub!(/foo/, "bar")
sftp.file.open("/path/to/remote", "w") do |f|
f.puts filedata
end
end
see http://net-ssh.rubyforge.org/sftp/v2/api/
I've installed Syntastic plugin in vim. I can't get it to work. I've tried :SyntasticEnable but no luck. SyntasticEnable python in my vimrc doesn't work either (in fact, it doesn't even parse the command, an error is shown when I try to add it to my .vimrc : Not an editor command: SyntasticEnable python).
How can I know what's going on? Syntastic isn't showing errors when I call it from vim. Does the first error (not and editor command in my .vimrc) indicates something I'm unaware of?
I have in my .vimrc:
set statusline+=%#warningmsg#
set statusline+=%{SyntasticStatuslineFlag()}
set statusline+=%*
let g:syntastic_enable_signs=1
let g:syntastic_auto_loc_list=1
I have a python.vim in ~/.vim/syntax_checkers as well. I can already use Pyflakes for python files, it works GREAT but I would like to have Syntastic functionality in other files/extensions I need when developing applications.
Is pyflakes on your environment path? If it is not, then you must add it to your path, or modify syntax\checkers\python.vim and add the full path to the binary.
There are two lines you have to modify:
if !(executable("pyflakes"))
and also
let makeprg = 'pyflakes %'
In my case. I wanted Syntastic to work with PHP on my Windows machine. So I had to modify those two similar lines in php.vim to be:
let s:php_executable = "C:\\Uniserver\\usr\\local\\php\\php.exe"
if !(executable(s:php_executable))
and
let makeprg = php_executable . " -l %"
If your path contains spaces, you'll have to surround them in double quotes in the makeprg variable. Also with html.vim, the single quotes in makeprg must be replaced with double quotes, (you'll have to re-escape everything inside).
let s:libfolder = "C:\\Program Files (x86)\\GnuWin32\\bin\\"
let s:tidyexe = s:libfolder . "tidy.exe"
let s:grepexe = s:libfolder . "grep.exe"
if !executable(s:tidyexe) || !executable(s:grepexe)
and
let makeprg="\"".s:tidyexe."\" -e % 2>&1 \\| \"".s:grepexe."\" -v \"\<table\> lacks \\\"summary\\\" attribute\""
Have you tried to make sure Vim has the right value set for $PATH?
:echo $PATH
If not, try putting this in your .vimrc:
let $PATH=substitute(system("echo \$PATH"), "\r\*\n", "", "g")
Then shutdown and restart vim. If it works, you won't have to manually update the plugin's source files to manually include the path to the executables.
(found this via: https://superuser.com/questions/380535/getting-man-path-through-vim )