I have a CakePHP database.php config file that I'd like to leverage to open a command line mysql client instance.
The goal would be to be able to run a shell script like ./db-login.sh from the project root and have it pull the database details (host, port, database name, username, password) from the PHP config file and pass them to the mysql command line as arguments. This is to avoid having to enter the details every time.
I understand it's possible to create a shell alias that has the values hard-coded: I would like a portable script that could be included with any of my Cake projects. I also would like to keep the task of getting the DB credentials into bash variables separate from launching the mysql client. This would open up the ability to re-use the DB credentials in other shell scripts easily (such as a mysqldump backup script.)
Here's what I have so far:
database.php
Consider this file immutable for the purposes of this question. It must exist exactly as you see it.
<?php
class DATABASE_CONFIG {
public $default = array(
'host' => 'localhost',
'login' => 'cakephpuser',
'password' => 'c4k3 roxx!',
'database' => 'my_cakephp_project',
);
}
db-cred.sh
Acts as middleware to convert the PHP variables into bash (friendly) variables.
#!/usr/bin/env php
<?php
include 'database.php';
$db = new DATABASE_CONFIG();
// Selectively wrap quotes. Has no real effect(?)
$p = (!empty($db->default['password']) ? "\"$db->default['password']\"" : '');
echo <<<EOD
DB_HOST="{$db->default['host']}"
DB_NAME="{$db->default['database']}"
DB_USER="{$db->default['login']}"
DB_PASS={$p}
EOD;
db-login.sh
#!/usr/bin/env bash
# Uses Cake's database.php file to log into mysql on the
# command line with the correct arguments.
# Holy yuck, but nothing else works!
eval $( db-cred.sh )
# Try to set an appropriate password clause.
PATTERN=" |'"
if [ -n "$DB_PASS" ]; then
if [[ $DB_PASS =~ $PATTERN ]]; then
PASS_CLAUSE=" -p'${DB_PASS}'"
else
PASS_CLAUSE=" -p${DB_PASS}"
fi
else
PASS_CLAUSE=""
fi
# Get a rough look at what we're about to run.
echo mysql --host=${DB_HOST} --database=${DB_NAME} --user=${DB_USER}${PASS_CLAUSE}
# Call MySQL.
mysql --host=${DB_HOST} --database=${DB_NAME} --user=${DB_USER}${PASS_CLAUSE}
Things that need help:
The db-login.sh script uses eval to suck in the variables. Yuck.
The MySQL password is leaked on the command line. This is not a dealbreaker, but if there's a clean/portable way to avoid it, I'm open to it.
Spaces and quotes in the mysql password don't get passed to bash properly.
A password like my Pass will cause DB_PASS to be set to my in db-login.sh.
Quoting the string in db-cred.sh produces "my instead.
Likewise, my attempts at quoting PASS_CLAUSE are not effective: Only passwords without spaces/quotes will successfully log into the mysql client.
The db-login.sh script uses eval to suck in the variables. Yuck.
Instead of eval, you can use the source command:
source db-cred.sh
The MySQL password is leaked on the command line. This is not a dealbreaker, but if there's a clean/portable way to avoid it, I'm open to it.
You can just unset the MySQL password after it is used:
unset DB_PASS
However, you will always have this problem if you are supplied the password to MySQL via the command line. You could remove the -pPassword clause and force the user to enter the password when MySQL prompts for it, but from your use-case this does not seem feasible.
Spaces and quotes in the mysql password don't get passed to bash properly.
Likewise, my attempts at quoting PASS_CLAUSE are not effective: Only passwords without spaces/quotes will successfully log into the mysql client.
For 3 and 4, I believe all you need to do is properly quote your variable values.
In PHP:
echo <<<EOD
DB_HOST="{$db->default['host']}"
DB_NAME="{$db->default['database']}"
DB_USER="{$db->default['login']}"
DB_PASS="{$p}"
EOD;
Note the added quotes in DB_PASS="{$p}".
As a reminder, you can also escape the quotes in PHP with addslashes().
Okay, so #nickb's suggestions led me down the right track with enough modifications.
Explanation
Issues arise if you try to pass the MySQL password through multiple bash variables. Here's a stripped down example:
#!/bin/bash
set -x
DB_PASS="pass with space and 'quote"
PASS_CLAUSE=" -p'$DB_PASS'"
# Doesn't work.
mysql $PASS_CLAUSE
# Works.
mysql -p"$DB_PASS"
Note that with the set -x we're able to see the commands that are actually being run. This is what helped me identify the double-escaping. The string in $DB_PASS gets re-escaped when it gets saved into $PASS_CLAUSE. Since it gets passed to MySQL with one level of escaping still in place the first connection fails, but using the original variable succeeds.
Final Scripts
db-cred.sh
#!/usr/bin/php
<?php
include 'database.php';
$db = new DATABASE_CONFIG();
echo <<<EOD
DB_HOST="{$db->default['host']}"
DB_NAME="{$db->default['database']}"
DB_USER="{$db->default['login']}"
DB_PASS="{$db->default['password']}"
EOD;
db-login.sh
#!/bin/bash
# Holy yuck, but nothing else works!
eval $( db-cred.sh )
CMD="mysql --host=${DB_HOST} --database=${DB_NAME} --user=${DB_USER}"
PATTERN=" |'"
if [[ ${DB_PASS} =~ ${PATTERN} ]]; then
${CMD} -p"${DB_PASS}"
elif [ -n "${DB_PASS}" ]; then
${CMD} -p${DB_PASS}
else
${CMD}
fi
Final Thoughts
I haven't specifically tested for it, but there may still be issues with passwords that contain double-quotes.
This could still be improved by somehow removing the need for eval. #Dave's idea about using CSV as a go between has potential, but in my tests it further complicated things by adding another level of escaping/unescaping necessary to get the variables into bash cleanly.
The MySQL password is still used on the command line. Writing a mysql config file, using it with --defaults-extra-file=filename and afterwards deleting it might be a way around that.
db-login.sh now represents a pattern for executing command lines with the database credentials and can be easily modified/extended for other tools like mysqldump.
Related
I just cannot fathom how to get the PHP exec() or shell_exec() functions to treat a '*' character as a wildcard. Is there some way to properly encode / escape this character so it makes it through to the shell?
This is on windows (via CLI shell script if that matters, Terminal or a git-bash yields the same results).
Take the following scenario:
C:\temp\ contains a bunch of png images.
echo exec('ls C:\temp\*');
// output: ls: cannot access 'C:\temp\*': No such file or directory
Permissions is not the problem:
echo exec('ls C:\temp\exmaple.png');
// output: C:\temp\example.png
Therefore the * character is the problem and is being treated as a literal filename rather than a wildcard. The file named * does not exist, so from that point of view, it's not wrong...
It also does not matter if I use double quotes to encase the command:
echo exec("ls C:\temp\*");
// output: ls: cannot access 'C:\temp\*': No such file or directory
I have also tried other things like:
exec(escapeshellcmd('ls C:\temp\*'));
exec('ls C:\temp\\\*');
exec('ls "C:\temp\*"');
exec('ls "C:\temp\"*');
And nothing works...
I'm pretty confused that I cannot find any other posts discussing this but maybe I'm just missing it. At this point I have already worked around the issue by manually programming a glob loop and using the internal copy() function on each file individually, but it's really bugging me that I do not understand how to make the wildcard work via shell command.
EDIT:
Thanks to #0stone0 - The answer provided did not particularly answer my initial question but I had not tried using forward slashes in the path and when I do:
exec('ls C:/temp/*')
It works correctly, and as 0stone0 said, it only returns the last line of the output, which is fine since this was just for proof of concept as I was not actually attempting to parse the output.
Also, on a side note, since posting this question my system had been updated to Win11 22H2 and now for some reason the original test code (with the backslashes) no longer returns the "Cannot access / no file" error message. Instead it just returns an empty string and has no output set to the &$output parameter either. That being said, I'm not sure if the forward slashes would have worked on my system prior to the 22H2 update.
exec() only returns the last output line by default.
The wildcard probably works, but the output is just truncated.
Pass an variable by ref to exec() and log that:
<?php
$output = [];
exec('ls -lta /tmp/*', $output);
var_dump($output);
Without any additional changes, this returns the same as when I run ls -lta /tmp/* in my Bash terminal
That said, glob() is still the preferred way of getting data like this especcially since
You shouldn't parse the output of ls
I am modifying a set of bash scripts that process video files and reports the processing steps to a mysql database (here is the original code in question).
The function that does the database reporting is called from the main processing script and looks like this in the original:
_report_to_db(){
if [ "${REPORT_TO_DB}" = "Y" ] ; then
echo "INSERT IGNORE INTO tableA (objectIdentifierValue,object_LastTouched) VALUES ('${MEDIA_ID}',NOW()) ON DUPLICATE KEY UPDATE object_LastTouched = NOW()" | mysql --login-path="${LOGIN_PROFILE}" "${DB_NAME}" 2> /dev/null
_db_error_check
fi
}
Since the scripts are meant to be run directly from the command line, when you run them that way it works fine. But I'm running them via php from a web interface and there's some shenanigans going on with the quoting/escaping of whitespace and/or variables.
For instance, the script breaks on the whitespace after ...| mysql and it thinks I'm trying to run mysql as root without a password and totally ignores the --login-path and the other stuff I'm piping to it.
When I call mysql from a variable like so:
_report_to_db(){
if [ "${REPORT_TO_DB}" = "Y" ] ; then
SQL_ARRAY=(INSERT IGNORE INTO tableA (columnA,lastTouched) VALUES ("${SOME_PASSED_VALUE}",NOW()) ON DUPLICATE KEY UPDATE object_LastTouched = NOW();)
MYSQL_COMMAND_ARRAY=(mysql --login-path="${LOGIN_PROFILE}" -e "${SQL_ARRAY[#]}" "${DB_NAME}")
echo "$(${MYSQL_COMMAND_ARRAY[#]})"
_db_error_check
fi
}
... I am able to log into mysql correctly but the SQL query is ignored (when it echos the result you get the standard MySQL --help output.
So far I have tried all kinds of variations on quoting, escaping, referencing the query as a separate string variable, as an array (as you see here).
What is also not helpful is that the original _db_error_check() function only checks the value of the pipe exit status. So if the pipe is ok, but there's a problem further down the path, it fails silently.
_db_error_check(){
if [ "$?" != "0" ] ; then
# reports an error if the pipe exit value ≠ 0
else
# everything is ok! even if there was a mysql error
fi
}
This is not a file or database permissions issue (already triple checked that). Are there quotes or some other stupid thing that I am missing?? Thanks! Oh, I am running OSX El Capitan.
UPDATE
Lol, I was going to post the PHP that calls the script and then I remembered that the PHP is actually calling a Pyhton script that does some other processing too, and that is what calls the bash script. Here it all is:
PHP
$command = escapeshellcmd("/usr/local/bin/python3 /Users/user/path/to/ingest.py " . $user . " 2>&1");
while (# ob_end_flush());
$proc = popen($command, 'r');
echo '<pre>';
while (!feof($proc))
{
echo fread($proc, 4096);
# flush();
}
echo '</pre>';
PYTHON
for item in os.listdir(ingestDir):
if not item.startswith("."):
filePath = os.path.abspath(ingestDir+"/"+item)
fileNameForMediaID = os.path.splitext(item)[0]
try:
ingest = subprocess.Popen(['/usr/local/bin/ingestfile','-e','-u',user,'-I',filePath,'-m',fileNameForMediaID])
ingest.wait()
os.remove(filePath)
except IOError as err:
print("OS error: {0}".format(err))
UPDATE 2
I think this might actually be a weird quirk of my installation (go figure). Using mysql --login-path=myDbUser [etc...] from a shell on my host machine I keep getting the error ERROR 1045 (28000): Access denied for user 'ADMIN'#'localhost' (using password: NO) where the client user is ADMIN and I am trying to login as myDbUser.
I actually uninstalled and reinstalled mysql (via Homebrew) and still have the same results. Using a different machine (running Sierra, but the same mysql version) I can run the above shell command successfully and log into mysql as the target user.
Also on the host machine, I can sudo -u _www zsh and run the command as the Apache user (which is the user running the whole show) without a problem. SO WHY IS IT NOT RUNNING CORRECTLY EITHER IN THE SCRIPT OR EVEN RUN FROM SHELL AS MY MAIN CLIENT USER???
Any ideas? $PATH is identical in all cases mentioned above. Same ~/.mylogin.cnf setups. Is there anything else stupid obvious I missed?
You need to use indirect expansion here:
echo "$(${MYSQL_COMMAND_ARRAY[#]})"
the man says:
If the first character of parameter is an exclamation point (!), and
parameter is not a nameref, it introduces a level of variable
indirection. Bash uses the value of the variable formed from the rest
of parameter as the name of the variable; this variable is then
expanded and that value is used in the rest of the substitution,
rather than the value of parameter itself. This is known as indirect
expansion. If parameter is a nameref, this expands to the name of the
variable referenced by parameter instead of performing the complete
indirect expansion. The exceptions to this are the expansions of
${!prefix*} and ${!name[#]} described below. The exclamation point
must immediately follow the left brace in order to introduce
indirection.
${!name[#]}
${!name[*]}
If name is an array variable, expands to the list of array indices
(keys) assigned in name. If name is not an array, expands to 0 if name
is set and null otherwise. When ‘#’ is used and the expansion appears
within double quotes, each key expands to a separate word.
PS: If I may put forward a piece of my personal opinion, having a chain of php -> python -> bash is the worst coding style one can ever met, you may want to rewrite it into single langue so it will be easier to track down further issues at least.
I am configuring Jenkins in order to make it build my project.
It checks out source code, installs all required libs etc..
My application configuration is stored in params.php file (Yii 1.x style), it looks the following way:
return array(
'dbHost' => 'dbHostValuePlaceholder',
'dbPasssword' => 'dbPasswordValuePlaceholder',
/* other values go here */
);
Obviously I need to replace dbPasswordValuePlaceholder with password that I set when starting parametrised build.
Current solution I've found is running this as shell command:
sed -i s/dbPasswordValuePlaceholder/$DB_PASSWORD/g $WORKSPACE/protected/config/params.php
where $DB_PASSWORD is corresponding variable.
This solution work perfectly with simple strings, but password may look like a1#2";$&^*&-=+/*-:'/.,,,,,,,,^^^^^ - in this case sed fails because of unescaped characters.
I have searched for different Jenkins plugins, but there are only plugins that work with other formats like .xml etc, and neither works with .php files.
I need to avoid moving my config values out of .php file to some other format.
Is there a way to replace the described tokens in .php file with build parameters values ?
UPDATE
As #Stanjer recommended, I have tried to set param value to:
a1#2";$&^*&-=+/*-:\'/.,,,,,,,,^^^^^
So, in this case, the following code in PHP would be right, valid:
'universalPassword' => 'a1#2";$&^*&-=+/*-:\'/.,,,,,,,,^^^^^',
It does not help, here is output:
[build.int] $ /bin/sh -xe /tmp/hudson642312569695847678.sh
+ sed -i s/defaultUniversalPassword/a1#2";$&^*&-=+/*-:\'/.,,,,,,,,^^^^^/g /var/www/build.int/protected/config/params.php
sed: -e expression #1, char 43: unknown option to `s'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Thanks for the updated question.
In your case you should show that it's already escaped by providing it as $'string'.
So the command will look like:
$ sed s#dbPasswordValuePlaceholder#$'"a1#2";$&^*&-=+/*-:\'/.,,,^^^"'#g test.php
and output will be:
<?php
return array(
'dbHost' => 'dbHostValuePlaceholder',
'dbPassword' => '"a1#2";$dbPasswordValuePlaceholder^*dbPasswordValuePlaceholder-=+/*-:'/.,,,,,,,,^^^^^"',
/* other values go here */
);
?>
If using with variable:
export NEWPASS=$'"a1#2";$&^*&-=+/*-:\'/.,,,^^^"'
sed "s#dbPasswordValuePlaceholder#$NEWPASS#g" test.php
I am not sure about any jenkins plugin which works that way , but I needed to know if its possible to make sed ignore the special characters using this :
sed with special characters
And I suppose you must be using a Password Parameter with your parameterized job, which hides the password.
How do I properly execute commands in the command line using php? For example I'm using the command below in the command line to convert a docx file into a pdf file:
pdfcreator.exe /PF"D:\Documents\sample.docx
Now using PHP code I want to be able to execute the same command but nothing seems to be happening:
<?php
shell_exec('pdfcreator.exe /PF"D:\Documents\sample.docx"');
?>
Is this possible in PHP?If yes, how do I do it?
system("c:\\path\\to\\pdfcreator.exe /PF\"D:\\Documents\\sample.docx"");
try this.
Don't forget to escape your command with escapeshellcmd(). This will prevent you from having to use ugly backslashes and escape characters.
There are also other alternatives which may work:
`command` // back ticks drop you out of PHP mode into shell
exec('command', $output); // exec will allow you to capture the return of a command as reference
shell_exec('command'); // will return the output to a variable
system(); //as seen above.
Also, make sure your .exe is included within your $PATH variable. If not, include the full path for the command.
I have spent several hours trying to find a means of writing a cross platform password prompt in php that hides the password that is input by the user. While this is easily accomplished in Unix environments through the use of stty -echo, I have tried various means of passthru() and system() calls to make windows do the same thing to no avail.
I have tried:
passthru('set /p pass=Password: ');
system('echo %pass% > out.txt');
$pass = file_get_contents('out.txt', 'r');
This seems to hang on the passthru('set /p pass=Password: '); line without allowing me to input any text and must be killed with a Ctrl-c.
I have also tried various methods of fgetc and fgets and printing backspace characters to hide the input since this works in other languages. However, PHP does not appear to be able to interact with text prior to a carriage return.
I would really like to find a way to make this work, is this an impossible task or is this something that can be done?
Note that I am aware that I could wrap the php script in a batch file and pass the password as a command line argument, but that doesn't work for me in this case.
There doesn't seem to be an IOCTL or STTY extension for PHP. I found the following trick here:
<?php
echo 'Password: ';
$pwd = preg_replace('/\r?\n$/', '', `stty -echo; head -n1 ; stty echo`);
echo "\n";
echo "Your password was: {$pwd}.\n";
?>
Here's a Windows solution, using the COM extension for PHP. I tested this on Windows XP with PHP 5.2.6.
<?php
$pwObj = new Com('ScriptPW.Password');
print "Password: ";
$passwd = $pwObj->getPassword();
echo "Your password is $passwd\n";
?>
I think you cannot do that in PHP with the standard library but you can do better :
Catch the first letter, then display a *. Catch the second one, then display two *...
Ergonomically, this is handy because the user can see what he have entered. Security is not at risk because if somebody can see the password one letter by one letter, he can see the guy typing it on the keyboard anyway. But it still prevent somebody from seeing it by accident in one time.