I have been trying to setup a build system for PHP and has the following Jenkins pipeline.
pipeline {
agent { docker { image 'my-custom-image' } }
stages {
stage('Build') {
steps {
sh 'composer config --global --auth http-basic.repo.packagist.com token mytokencode'
}
}
}
}
Here it is pulling the docker image, but when the composer command is executed, it it throwing following error :
Cannot create cache directory /.composer/cache/repo/https---repo.packagist.com-domain/, or directory is not writable. Proceeding without cache
Cannot create cache directory /.composer/cache/files/, or directory is not writable. Proceeding without cache
[ErrorException]
touch(): Unable to create file /.composer/config.json because No such file or directory
and the build is failing. How can I fix this ?
You could set the COMPOSER_HOME environment variable to the actual directory, them Composer will save all the cache data there. Something like:
pipeline {
agent { docker { image 'my-custom-image' } }
stages {
stage('Build') {
steps {
sh 'COMPOSER_HOME=$(pwd) composer config --global --auth http-basic.repo.packagist.com token mytokencode'
}
}
}
}
I was able to use composer cache by creating a volume between my Jenkins machine and the Docker image. You can try to reproduce:
At your pipeline script, create the volume using an argument:
agent {
docker {
image 'my-custom-image'
args '-v $HOME/.composer/cache:/usr/local/share/composer/cache'
}
}
Check where your $HOME directory is pointing to, it can be different depending on how you installed Jenkins. I'm using it as an apt package on Ubuntu 16. For me, it's pointing to /var/lib/jenkins
Inside your $HOME directory, a folder named ".composer" is needed with right permissions so the docker can use it. I set it up with chmod -R 775.
Related
UPDATE: Thank you all! I have solved this by creating a custom runtime for my PHP Lambda.
I am currently using Node.js 8.10 Runtime with a php.handler and my Lambda function works fine, but when I change the Runtime to 12.x, I get the following error:
"php-7-bin/bin/php: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory"
exports.handler = function(event, context, callback) {
var php = spawn('php-7-bin/bin/php',['--php-ini', 'user.ini', process.env['PHPFILE']], {maxBuffer: 200 * 1024 * 200});
var output = "";
var statusCode = 0;
php.stdin.write(JSON.stringify(event));
php.stdin.end();
php.stdout.on('data', function(data) {
console.log("CHUNK: " + data);
output+=data;
});
php.stderr.on('data', function(data) {
console.log(data);
});
php.on('close', function(code) {
var obj = JSON.parse(output);
statusCode = obj.status.statusCode;
if(statusCode !== 0){
callback(output);
}else{
context.succeed(obj);
}
});
}
I need to update my Lambda to the latest node.js version, but I have no idea how to overcome this error, so any help would be greatly appreciated!
First, why on earth are you using node to load php?
But if you had this working before, why do you need to update to node 12?
If you are upgrading from Node 8, the runtime is different:
https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
So then take a look here:
https://aws.amazon.com/blogs/apn/aws-lambda-custom-runtime-for-php-a-practical-example/
You may need to create a new custom runtime based off the node12 built-in runtime for AWS.
Easy fix is to add on top of your PHP code:
set_include_path('/opt/lib64’);
If that won't work you need to compile/build/install the missing modules/libraries by yourself:
Run two docker instances that will have mounted the same “local” Layer folder.
First container is going to be your lambda container while second one is Amazon linux used to build items.
Test your code with Lambda container and in case something is missing switch to Amazon Linux and build/extract binaries/libraries into shared Layer folder structure.
Make sure the Lambda code have proper PATH defined to use Layer folder.
Install docker.
In first terminal tab go to your lambda folder and start the lambda docker container:
docker run --rm -it --entrypoint=/bin/bash -v "$PWD":/var/task:ro,delegated -v /your/path/to/Layer/folder/:/opt:rw,delegated -e AWS_ACCESS_KEY_ID=[ACCESS_KEY_PASTE_HERE -e AWS_SECRET_ACCESS_KEY=[SECRET_GOES_HERE] lambci/lambda:nodejs12.x
In second terminal tab run another container with Amazon linux:
docker run --rm -it -v /your/path/to/Layer/folder/:/opt:rw,delegated amazonlinux:latest
(Keep in mind that the Layer folder is mounted with read/write permissions).
Test your lambda code in your favourite way or just by simple run (make sure to check if your handler module name is “handler” and file name is “index.js”):
cd /var/task
node index.js; node "var func = require('./index.js');func.handler({},function() {},function(){console.log('Lambda finished')});"
In case you find some missing libraries, make sure to add to your PHP code:
set_include_path('/opt/lib');
Then on Amazon Linux terminal tab and install/build your library and then copy it to Layer folder:
cp /usr/lib64/[here is your library name] /opt/lib
Test again your code in Lambda container.
When you will be done just zip the content of your Lambda Layer structure, keep in mind that your \bin ora \lib folders need to be in the root folder of the zip file.
Add the zip file as a Layer for you lambdas and attach it.
I Fix this problem by adding extra library folder in my function's zip.
Make a directory name extra-libs
Copy all required libraries from Amazon Linux 2 to Extra-libs by using following steps :
Run amazon Linux 2's docker instance by following command
docker run --rm -it -v :/opt:rw,delegated amazonlinux:latest
Then in docker instance make directory using
mkdir deps
Copy all required libraries from lib64 to deps directory using
cp -f lib64/libcrypt.so.1 deps (Taken libcrypt.so.1 for example purpose)
Then open another terminal window and move all library files to local extra-libs
docker cp <DOCKER_CONTAINER_ID>:/deps/ . && mv deps/* ./extra-libs
Get container id by using docker ps
Then in index.js file , add following line to php's env setting.
LD_LIBRARY_PATH:path.join(__dirname, '/extra-libs')
Zip extra-libs folder with your lambda function and upload it.
Hope this helps.
I have this error when the deploy script tries to install the assets:
(Setup PHP 7.2, Symfony 4.4)
[Symfony\Component\Process\Exception\ProcessFailedException]
The command "ssh -A root#vps.net -p 21 '(export APP_ENV=prod; cd /var/www-deploy/site.com/releases/20191129224025
&& php /var/www-deploy/site.com/releases/20191129224025/bin/console assets:install /var/www-deploy/site.com/releases/20191129224025/public --symlink --no-debug --env=prod)'" failed.
Exit Code: 1(General error)
Working directory: /Users/coil/Sites/site.com
Output:
================
Error Output:
================
In EnvVarProcessor.php line 162:
Environment variable not found: "APP_SERVER".
I have put the .env file in the shared files, so it's copied when the release directory is created, this file is in the directory.
public function configure(): DefaultConfiguration
{
return $this->getConfigBuilder()
// ...
->sharedFilesAndDirs(['.env'])
// ...
}
I also tried to put the env vars in the .bashrc file of the user who deploys:
printenv | grep -i app
APP_DEBUG=0
APP_SECRET=whatever
APP_ENV=prod
APP_SERVER=prod
So, I don't understand, why the environment variables are still not found. And why they are not taken from the .env file.
If I interrupt the deployment and if I manually run the same command on the server it works well. Thanks.
[Edit 2019-11-30] I think I understand know, when running the command, there is an export of the APP_ENV that is done. And, if the application detects this env variable it will just skip the .env file. So, I need to avoid setting this env var, or I must set all instead of this one only.
[Edit 2021-02-13] I have fixed the bin/console file and removed the "hack". Instead of testing the APP_ENV var I test the APP_SECRET one which is not set by EasyDebug:
if (!isset($_SERVER['APP_SECRET'])) {
(new Dotenv())->load(__DIR__.'/../.env');
}
Old solution:
It now works with the following modififications, in src/Configuration/DefaultConfiguration.php:
} elseif (4 === $symfonyMajorVersion || (3 === $symfonyMajorVersion && 4 >= $symfonyMinorVersion)) {
$this->_symfonyEnvironmentEnvVarName = 'APP_ENV';
I have passed an unkown env var to Symfony:
$this->_symfonyEnvironmentEnvVarName = 'APP_FAKE_ENV';
So the .env file is not ignored any-more. I think there is a missing option to set or not this environment var based on the setup of the application. I'll try to do a PR to the bundle.
So, as the bundle is not modified yet, I have modified the private property, in order to make the command use my .env files instead of environment variables:
/**
* #see https://stackoverflow.com/q/1762135/633864
*/
private function forceEnvironmentEnvVarName(DefaultConfiguration $configuration): DefaultConfiguration
{
$myClassReflection = new ReflectionClass(get_class($configuration));
$symfonyEnvironmentEnvVarName = $myClassReflection->getProperty('_symfonyEnvironmentEnvVarName');
$symfonyEnvironmentEnvVarName->setAccessible(true);
$symfonyEnvironmentEnvVarName->setValue($configuration, 'FAKE_APP_ENV');
$symfonyEnvironmentEnvVarName->setAccessible(false);
return $configuration;
}
I'm experimenting with Deployer to deploy Laravel application into shared hosting (using laravel recipe) from my local ~/Code/project_foo.
The point is that when I'm connected to my shared hosting server via ssh, then default php -v version is 5.6.33. I confirmed that I can change php version on fly by calling php70 -v or even whole path like /usr/local/bin/php70 whatever.
The point is that I don't know how to tell deployer to call commands using php70 which is required, otherwise composer install fails.
So in Terminal I'm inside root of the Laravel project and I simply call:
dep deploy
My deploy.php is messy and very simple but this is just a proof of concept. I'm trying to figure out everything and then I will make it look nicer.
I checked the source code of the laravel recipe, and I saw that there is:
{{bin/php}}
but I don't know how to override the value to match what my hosting tells me to use:
/usr/local/bin/php70
Please, give me any hints how to force the script use different PHP version once connected to the remote host / server.
This is whole script:
<?php
namespace Deployer;
require 'recipe/laravel.php';
//env('bin/php', '/usr/local/bin/php70'); // <- I thought that this will work but it doesn't change anything
// Project name
set('application', 'my_project');
// Project repository
set('repository', 'git#github.com:xxx/xxx.git');
// [Optional] Allocate tty for git clone. Default value is false.
set('git_tty', true);
// Shared files/dirs between deploys
add('shared_files', []);
add('shared_dirs', []);
// Writable dirs by web server
add('writable_dirs', []);
// Hosts
host('xxx')
->user('xxx')
->set('deploy_path', '/home/slickpl/projects/xxx');
// Tasks
task('build', function () {
run('cd {{release_path}} && build');
});
// [Optional] if deploy fails automatically unlock.
after('deploy:failed', 'deploy:unlock');
// Migrate database before symlink new release.
before('deploy:symlink', 'artisan:migrate');
OK, I've found the solution.
I added (after require):
set('bin/php', function () {
return '/usr/local/bin/php70';
});
For anybody who searches for changing Composer's PHP version:
set('bin/composer', function () {
return '/usr/bin/php7.4 /usr/local/bin/composer';
});
There is function locateBinaryPath()
so result is:
set('bin/php', function () {
return locateBinaryPath('php7.4');
});
first found php path and composer path use this
for more information Setting PHP versions in Deployer deployments
find / -type f -name "php" 2>&1 | grep -v "Permission denied"
find / -type f -name "composer" 2>&1 | grep -v "Permission denied"
then
set('bin/composer',
function () {
return 'php_path composer_path';
});
like this
set('bin/composer',
function () {
return '/opt/remi/php73/root/usr/bin/php /usr/bin/composer';
});
I am trying to use easy-deploy-bundle to deploy a symfony project cannot deploy any symfony project, even to my local machine. The symlinking for the parameters.yml file from the shared directory always fails with the error below.
I have cloned the symfony-demo project, this is a symfony 3.3.10, the parameters.yml file is empty but present in the shared/app/config dir.
Error:
[Symfony\Component\Process\Exception\ProcessFailedException]
The command "(export SYMFONY_ENV=prod; cd /home/patrick/code/testedb-deploy/releases/20171015162057 && ln -nfs /home/patrick/code/testedb-deploy/shared/app/config/parameters.yml /home/patrick/code/testedb-deploy/releases/20171015162057/app/config/parameters.yml)" f
ailed.
Exit Code: 1(General error)
Working directory: /home/patrick/code/testedb
Output:
================
Error Output:
================
ln: failed to create symbolic link '/home/patrick/code/testedb-deploy/releases/20171015162057/app/config/parameters.yml': No such file or directory
This is my deploy file:
<?php
use EasyCorp\Bundle\EasyDeployBundle\Deployer\DefaultDeployer;
return new class extends DefaultDeployer
{
public function configure()
{
return $this->getConfigBuilder()
->server('localhost')
->deployDir('/home/patrick/code/testedb-deploy')
->repositoryUrl('git#github.com:superpatty/symfony-demo.git')
->symfonyEnvironment('prod')
->resetOpCacheFor('https://demo.symfony.com')
;
}
public function beforeStartingDeploy()
{
$this->log('Checking that the repository is in a clean state.');
$this->log('Running tests, linters and checkers.');
$this->runLocal('./bin/console security:check --env=dev');
$this->runLocal('./bin/console lint:twig app/Resources/ --no-debug');
$this->runLocal('./bin/console lint:yaml app/ --no-debug');
$this->runLocal('./bin/console lint:xliff app/Resources/ --no-debug');
$this->runLocal('./vendor/bin/simple-phpunit');
}
public function beforeFinishingDeploy()
{
}
};
Has anyone else come across this issue and how to get around it?
I have trouble deploying with Deployer 4.0.2 and I am in need for help of somebody more experienced than me in this.
I want to deploy a repository of mine to a Ubuntu 16.04 server.
I am using laravel homestead as a development environment, where I also installed deployer. From there I ssh into my remote server.
I was able to deploy my code with the root user, until I hit a RuntimeExceptionthat aborted my deployment.
Do not run Composer as root/super user! See https://getcomposer.org/root for details
That made me create another user called george, whom I gave superuser rights. I copied my public key from my local machine to a newly generated ~/.ssh/authorized_keys file, that gave me permission to access the server via ssh.
Yet when I run dep deploy with the new user:
server('production', '138.68.99.157')
->user('george')
->identityFile()
->set('deploy_path', '/var/www/test');
I get another RuntimeException:
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Now it looks like the new user george cannot access the ~/.ssh/id_rsa.pubkey. So I copy them from the root folder into my home folder and also add the public key in the Github SSH settings.
cp root/.ssh/id_rsa.pub home/george/.ssh/id_rsa.pub
cp root/.ssh/id_rsa home/george/.ssh/id_rsa
Only to get the same error as before.
In the end I had to add github to my list of authorized hosts:
ssh-keyscan -H github.com >> ~/.ssh/known_hosts
Only to get the next RuntimeException
[RuntimeException]
sudo: no tty present and no askpass program specified
I managed to comment this code in the deploy.php
// desc('Restart PHP-FPM service');
// task('php-fpm:restart', function () {
// // The user must have rights for restart service
// // /etc/sudoers: username ALL=NOPASSWD:/bin/systemctl restart php-fpm.service
// run('sudo systemctl restart php-fpm.service');
// });
// after('deploy:symlink', 'php-fpm:restart');
to get the deployment process finally done, and now I ask myself, if the restart of php-fpm is really necessary, for me to continue debugging this deployment tool? Or can I live without it?
And if I need it, can somebody help me understand what I need it for? And maybe as a luxury also provide the solution to the RuntimeException?
Try this:
->identityFile('~/.ssh/id_rsa.pub', '~/.ssh/id_rsa', 'pass phrase')
It works great for me - no need for an askpass program.
It helps to be explicit in my experience.
As for your phpfm restart task .. I haven't seen that before. Shouldn't be needed. :)
EDIT:
That you provide a password is probably a good sign that you ought to refactor your Deployer code a bit if you keep it under source control.
I am loading site specific data from a YAML file - which I am not submitting to source control.
The first bit of my stage.yml :
# Site Configuration
# -------------
prod_1:
host: hostname
user: username
identity_file:
public_key: /home/user/.ssh/key.pub
private_key: /home/user/.ssh/key
password: "password"
stage: production
repository: https://github.com/user/repository.git
deploy_path: /var/www
app:
debug: false
stage: 'prod'
And then, in my deploy.php :
if (!file_exists (__DIR__ . '/deployer/stage/servers.yml')) {
die('Please create "' . __DIR__ . '/deployer/stage/servers.yml" before continuing.' . "\n");
}
serverList(__DIR__ . '/deployer/stage/servers.yml');
set('repository', '{{repository}}');
set('default_stage', 'production');
Notice that, when you use serverList, it replaces your server setup in deploy.php