I have a very simple pipelines, I am just trying to get the app to build before I look to adding anything else to it.
But, even in its simple state, I can't get composer to successfully patch a file.
The entire composer install works flawlessly until this point:
- Applying patches for drupal/core
https://www.drupal.org/files/issues/2021-03-05/3007386-language-negotiator-weight-12.patch (Language negotiation weight)
Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2021-03-05/3007386-language-negotiator-weight-12.patch
[Exception]
Cannot apply patch Language negotiation weight (https://www.drupal.org/files/issues/2021-03-05/3007386-language-negotiator-weight-12.patch)!
After it started failing, I added some debug to the config including trying to cat the file that was to be patched, but it is not there.
+ cat docroot/core/modules/language/src/LanguageNegotiator.php
cat: can't open 'docroot/core/modules/language/src/LanguageNegotiator.php': No such file or directory
This is the composer patch chunk:
"extra": {
"patches": {
"drupal/core": {
"Language negotiation weight": "https://www.drupal.org/files/issues/2018-10-17/3007386-language-negotiator-weight-2.patch"
}
},
This is the pipeline yml:
# Template PHP Build
# This template allows you to validate your PHP application.
# The workflow allows running tests and code linting on the default branch.
image: php:7.3-alpine
pipelines:
default:
- parallel:
- step:
name: Prepare environment
script:
- /sbin/apk update
- /sbin/apk add git
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- docker-php-ext-install mysqli pdo pdo_mysql
- /sbin/apk add libpng-dev
- docker-php-ext-install mbstring
- docker-php-ext-install gd
- pwd
- ls -la
- cat docroot/core/modules/language/src/LanguageNegotiator.php
- /usr/local/bin/composer install
caches:
- composer
Do I need to change directories or do something similar?
Related
Github Action.
I have a PHP Project (using Yii2 Framework) that I need to build into a docker image, which is those project also need MongoDB extension.
My plan is in Github Action - CI phase is like this:
Commit to main branch
Enable php extension
Run composer install --no-dev
Build docker image using dockerfile, which is in those dockerfile, I copy vendor`s folder into an image container,
..so on
So, for those, I start with a configuration: 01 - Init Composer.yml like this:
name: Init composer
on:
push:
branches: ["main"]
jobs:
run-composer:
name: composer initialization
runs-on: ubuntu-latest
container:
image: composer:latest
volumes:
- ${{ github.workspace }}:/app
steps:
- name: Check out the repo
uses: actions/checkout#v3
- name: PHP setup with Extension
id: setup-php
uses: shivammathur/setup-php#v2
with:
php-version: '8.1'
extensions: mongodb-mongodb/mongo-php-driver#v1.9
- name: Cache Composer packages
id: composer-cache
uses: actions/cache#v3
with:
path: vendor
key: ${{ runner.os }}-php-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-php-
- name: Running composer install
run: composer install --no-dev
I got error like this:
Run shivammathur/setup-php#v2
/usr/bin/docker exec fccfaf138a6147da33e0f9ca9add2947ae76ff4b439c3fe9c35f53042e6419fa sh -c "cat /etc/*release | grep ^ID"
/bin/bash /__w/_actions/shivammathur/setup-php/v2/src/scripts/run.sh
==> Setup PHP
/__w/_actions/shivammathur/setup-php/v2/src/scripts/linux.sh: line 216: sudo: command not found
✗ PHP Could not setup PHP 8.1
Error: The process '/bin/bash' failed with exit code 1
Any advise is so appreciated.
Running inside the composer image, it doesn't have the sudo command to perform actions to compile the extension.
In action shivammathur/setup-php it is possible enable composer, see tools support
Try running it directly in ubuntu, setting the php and composer extension first.
I had a similar problem here when running it in self hosted server.
Looks like shivammathur already fixed in the "develop" branch (see here)
It will be fixed in the next version, you can follow the issue about it here
I would be grateful if someone could help me with setting up bitbucket-pipeline for my php env. What I'm trying to succeed is:
Build image in the first step
In the second step run some QA stuff like unit test, code sniffer etc.
Deploy to preprod envirement
Currently I'm stuck with re-using image from the first step. This is how my bitbucket-pipelines.yml looks like:
pipelines:
branches:
develop:
- step:
name: Build docker image
caches:
- composer
script:
- apt-get update && apt-get install -y unzip
- docker-php-ext-install mysqli pdo_mysql json sockets
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- composer install
- parallel:
- step:
caches:
- composer
name: Unit tests
script:
- vendor/bin/phpunit
- step:
caches:
- composer
name: Code sniffer
script:
- composer phpcs:all
- step:
name: Deploy to preprod
script:
- echo "Deployment"
What I get here is:
bash: vendor/bin/phpunit: No such file or directory - from "Unit tests" step
and bash: composer: command not found - from "Code sniffer" step
I already tried to set docker/composer to cache, save image in the first step and import it in the second, but still not working.
One way to share stuff between different steps is to make use of Artifacts.
Basically, bitbucket spins up each step in a separate docker container. So a way to re-use material between steps is create an artifact.
The link should give you enough information.
For example:
- step: &build
caches:
- node
name: Build
script:
- npm install
- npm run build
artifacts: # defining the artifacts to be passed to each future step.
- dist/**
- step: &read-from-artifact
name: Read from the artifact saved in the previous step
script:
- echo "Read artifact (dist folder) saved in previous step"
- ls -la dist/
I am currently learning about Docker, and using it for 2 weeks. Now i have a very simple task, installing PHP Libraries via Composer. That is usually, when working without Docker:
composer install
Now since i am using Docker, i found there is a Docker Container, that is holding composer for me:
docker run --rm -v $(pwd):/app composer/composer install
This is working pretty good, but there is some libraries out there, that require specific php libraries to be installed, like bcmath, so i add this to my Dockerfile
FROM php:7.0-apache
RUN docker-php-ext-install bcmath <-- added this line
COPY . /var/www/html
WORKDIR /var/www/html
EXPOSE 80
When i rebuild my container, this code returns true
var_dump(extension_loaded('bcmath'))
Hooray! BCMath is installed correctly, but composer does not recognize it, because the library is not installed in the composer container!
Now i could ignore that by using
docker run --rm -v $(pwd):/app composer/composer install --ignore-platform-reqs
but this is, in my opinion, a dirty workound, and composer can not validate my platform. Is there any clean solution, besides downloading composer in my Dockerfile and not reusing an existing container?
You may use platform settings to mimic your PHP container configuration. This will work similar to --ignore-platform-reqs switch (it will use PHP and extensions configured in composer.json instead of real info from current PHP installation), but it gives you more granular control. Instead of "ignore all platform requirements checks" you may say "I really have bcmath installed, trust me". All other requirements will be checked, so you still be warned if new requirement will pop-up.
"config": {
"platform": {
"php": "7.1",
"ext-bcmath": "*"
}
},
You need PHP + PHP Extensions + Composer in the same(!) container = DevContainer.
Simply install Composer using the command provided here.
I work on several Symfony bundles hosted on GitHub and tested automatically with Travis CI.
The longest part of the tests is the installation of the requirements by Composer.
I configured Travis CI to install packages with composer update --prefer-dist and cache the $HOME/.composer/cache directory. Tests took a total time of 18 minutes, thanks to caching:
Installing doctrine/lexer (v1.0.1)
Loading from cache
But a week ago I saw a message from Composer:
Installing doctrine/lexer (v1.0.1)
Downloading: Connecting... Failed to download doctrine/lexer from dist: Could not authenticate against github.com
I changed the configuration to composer update --prefer-source because of this. This seems to be a common practice across Symfony bundles. The tests suite took 28 minutes.
I know that I can register GitHub keys in Travis CI in order to avoid the API limit while using the Composer --prefer-dist option.
Are they some other ways to cache the dependencies? E.g by cloning the dependencies repositories in a cache?
GitHub has removed the API rate limits, composer can now be used with --prefer-dist and the zip files can be cached. Here is an example of configuration in .travis.yml:
cache:
directories:
- $HOME/.composer/cache
# …
install:
- composer update --prefer-dist
Here is the announce:
Hi Niels and Jordi,
We've spent some time digging in, considering all the options to resolve your issues, weighing your needs against infrastructure strain and availability.
I'm happy to report that our Infrastructure team believes that due to their work on our Git backend since these APIs were introduced, we're now able to drop these rate limits on the API side. We deployed those changes a couple of hours ago. Getting an archive link 1 via the API will no longer count against your hourly rate limit (authenticated or not). This should make Composer installs happy.
Let us know if you see any funny business.
Cheers,
Wynn Netherland
Platform Engineering Manager, GitHub
Source.
I tested caching of the vendor/ directory and it worked. I used tar in order to create an uncompressed archive $HOME/vendor-cache/ and configured Travis CI for this directory.
Commands have two goals:
Extract the vendor/ from cache if it's available
Put the vendor/ in the cache after the tests
Here is an example .travis.yml file:
sudo: false
language: php
cache:
directories:
- $HOME/.composer/cache
# This is where vendor/ backups will be stored
- $HOME/vendor-cache/
php:
- […]
env:
- SYMFONY_VERSION="2.7.*"
- SYMFONY_VERSION="2.8.*"
- SYMFONY_VERSION="3.0.*"
before_install:
# Create an hash corresponding to the PHP version and the dependencies
- tohash="${SYMFONY_VERSION}"
- cachefile="`echo -n "$tohash" | sha1sum | cut -d " " -f 1`.tar"
# Extract cache archive if the file exists
- if [[ -f $HOME/vendor-cache/$cachefile ]]; then tar -xf $HOME/vendor-cache/$cachefile ; fi
install:
- composer self-update
- composer update --profile --no-progress
script: php ./vendor/bin/phpunit
# Create cache archive from vendor/ directory
before_cache:
- if [[ -f $HOME/vendor-cache/$cachefile ]]; rm -fv $HOME/vendor-cache/$cachefile ; fi
- tar -cf $HOME/vendor-cache/$cachefile vendor/
Here is a fully annotated .travis.yml file with more verbose output:
sudo: false
language: php
cache:
directories:
- $HOME/.composer/cache
# This is where vendor/ backups will be stored
- $HOME/vendor-cache/
git:
depth: 5
php:
- […]
env:
- SYMFONY_VERSION="2.7.*"
- SYMFONY_VERSION="2.8.*"
- SYMFONY_VERSION="3.0.*"
before_install:
# Create an hash corresponding to the PHP version and the dependencies
- echo "Vendor cache content:" ; ls -lh $HOME/vendor-cache/
- echo "Values used for hash:"
- tohash="${SYMFONY_VERSION}"
- echo "$tohash"
- cachefile="`echo -n "$tohash" | sha1sum | cut -d " " -f 1`.tar"
- echo "cachefile = ${cachefile}"
# Extract cache archive if the file exists
- if [[ -f $HOME/vendor-cache/$cachefile ]]; then echo "Extract cache archive"; tar -xf $HOME/vendor-cache/$cachefile ; echo "Done" ; else echo "Cache archive does not exist" ; fi
- if [[ -d vendor/ ]]; then echo "Size of vendor directory extracted from cache:" ; du -hs vendor/; else echo "vendor/ directory does not exist"; fi
install:
- composer self-update
- composer update --profile --no-progress
script: php ./vendor/bin/phpunit
# Create cache archive from vendor/ directory
before_cache:
- if [[ -f $HOME/vendor-cache/$cachefile ]]; then echo "Delete previous cache archive"; rm -fv $HOME/vendor-cache/$cachefile ; echo "Done" ; else echo "No cache archive to delete" ; fi
- echo "Create cache archive" ; tar -cf $HOME/vendor-cache/$cachefile vendor/ ; echo "Done"
- echo "Size of cache archive:" ; ls -lh $HOME/vendor-cache/$cachefile
By using this method, composer update took 30 seconds, instead of about 2 minutes without cache (note that the comparison is not perfect, some other changes are applied but that's still a good estimate).
It's preferable to limit the number of parallel builds the first time you launch builds, so that caches won't suffer from a race condition.
While working on a laravel 5.1+ package I have this need to run automated tests through travis-ci.org. The difference with regular automated tests is the requirement to include this package into a framework and set specific configuration options to run the tests.
So the requirement would be:
install laravel
add my package as dependency
set some travis specific configurations like the travis database access
run migrations of laravel
run migrations specific for package or run an artisan command
run package specific unit tests
I searched everywhere; asked on laravel forums, asked in a travis community chat and saw this topic being closed as too localized (although an answer would have certainly been helpful now). I'm hoping my question is fit to remain open.
At this time I have the following configuration:
language: php
php:
- 5.5
- 5.6
- hhvm
addons:
hosts:
- system.hyn.me
- tenant.hyn.me
before_install:
- sudo composer self-update
install:
- composer create-project laravel/laravel
- cd ./laravel
- composer require hyn-me/multi-tenant ~0.1.0
- composer update
before_script:
- cp .env.travis .env
- export APP_ENV="testing"
- php artisan migrate -q -n --path ./vendor/hyn-me/multi-tenant/src/migrations
- cd ./vendor/hyn-me/multi-tenant
script: phpunit
Yet my knowledge of travis (thus far) is limited and before I send in an unneeded number of commits to fix my problems I'd rather have your opinion on what would be a good method to test integration into a framework.
Ps. this concerns the package hyn/multi-tenant.
Advise on how to keep this question as generic as possible would be helpful. I hope without explicitly mentioning best practice and requesting integration into framework examples helps in defining the scope of the answers.
So after weeks of pushing commits into travis, I finally made this work.
The .travis.yml:
language: php
sudo: true
php:
- 5.5
- 5.6
- 7.0
- hhvm
addons:
hosts:
- system.hyn.me
- tenant.hyn.me
install:
# fix ipv6 issue that prevented composer requests and resulted in failing builds
- sudo sh -c "echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf"
# updates composer on travis
- travis_retry composer self-update
# clear composer cache, might speed up finding new tags
- travis_retry composer clear-cache
# set the global github token, so connections won't be cancelled
- composer config -g github-oauth.github.com $GITHUB_TOKEN
# create a new database for the hyn connection
- mysql -e 'create database hyn;' -uroot
- mysql -e "grant all privileges on *.* to 'travis'#'localhost' with grant option;" -uroot
# create a new laravel project in the subfolder laravel (default composer behaviour)
- composer create-project laravel/laravel
# set global variables
- export DB_USERNAME=travis DB_DATABASE=hyn DB_PASSWORD= QUEUE_DRIVER=sync
script:
# run the script calling unit tests and so on
- ./scripts/travis.sh
after_script:
- if [[ $TRAVIS_PHP_VERSION != '7.0' ]]; then php vendor/bin/ocular code-coverage:upload --format=php-clover ${TRAVIS_BUILD_DIR}/coverage.clover; fi
And the scripts/travis.sh
#!/bin/bash
# e causes to exit when one commands returns non-zero
# v prints every line before executing
set -ev
cd ${TRAVIS_BUILD_DIR}/laravel
BRANCH_REGEX="^(([[:digit:]]+\.)+[[:digit:]]+)$"
if [[ ${TRAVIS_BRANCH} =~ $BRANCH_REGEX ]]; then
echo "composer require ${TRAVIS_REPO_SLUG}:${TRAVIS_BRANCH}"
composer require ${TRAVIS_REPO_SLUG}:${TRAVIS_BRANCH}
else
echo "composer require ${TRAVIS_REPO_SLUG}:dev-${TRAVIS_BRANCH}"
# development package of framework could be required for the package
composer require hyn-me/framework "dev-master as 0.1.99"
composer require "${TRAVIS_REPO_SLUG}:dev-${TRAVIS_BRANCH}#${TRAVIS_COMMIT}"
fi
# moves the unit test to the root laravel directory
cp ./vendor/${TRAVIS_REPO_SLUG}/phpunit.travis.xml ./phpunit.xml
phpunit
# phpunit --coverage-text --coverage-clover=${TRAVIS_BUILD_DIR}/coverage.clover
This code might change due to new Laravel versions or changes in travis. If this is the case, you will find the latest release here.