Use a postgres database with symfony3 - php

We are retro-engineering a web application using a postgres database. We want to refactor the code using the symfony framework but we are facing a problem to use the database with it.
For the moment our project is working with a MySQL database just using the same structure for the used tables. The problem is that, now, we need to use the postgresql database because there is a lot of data and it's not really adapted to MySQL as far as we know.
We made a lot of research but we didn't succeed to create a postgresql database in the symfony project.
First, we tried to apply this tutorial : http://www.tutodidacte.com/symfony2-utiliser-une-base-de-donnee-postgresql we adapted it as much as possible for our project.
Here is our config.yml
imports:
- { resource: parameters.yml }
- { resource: security.yml }
- { resource: services.yml }
# Put parameters here that don't need to change on each machine where the app is deployed
# http://symfony.com/doc/current/best_practices/configuration.html#application-related-configuration
parameters:
locale: en
framework:
#esi: ~
#translator: { fallbacks: ["%locale%"] }
secret: "%secret%"
router:
resource: "%kernel.root_dir%/config/routing.yml"
strict_requirements: ~
form: ~
csrf_protection: ~
validation: { enable_annotations: true }
#serializer: { enable_annotations: true }
templating:
engines: ['twig']
default_locale: "%locale%"
trusted_hosts: ~
trusted_proxies: ~
session:
# http://symfony.com/doc/current/reference/configuration/framework.html#handler-id
handler_id: session.handler.native_file
save_path: "%kernel.root_dir%/../var/sessions/%kernel.environment%"
fragments: ~
http_method_override: true
assets: ~
# Twig Configuration
twig:
debug: "%kernel.debug%"
strict_variables: "%kernel.debug%"
# Doctrine Configuration
doctrine:
dbal:
default_connection: default
connections:
#Mysql
default:
driver: pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
#Postgresql
pgsql:
driver: pdo_pgsql
host: localhost
port: 5432
dbname: "%psql_database_name%"
user: root
password: "%psql_database_password%"
charset: UTF8
#mapping_types:
#geometry: string
orm:
default_entity_manager: default
auto_generate_proxy_classes: "%kernel.debug%"
#naming_strategy: doctrine.orm.naming_strategy.underscore
#auto_mapping: true
entity_managers:
default:
connection: default
# lister les Bundles utilisant la connexion par defaut
#mappings:
#monprojetmysqlBundle: ~
#tutoUserBundle: ~
pgsql:
connection: pgsql # connection name for your additional DB
# bundles utilisant la connexion Postgresql
#mappings:
# PostgresqlBundle: ~
# Swiftmailer Configuration
swiftmailer:
transport: "%mailer_transport%"
host: "%mailer_host%"
username: "%mailer_user%"
password: "%mailer_password%"
spool: { type: memory }
but when we launched this command : php bin/console doctrine:database:create --connection=pgsql we had this answer :
[Doctrine\DBAL\Exception\DriverException]
An exception occured in driver: could not find driver
[Doctrine\DBAL\Driver\PDOException]
could not find driver
[PDOException]
could not find driver
doctrine:database:create [--connection [CONNECTION]] [--if-not-exists] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-e|--env ENV] [--no-debug] [--] <command>
It seems that we don't have the module pdo_pgsql so we searched how to install it.
To do it, we applied the script proposed on this github page : https://gist.github.com/doole/8651341#file-install_psql_php-sh
We changed the version of postgres.app to 9.5.
After a few tries, we finally succeeded to have pdo_pgsql on the result of php -m.
But know, we have this answer when we launch the command : php bin/console doctrine:database:create --connection=pgsql
PHP Warning: PHP Startup: pdo_pgsql: Unable to initialize module
Module compiled with module API=20131226
PHP compiled with module API=20121212
These options need to match
in Unknown on line 0
PHP Warning: PHP Startup: pgsql: Unable to initialize module
Module compiled with module API=20131226
PHP compiled with module API=20121212
These options need to match
in Unknown on line 0
[Doctrine\DBAL\Exception\DriverException]
An exception occured in driver: could not find driver
[Doctrine\DBAL\Driver\PDOException]
could not find driver
[PDOException]
could not find driver
doctrine:database:create [--connection [CONNECTION]] [--if-not-exists] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-e|--env ENV] [--no-debug] [--] <command>
We tried to do this : Install PHP with Postgresql on MAC using homebrew
but it didn't change anything. Now we have 5 PHP WARNING for pdo_pgsql and pgsql when we launch the command php bin/console doctrine:database:create --connection=pgsql
We have also seen this :
How to change a database to postgresql with Symfony 2.0? and this : How to create 2 connections (mysql and postgresql) with Doctrine2 in Symfony2 but it didn't really help because the first one concern debian and we are working on OS X El Capitan and the second one don't tell more than the previous tutorial.
Finally, the only hope we have is that someone can help us...
Thank you in advance.

Finally, I fixed it by removing all the files that manage php and reinstalling php with homebrew.
---- Removing php ----
First, I used the following command from root (cd /) to find all the files starting by "php"
find . -name "php*"
Depending on the results (you might have a lot), remove all the files that need to be removed (at this point it's your judgement that matter). For example, I removed files in /usr/local and /usr/bin but not in /Applications or /Homebrew.
Examples :
rm -Rf /usr/bin/php*
rm -Rf /usr/local/php*
Sometimes, you can have a "permission denied" error even with sudo but it didn't make problem at the end.
---- Reinstalling php ----
Once everything concerning php is removed, you can reinstall it using the following command line :
brew install php56 --with-postgresql
If you have a "libz not found" error, you will need to launch the following command :
xcode-select --install
and relaunch the installation with :
brew reinstall php56 --with-postgresql
If everything went well, you will only have to define the field date.timezone in php.ini and you will have new php system. You can check that you have the pdo_pgsql module installed using this commande line : php -m.
---- Connect your database to your symfony project ----
First, you need to modify the file app/config/parameters.yml in your project by adding the following code :
# Postgresl
psql_database_driver: pdo_pgsql
psql_database_host: 127.0.0.1
psql_database_port: 5432
psql_database_name: your_database_name
psql_database_user: your_user_name
psql_database_password: your_password
The fields host and port can be different but this two are the default values for symfony and a postgres database.
Then, you will have to modify the file app/config/config.yml at the Doctrine Configuration level this way :
# Doctrine Configuration
doctrine:
dbal:
default_connection: pgsql
connections:
#Mysql
default:
driver: pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
#Postgresql
pgsql:
driver: pdo_pgsql
host: "%psql_database_host%"
port: "%psql_database_port%"
dbname: "%psql_database_name%"
user: "%psql_database_user%"
password: "%psql_database_password%"
charset: UTF8
#mapping_types:
#geometry: string
orm:
auto_generate_proxy_classes: "%kernel.debug%"
naming_strategy: doctrine.orm.naming_strategy.underscore
auto_mapping: true
This is an example you can adapt as you wish.
Now, you can connect your database to your project with this command line :
php bin/console doctrine:database:create --connection=pgsql
If you already have entities in your src/AppBundle/Entity you can create your tables with :
php bin/console doctrine:schema:update --force
Everything must be alright now. I hope, it will help someone else who faces this kind of problems.

Related

Problem to create two databases with doctrine in Symfony

I'm working on a Symfony project, using doctrine as ORM with MariaDB as a driver.
To test my different job, I'm working with gitlab-runner in local on MacOs.
Problem
I create a job to test the creation of my different databases and the migrations for both.
I'm using the symfony-cli command line to create my databases from the doctrine's file configuration (see in the section Source File)
The command line to create a database with symfony-cli
$ php bin/console doctrine:database:create --env=test --if-not-exists --connection=[CONNECTION_NAME]
The error prompt when the command line from is asked to run
$ php bin/console doctrine:database:create --env=test --if-not-exists --connection=CleanerFuture
Could not create database `cf_CleanerFuture` for connection named CleanerFuture
An exception occurred while executing 'CREATE DATABASE `cf_CleanerFuture`':
SQLSTATE[42000]: Syntax error or access violation: 1044 Access denied for user 'myapptest'#'%' to database 'cf_CleanerFuture'
Goal
Well my goal it's to try to create the database base with this command line to run my job and past to the next one
Source File
.gitlab-ci.yml
This is the job I try to execute
doctrine-migrations:
image: php:7.3
stage: Migrations
services:
- name: mysql:5.7
alias: mysql
variables:
ENV: test
MYSQL_ROOT_PASSWORD: pass_test
MYSQL_DATABASE: cf_Central
MYSQL_USER: myapptest
MYSQL_PASSWORD: myapptest
DATABASE_URL: 'mysql://myapptest:myapptest#mysql:3306/'
before_script:
- apt-get update
- apt-get install -y git libzip-dev
- curl -sSk https://getcomposer.org/installer | php -- --disable-tls && mv composer.phar /usr/local/bin/composer
- docker-php-ext-install mysqli pdo pdo_mysql zip
- curl -sS https://get.symfony.com/cli/installer | bash
- mv /root/.symfony/bin/symfony /usr/local/bin/symfony
- composer remove ext-xdebug
- composer install
script:
- php bin/console doctrine:database:drop --force --if-exists --env=test --connection=default
- php bin/console doctrine:database:drop --force --if-exists --env=test --connection=CleanerFuture
- php bin/console doctrine:database:create --env=test --if-not-exists --connection=CleanerFuture
- php bin/console doctrine:migrations:migrate --env=test
allow_failure: false
config/test/doctrine.yaml
doctrine:
dbal:
default_connection: default
connections:
default:
url: '%env(resolve:DATABASE_URL)%cf_central'
server_version: "mariadb-10.4.11"
driver: 'pdo_mysql'
charset: utf8
CleanerFuture:
url: '%env(resolve:DATABASE_URL)%cf_CleanerFuture'
server_version: "mariadb-10.4.11"
driver: 'pdo_mysql'
charset: utf8
orm:
auto_generate_proxy_classes: true
default_entity_manager: default
entity_managers:
default:
naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
connection: default
mappings:
Central:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity/Central'
prefix: 'App\Entity\Central'
CleanerFuture:
naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
connection: CleanerFuture
mappings:
Client:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity/Client'
prefix: 'App\Entity\Client'
url: '%env(resolve:DATABASE_URL)%cf_CleanerFuture' in your config/test/doctrine.yaml file:
The part cf_CleanerFuture will be added to the string value of the variable DATABASE_URL which is defined in your .env file.
If your .env file value for the datbase connection contains something like
DATABASE_URL="mysql://symfony:symfony#database:3306/symfony?serverVersion=8.0"
the processed content of your config will be
doctrine:
dbal:
default_connection: default
connections:
default:
url: 'mysql://symfony:symfony#database:3306/symfony?serverVersion=8.0cf_central'
'mysql://symfony:symfony#database:3306/symfony?serverVersion=8.0cf_central' seems to be valid (doctrine can connect and the parameter serverVersion will be ignored if not usable), but the value of the parameter serverVersion is now 8.0cf_central. That makes no sense, is not a valid version number and you haven't configured a different database connection.
It is more recommended to use two different environment variables and store the whole connection URL in these variables:
# config/packages/doctrine.yaml
doctrine:
dbal:
default_connection: default
connections:
default:
# configure these for your database server
url: '%env(resolve:DATABASE_URL)%'
driver: 'pdo_mysql'
server_version: '5.7'
charset: utf8mb4
customer:
# configure these for your database server
url: '%env(resolve:DATABASE_CUSTOMER_URL)%'
driver: 'pdo_mysql'
server_version: '5.7'
charset: utf8mb4
See also the Symfony documentation which describes how to configure multiple database connections: https://symfony.com/doc/current/doctrine/multiple_entity_managers.html

Symfony 4.2 doesnt connect to Microsoft SQL Server

I'm setting up a new project on Symfony 4.2, and I'm trying to connect to Microsoft SQL Server as database, so I have the following files:
/config/packages/doctrine.yaml:
parameters:
# Adds a fallback DATABASE_URL if the env var is not set.
# This allows you to run cache:warmup even if your
# environment variables are not available yet.
# You should not need to change this value.
env(DATABASE_URL): ''
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_sqlsrv'
#server_version: '5.7'
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
url: '%env(resolve:DATABASE_URL)%'
orm:
auto_generate_proxy_classes: true
naming_strategy: doctrine.orm.naming_strategy.underscore
auto_mapping: true
mappings:
App:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity'
prefix: 'App\Entity'
alias: App
/.env part:
###> doctrine/doctrine-bundle ###
# Format described at http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/configuration.html#connecting-using-a-url
# For an SQLite database, use: "sqlite:///%kernel.project_dir%/var/data.db"
# Configure your db driver and server_version in config/packages/doctrine.yaml
DATABASE_URL=mssql://sa:root#DESKTOP-A9GIM4A\ISTANZA:1433/linkontro
###< doctrine/doctrine-bundle ###
MSSQL Service is running correctly, but when I try to run on CLI:
php bin/console doctrine:schema:validate
it returns this error:
Mapping
-------
[OK] The mapping files are correct.
Database
--------
In PDOConnection.php line 31:
SQLSTATE[08001]: [Microsoft][ODBC Driver 13 for SQL Server]TCP Provider: No connection could be made because the target machine actively refused it.
In PDOConnection.php line 27:
SQLSTATE[08001]: [Microsoft][ODBC Driver 13 for SQL Server]TCP Provider: No connection could be made because the target machine actively refused it.
doctrine:schema:validate [--skip-mapping] [--skip-sync] [--em [EM]] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-e|--env ENV] [--no-debug] [--] <command>
Any clue of what's going on?
I tried to write a simple PHP script to check the connection, and it worked.
So credentials are correct, MSSQL Service is on, but the only one who can't connect is Symfony.
From the error description: the php-initiated connection is refused by the MS SQL server. This could mean:
there is no server listening on the address/port you are trying to connect to. Maybe DATABASE_URL is not set correctly and doctrine is trying to connect to the driver's default (i.e. localhost:1433)
the server you are connecting to (or some firewall in between you and the server) is refusing connections - but this is less likely.
Try to add a -v or -vv to the command for more verbosity.
Because of the \ and some unknown reasons,
I find it easear to have it to work using the yaml configuration like following.
doctrine:
dbal:
#url: '%env(resolve:DATABASE_URL)%'
#driver_class: Realestate\MssqlBundle\Driver\PDODblib\Driver
driver: 'pdo_sqlsrv'
host: DESKTOP-A9GIM4A\ISTANZA
dbname: linkontro
user: sa
password: root

Symfony2 gives a ParameterNotFoundException on a database driver

Lately I have created an extra database with it's own user. Therefore I have created an extra database driver in the parameters.yml. This is, as far as I know, the standard approach for this kind of situations. So far, it works. In one of the services I created, I can use this database driver. When running the code on the website, there are no problems at all.
But of course there is a problem, otherwise I won't asking for your guys help.
I'm trying to install a plugin by running the following command:
$ ./composer.phar require pugx/autocompleter-bundle
This gives the following error:
[Symfony\Component\DependencyInjection\Exception\ParameterNotFoundException]
You have requested a non-existent parameter "database_driver_geo". Did you mean this: "database_driver"?
Script Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache handling the post-update-cmd event terminated with an exception
Installation failed, reverting ./composer.json to its original content.
[RuntimeException]
An error occurred when executing the "'cache:clear --no-warmup'" command.
Some other posts say that the error about the cache has something to do with the file/dir rights. But that doesn't seem to be the problem, because when the configuration of the geo driver is removed, these kind of errors do not appear.
I'm running Symfony 2.5
[EDIT: Added parameters.yml file]
My parameters.yml looks like this:
# This file is auto-generated during the composer install
parameters:
# Default database
database_driver: pdo_mysql
database_host: ***
database_port: ***
database_name: ***
database_user: ***
database_password: ***
# Geo database
database_driver_geo: pdo_mysql
database_host_geo: ***
database_port_geo: ***
database_name_geo: ***
database_user_geo: ***
database_password_geo: ***
mailer_transport: ***
mailer_host: ***
mailer_user: ***
mailer_password: ***
locale: ***
secret: ***
[EDIT: Added config.yml file]
The doctrine section in the config.yml file:
# Doctrine Configuration
doctrine:
dbal:
default_connection: default
connections:
default:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
mapping_types:
enum: string
bit: integer
# if using pdo_sqlite as your database driver, add the path in parameters.yml
# e.g. database_path: %kernel.root_dir%/data/data.db3
# path: %database_path%
geo:
driver: %database_driver_geo%
host: %database_host_geo%
port: %database_port_geo%
dbname: %database_name_geo%
user: %database_user_geo%
password: %database_password_geo%
charset: UTF8
mapping_types:
enum: string
bit: integer
# if using pdo_sqlite as your database driver, add the path in parameters.yml
# e.g. database_path: %kernel.root_dir%/data/data.db3
# path: %database_path%
orm:
default_entity_manager: default
entity_managers:
default:
connection: default
mappings:
***CoreBundle: ~
geo:
connection: geo
mappings:
***GeoBundle: ~
auto_generate_proxy_classes: %kernel.debug%
I hope there's someone that can help me fix this problem.
Kind regards,
Malcolm
As mentioned in comments, parameters.yml file is autmatically rebuilt after composer update or install command. You can see that in your composer.json file in scripts section:
"scripts": {
"post-install-cmd": [
"Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
// other commands...
],
"post-update-cmd": [
"Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
// other commands...
]
},
You can of course turn off this feature if you don't like it. But it may be useful when used properly.
Therefore when you install some package via composer you're losing parameters that you put directly into parameters.yml.
What you should do is to make use of parameters.yml.dist file which is used to build parameters.yml. It should provide application parameters values (if they are the same for every instance of app) or default values if parameters are different for every environment (prod/dev).
In your case it's the second use case (default values), since DB credentials will change for every server. It's actually exactly the same as configuration of default DB connection. parameters.yml.dist contains some default values for these params.

Setup Symfony2 and Postgresql on OS X 10.10 Yosemite?

I can't found a way to setup Symfony with a postgresql database on OS X 10.10 (clean install). Here is what I have done:
1) Install PHP 5.6 from Liip (specially built for Symfony): http://php-osx.liip.ch/
2) Install Postgres.app: http://postgresapp.com/
3) Install Symfony: http://symfony.com/
4) Setup my parameters.yml
parameters:
database_driver: pdo_pgsql
database_host: 127.0.0.1
database_port: 5432
database_name: bachelor
database_user: username
database_password: ~
I have created the DB, verified that I have the right PHP cli, the right pgql cli, etc.
I have created a bundle in Symfony and some entities (no errors here, it works with MySQL).
But when I launch any command to interact with the DB like "php app/console doctrine:schema:update --dump-sql", I have this error:
[PDOException]
SQLSTATE[HY000] [2006] MySQL server has gone away
[Symfony\Component\Debug\Exception\ContextErrorException]
Warning: PDO::__construct(): MySQL server has gone away
But why? I don't use a MySQL server...
Does someone have any solution?
Thanks!
Can you show us your /app/config/config.yml ? please
Did you change the dbal connection value ?
# Doctrine Configuration
doctrine:
dbal:
default_connection: pgsql
connections:
#Postgresql
pgsql:
driver: "%database_driver%"
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8

How do I use Symfony doctrine:fixtures:load with multiple entity managers?

I have setup a new project with multiple entity managers, when I try to load the data fixtures I get an MappingException as the console tries to load Fixtures for everything rather than the entity manager I specified.
Here is the doctrine section from my config.yml
doctrine:
dbal:
connections:
default:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: %database_charset%
symblog:
driver: %database_driver_blog%
host: %database_host_blog%
port: %database_port_blog%
dbname: %database_name_blog%
user: %database_user_blog%
password: %database_password_blog%
charset: %database_charset_blog%
orm:
auto_generate_proxy_classes: %kernel.debug%
entity_managers:
default:
connection: default
mappings:
IncompassAuthBundle: ~
IncompassUserBundle: ~
IncompassApiBundle: ~
IncompassSurgeryBundle: ~
IncompassVendorBundle: ~
IncompassHospitalBundle: ~
dql:
datetime_functions:
date: Mapado\MysqlDoctrineFunctions\DQL\MysqlDate
symblog:
connection: symblog
mappings:
IncompassBlogBundle: ~
dql:
datetime_functions:
date: Mapado\MysqlDoctrineFunctions\DQL\MysqlDate
As you can see I've setup a separate connection and entity manager for the symblog tutorial stuff.
When I try
php app/console doctrine:fixtures:load --em=default
I get this
Careful, database will be purged. Do you want to continue Y/N ?Y
> purging database
> loading [1] Incompass\BlogBundle\DataFixtures\ORM\BlogFixtures
[Doctrine\Common\Persistence\Mapping\MappingException]
The class 'Incompass\BlogBundle\Entity\Blog' was not found in the chain configured namespaces Incompass\AuthBundle\Entity, Incompass\UserBundle\Entity, Incompass\
SurgeryBundle\Entity, Incompass\VendorBundle\Entity, Incompass\HospitalBundle\Entity, FOS\UserBundle\Model
doctrine:fixtures:load [--fixtures[="..."]] [--append] [--em="..."] [--purge-with-truncate]
When I try
php app/console doctrine:fixtures:load --em=symblog
I get
Careful, database will be purged. Do you want to continue Y/N ?Y
> purging database
> loading [1] Incompass\BlogBundle\DataFixtures\ORM\BlogFixtures
> loading [1] Incompass\SurgeryBundle\DataFixtures\ORM\SurgeryStatusFixtures
[Doctrine\Common\Persistence\Mapping\MappingException]
The class 'Incompass\SurgeryBundle\Entity\SurgeryStatus' was not found in the chain configured namespaces Incompass\BlogBundle\Entity
doctrine:fixtures:load [--fixtures[="..."]] [--append] [--em="..."] [--purge-with-truncate]
So the console command appears to be ignoring the "--em=foobar" option and is trying to load up all the data fixtures it finds.
How can I get doctrine:fixtures:load to only use the specified entity manager?
After Phils comment I moved all my Fixtures into a FixturesBundle and did this
php app/console doctrine:fixtures:load --fixtures=src/Incompass/FixturesBundle/DataFixtures/ORM
For the Blog Fixtures I also had to specify the entity manager
php app/console doctrine:fixtures:load --fixtures=src/Incompass/BlogBundle/DataFixtures/ORM --em=symblog

Categories