I have a reporting application build with Symfony 2.8.14/ Doctrine. One of my reports takes about 2 minutes to run and executes a series of queries (https://dba.stackexchange.com/questions/157981/reporting-query-blocks-other-query-but-isolation-level-read-uncommitted-set/159495#159495).
I have found that the locking appears to be happening at Symfony level, because the same pages can be loaded no problem if I switch into app_dev.php/ or run the MySQL queries on the command line, while the report is running.
Is there a connection limit or other locking I could have turned on accidentally?
My Doctrine configuration
# Doctrine Configuration
doctrine:
dbal:
default_connection: default
connections:
default:
driver: pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
options:
1001: true
orm:
auto_generate_proxy_classes: "%kernel.debug%"
default_entity_manager: default
entity_managers:
default:
connection: default
naming_strategy: doctrine.orm.naming_strategy.underscore
mappings:
AppBundle: ~
FOSUserBundle: ~
errorlog:
connection: default
naming_strategy: doctrine.orm.naming_strategy.underscore
mappings:
AppBundle: ~
Apache2 configuration - mpm-itk
<IfModule mpm_itk_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestWorkers 150
MaxConnectionsPerChild 0
</IfModule>
Not using PHP-FPM as stated in comments, but using mod_php:
<FilesMatch \.php$>
SetHandler application/x-httpd-php
</FilesMatch>
I'm running MySQL 5.7.11 on m4.2xlarge which according to this: http://pushentertainment.com/rds-connections-by-instance-type/ allows for 2500+ connections.
Almost every time I've encountered locking that wasn't in the database, it has been PHP session serialization.
With the default Symfony NativeFileSessionHandler, PHP will wait to obtain a file lock before opening the session file; the lock won't be released until session is closed (ie the request is finished). This helps avoid race conditions between processes reading/writing session data.
If this is the cause then opening 2 tabs in the same browser (print $session->getId to confirm the session ID is shared) will block, but trying with different browsers (different session IDs) will not block. Be aware that depending on the state of ignore_user_abort() previous requests cancelled in the browser but still processing will also block any new requests.
As to why this would work on dev/prod, differences in session handler settings in your config file will do this.
Alternately, if that's not the cause I'd use strace -p PID and /proc/PID to determine what system call the apache/PHP process is blocked on (can be annoying to work out which is the blocked process but you only have 5 apache processes and a 2 minute window to find the right one)
It's not caused by the symfony, its caused by the webserver. You can refer to my post about the laravel another php mvc frame requests.
To solve this run the time cost process in the background.
Related
Hey guys so im trying to connect to my databse that's already created in postgresql (I'm using PGAdmin4 for the user interface of postgresql).
I'm on Symfony CLI version v4.16.3 get three different Timeout.
In details i have a .env file that take the my database url :
DATABASE_URL=postgresql://root:password#127.0.0.1:35583/nd2d?serverVersion=12.3
My doctrice.yaml is set up like this :
doctrine:
dbal:
url: '%env(resolve:DATABASE_URL)%'
orm:
auto_generate_proxy_classes: true
naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
auto_mapping: true
mappings:
App:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity'
prefix: 'App\Entity'
alias: App
I had issue but found a way to fix they were related to pdo drivers for postgresql.
but after i launch the command bin/console make:migration i get those 3 errors:
In AbstractPostgreSQLDriver.php line 73:
An exception occurred in driver: SQLSTATE[08006] [7] timeout expired
In PDOConnection.php line 31:
SQLSTATE[08006] [7] timeout expired
In PDOConnection.php line 27:
SQLSTATE[08006] [7] timeout expired
I avoided working on heavyn program with PHP or any framework related to it but this time i don't have a choice, soo much configuration i cant take it anymore, been on this for soo many hours now. What am i missing.
Thank you :)
Error came from listening on the wrong port, i was listening to the port 35583, which is the port that PgAdmin is showed on the web browse.
DATABASE_URL=postgresql://root:password#127.0.0.1:35583/nd2d?serverVersion=12.3
The port i should have been listening to is the default postgresql port of 5432
In a Symfony 4.3 application with Elastcsearc 6.8 plus friendsofsymfony/elastica-bundle v5.1.0
an index creation task takes 18 minutes to complete either with or without enqueue/enqueue-bundle 0.9.12 and enqueue/fs 0.9.12. Is there a package I'm missing (altho' enqueue says it's a complete solution) or a configuration error?
fos_elasitica.yaml:
fos_elastica:
serializer: ~
clients:
default: { host: localhost, port: 9200 }
indexes:
house_date:
types:
house_date:
serializer:
groups: [house_date]
persistence:
# the driver can be orm, mongodb or phpcr
driver: orm
model: App\Entity\Contact
provider: ~
finder: ~
enqueue.yaml:
enqueue:
default:
transport: '%env(resolve:ENQUEUE_DSN)%'
client: ~
enqueue_elastica:
transport: '%enqueue.default_transport%
'
Edit:
After much exploration I've inched along but without ultimate success. Added was enqueue/elastica-bundle and enqueue.yaml has been edited to appear as above.
[An identical installation in Windows reaches a 256M memory limit at about 54% completion, again regardless of the presence of the enqueue components.]
It is likely true that the seemingly long time to populate the index was the result of an improper definition. The definition incorporated four entities (via relationships). By changing Contact to Household, which has a one to many relationship with Contact, time to populate the index was reduced was reduced by a factor of 10. As a result I'm abandoning this question and marking it Answered.
I have an existing entity in a Symfony3 project that I am trying to add an entity manager to. My doctrine.yml file looks like this
doctrine:
dbal:
default_connection: default
connections:
default:
driver: pdo_mysql
host: localhost
port: 3306
dbname: fullstackdb
user: root
password: root
charset: UTF8
lego:
driver: pdo_mysql
host: localhost
port: 3306
dbname: fullstackdb
user: root
password: root
charset: UTF8
orm:
default_entity_manager: default
entity_managers:
auto_mapping: true
default:
connection: default
mappings:
lego:
connection: lego
mappings:
AppBundle: ~
However anytime I try to access this entity manager through php bin/console doctrine:database:create --connection=lego
, it says the manager does not exist! I'm not sure if I'm missing a step - the only thing I've done to create the manager is to make the yml file.
It doesn't show up when I run bin/console debug:container.
Grateful for any help!
1) You are using the same schema name (database name) on the same host in two different entity managers without schema_filters. This means that when you do an update to one of the entity managers it will attempt to delete the data of the other entity manager.
2) You are asking about a entity manager but in the command you are passing a connection.
3) Run php bin/console debug:container | grep doctrine
This will give you all the doctrine services. Update your question accordingly.
4) Provide the exact command and exact error you get so we can track it down to the origin
5) For a more information run the command in verbose mode adding -v at the end
6) Are you importing your doctrine.yml in your config.yml ?
I have Entity(s) and EntityHtml(s) entities which has one-to-one relashionship (Entity stores metadata and EntityHtml acts like a cache, storing ready HTML chunks for rendering).
I have defined a relationship in Entity class:
/**
* #ORM\OneToOne(targetEntity="EntityHtml")
* #ORM\JoinColumn(name="entityId", referencedColumnName="entityId")
*/
private $entityHtml;
but it isn't working. Also I have a kind of feeling, that annotations don't work at all, because changing them has no effect upon workability of the application.
On the other hand, messing with .orm.xml(s) reflects in how application works.
Can I tell Symfony to update ORM XMLs based on changes to annotations?
Should I duplicate relation meta to XML?
Does Symfony use info at both XML and annotations or does it choose one source?
config.yml is default one:
# Doctrine Configuration
doctrine:
dbal:
driver: pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
# if using pdo_sqlite as your database driver:
# 1. add the path in parameters.yml
# e.g. database_path: "%kernel.root_dir%/data/data.db3"
# 2. Uncomment database_path in parameters.yml.dist
# 3. Uncomment next line:
# path: "%database_path%"
orm:
auto_generate_proxy_classes: "%kernel.debug%"
naming_strategy: doctrine.orm.naming_strategy.underscore
auto_mapping: true
Does it make any difference if you try this:
/**
* #ORM\OneToOne(targetEntity="EntityHtml")
* #ORM\JoinColumn(name="entity_Id", referencedColumnName="entityId")
*/
I'm thinking that "entityId" is the Id in EntityHtml, and you need to specify a different "name" value in JoinColumn. I think I ran into that problem.
Try it - I'm not sure if it will work.
Figured this one out. It's a configuration issue. In order to make annotations work (xml is default option), you have to explicitly configure it:
# Doctrine Configuration
doctrine:
...
orm:
...
mappings:
AppBundle:
type: annotation
Unfortunatelly that's not specified in tutorials.
I use Doctrine Entity Manager in my code several times, I used in this way for example:
$em = $this->getDoctrine()->getManager();
$countries = $em->getRepository('CommonBundle:Country')->findAll();
It's possible to use Doctrine Cache in this format? I see a lot of docs but all of them relative to Doctrine Query Builder, any advice? How to enable cache for SELECT queries?
Caching is not working (tough)
I'm dealing with caching results and I do this in order to cache some queries:
Enable APC under orm configuration at config.yml:
doctrine:
dbal:
driver: "%database_driver%"
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
orm:
auto_generate_proxy_classes: "%kernel.debug%"
auto_mapping: true
metadata_cache_driver: apc
result_cache_driver: apc
query_cache_driver: apc
Attach a repository to my entity:
/**
* #ORM\Entity(repositoryClass="CommonBundle\Entity\Repository\CountryRepository")
*/
class Country {
....
Write a method inside repository class for get the data and cache the results:
public function getCountries() {
$qb = $this->createQueryBuilder('c');
$query = $qb->getQuery();
$query->useResultCache(true, 3600, 'countries_cache');
return $query->getResult();
}
But any time I reload the page the query is executed as shown in the profiler (see picture below):
What I did wrong? The cache only works for production or also works for development? Should't get cached results instead of query the database once and once?
EntityRepository API does NOT allow to enable ResultCache with find* methods.
You have to use a QueryBuilder and/or DQL requests.
I wrote an article about Doctrine caching, maybe it can help you : http://blog.alterphp.com/2014/05/doctrine2-optimization-with-apc-cache.html
You have to enable cache for results in your symfony configuration :
doctrine:
orm:
metadata_cache_driver: apc # or memcache
result_cache_driver: apc # or memcache
query_cache_driver: apc # or memcache
Then on every request that need to be cached, call
$query->useResultCache(true)
More advanced options available here : http://docs.doctrine-project.org/en/2.0.x/reference/caching.html#result-cache