Doctrine 2 query caching was a little surprising for me. I had two update queries that follow one-by-one:
function upd($user, $passwordHash) {
$qb = $this->entityManager->createQueryBuilder()
->update(UserEntity::__class, 'u')
->set('u.password', '?1')
->where('u = ?0')
->setParameters(array($user, $passwordHash));
$qb->getQuery()->execute();
}
I updated password with 2 different values (ex. A, B), but user was the same:
upd($user, 'A');
upd($user, 'B');
The first query really updated database row. But the second just didn't make any changes in DB after execution.
Trying to solve this problem I found some workaround:
$qb->getQuery()->useQueryCache(false)->execute();
After disabling QueryCache both two queries changes DB row.
So, the questions are: why doctrine uses 1st query cache in 2nd UPDATE query? And why doctrine uses cache while it's two queries with different parameter ($passwordHash)?
Problem was found. It is an issue in doctrine 2 ORM code. My $user entity has inheritance, so the update uses Doctrine\ORM\Query\Exec\MultiTableUpdateExecutor. In that executor:
//FIXME (URGENT): With query cache the parameter is out of date. Move to execute() stage.
Source.
So the only workaround for now is to disable query cache for multitable updates:
$qb->getQuery()->useQueryCache(false)->execute();
I created new bug for it.
Related
i hope you are having a good time. i am learning laravel and the inscuctor talked about when you load relationships in laravel, like so
public function timeline()
{
$ids = $this->follows()->pluck('id');
$ids->push($this->id);
return Tweet::whereIn('user_id', $ids)->latest()->get();
}
and i have a follows relationship in my model, and he talked about this line
$ids = $this->follows()->pluck('id');
being better for performance than this line
$ids = $this->follows->pluck('id');
my question is, how does laravel pluck the ids in the first case, and how it queries the database
i hope im making sense, thanks for your time, and answer.
the following one executes a select query on database
$this->follows()->pluck('id');
the follows() returns a query builder (which is a not yet executed sql statement) and then on the result select the id column and returns a collection of ids
you can see the query by dumping the query builder by $this->follows()->dd()
Whereas in the second option
$this->follows->pluck('id')
up until $this->follows laravel executes a query and returns all the records as a collection instance, You will be able to see all the attributes on each of the records. And then ->pluck('id') is getting executed on the laravel collection class, which will do an operation I assume similar to the array_column function does and returns only the id column.
as you can easily see in the second operation the whole data set was retrieved first from the DB and then selected the required attribute/column (2 distinct and heavy operations). Where as in the first option we directly told eloquent to select only the required column, which is only one lighter operation compared to the second option.
I'm encountering a strange issue with Doctrine.
I need to query a simple table with only 1 inner join, which I something I have already done many times. But in this case something's odd: there are a lot of rows missing.
I have an entity called Policy. It is linked to a table on my Oracle database. There are 81k+ rows in this table. When querying this entity with the Doctrine query builder, I only get 5k results. I made this query as simple as possible for testing :
$qb = $em->createQueryBuilder();
$qb->select('p')->from('ErwMonitoringExtranetBundle:Policy', 'p');
$query = $qb->getQuery();
$policiesFull = $query->getResult();
The $policiesFull variable only contains 5k elements. There are no duplicates in the table.
The SQL query that is generated by Doctrine looks like this :
SELECT
r0_.node_group_name AS NODE_GROUP_NAME0,
r0_.policy_name AS POLICY_NAME1,
r0_.policy_description AS POLICY_DESCRIPTION2,
r0_.policy_group_name AS POLICY_GROUP_NAME3,
r0_.policy_type_name AS POLICY_TYPE_NAME4,
r0_.policy_name_on_agent AS POLICY_NAME_ON_AGENT5,
r0_.date_last_maj AS DATE_LAST_MAJ6,
r0_.om_name AS OM_NAME7,
r0_.id_node AS ID_NODE8
FROM
ewintranet.ref_monitored_ci;
Running the same exact query on Oracle returns the full table content.
Counting results through a doctrine query returns the correct number of rows :
$qb = $em->createQueryBuilder();
$qb->select('count(p)')->from('ErwMonitoringExtranetBundle:Policy', 'p');
$query = $qb->getQuery();
echo $query->getSingleScalarResult();
This returns 81k.
Does anybody know why all these rows disappear after using getResult() ?
This is what I would do:
Check the result using createQuery or createNativeQuery
Run the query from simple php script outside of symfony
If with all 3 methods you get the same issue then it might be problem with the size of data limitation.
I would start with max_execution_time and memory_limit settings in php.ini
I found also some oracle limitations that might be set at:
oci8.statement_cache_size in php.ini or in file oraaccess.xml in Oracle directory. It is also significant if you use APC for query caching.
Anyway what does app_dev.php say?
Okay, I found out what was causing my issue. The primary ID was wrong in my Entity declaration.
The oracle table had a composed primary key while in my entity the ID was only on one column. GetResult was making a DISTINCT on this column.
I'm developing app with FuelPHP & mySql and I'm using the provided ORM functionality. The problem is with following tables:
Table: pdm_data
Massive table (350+ columns, many rows)
Table data is rather static (updates only once a day)
Primary key: obj_id
Table: change_request
Only few columns
Data changes often (10-20 times / min)
References primary key (obj_id from table pdm_data)
Users can customize datasheet that is visible to them, eg. they can save filters (eg. change_request.obj_id=34 AND pdm_data.state = 6) on columns which then are translated to query realtime with ORM.
However, the querying with ORM is really slow as the table pdm_data is large and even ~100 rows will result in many mbs of data. The largest problem seems to be in FuelPHP ORM: even if the query itself is relatively fast model hydration etc. takes many seconds. Ideal solution would be to cache results from pdm_data table as it is rather static. However, as far as I know FuelPHP doesn't let you cache tables through relations (you can cache the complete result of query, thus both tables or none).
Furthermore, using normal SQL query with join instead of ORM is not ideal solution, as I need to handle other tasks where hydrated models are awesome.
I have currently following code:
//Initialize the query and use eager-loading
$query = Model_Changerequest::query()->related('pdmdata');
foreach($filters as $filter)
{
//First parameter can point to either table
$query->where($filter[0], $filter[1], $filter[2]);
}
$result = $query->get();
...
Does someone have a good solution for this?
Thanks for reading!
The slowness of the version 1 ORM is a known problem which is being addressed with v2. My current benchmarks are showing that v1 orm takes 2.5 seconds (on my machine, ymmv) to hydrate 40k rows while the current v2 alpha takes around 800ms.
For now I am afraid that the easiest solution is to do away with the ORM for large selects and construct the queries using the DB class. I know you said that you want to keep the abstraction of the ORM to ease development, one solution is to use as_object('MyModel') to return populated model objects.
On the other hand if performance is your main concern then the ORM is simply not suitable.
Is it possible (and how) in Doctrine 2 to create a query with a QueryBuilder and acquire a write lock for each of the matching rows? LockMode::PESSIMISTIC_WRITE can be used when fetching individual items with EntityRepository->find(), but I haven't been able to find such property for QueryBuilder.
The answer is to call setLockMode() on the Query object.
$qb = $em->createQueryBuilder();
$query = $qb->getQuery();
$query->setLockMode(LockMode::PESSIMISTIC_WRITE);
$results = $query->getResult();
The very last line of the Transactions and Concurrency documentation page shows that Query->setLockMode() is supported. A bit hard to notice, but it's there... ;)
I want to update one column for all rows in a table with a random number. As far as my research goes, there is no rand() in Doctrine by default. The options I see are 1. Add a custom DQL-Function, this would be MySQL specific, 2. Update every row with a PHP generated value.
Both options seem like bad practice to me. Is there something I'm missing?
I would go with native query. It is much simpler than creating custom DQL function.
$em = getEntityManager();
$tableName = $em->getClassMetadata('Your:Entity')->getTableName();
$em->getConnection()->exec('UPDATE '.$tableName.' SET column=RAND()');
But if You prefer DQL go with it.
But doing it in PHP will be the worst.
You will have to fetch all records first
You will have to update each row one by one
Database is not something You change every week so don't be afraid of using vendor specific functions.