I'm encountering a strange issue with Doctrine.
I need to query a simple table with only 1 inner join, which I something I have already done many times. But in this case something's odd: there are a lot of rows missing.
I have an entity called Policy. It is linked to a table on my Oracle database. There are 81k+ rows in this table. When querying this entity with the Doctrine query builder, I only get 5k results. I made this query as simple as possible for testing :
$qb = $em->createQueryBuilder();
$qb->select('p')->from('ErwMonitoringExtranetBundle:Policy', 'p');
$query = $qb->getQuery();
$policiesFull = $query->getResult();
The $policiesFull variable only contains 5k elements. There are no duplicates in the table.
The SQL query that is generated by Doctrine looks like this :
SELECT
r0_.node_group_name AS NODE_GROUP_NAME0,
r0_.policy_name AS POLICY_NAME1,
r0_.policy_description AS POLICY_DESCRIPTION2,
r0_.policy_group_name AS POLICY_GROUP_NAME3,
r0_.policy_type_name AS POLICY_TYPE_NAME4,
r0_.policy_name_on_agent AS POLICY_NAME_ON_AGENT5,
r0_.date_last_maj AS DATE_LAST_MAJ6,
r0_.om_name AS OM_NAME7,
r0_.id_node AS ID_NODE8
FROM
ewintranet.ref_monitored_ci;
Running the same exact query on Oracle returns the full table content.
Counting results through a doctrine query returns the correct number of rows :
$qb = $em->createQueryBuilder();
$qb->select('count(p)')->from('ErwMonitoringExtranetBundle:Policy', 'p');
$query = $qb->getQuery();
echo $query->getSingleScalarResult();
This returns 81k.
Does anybody know why all these rows disappear after using getResult() ?
This is what I would do:
Check the result using createQuery or createNativeQuery
Run the query from simple php script outside of symfony
If with all 3 methods you get the same issue then it might be problem with the size of data limitation.
I would start with max_execution_time and memory_limit settings in php.ini
I found also some oracle limitations that might be set at:
oci8.statement_cache_size in php.ini or in file oraaccess.xml in Oracle directory. It is also significant if you use APC for query caching.
Anyway what does app_dev.php say?
Okay, I found out what was causing my issue. The primary ID was wrong in my Entity declaration.
The oracle table had a composed primary key while in my entity the ID was only on one column. GetResult was making a DISTINCT on this column.
Related
i hope you are having a good time. i am learning laravel and the inscuctor talked about when you load relationships in laravel, like so
public function timeline()
{
$ids = $this->follows()->pluck('id');
$ids->push($this->id);
return Tweet::whereIn('user_id', $ids)->latest()->get();
}
and i have a follows relationship in my model, and he talked about this line
$ids = $this->follows()->pluck('id');
being better for performance than this line
$ids = $this->follows->pluck('id');
my question is, how does laravel pluck the ids in the first case, and how it queries the database
i hope im making sense, thanks for your time, and answer.
the following one executes a select query on database
$this->follows()->pluck('id');
the follows() returns a query builder (which is a not yet executed sql statement) and then on the result select the id column and returns a collection of ids
you can see the query by dumping the query builder by $this->follows()->dd()
Whereas in the second option
$this->follows->pluck('id')
up until $this->follows laravel executes a query and returns all the records as a collection instance, You will be able to see all the attributes on each of the records. And then ->pluck('id') is getting executed on the laravel collection class, which will do an operation I assume similar to the array_column function does and returns only the id column.
as you can easily see in the second operation the whole data set was retrieved first from the DB and then selected the required attribute/column (2 distinct and heavy operations). Where as in the first option we directly told eloquent to select only the required column, which is only one lighter operation compared to the second option.
I am working on one module in yii2 and please count me as a noob in this. I usually work in Laravel.
My query looks as below.
$query = SomeModel::find()
->joinWith(['relation1 s', 'relation2 r','relation3','relation4 c'])
->where(some where contions)
->where(some where contions);
The issue is in the view the yii2 debugger (Database Queries) shows "187 duplicated queries found."
My model doesn't have any loop around the query.
I have tried with removing joinWith relations but it will increase the duplicate queries to 220.
The query which gets repeated is one SELECT query
SELECT * FROM `relation4` WHERE `id`= 2159
I have checked the "relation4" model but it has nothing suspicious in it.
What should I do or where should I look to resolve this issue??
Is it possible (and how) in Doctrine 2 to create a query with a QueryBuilder and acquire a write lock for each of the matching rows? LockMode::PESSIMISTIC_WRITE can be used when fetching individual items with EntityRepository->find(), but I haven't been able to find such property for QueryBuilder.
The answer is to call setLockMode() on the Query object.
$qb = $em->createQueryBuilder();
$query = $qb->getQuery();
$query->setLockMode(LockMode::PESSIMISTIC_WRITE);
$results = $query->getResult();
The very last line of the Transactions and Concurrency documentation page shows that Query->setLockMode() is supported. A bit hard to notice, but it's there... ;)
Doctrine 2 query caching was a little surprising for me. I had two update queries that follow one-by-one:
function upd($user, $passwordHash) {
$qb = $this->entityManager->createQueryBuilder()
->update(UserEntity::__class, 'u')
->set('u.password', '?1')
->where('u = ?0')
->setParameters(array($user, $passwordHash));
$qb->getQuery()->execute();
}
I updated password with 2 different values (ex. A, B), but user was the same:
upd($user, 'A');
upd($user, 'B');
The first query really updated database row. But the second just didn't make any changes in DB after execution.
Trying to solve this problem I found some workaround:
$qb->getQuery()->useQueryCache(false)->execute();
After disabling QueryCache both two queries changes DB row.
So, the questions are: why doctrine uses 1st query cache in 2nd UPDATE query? And why doctrine uses cache while it's two queries with different parameter ($passwordHash)?
Problem was found. It is an issue in doctrine 2 ORM code. My $user entity has inheritance, so the update uses Doctrine\ORM\Query\Exec\MultiTableUpdateExecutor. In that executor:
//FIXME (URGENT): With query cache the parameter is out of date. Move to execute() stage.
Source.
So the only workaround for now is to disable query cache for multitable updates:
$qb->getQuery()->useQueryCache(false)->execute();
I created new bug for it.
I have two entities in Doctrine 2.1: Category and Site each category has many sites and each site has a parent category.
I would like to make a single update query (in DQL) which will update a field called count of the Category entity with the number of related sites.
So in SQL I would do something like this:
UPDATE categories c SET c.count = (SELECT COUNT(s.id) FROM sites s WHERE s.category_id = c.id);
This would work beautifuly, in DQL it might something like this:
UPDATE PackageNameBundle:Category c SET c.count = (SELECT COUNT(s.id) FROM PackageNameBundle:Site s WHERE s.category = c)
Such attempt raises [Syntax Error] line 0, col 61: Error: Expected Literal, got 'SELECT'.
Subqueries DO work in DQL, but the problem here (as far as I see it) is that Doctrine cannot assign the returned value from the subquery, to the c.count. This is understandable since I might fetch more than 1 field in the subquery and even more than one row. It magicaly works in MySQL since it sees one row, one field and for convenience returns a single integer value. Doctrine on the other hand has to be object oriented and has to work with different engines where such convertions might not be supported.
Finally, my question is:
What is the best way to do this in Doctrine, should I go with Native SQL or it can be done with DQL and how?
Thanks in advance!
EDIT: I just found this quote in the DQL Docs:
References to related entities are only possible in the WHERE clause and using sub-selects.
So, I guess assigning anything but a scalar value is impossible?
The main question remains though..
You can use native sql queries in Doctrine also, for that kind of specific queries. DQL is powerful in its own way, but it's also limited due to performance constraints. Using native sql queries and mapping the results will achieve the same thing, and there is no disadvantage in doing that.
The documentation explains it in detail.