I'm working with Entity objects from Doctrine queries and i end up with a very big array, with all information from all entities related. This ends up being a huge data tree... how can i limit this? Avoid listing all data from all relationships?
You can always remove not needed associations (this is a best practice for speeding up Doctrine). Or you can select only fields that you need in your presentation layer (as read-only data):
public function getAll()
{
$qb = $this->createQueryBuilder('u'); // Where are in User custom repository
return $qb
->select(array('u.id', 'u.first', 'u.last'))
->getQuery()
->getResult();
}
If you still need to work with objects (or for complex queries that needs plain SQL) a possibility is filling only needed properties (and eventually, associations/nested collections) of your domain object.
An example, more on native SQL:
public function getAll()
{
$mapping = new \Doctrine\ORM\Query\ResultSetMapping();
$mapping->addEntityResult('Acme\HelloBundle\User', 'e');
$mapping->addFieldResult('e', 'id', 'id');
$mapping->addFieldResult('e', 'first', 'first');
$mapping->addFieldResult('e', 'last', 'last');
$sql = "SELECT id, first, last FROM user ";
$result = $this->_em->createNativeQuery($sql, $mapping)->getResult();
// Or hust return $result itself (array)
return new \Doctrine\Common\Collections\ArrayCollection($result);
}
Of course the disadvance (?) is use of native SQL. I don't believe that ResultSetMapping can be used with DQL.
EDIT: take a look at http://docs.doctrine-project.org/projects/doctrine-orm/en/2.0.x/reference/best-practices.html
Related
I have following code:
$em = $this->getDoctrine()->getManager();
$evaluation->getQuestions()->clear();
foreach ($questions_data as $data) {
$id = (int) $data['id'];
if ($id > 0) {
$question = $em->getRepository('MyBundle:Question')->find($id);
if ($question)
$evaluation->getQuestions()->add($question);
}
}
$em->persist($evaluation);
$em->flush();
Here is $questions_data — array(['id' => 2], ['id' => 1], ['id' => 3]).
And here is how doctrine persist questions to database:
So, how to make doctrine to don't sort questions?
Evaluation entity has ManyToMany relation with the Question entity, so ORDER BY couldn't help, because table evaluations_questions was created automatically by Doctrine and don't have field id.
When you flush newly persisted items in Doctrine, Doctrine must determine which order to commit them to the database using an internal function getCommitOrder. The purpose of that function is to ensure that an object's dependencies are committed before the object itself is committed. This is done to comply with any foreign key constraints that might be set up. As you observed, a consequence of ordering data to commit like this is that you lose the ability to finely tune the order that items are committed - this isn't necessarily a bad thing.
In SQL the only way you can order your results is by issuing a query with ORDER BY. If you choose not to specify a sorting method, you cannot expect the results to come back in any particular order. The PostgreSQL docs explain this:
If sorting is not chosen, the rows will be returned in an unspecified order. The actual order in that case will depend on the scan and join plan types and the order on disk, but it must not be relied on.
In other words, it shouldn't matter what order the content is stored in your database.
In your problem, the order by which questions appear to a user is an issue. You can't allow the questions to appear randomly in an evaluation - they must follow a preset order. Frankbeen touches on this in a comment, but the best solution would be to add a new field on the Evaluation that stores an array of the Questions in the proper order. The order can then be read when you present the evaluation to a user.
If you absolutely must order them in a specific order in your database, you should be able to just flush the new objects individually as they are persisted instead of scheduling them to be flushed together.
$em = $this->getDoctrine()->getManager();
$evaluation->getQuestions()->clear();
foreach ($questions_data as $data) {
$id = (int) $data['id'];
if ($id > 0) {
$question = $em->getRepository('MyBundle:Question')->find($id);
if ($question) {
$evaluation->getQuestions()->add($question);
$em->persist($evaluation);
$em->flush();
}
}
}
Please be aware, this will take much more time to complete and is a pretty poor solution to your problem.
new records are stored at the end of the table, always. If you want to get the questions in natural order (by id) then you should use ORDER BY. I guess that the Evaluation entity has a OneToMany relation with the Question entity.
In that case you can simple add a optional annotation above the $questions property in the Evalutation entity:
/**
* #ORM\OneToMany(targetEntity="Question")
* #ORM\OrderBy({"id" = "ASC"})
*/
private $questions;
The EntityManagerInterface have transaction support and I recommend using that feature for all multi statement operations. This will allow you to create multiple statements and commit them in order all att once. Each statement will be executed in the order that they were added and if one statement fails, no changes will be done to the database at all.
That being said, all database implementations might not store data in the order that they were added, so if data is expected to be returned in a certain order, you should always query with ORDER BY explicitly as mentioned in another reply.
Say that you want to abort the operation if you encounter a question with an unknown id. This can quite easily be accomplished with something like the following:
function storeQuestions($questions_data): bool
{
$em = $this->getDoctrine()->getManager();
$em->beginTransaction();
$evaluation->getQuestions()->clear();
foreach ($questions_data as $data) {
$id = (int) $data['id'];
if ($id > 0) {
$question = $em->getRepository('MyBundle:Question')->find($id);
if ($question) {
$evaluation->getQuestions()->add($question);
$em->persist($evaluation);
} else {
$em->rollback();
return false;
}
}
}
$em->flush();
$em->commit();
return true; // Success
}
I am trying to use native query in doctrine and for now created something really simple:
$rsm = new ResultSetMapping();
$rsm->addEntityResult('ObjectA', 'a');
$rsm->addFieldResult('a', 'id', 'id');
$query = $em->createNativeQuery('SELECT * FROM table a', $rsm);
What I try using this code, I am getting an error that ObjectA is not a valid entity or mapped super class. Which is totally true.
My question is: Is there any way to mad result of a native query to any arbitrary class (not Entity), but still user Doctrine's tools to do it.
Note: I am trying to avoid usage of lower level PDO.
Thank you.
Nothing like that, neither in the doc nor in the source code Doctrine\ORM\Query\ResultSetMapping (and it happens that some features are not documented).
I'd go with using scalar results and mapping the query result back to the object. Something like this:
$rsm = new ResultSetMapping();
$rsm->addScalarResult('a', 'a');
$rsm->addScalarResult('b', 'b');
$query = $em->createNativeQuery('SELECT a, b FROM table LIMIT 1', $rsm);
$result = $query->getSingleResult();
$a = new ObjectA();
$a->setA($result['a']);
// or
$a = new ObjectA($result); // with mapping passed to the constructor
I've got below function in my Organisation model.
public function getOrganisations($where=null,$order='name ASC',$offset=null,$limit=null){
$Result = $this->fetchAll($where,$order,$limit,$offset);
if (!$Result){
return array();
}
return $Result->toArray();
}
but How can I include my organisation_types model so I can left join to organisation_type_id in organisation table?
This is the core of the argument in favor of using the data mapper pattern.
With the structure you seem to be using in your example you'll have huge trouble trying pass the organisation_types object into your Organisation model. You can however do a join in your query to join to the organisation_types table, but joining on the object is not likely to be reasonable.
to join on the organisation_types table:
//assuming this is a DbTable model that extends Zend_Db_Table_Abstract
function getOrganisations($where=null,$order='name ASC',$offset=null,$limit=null){
$select = $this->select()->setIntegrityCheck(FALSE);//This locks the table to allow joins
$select->joinLeft('organisation_types', 'organisation_types.id = organisation.organisation_type_id');//This will join the tables with all feilds, use the join type you like.
if (!is_null($where) {
$select->where($where);
}
if (!is_null($order) {
$select->order($order);
}
if (!is_null($offset) {
$select->limit(null,$offset);//offset is second arg in limit() in select()
}
if (!is_null($limit) {
$select->limit($limit);
}
$Result = $this->fetchAll($select);
if (!$Result){
return array();
}
return $Result->toArray();
}
This should give you an idea of how a table join would work. If you want to use the objects you'll need to begin again with a different structure.
I found a couple of good tutorials on PHPMaster that helped me get my head around the data mappers and domain models.
Building A Domain Model, Introduction
Integrating data mappers
Also The online book Survive The Deepend has a good example of the data mapper pattern and how to test it.
Good Luck...
Maybe using Zend_Db_Select with joinLeft() would be more appropriate:
http://framework.zend.com/manual/en/zend.db.select.html#zend.db.select.building.join
$shops = $this->em->getRepository('models\Shop')->findAll();
Gives my an array with entities but I need the entity as array.
How do I convert an entity to an array?
Doctrine allows you to specify a hydration mode when executing queries, which let's you change the data type of the results returned. In this case, you need Query::HYDRATE_ARRAY. It does not let you specify this on the default findAll() method, found on the repositories. You will need to write your own DQL for it.
If you need a collection of entites as arrays:
$query = $em->createQuery('SELECT u FROM User u');
$entites = $query->execute(array(), Query::HYDRATE_ARRAY);
// If you don't have parameters in the query, you can use the getResult() shortcut
$query = $em->createQuery('SELECT u FROM User u');
$entities = $query->getResult(Query::HYDRATE_ARRAY);
If you need a single entity as an array, eg. for a specific ID:
$query = $em->createQuery('SELECT u FROM User u WHERE u.id = ?1');
$query->setParameter(1, $id);
$entity = $query->getSingleResult(Query::HYDRATE_ARRAY);
These methods are defined on Query, and AbstractQuery.
I had the same issue.return get_object_vars($this) is not a good solution because it also converts internal doctrine object/properties too.After some research i found this class: EntitySerializer which creates clean array or JSON from your entities and removes unnecessary items.The documentation is located here.For example i used the following code:
$patientProfile = $this->em->getRepository('Entities\Patientprofile')->findOneByuserid('2222222');
$entitySerializer=new Bgy\Doctrine\EntitySerializer($this->em);
$patientProfile=$entitySerializer->toArray($patientProfile);
When using the ACL implementation in Symfony2 in a web application, we have come across a use case where the suggested way of using the ACLs (checking a users permissions on a single domain object) becomes unfeasible. Thus, we wonder if there exists some part of the ACL API we can use to solve our problem.
The use case is in a controller that prepares a list of domain objects to be presented in a template, so that the user can choose which of her objects she wants to edit. The user does not have permission to edit all of the objects in the database, so the list must be filtered accordingly.
This could (among other solutions) be done according to two strategies:
1) A query filter that appends a given query with the valid object ids from the present user's ACL for the object(or objects). I.e:
WHERE <other conditions> AND u.id IN(<list of legal object ids here>)
2) A post-query filter that removes the objects the user does not have the correct permissions for after the complete list has been retrieved from the database. I.e:
$objs = <query for objects>
$objIds = <getting all the permitted obj ids from the ACL>
for ($obj in $objs) {
if (in_array($obj.id, $objIds) { $result[] = $obj; }
}
return $result;
The first strategy is preferable as the database is doing all the filtering work, and both require two database queries. One for the ACLs and one for the actual query, but that is probably unavoidable.
Is there any implementation of one of these strategies (or something achieving the desired results) in Symfony2?
Assuming that you have a collection of domain objects that you want to check, you can use the security.acl.provider service's findAcls() method to batch load in advance of the isGranted() calls.
Conditions:
Database was populated with test entities, with object permissions of MaskBuilder::MASK_OWNER for a random user from my database, and class permissions of MASK_VIEW for role IS_AUTHENTICATED_ANONYMOUSLY; MASK_CREATE for ROLE_USER; and MASK_EDIT and MASK_DELETE for ROLE_ADMIN.
Test Code:
$repo = $this->getDoctrine()->getRepository('Foo\Bundle\Entity\Bar');
$securityContext = $this->get('security.context');
$aclProvider = $this->get('security.acl.provider');
$barCollection = $repo->findAll();
$oids = array();
foreach ($barCollection as $bar) {
$oid = ObjectIdentity::fromDomainObject($bar);
$oids[] = $oid;
}
$aclProvider->findAcls($oids); // preload Acls from database
foreach ($barCollection as $bar) {
if ($securityContext->isGranted('EDIT', $bar)) {
// permitted
} else {
// denied
}
}
RESULTS:
With the call to $aclProvider->findAcls($oids);, the profiler shows that my request contained 3 database queries (as anonymous user).
Without the call to findAcls(), the same request contained 51 queries.
Note that the findAcls() method loads in batches of 30 (with 2 queries per batch), so your number of queries will go up with larger datasets. This test was done in about 15 minutes at the end of the work day; when I have a chance, I'll go through and review the relevant methods more thoroughly to see if there are any other helpful uses of the ACL system and report back here.
Itinerating over the entities is not feasible if you have a couple of thousandth entities - it will keep getting slower and consuming more memory, forcing you to use doctrine batching capabilities, thus making your code more complex (and innefective because after all you need only the ids to make a query - not the whole acl/entities in memory)
What we did to solve this problem is to replace acl.provider service with our own and in that service add a method to make a direct query to the database:
private function _getEntitiesIdsMatchingRoleMaskSql($className, array $roles, $requiredMask)
{
$rolesSql = array();
foreach($roles as $role) {
$rolesSql[] = 's.identifier = ' . $this->connection->quote($role);
}
$rolesSql = '(' . implode(' OR ', $rolesSql) . ')';
$sql = <<<SELECTCLAUSE
SELECT
oid.object_identifier
FROM
{$this->options['entry_table_name']} e
JOIN
{$this->options['oid_table_name']} oid ON (
oid.class_id = e.class_id
)
JOIN {$this->options['sid_table_name']} s ON (
s.id = e.security_identity_id
)
JOIN {$this->options['class_table_nambe']} class ON (
class.id = e.class_id
)
WHERE
{$this->connection->getDatabasePlatform()->getIsNotNullExpression('e.object_identity_id')} AND
(e.mask & %d) AND
$rolesSql AND
class.class_type = %s
GROUP BY
oid.object_identifier
SELECTCLAUSE;
return sprintf(
$sql,
$requiredMask,
$this->connection->quote($role),
$this->connection->quote($className)
);
}
Then calling this method from the actual public method that gets the entities ids:
/**
* Get the entities Ids for the className that match the given role & mask
*
* #param string $className
* #param string $roles
* #param integer $mask
* #param bool $asString - Return a comma-delimited string with the ids instead of an array
*
* #return bool|array|string - True if its allowed to all entities, false if its not
* allowed, array or string depending on $asString parameter.
*/
public function getAllowedEntitiesIds($className, array $roles, $mask, $asString = true)
{
// Check for class-level global permission (its a very similar query to the one
// posted above
// If there is a class-level grant permission, then do not query object-level
if ($this->_maskMatchesRoleForClass($className, $roles, $requiredMask)) {
return true;
}
// Query the database for ACE's matching the mask for the given roles
$sql = $this->_getEntitiesIdsMatchingRoleMaskSql($className, $roles, $mask);
$ids = $this->connection->executeQuery($sql)->fetchAll(\PDO::FETCH_COLUMN);
// No ACEs found
if (!count($ids)) {
return false;
}
if ($asString) {
return implode(',', $ids);
}
return $ids;
}
This way now we can use the code to add filters to DQL queries:
// Some action in a controller or form handler...
// This service is our own aclProvider version with the methods mentioned above
$aclProvider = $this->get('security.acl.provider');
$ids = $aclProvider->getAllowedEntitiesIds('SomeEntityClass', array('role1'), MaskBuilder::VIEW, true);
if (is_string($ids)) {
$queryBuilder->andWhere("entity.id IN ($ids)");
}
// No ACL found: deny all
elseif ($ids===false) {
$queryBuilder->andWhere("entity.id = 0")
}
elseif ($ids===true) {
// Global-class permission: allow all
}
// Run query...etc
Drawbacks: This methods have to be improved to take into account the complexities of ACL inheritance and strategies, but for simple use cases it works fine. Also a cache has to be implemented to avoid the repetitive double query (one with class-level, another with objetc-level)
Coupling Symfony ACL back to application and using it as sorting, is not good approach. You are mixing and coupling 2 or 3 layers of application together.
ACL functionality is to answer "YES/NO" to question "Am I allowed to do this?" If you need some sort of owned/editable articles, you can use some column like CreatedBy or group CreatedBy by criteria from another table. Some usergroups or accounts.
Use joins, and in case you're using Doctrine, get it to generate joins for you, as they are almost always faster. Therefore you should design your ACL schema that doing these fast filters are feasible.