Symfony2: use flush() or persist() to optimize data handling - php

Question

Im working with Symfony (const version="2.5.10") and using XAMPP with PHP version of 5.5.19
My problem is I keep on getting out of memory error. Because I think I query thousand of data (rows) in the database. For info, I have a lot of data in the database. I want to use flush() or anything that could be use to optimize my data handling.
Here is my code in my indexAction controller:
public function indexAction(){
$em = $this->getDoctrine()->getManager();
$po = $em->getRepository('MatrixEdiBundle:EdiTransactionDetail')->findDocs('850');
return $this->render('MatrixEdiBundle:Matrix:index.html.twig', array('po' => $po));
}
index.html.twig
{% extends '::layout.html.twig' %}
{# {% include 'MatrixEdiBundle:Matrix:header.html.twig'%} #}
{% block body %}
<div class="content">
</br>
<table id="datTable" class="table table-bordered table-condensed table-hover">
<thead>
<th colspan="8">Edi Matrix</th>
<tr>
<th>Po Numbers</th>
<th>Trading Partner Id</th>
<th>PO 855 Acknowledgement</th>
<th>PO 997 Acknowledgement</th>
<th>Advance Shipment Notice</th>
<th>Invoice</th>
<th>Purchase Order Change</th>
<th>Order Status</th>
</tr>
</thead>
<tbody>
{% for tran in po %}
<tr>
<td><a href="{{ path('matrix_edi_showpo', {'po_num': tran.poNumber}) }}"data-toggle="modal" data-target="#myModal">{{tran.poNumber}}</td>
<td>{{tran.ediTransaction.senderId}}</td>
<td><a href="{{ path('matrix_edi_findAll', {'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'doc_type': 855}) }}"data-toggle="modal"data-target="#myModal">
{{ render(controller('MatrixEdiBundle:Matrix:matrix', {
'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'reciever_id': tran. ediTransaction.receiverId, 'doc_type': 855, 'gs_number': tran.ediTransaction.gsNumber })) }}</a>
</td>
<td><a href="{{ path('matrix_edi_poack', {'gs_number': tran.ediTransaction.gsNumber, 'receiver_id': tran.ediTransaction.receiverId, 'sender_id': tran.ediTransaction.senderId}) }}"data-toggle="modal"data-target="#myModal">
{{ render(controller('MatrixEdiBundle:Matrix:matrix', {
'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'reciever_id': tran. ediTransaction.receiverId, 'doc_type': 997, 'gs_number': tran.ediTransaction.gsNumber })) }}</a>
</td>
<td><a href="{{ path('matrix_edi_findAll', {'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'doc_type': 856}) }}"data-toggle="modal"data-target="#myModal">{{ render(controller('MatrixEdiBundle:Matrix:matrix', {
'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'reciever_id': tran.ediTransaction.receiverId, 'doc_type': 856, 'gs_number': tran.ediTransaction.gsNumber }))}}</a>
</td>
<td>{{ render(controller('MatrixEdiBundle:Matrix:matrix', {'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'reciever_id': tran.ediTransaction.receiverId, 'doc_type': 810, 'gs_number': tran.ediTransaction.gsNumber})) }}
</td>
<td>{{ render(controller('MatrixEdiBundle:Matrix:matrix', {'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'reciever_id': tran.ediTransaction.receiverId, 'doc_type':860, 'gs_number': tran.ediTransaction.gsNumber})) }}
</td>
<td><a href="{{ path('matrix_edi_findAll', {'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'doc_type': 870}) }}"data-toggle="modal"data-target="#myModal">{{ render(controller('MatrixEdiBundle:Matrix:matrix', {
'po_num': tran.poNumber, 'sender_id': tran.ediTransaction.senderId, 'reciever_id': tran.ediTransaction.receiverId, 'doc_type': 870, 'gs_number': tran.ediTransaction.gsNumber
})) }}</a></td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<div class="modal fade" id="myModal" tabindex="-1" role="dialog" aria-labelledby="myModalLabel" aria-hidden="false" >
<div class="modal-dialog">
<div class="modal-content" style="width: 650px;">
<div class="modal-header" style="background-color: #2d6ca2; color: white;">
<button type="button" class="close" data-dismiss="modal"><span aria-hidden="true" style="color: white;">×</span><span class="sr-only">Close</span></button>
<h4 class="modal-title" id="myModalLabel" >Details</h4>
</div>
<div class="modal-body">
Loading, please wait......
<div class="bootstrap-table">
<div class="fixed-table-container" style="height: 299px; padding-bottom: 37px;">
<div class="fixed-table-body">
<div class="fixed-table-loading" style="top: 27px; display: none;">Loading, please wait…</div>
</div>
<div class="fixed-table-pagination"></div>
</div>
</div><div class="clearfix"></div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-primary" data-dismiss="modal" >Close</button>
</div>
</div><!-- /.modal-content -->
</div><!-- /.modal-dialog -->
</div><!-- /.modal -->
{% endblock %}
{% block javascripts %}
{% javascripts
'bundles/matrixdoc/js/jQuery.js'
'bundles/matrixdoc/js/jquery.dataTables.min.js'
'bundles/matrixdoc/js/dataTables.bootstrap.js'
'bundles/matrixdoc/js/bootstrap.js'
%}
<script src="{{ asset_url }}"></script>
<script type="text/javascript">
$(document).ready(function() {
$('#datTable').dataTable( {
"scrollY": "400px",
"scrollCollapse": true,
"pagingType": "simple",
});
$('body').on('hidden.bs.modal', '.modal', function () {
$(this).removeData('bs.modal');
});
$(document).on("hidden.bs.modal", function (e) {
$(e.target).removeData("bs.modal").find(".modal-content").empty();
});
});
</script>
{% endjavascripts %}
{% endblock %}

Answer 1

You should use pagination getting your data with limits and coupled with partial to get your needed fields if you don't need everything.
http://zrashwani.com/pagination-optimization-symfony2-doctrine/#.VV1av_mSwbg
for partial in DQL , simple example for user:
//partial_fields is an array, which can be passed like this $repo->getPartialUser($user_id, array('field1', 'field2', 'etc..');
public function getPartialUser($id, $partial_fields){
$qb = $this->_em->createQueryBuilder()
->select('partial u.{'.implode(',',$partial_fields).'}')
->from('AcmeUserBundle:User', 'u')
->where('u.id = :id')
->setParameter('id', $id);
$result = $qb->getQuery()->getOneOrNullResult();
return $result;
}
EDIT EXAMPLE CODE :
i will give you an example but you will probably still need to adapt it to your code or maybe it will work out of the box.
First you need a new twig with simple piece of code to show pages numbers for navigation below the table (note that i will not use datatables paginator but we will keep it for table layout rendering) :
pager.html.twig
<div class="pagination">
<div class="pagination-buttons">
{% if pagination.page>1 %}
<a href="{{ path(pagination.route,
pagination.route_params|merge({'page': 1})) }}"><<</a>
<a href="{{ path(pagination.route,
pagination.route_params|merge({'page': pagination.page-1})) }}"><</a>
{% endif %}
{#display p numbers only from p-4 to p+4 but don't go <1 or >pages_count#}
{% for p in range(max(pagination.page-4, 1),
min(pagination.page+4, pagination.pages_count)) %}
<a{% if p == pagination.page %} class="current-page"{% endif %}
href="{{ path(pagination.route,
pagination.route_params|merge({'page': p})) }}">{{ p }}</a>
{% endfor %}
{% if pagination.page<pagination.pages_count %}
<a href="{{ path(pagination.route,
pagination.route_params|merge({'page': pagination.page+1})) }}">></a>
<a href="{{ path(pagination.route,
pagination.route_params|merge({'page': pagination.pages_count})) }}">>></a>
{% endif %}
</div>
</div>
EdiTransactionDetailRepository:
add a function with your field that you use in your table :
public function getPagedDocs($page = 1, $limit = 50)
{
$qb = $this->_em->createQueryBuilder()
->select('partial d.{field1,field2 , etc..},
partial et.{field1,field2, etc..}'
))//select doc fields and second partial for your join select edittransaction fields if you have other joins add another partial etc..
->from('YourBundle:DocsEntity', 'd')
->innerJoin('d.ediTransaction', 'et')//not sure which join you need i have no clue but i believe you will want an inner one since you will want a doc to have a ediTransaction because i didn't see any checks in your twig for it if null.
->setFirstResult(($page - 1) * $limit)
->setMaxResults($limit);
$paginator = new Paginator($qb, $fetchJoinCollection = false);//for more performance fetchjoincollection to false if you have joined tables
$paginator->setUseOutputWalkers(false);//for more performance set to false for more information http://www.doctrine-project.org/jira/browse/DDC-2890
return $paginator;
}
PS: Change limit to how many records you want to show in each page depending on the load of data you have make it lower if it is slow for 50 this is depend on how much data each row have.
Next the indexAction in your controller :
public function indexAction($page){
$em = $this->getDoctrine()->getManager();
$po = $em->getRepository('MatrixEdiBundle:EdiTransactionDetail')->getPagedDocs($page, 50);
$count = $po->count();
$pagination = array(
'page' => $page,
'route' => 'docs_index_route_name', //i dont know which name you have but it should be the route name of this indexAction
'route_params' => array()
);
if ($max_records > 0)
$pagination['pages_count'] = max(ceil($count / $max_records), 1);
return $this->render('MatrixEdiBundle:Matrix:index.html.twig', array('po' => $po,
'pagination' => $pagination
));
}
Routing:
You need to modify the route of your indexAction as we added a parameter page you need to add it for the route too :
index_docs:
pattern: /index/{page}
defaults: { _controller: "YourBundle:Controller:index", page: 1 }
NOTE : you will need to change the names and stuff i dont know you controller names and bundles so normally you will only need to add /{page} to the pattern and , page:1 to your _controller config which is the default value.
Last thing to do is your index.html.twig which need to include our pager.html.twig
include this piece of code after you close your table :
{% if po|length > 0 and pagination['pages_count'] is defined and pagination['pages_count'] > 0 %}
{#---------Pager----------#}
<div style="text-align:center;">
{% include 'YourBundle:EntityDoc:pager.html.twig' %}
</div>
{% endif %}
NOTE : The include is the path to the file where you create the pager.html.twig i assume you are familiar with how this works.
So if i didn't forget anything this should work out of the box , it should enhance greatly your performance but i am afraid that your render controller there for each row may still be problematic. But try this solution first and see, it depends what you put inside those controllers.