Show the catalog is too slow in Symfony2 - php

Question

I'm working on an app that has a product catalog with a n-level category tree.
I have the next structure:
Category 1
--Category 1.1
----Category 1.1.1
------Product 1.1.1.1
------Product 1.1.1.2
----Product 1.1.1
--Category 1.2
----Product 1.2.1
--Product 1.1
And I need something like a flat array:
Category 1
Category 1.1
Category 1.1.1
Product 1.1.1.1
Product 1.1.1.2
Product 1.1.1
Category 1.2
Product 1.2.1
Product 1.1
But the problem is that I need too much queries and too much time to generate this (I have 700 products and 200 categories more or less and it takes about 846 queries and 2100 ms to execute them). I can add more joins, but the execution time grows significantly. I'm working with translatable entities.
Any idea to minimize the queries number and the execution time? Categories has a self-relationship and a relationship with products.
I think that an option can be to get all categories and make a ordered tree and later redo the flat array again, but I don't know if it could be the best option...
Thank you in advance for any help!
EDIT:
This is my function to generate the "flat" array.
public function generateCategoriesTree($categories, $result = array()) {
$tempResult = array();
if(count($categories)) {
foreach($categories as $k => $v) {
$tempResult[] = $v;
if(count($v->getCategories())) {
$temp = $this->generateCategoriesTree($v->getCategories(), $result);
foreach($temp as $kk => $vv) {
$tempResult[] = $vv;
}
}
}
}
return array_merge($result, $tempResult);
}
But the problem is not here, the problem is when I trying to acces to the second level of the tree, when call to the method getcategories, that execute a query in the database.
UPDATE
With this solution, I reduced the number of queries from 700 to 100 and the execution time from 2100ms to 350ms. I think that it could be enough for now, becouse this actions is used a couple per hour.
if ($this->isGranted('ROLE_EMPLOYEE') || $currentUser->isXXXXXXSeller()) {
$XXXXXXCategories = $em->getRepository('CatalogBundle:Category')->getMainCategoriesByBrand(1);
if(is_array($XXXXXXCategories)) {
$categoriesXXXXXX = $this->get('core_tools')->generateCategoriesTree($XXXXXXCategories);
$categories = new ArrayCollection(
array_merge($categories->toArray(), $categoriesXXXXXX)
);
}
}
$categoriesToShow = array();
$products = $em->getRepository('CatalogBundle:Product')->getAllOrderedByPosition();
if(is_array($products) && is_array($categories)) {
foreach($categories as $category) {
$categoriesToShow["c_".$category->getId()] = array(
"category" => $category,
"products" => array(),
);
}
foreach($products as $product) {
if($product->getCategories()) {
foreach($product->getCategories() as $productCategory) {
if(array_key_exists("c_".$productCategory->getCategory()->getId(), $categoriesToShow)) {
$categoriesToShow["c_".$productCategory->getCategory()->getId()]['products'][] = $product;
}
}
}
}
}

Answer 1

If the catalog doesn't change very often (ie: more than every few seconds) - then do the hard work once, and then cache the output ready to be used instantly.
When the catalog changes - rebuild the cache.

Answer 2

If you are using MySQL you might like to consider the "Nested Sets" model to store your category hierarchy. The complexity of your SELECTs will usually stay constant no matter how many levels your category tree has. There are Doctrine extensions available that help you setting it up (e.g. https://github.com/blt04/doctrine2-nestedset).
If you are using a DBS supporting hierarchical queries you can consider using them (https://en.wikipedia.org/wiki/Hierarchical_and_recursive_queries_in_SQL).