Why is my controller so slow? - php

Question

I'm on an app that retrieve datas (a 7k lines CSV formated string) from an external server to update my own entity. Each row is an item in a stock.
Today the job is nicely done but it's very very very slow: more than 60s (prod env) to retrieve datas, push it in a 2D array, update the BDD, and finally load a page that display the bdd content.
When only displaying the page it's about 20s (still prod).
This the profiler's timeline result while only displaying records : Symfony's profiler timeline
Anymore, i'm not able to profile the "updateAction" cause i't don't appear in the last ten request list.
2 days ago I was checking each row of the CSV file to add it only if needed, I was soft-deleting items to restore it later when back in the stock etc. but with that speed I tried many things to have normal performances.
At the begening everything was in the controler, I moved the function that add/remove in a dedicated service, then in the repository to finally get it back in my controler. To have decent results I tried to empty the database and then refill it without checking. First, using LOAD DATA LOCAL INFILE but it is not compatible with my table pattern (or I mis understood something) and now I'm simply emptying the table before filling it with the CSV (without any control). The time score I gave earlier was with this last try (which is the best one).
But enought talk
here is my controler:
public function majMatosClanAction()
{
$resMaj = $this->majClanCavernes();
if ($resMaj === NULL)
{
$this->get('session')->getFlashBag()->add('alert-danger', 'Unidentified');
return $this->redirect($this->generateUrl('loki_gbl'));
} else if ($resMaj === FALSE)
{
$this->get('session')->getFlashBag()->add('alert-warning','password update required');
return $this->redirect($this->generateUrl('loki_gbl_ST'));
} else
{
$this->get('session')->getFlashBag()->add('alert-success','success');
return $this->redirect($this->generateUrl('loki_gbl_voirMatosClan'));
}
}
here is the function that my controller call:
public function majClanCavernes()
{
$user = $this->get('security.token_storage')->getToken()->getUser();
$outils = $this->container->get('loki_gbl.outils');
if ($user !== NULL)
{
$pwd = $user->getGob()->getPwd();
$num = $user->getGob()->getNum();
if($outils->checkPwd($num, $pwd) !== TRUE) return FALSE;
$em = $this->getDoctrine()->getManager();
//This is a temporary solution
//////////////////////////////////////////////
$connection = $em->getConnection();
$platform = $connection->getDatabasePlatform();
$connection->executeUpdate($platform->getTruncateTableSQL('MatosClan', true ));
//////////////////////////////////////////////
$repository = $em->getRepository('LokiGblBundle:MatosClan');
$urlMatosClan = "http://ie.gobland.fr/IE_ClanCavernes.php?id=".$num."&passwd=".$pwd;
//encode and format the string via a service
$infosBrutes = $outils->fileGetInfosBrutes($urlMatosClan);
//$csv is a 2D array containing the datas
$csv = $outils->getDatasFromCsv($infosBrutes);
foreach($csv as $item)
{
$newItem = new MatosClan;
$newItem->setNum($item[0]);
$newItem->setType($item[1]);
[...]
$em->persist($newItem);
}
$em->flush();
return TRUE;
}
else{
return NULL;
}
}
What is wrong? 7k lines is not that big!
Could it be a lack of hardware issue?

Answer 1

Check out doctrine's batch processing documentation here.
You can also disable logging:
$em->getConnection()->getConfiguration()->setSQLLogger(null);