How to improve query performance in symfony for multiple relationship? - php

Hi I am using symfony 2 for my application and using sg-datatable for displaying my data, now to feed the data table i used client side not server side and using this query:
public function indexAction()
{
$em = $this->getDoctrine()->getManager();
$entities = $em->getRepository('MyAppBundle:Artist')
->createQueryBuilder('c')
->select('c.id, c.name, c.sex, c.priority, c.bday, c.bmonth, c.byear')
->getQuery()
->getArrayResult();
$cacsData = $em->createQueryBuilder()
->select('a.id, r.name as role, c.title as work')
->from('MyAppBundle:CastAndCrew', 'q')
->innerJoin('q.role', 'r')
->innerJoin('q.artists', 'a')
->innerJoin('q.content', 'c')
->getQuery()
->getArrayResult();
$cacs = array();
foreach ($cacsData as $cacData) {
if (!array_key_exists($cacData['id'], $cacs)) {
$cacs[$cacData['id']] = array();
}
$cacs[$cacData['id']][] = $cacData;
}
foreach ($entities as &$artist) {
if (!array_key_exists($artist['id'], $cacs)) {
continue;
}
$roles = array();
$contents = array();
foreach ($cacs[$artist['id']] as $cac) {
$roles[] = $cac['role'];
$contents[] = $cac['work'];
}
$artist['role'] = implode('/', array_unique($roles));
$artist['works'] = implode('<br>', array_unique($contents));
}
$encoders = array(new JsonEncoder());
$normalizers = array(new GetSetMethodNormalizer());
$serializer = new Serializer($normalizers, $encoders);
$datatable = $this->get("bbd_datatables.artist");
$datatable->setData($serializer->serialize($entities, "json"));
return $this->render('MyAppBundle:Artist:index.html.twig', array(
"datatable" => $datatable
));
}
everything is fine but the problem is query time, i have 20000+ data in my table and it's taking 250+ ms and because of the client side it paused few seconds then displays the data, can any one suggest me how i can improve the performance?

Related

How to add an extra row to each object returned from a Laravel database call?

I am new to PHP and new to Laravel. I'm taking over an old project.
We had this code and it was working fine:
public function getList(Request $request)
{
$apiFormat = array();
try
{
$perPage = Input::get('page_size', 10);
$filters = $request->input();
$postRepo = new PostRepo();
$user = $_SESSION["user"];
$listPost = $postRepo->getlist($user->id, $perPage, 0);
$apiFormat['status'] =\Config::get('constants.api.STATUS_OK');
$apiFormat['message'] = \Config::get('constants.api.SUCCESS');
$apiFormat['data'] = $listPost;
} catch (\Exception $ex) {
$apiFormat['status'] =
\Config::get('constants.api.STATUS_ERROR');
$apiFormat['message'] = $ex->getMessage();
}
return response()->json($apiFormat);
}
This returned 10 items that would show up on our newsfeed.
This line:
$listPost = $postRepo->getlist($user->id, $perPage, 0);
Did a database call like this:
$sqlCheckHasLike = "SELECT count(*) > 0 FROM `likes` WHERE `post_id` = `posts`.`id` and `user_id` = '".$user_id."'";
$query = DB::table('posts')
->select('posts.*',DB::raw("($sqlCheckHasLike) as is_like"), 'users.full_name', 'users.avatar', DB::raw("post_categories.name as post_categories_name"))
->leftJoin('users', 'posts.user_id', '=', 'users.id')
->leftJoin('post_categories', 'post_categories.id', '=', 'posts.post_category_id')
->where('posts.status',$post_status)
->where('users.status', 1)
->where(function ($query) use ($user_id, $list_user) {
$query->whereIn('user_id', $list_user)
->orwhere('user_id', $user_id);
})
->orderBy('created_at', 'desc')->paginate($pageSize);
return $query;
However, we've just added comments to each item in the newsfeed. That code is handled elsewhere, but here I want to find, given a post.id, how many comments belong to it? This needs to be included in the response.
Replacing the first method, I tried this:
try {
$perPage = Input::get('page_size', 10);
$filters = $request->input();
$postRepo = new PostRepo();
$user = $_SESSION["user"];
$listPost = $postRepo->getlist($user->id, $perPage, 0);
$array_of_rows = [];
foreach ($listPost as $lp) {
$row_as_array = (array) $lp;
$post_id = $lp->id;
$query = "select count(id) as how_many from comments where
post_id = '". $post_id ."'";
$result_array = DB::select($query);
$result_obj = $result_array[0];
$how_many = $result_obj->how_many;
$row_as_array['how_many_comments'] = $how_many;
$array_of_rows[] = $row_as_array;
}
$merged_list_post = (object) $array_of_rows;
file_put_contents("/var/log/api_debugging", "\ncomment count:\n",
FILE_APPEND | LOCK_EX);
file_put_contents("/var/log/api_debugging",
print_r($merged_list_post, true), FILE_APPEND | LOCK_EX);
$apiFormat['status'] = \Config::get('constants.api.STATUS_OK');
$apiFormat['message'] = \Config::get('constants.api.SUCCESS');
$apiFormat['data'] = $merged_list_post;
} catch (\Exception $ex) {
$apiFormat['status']=\Config::get('constants.api.STATUS_ERROR');
$apiFormat['message'] = $ex->getMessage();
}
This does not throw an error (there are no Exceptions) but it changed the returned data, such that the apps which consume this feed no longer get what they need.
Is there an official approach in Laravel that makes it easy to do several database queries, and combine that data at a per inner object level? I'd like to avoid returning an object that is of a different type than the one in the original example.
You could just modify the original query (I've just included the first few lines here) to JOIN the comments table and get the count:
$query = DB::table('posts')
->select('posts.*',DB::raw("($sqlCheckHasLike) as is_like"), 'users.full_name', 'users.avatar', DB::raw("post_categories.name as post_categories_name"), DB::raw("count(comments.id) as how_many"))
->leftJoin('users', 'posts.user_id', '=', 'users.id')
->leftJoin('post_categories', 'post_categories.id', '=', 'posts.post_category_id')
->leftJoin('comments', 'comments.post_id', '=', 'posts.id')
You will also need to add a groupBy clause:
->groupBy('posts.id')
prior to the orderBy.

Find Entity for field another entity

I have entity project and when I create new project I write skill and mainskill for this project in form. Then I have entity developer he have skill and mainskill too, early, when I create project, we find developer who have skill = 50% projectskill+project mainskill. Now this find still and I nedd addfind developer who have skill or mainskill = project mainskill
This is I try:
But now I have error:
Invalid parameter number: number of bound variables does not match number of tokens
Whats wrong with createQueryBuilder????
$query = $em->getRepository('ArtelCustomerBundle:Developers')->createQueryBuilder('d')
->where('(d.skills LIKE :skills OR d.main_skill LIKE :main_skill) AND d.unsubscribeDate <= :date')
->groupBy('d.email')
//->where('d.active = 1 AND d.approved = 1 AND d.skills LIKE :skills AND d.unsubscribeDate <= :date')
->setParameter('main_skill', '%'.$skills.'%')
->setParameter('date', new \DateTime('now'));
I change
$query = $em->getRepository('ArtelCustomerBundle:Developers')->createQueryBuilder('d')
->where('(d.skills LIKE :skills OR d.main_skill LIKE :skills) AND d.unsubscribeDate <= :date')
->groupBy('d.email')
//->where('d.active = 1 AND d.approved = 1 AND d.skills LIKE :skills AND d.unsubscribeDate <= :date')
->setParameter('main_skill', '%'.$skills.'%')
->setParameter('date', new \DateTime('now'));
developer have field main_skill and I get mainskill for project (instring) and I want compare this filed: field developera(main_skill) and field project (mainskill = $skills = $project->getMainSkill();)
private function foundSubscribedDevelopersBySkills($project)
{
$em = $this->getDoctrine()->getManager();
$developers = [];
$emails = [];
$skills = $project->getSkills();
$skills [] = $project->getMainSkill();
$main_skill = $project->getMainSkill();
$except_emails = [];
if ($this->notAcceptTags) {
foreach ($this->notAcceptTags as $tag) {
$q = $em->createQuery("SELECT n.email FROM ArtelCustomerBundle:Developers n WHERE n.tags LIKE :tag")->setParameter('tag', '%' . $tag . '%');
$entities = $q->getArrayResult();
foreach ($entities as $data)
array_push($except_emails, $data['email']);
}
}
$q = $em->createQuery("SELECT n.email FROM ArtelCustomerBundle:Developers n WHERE n.unsubscribeDate >= :date")->setParameter('date', new \DateTime('now'));
$entities = $q->getArrayResult();
foreach ($entities as $data)
array_push($except_emails, $data['email']);
foreach ($skills as $skill) {
$query = $em->getRepository('ArtelCustomerBundle:Developers')->createQueryBuilder('d')
->where('(d.skills LIKE :skills OR d.main_skill LIKE :skills) AND d.unsubscribeDate <= :date')
->groupBy('d.email')
//->where('d.active = 1 AND d.approved = 1 AND d.skills LIKE :skills AND d.unsubscribeDate <= :date')
->setParameter('skills', '%'.$skill.'%')
->setParameter('date', new \DateTime('now'));
if ($except_emails)
$query->andWhere($query->expr()->notIn('d.email', $except_emails));
$developers_tmp = $query->getQuery()->getArrayResult();
$developers = array_merge($developers, $developers_tmp);
}
foreach ($developers as $key => $developer_data) {
if (isset($emails[$developer_data['email']]))
unset($developers[$key]);
else
$emails[$developer_data['email']] = $developer_data['email'];
$developerSkills = $developer_data['skills'];
$developerSkills[] = $developer_data['main_skill'];
if ($developer_data['tags'] && in_array('only main skill', $developer_data['tags']))
$developerSkills = [$developer_data['main_skill']];
$this->overlapSkills($skills, $developerSkills), $skills, $developerSkills);exit;
if (!$this->overlapSkills($skills, $developerSkills))
unset($developers[$key]);
}
return $developers;
}
private function foundSubscribedDevelopersByMainSkills($project)
{
$em = $this->getDoctrine()->getManager();
$developers = [];
$emails = [];
$skills = $project->getMainSkill();
$except_emails = [];
if ($this->notAcceptTags) {
foreach ($this->notAcceptTags as $tag) {
$q = $em->createQuery("SELECT n.email FROM ArtelCustomerBundle:Developers n WHERE n.tags LIKE :tag")->setParameter('tag', '%' . $tag . '%');
$entities = $q->getArrayResult();
foreach ($entities as $data)
array_push($except_emails, $data['email']);
}
}
$q = $em->createQuery("SELECT n.email FROM ArtelCustomerBundle:Developers n WHERE n.unsubscribeDate >= :date")->setParameter('date', new \DateTime('now'));
$entities = $q->getArrayResult();
foreach ($entities as $data)
array_push($except_emails, $data['email']);
$query = $em->getRepository('ArtelCustomerBundle:Developers')->createQueryBuilder('d')
->where('(d.skills LIKE :skills OR d.main_skill LIKE :main_skill) AND d.unsubscribeDate <= :date')
->groupBy('d.email')
//->where('d.active = 1 AND d.approved = 1 AND d.skills LIKE :skills AND d.unsubscribeDate <= :date')
->setParameter('main_skill', '%'.$skills.'%')
->setParameter('date', new \DateTime('now'));
if ($except_emails)
$query->andWhere($query->expr()->notIn('d.email', $except_emails));
$developers_tmp = $query->getQuery()->getArrayResult();
$developers = array_merge($developers, $developers_tmp);
foreach ($developers as $key => $developer_data) {
if (isset($emails[$developer_data['email']]))
unset($developers[$key]);
else
$emails[$developer_data['email']] = $developer_data['email'];
$developerSkills = $developer_data['skills'];
$developerSkills[] = $developer_data['main_skill'];
if ($developer_data['tags'] && in_array('only main skill', $developer_data['tags']))
$developerSkills = [$developer_data['main_skill']];
$this->overlapSkills($skills, $developerSkills), $skills, $developerSkills);exit;
}
return $developers;
}
private function sendEmailForDevelopers($project)
{
$developers = $this->foundSubscribedDevelopersBySkills($project);
$dispatcher = $this->get('hip_mandrill.dispatcher');
$em = $this->getDoctrine()->getManager();
if (!empty($developers)) {
foreach ($developers as $developer) {
$message = new Message();
if (!empty($developer[ 'unsubscribe_token' ])) {
$token = $developer[ 'unsubscribe_token' ];
} else {
$token = sha1($developer[ 'email' ]);
$developer_obj = $em->getRepository('ArtelCustomerBundle:Developers')->findOneById($developer[ 'id' ]);
$developer_obj->setUnsubscribeToken($token);
$em->persist($developer_obj);
$em->flush();
}
$parameters = [
'project' => $project,
'token' => $token,
'developer_name' => $developer[ 'firstName' ]
];
$message->setFromEmail($this->emailFrom)
->setFromName($this->emailAuthorName)
->addTo($developer['email'])
->setSubject($developer[ 'firstName' ].''.$this->emailSubject.' '.$project->getLevel().' '.$project->getMainSkill().' '.$project->getName())
->setHtml($this->get('twig')->render('ArtelSiteBundle:Mail:new_project_add.html.twig', $parameters));
$result = $dispatcher->send($message);
}
}
}
help people

Doctrine QueryBuilder Re-Use Parts

I want to count all fields that fits my conditions and get them page by page with doctrine query builder.
I'm generating the query depends my filter fields.
First section is counting the records so i can calculate the pages.
$qb = $em->createQueryBuilder();
$qb
->select('COUNT(m.id)')
->from('CSMediaBundle:MediaItem', 'm')
->where(
$qb->expr()->eq('m.media', $media->getId())
);
$filters = $request->request->get('filter');
if(!empty($filters['size'])) {
foreach($filters['size'] as $key => $value) {
if(!empty($value)) {
$qb->andWhere(
$qb->expr()->eq('m.'.$key, ':'.$key)
)->setParameter($key, $value);
}
}
}
if(!empty($filters['sliders'])) {
$qb
->leftJoin('CSSliderBundle:SliderItem', 's', 'ON', 'm.id = s.media_id')
->andWhere(
$qb->expr()->in('s.sliders', $filters['sliders'])
);
}
$media_count = $qb->getQuery()->getSingleScalarResult();
Second section is getting records by calculated page using same filters, just changing the select and final parts (getSingleScalarResult to getResult)
I wonder if is there any way to just change the select and the result parts so i would not use the filters again and again...
Yeah, that's what functions are for:
function filter($qb, $filters) {
if (!empty($filters['size'])) {
foreach($filters['size'] as $key => $value) {
if (!empty($value)) {
$qb->andWhere(
$qb->expr()->eq('m.'.$key, ':'.$key)
)->setParameter($key, $value);
}
}
}
if (!empty($filters['sliders'])) {
$qb
->leftJoin('CSSliderBundle:SliderItem', 's', 'ON', 'm.id = s.media_id')
->andWhere(
$qb->expr()->in('s.sliders', $filters['sliders'])
);
}
return $qb;
}
$filters = $request->request->get('filter');
// count
$qb = $em->createQueryBuilder();
$qb
->select('COUNT(m.id)')
->from('CSMediaBundle:MediaItem', 'm')
->where(
$qb->expr()->eq('m.media', $media->getId())
);
$media_count = filter($qb, $filters)->getQuery()->getSingleScalarResult();
// entities
$qb = $em->createQueryBuilder();
$qb
->select('m')
->from('CSMediaBundle:MediaItem', 'm')
->where(
$qb->expr()->eq('m.media', $media->getId())
);
$media_entities = filter($qb, $filters)->getQuery()->getResult();
Another way is to clone the query builder object:
$qb = $em->createQueryBuilder();
$qb->from('CSMediaBundle:MediaItem', 'm')
->where(
$qb->expr()->eq('m.media', $media->getId())
);
$filters = $request->request->get('filter');
if (!empty($filters['size'])) {
foreach($filters['size'] as $key => $value) {
if (!empty($value)) {
$qb->andWhere(
$qb->expr()->eq('m.'.$key, ':'.$key)
)->setParameter($key, $value);
}
}
}
if (!empty($filters['sliders'])) {
$qb
->leftJoin('CSSliderBundle:SliderItem', 's', 'ON', 'm.id = s.media_id')
->andWhere(
$qb->expr()->in('s.sliders', $filters['sliders'])
);
}
$qb2 = clone $qb;
$qb->select('COUNT(m.id)')
$media_count = $qb->getQuery()->getSingleScalarResult();
$qb2->select('m')
$media_entities = $qb2->getQuery()->getResult();
In Javascript-TypeORM you can clone the existing query builder as shown below.
let baseQuery=this.Repository.createQueryBuilder("user");
//Add conditions
if(user_type==="admin"){
baseQuery.where("user.user_type = : user_type",{user_type:"admin"})
}
//Clone query builder
let count_query= new SelectQueryBuilder(baseQuery);
const count= count_query.select('COUNT(*)', 'count').getRawOne();
let sum_query= new SelectQueryBuilder(baseQuery);
const sum= sum_query.select('SUM(user.amount)', 'amount').getRawOne();

How to print certain attributes using CActive record in yii?

How to print certain properties in the model while returning data to user
I have Users model , which has Username, Password, Email
I can get all users username, email
Now i need to send Username and email to user
while trying to send data to the user as json from the following code
$models = Users::model()->findAll($criteria);
if(is_null($models)) {
$this->_sendResponse(200, sprintf('No users found'));
} else {
$rows = array();
foreach($models as $model)
{
$Users[] = $model->attributes;
}
$SandboxObj->_sendResponse(200, CJSON::encode($Users));
}
It shows like [{"Username":"foo","Email":"foo#foo.com","Password":"null"}]
I need to send like [{"Username":"foo","Email":"foo#foo.com"}]
How to do this in yii?
I know this question is already been answered and accepted but i do not like to teach people the lazy way of coding. there for, as a correction on Alireza Fallah's answer:
$models = Users::model()->findAll($criteria);
if(is_null($models)) {
$this->_sendResponse(200, sprintf('No users found'));
} else {
$userData = array();
foreach($models as $model)
{
$tempData = new StdClass;
$tempData->Username = $model->Username;
$tempData->Email= $model->Email;
$userData[] = $tempData;
}
$SandboxObj->_sendResponse(200, CJSON::encode($userData));
}
or as said before only query what you need in the first place, then you can be lazy all you want in the rest of your code.
Edit: another way of doing the foreach with less resources is :
foreach($models as &$model)
{
$tempData = new StdClass;
$tempData->Username = $model->Username;
$tempData->Email= $model->Email;
$model = $tempData;
}
and then you can use $models in stead of $userData
Do this :
$models = Users::model()->findAll($criteria);
if(is_null($models)) {
$this->_sendResponse(200, sprintf('No users found'));
} else {
$rows = array();
foreach($models as $model)
{
unset($model->attributes['Password']); //add this line
$Users[] = $model->attributes;
}
$SandboxObj->_sendResponse(200, CJSON::encode($Users));
}
Or simply don't select Password field in your query if you don't need it .

How to insert data in symfony table?

I have made a symfony form and I used:
$conn->insert('Invoiceshasitems', array('Invoiceitemsid' => '$items'));
for data insertion but it not works ho i do insert data from symfony controller.
here is my code:
$itemscounter = $request->request->get('itemscounter');
if(isset($_SESSION['invoiceid'])) {
$invoiceid=$_SESSION['invoiceid'];
//echo $invoiceid;
//exit;
}
//$entity= new Invoiceshasitems();
if($request->isXmlHttpRequest()) {
if($itemscounter > 1){
for($i=1; $i<=$itemscounter; $i++){
if($i==1){
$items = $_POST['items'];
}else {
$items.$i = $_POST['items'.$i];
}
}
}else{
$items = $_POST['items'];
$conn->insert('Invoiceshasitems', array('Invoiceitemsid' => '$items'));
}
}
You can use native query in Doctrine, there is some docs here: http://docs.doctrine-project.org/en/latest/reference/native-sql.html
For your example the code is something like that:
$rsm = new ResultSetMapping();
$query = $this->_em->createNativeQuery('INSERT INTO Invoiceshasitems SET Invoiceitemsid = ?', $rsm);
$query->setParameter(1, $items);
$result = $query->getResult();

Categories