ORM removes duplicates. But not everywhere - php

I am making some DB query via repository (ORM), I want to get a list of objects.
What's my problem?
I have two methods in the repository: getList() to get a list and getCount() to get the number of total records for information.
getList() gives me 5 records, and getCount() gives the number 9.
public function getList($criteria, $page, $limit, $orderBy, $joins = [])
{
$qb = $this->createQueryBuilder('a')->select('a');
if(!empty($joins)) {
$qb = $this->setCriteriaByJoins($qb, $joins);
}
$qb = $this->setCriteriaToQuery($qb, $criteria);
foreach($orderBy as $order) {
$qb->addOrderBy('a.' . $order[0], $order[1]);
}
if($page && $limit) {
$offset = ($page - 1) * $limit;
$qb->setFirstResult($offset)
->setMaxResults($limit);
}
$result = $qb->getQuery()->getResult();
return $result;
}
public function getCount($criteria = [], $joins = []) {
try {
$qb = $this->createQueryBuilder('a')->select('count(a.id)');
if(!empty($joins)) {
$qb = $this->setCriteriaByJoins($qb, $joins);
}
$qb = $this->setCriteriaToQuery($qb, $criteria);
$result = $qb->getQuery()->getSingleScalarResult();
} catch(\Exception $e) {
$result = 0;
}
return $result;
}
In debugging, I see that the getList() request was built like this:
SELECT r0 _. * FROM film r0_ LEFT JOIN person_film r1_ ON (r0_.id = r1_.film_id) WHERE r1_.person_id IN (45793) AND r0_.status = 1 ORDER BY r0_.`release` DESC, r0_.id DESC LIMIT 48
and getCount()
SELECT count (r0_.id) AS sclr_0 FROM film r0_ LEFT JOIN person_film r1_ ON (r0_.id = r1_.film_id) WHERE r1_.person_id IN (45793) AND r0_.status = 1
Requesting getList() from the repository gives me 5 records. However, if I directly ask the database with this query (without ORM), then I will get 9 records. Of these 9 records, 4 are duplicated due to the fact that the person_film table can contain more than one record with the same film_id and person_id (these records differ in other fields). I found that /vendor/doctrine/orm/lib/Doctrine/ORM/Internal/Hydration/ObjectHydrator.php:hydrateRowData() removes duplicates.
How to do to make the ORM in the getList() request return all 9 records, even if they are duplicates?
And vice versa. How to do to the getCount() to return 5.

it seems like this is the answer to the second question
public function getCount($criteria = [], $joins = [], $distinct = false) {
try {
$sq = ($distinct) ? 'count(distinct(a.id))' : 'count(a.id)';
$qb = $this->createQueryBuilder('a')->select($sq);
if(!empty($joins)) {
$qb = $this->setCriteriaByJoins($qb, $joins);
}
$qb = $this->setCriteriaToQuery($qb, $criteria);
$result = $qb->getQuery()->getSingleScalarResult();
} catch(\Exception $e) {
$result = 0;
}
return $result;
}

Related

What causes this failure to get the data from the next row in a MySQL table, with Codeigniter 3?

I am working on an online newspaper/blogging application with CodeIgniter 3.1.8 and Bootstrap 4.
At the bottom of the single post view, I want to add a link to the next post (as well as one to the previous post). For this, I need to get the data (slug, title, etc), of the next post (row in the posts table).
For this purpose, I have added this method to my Posts_model model:
/* Next post */
public function get_next_post($slug) {
$query = $this->db->get_where('posts', array('slug' => $slug));
if ($query->num_rows() > 0) {
$data = $query->next_row();
return $data;
}
}
In the controller I have:
public function post($slug) {
//more code
$data['post'] = $this->Posts_model->get_post($slug);
$data['next_post'] = $this->Posts_model->get_next_post($slug);
print_r($data['next_post']);
//more code
}
EDIT: In the Posts_model, I now have:
/* Next post */
public function get_next_post($slug) {
$query = $this->db->get('posts');
$row_index = 6;
$data = $query->row_array($row_index);
if ($query->num_rows() > 0) {
$data = $query->next_row();
return $data;
}
}
/* Prev post */
public function get_prev_post($slug) {
$query = $this->db->get('posts');
$row_index = 6;
$data = $query->row_array($row_index);
if ($query->num_rows() > 0) {
$data = $query->previous_row();
return $data;
}
}
That means that if I could get the current post's index by slug, I could replace this hardcoded index of the 7th post - $row_index = 6 - and the problem would be solved.
How do I do that?
// Your_model.php
...
public function getPost($slug) {
$this->db->where('slug', $slug);
return $this->db->get('posts_table')->row_array();
}
public function getPrevPost($currentPostId) {
$this->db->where('id <', $currentPostId);
$this->db->order_by('id', 'desc');
$this->db->limit(1);
return $this->db->get('posts_table')->row_array();
}
public function getNextPost($currentPostId) {
$this->db->where('id >', $currentPostId);
$this->db->limit(1);
return $this->db->get('posts_table')->row_array();
}
// Yourcontroller.php
...
public function getPost($slug) {
$post = $this->your_model->getPost($slug);
$data = [
'thePost' => $post,
...
'prevPost' => $this->your_model->getPrevPost($post['id']),
'nextPost' => $this->your_model->getNextPost($post['id']),
...
];
...
}
EDIT: this post answers the original question. In the meantime below code was used in an edit by OP.
you need to return a result of your query: $data = $query->row_array();
And get_where() is limiting the record-set to one record, hence there is no next record. You need to return the complete record-set with $this->db->get('posts'). In case you know the row_number (e.g.: 5) of the row containing $slug, you can point to it. The next_row shown, is row number 6.
public function get_next_post($slug) {
$query = $this->db->get('posts'); // querying the whole data-set
$data = $query->row_array(5); // the missing line
if ($query->num_rows() > 0) {
$data = $query->next_row();
return $data;
}
}
now you should get your next row (if exists), see Result Rows

How do I call Codeigniters 2 Active record to get 2 sets of results from the same query?

I am essentially creating a chunking function for a file export, and I have a select->from->where that I have built already. I want to be able to call 'get' twice each time with different limit/offset values.
Here's a walk-through of the basic idea.
// BallReport.php
function ProcessData(){
//Report 1
$query = createSelectQuery();
$query = applyReportOneWhereValues($query);
$results1 = CSVTool::processLargeDataSet($query, 10, 1000);
//Report 2
$query = createSelectQuery();
$query = applyReportTwoWhereValues($query);
$results2 = CSVTool::processLargeDataSet($query, 10, 1000);
}
function createSelectQuery(){
// the select is complicated having multiple joins and sub queries
// so I only want to have to write this once
$query = $this->db->select('ball.name,
color.name,
size.name,
shape.name')
->from('ball')
->join('color', 'ball.color_id = color.id')
->join('size', 'ball.size_id = size.id')
->join('shape', 'ball.shape_id = shape.id');
return $query;
}
function applyReportOneWhereValues($query){
// I have 2 different sets of where parameters
// But they are both using the same select
// so I separated them into these functions
// So I can apply the set of where statements
// all at once
$query = $query->where("table.color", "blue")
->where("table.size" , "large")
->where("table.shape", "round");
return $query;
}
function applyReportTwoWhereValues($query){
$query = $query->where("table.color", "red")
->where("table.size" , "small")
->where("table.shape", "round");
return $query;
}
//In CSVTool.php
public static function processLargeDataSet($query, $numberOfPages, $chunkSize){
// Since the data set is going to be so large we want to process in chunks
// So that we don't hit the limit and break mid way.
// To do that we only call the DB in sets of 1000 rows
for(int $i = 0; $i <= $numberOfPages: $i++){
processRows($query, $i * $chunkSize, $chunkSize);
}
}
function processRows($query, $offset, $limit){
// We limit in here so each time it's called we change the offset and limit
$query = $query->offset($offset)->limit($limit);
$valuesToProcess = $query->get()->result_array();
// process the rows here
}
this of course doesn't work because once processRows calls $query->get() the first time all subsequent calls throws a Query error: No tables used
Is there any solution for this? Is there a chunking function in Codeigniter 2 that I'm unaware of?
Here is a new answer to the revised question.
public function ProcessData()
{
//Report 1
$query_builder = $this->applyReportOneWhereValues($this->createSelectQuery());
$this->db->stop_cache();
$results1 = CSVTool::processLargeDataSet($query_builder, 10, 1000);
$this->db->flush_cache();
//Report 2
$query_builder = $this->applyReportTwoWhereValues($this->createSelectQuery());
$this->db->stop_cache();
$results2 = CSVTool::processLargeDataSet($query_builder, 10, 1000);
$this->db->flush_cache(); //just to be safe
}
public function createSelectQuery()
{
$this->db->start_cache();
return $this->db->select('ball.name, color.name, size.name, shape.name')
->join('color', 'ball.color_id = color.id')
->join('size', 'ball.size_id = size.id')
->join('shape', 'ball.shape_id = shape.id');
}
public function applyReportOneWhereValues($query_builder)
{
return $query_builder
->where("table.color", "blue")
->where("table.size", "large")
->where("table.shape", "round");
}
public function applyReportTwoWhereValues($query_builder)
{
return $query_builder
->where("table.color", "red")
->where("table.size", "small")
->where("table.shape", "round");
}
In CSVTool.php
/**
* Process the records in chunks
* #param CI_DB_query_builder $qb An instance of the CI_DB_query_builder class
* #param int $numberOfPages The number of pages to create in the set (1 to n)
* #param int $pageSize The number of records per page
*/
public static function processLargeDataSet($qb, $numberOfPages, $pageSize)
{
if($numberOfPages < 1)
{
$numberOfPages = 1;
}
for($i = 1; $i < $numberOfPages; $i++)
{
$valuesToProcess = $qb
->get('ball', $pageSize, $i - 1 * $pageSize)
->result_array();
// process the rows in $valuesToProcess
}
}
I think what you are looking for is "Active Record Caching". This could be managed from a couple different places. In this answer it is in ProcessData()
Note:
You were assigning lots of things to the same var $query and passing it around a lot for no good reason I can see. And you are often overwriting $query with the exact same value multiple times in a row. I have used $this->db in most of the places you used $query.
public function ProcessData()
{
//Report 1
$this->db->start_cache();
//createSelectQuery(); not needed if you want all fields from one table
applyReportOneWhereValues();
$this->db->stop_cache();
processLargeDataSet(10, 1000);
//Report 2
$this->db->flush_cache()
$this->db->start_cache();
//createSelectQuery(); not needed if you want all fields from one table
applyReportTwoWhereValues();
$this->db->stop_cache();
processLargeDataSet(10, 1000);
$this->db->flush_cache();
}
Your question uses select("*") and from("table_name") which can be eliminated if you really want all fields from one table. When get("table_name") is used and there is no select() call then all fields are assumed. IOW, the query statement would be SELECT * FROM 'table_name';
Based on the question's code it seems you don't need the createSelectQuery() function.
Your "apply where" functions but re-written using method chaining.
public function applyReportOneWhereValues()
{
$this->db
->where("table.color", "blue")
->where("table.size", "large")
->where("table.shape", "round");
}
public function applyReportTwoWhereValues()
{
$this->db
->where("table.color", "red")
->where("table.size", "small")
->where("table.shape", "round");
}
I have eliminated processRows() and incorporated that logic into processLargeDataSet(). Notice how get() is used - passing a table name, limit, and offset - to remove the need for select(), from(), limit(), and offset() calls.
/**
* Process the records in chunks
* #param int $numberOfPages The number of pages to create in the set (1 to n)
* #param int $pageSize The number of records per page
*/
function processLargeDataSet($numberOfPages, $pageSize)
{
if($numberOfPages < 1)
{
$numberOfPages = 1;
}
for($i = 1; $i < $numberOfPages; $i++)
{
$valuesToProcess = $this->db
->get('table', $pageSize, ($i-1) * $pageSize)
->result_array();
// process the rows in $valuesToProcess
}
}

Fatal error: Can't use function return value in write context in Codeigniter

This is the function I am using in my model
public function user_birthday() {
$this->db->select('birth_day')
->from('informations')
->where(DATE_FORMAT(FROM_UNIXTIME('birth_day'), '%m-%d') = DATE_FORMAT(NOW(), '%m-%d'));
$q = $this->db->get();
return $q->result();
}
function in controller like $this->data['user'] = $this->users_m->user_birthday(); that way
and code in the view is
if (!empty($user)):
echo $user;
else:
echo "No dob found";
endif;
This is one of those circumstances where Query Builder is more trouble that it is worth. Try this
$q= $this->db->query("SELECT birth_day FROM informations WHERE DATE_FORMAT(birth_day, '%m-%d') = DATE_FORMAT(NOW(), '%m-%d')");
//return empty array if not records found
return $q->num_rows() > 0 ? $q->result() : [];
If you really, really, really want to use Query Builder then this
$q = $this->db
->select('birth_day')
->where("DATE_FORMAT(birth_day, '%m-%d')=", "DATE_FORMAT(NOW(), '%m-%d')", FALSE)
->get('informations');
return $q->num_rows() > 0 ? $q->result() : [];

How to put pagination on search result using symfony2?

I am working on a project using symfony2. I already have my search function and it is already working. But what I want is to have pagination on the search results since there are times that there are a lots of data being retrieve. How can I possibly do it?
Here is my code :
public function searchAction (Request $request) {
$request = $this->getRequest();
$data = $request->get('search');
$em = $this->getDoctrine()->getManager();
$query = $em->createQuery (
'SELECT a,b,c FROM SampleBundle:transactDetail a
JOIN a.transact b
JOIN b.transactType c
WHERE b.pNumber LIKE :data
OR b.senderId LIKE :data'
)
->setParameter('data', "%$data%");
$result = $query->getResult();
if ($result == null ) {
return $this->render('SampleBundle:Sample:noresult.html.twig');
}
return $this->render('SampleBundle:Sample:search.html.twig', array('result' => $result));
}
Thanks in advance.
This is an example of your code with Paginator :
public function searchAction(Request $request, $firstResult = 0, $maxResults = 100)
{
$request = $this->getRequest();
$data = $request->get('search');
$em = $this->getDoctrine()->getManager();
$query = $em->createQuery(
'SELECT a,b,c FROM SampleBundle:transactDetail a
JOIN a.transact b
JOIN b.transactType c
WHERE b.pNumber LIKE :data
OR b.senderId LIKE :data'
)
->setParameter('data', "%$data%")
->setFirstResult($firstResult)
->setMaxResults($maxResults);
$result = new Paginator($query);
if ($result == null) {
return $this->render('SampleBundle:Sample:noresult.html.twig');
}
return $this->render('SampleBundle:Sample:search.html.twig', array('result' => $result));
}
And if you would like to count the total just count($result);.
Paginator has Countable et IteratorAggregate interfaces.
So you can loop on your result without problem.

How to use raw sql Pagination in Laravel5?

This is my Controller code:
$sql = "SELECT *,earth_distance(ll_to_earth(team.lat, team.lng), ll_to_earth(23.1215939329,113.3096030895)) AS distance FROM team where earth_box(ll_to_earth(23.1215939329,113.3096030895),1000) #> ll_to_earth(team.lat, team.lng); ";
$result = DB::select( \DB::raw( $sql ) );
How can I add pagination to this code to build my restful api?
iOS or android will send the "next page" parameter, how to use it and find the next section data?
As far as I know you can't paginate raw query, here's why:
$result = DB::select($sql);
$result here will have the array type and paginate() is the method from the Illuminate\Database\Query\Builder class.
Your case can be performed this way:
$items = DB::table('team')
->selectRaw('SELECT *,earth_distance(ll_to_earth(team.lat, team.lng), ll_to_earth(23.1215939329,113.3096030895)) AS distance')
->whereRaw('earth_box(ll_to_earth(23.1215939329,113.3096030895),1000) #> ll_to_earth(team.lat, team.lng)')
->paginate(10);
foreach($items as $item) {
echo $item->distance;
}
As you can see minimal effort is needed here to separate raw query to selectRaw() and whereRaw() methods.
Another option if you are trying to paginate dynamic columns that maybe you were processing calculations on for reporting is to create a sort method and pass in your array and params:
public function sort($array_of_objects, $sort_by=null, $order, $page)
{
$collection = collect($array_of_objects);
if ($sort_by)
{
if ($order=='desc') {
$sorted = $collection->sortBy(function($role) use ($sort_by)
{
return $role->{$sort_by};
})->reverse();
} else if ($order=='asc') {
$sorted = $collection->sortBy(function($role) use ($sort_by)
{
return $role->{$sort_by};
});
}
} else {
$sorted = $collection;
}
$num_per_page = 20;
if (!$page) {
$page = 1;
}
$offset = ( $page - 1) * $num_per_page;
$sorted = $sorted->splice($offset, $num_per_page);
return new Paginator($sorted, count($array_of_objects), $num_per_page, $page);
}

Categories