I am using aws PHP sdk to querying dynamodb.
Requirement is to fetch records in a batch of 10 items.
When user scroll screen down then fetch 10 previous items and so on.
How can I instruct Dynamodb to fetch only 10 items ? I use Limit option but as per documentations of dynamodb Limit option is the number of records to process , not number of records dynamodb will return.
How can I give limit of number of records to return so that in next run I can use LastEvaluatedKey as ExclusiveStartkey for next 10.
Here is the snippet of my code
// It means get the top $limit posts
$iterator = $dynamo->query(array(
'TableName' => 'posts',
'IndexName' => 'college_id-unix_timestamp-index',
'ScanIndexForward' => false,
'ExclusiveStartKey' => $lastEvaluatedKey,
'KeyConditions' => array(
'college_id' => array(
'AttributeValueList' => array(
array('S' => (string)$collegeId )
),
'ComparisonOperator' => 'EQ'
)
),
'Limit' => (Integer)$limit,
)
);
Related
I am using subquery for id field.
$db = $this->AccountRequest->getDataSource();
$subQuery = $db->buildStatement(
array(
'fields' => array('MAX(id)'),
'table' => $db->fullTableName($this->AccountRequest),
'alias' => 'MaxRecord',
'limit' => null,
'offset' => null,
'order' => null,
'group' => array("user_id")
),
$this->AccountRequest
);
$searching_parameters = array(
#"AccountRequest.id IN " => "(SELECT MAX( id ) FROM `account_requests` GROUP BY user_id)"
"AccountRequest.id IN " => "(".$subQuery.")"
);
$this->Paginator->settings = array(
#'fields' => array('AccountRequest.*'),
'conditions' => $searching_parameters,
'limit' => $limit,
'page' => $page_number,
#'group' => array("AccountRequest.user_id"),
'order' => array(
'AccountRequest.id' => 'DESC'
)
);
$data = $this->Paginator->paginate('AccountRequest');
This structure is producing a query is:
SELECT
`AccountRequest`.`id`,
`AccountRequest`.`user_id`,
`AccountRequest`.`email`,
`AccountRequest`.`emailchange`,
`AccountRequest`.`email_previously_changed`,
`AccountRequest`.`first_name`,
`AccountRequest`.`first_namechange`,
`AccountRequest`.`f_name_previously_changed`,
`AccountRequest`.`last_name`,
`AccountRequest`.`last_namechange`,
`AccountRequest`.`l_name_previously_changed`,
`AccountRequest`.`reason`,
`AccountRequest`.`status`,
`AccountRequest`.`created`,
`AccountRequest`.`modified`
FROM
`syonserv_meetauto`.`account_requests` AS `AccountRequest`
WHERE
`AccountRequest`.`id` IN '(SELECT MAX(id) FROM `syonserv_meetauto`.`account_requests` AS `MaxRecord` WHERE 1 = 1 GROUP BY user_id)'
ORDER BY
`AccountRequest`.`id` DESC
LIMIT 25
In the subquery, its add an extra single quote so it's producing an error.
So, How can I remove these single quotes from this subquery?
Thanks
What are you trying to achieve with the sub query?
The MAX(id) just means it will pull the id with the largest value AKA the most recent insert. The sub query is completely redundant when you can just ORDER BY id DESC.
using MAX() will return only one record, if this is what you want to achieve you can replicate by adding LIMIT 1
If the sub query is just an example and is meant to be from another table I would just run the query that gets the most recent id before running the main query. Getting the last inserted id in a separate query is very quick and I cant see much of a performance loss. I think it will result in cleaner code that`s easier to follow to.
edit 1: From the comments it sounds like all your trying to get is a particular users latest account_requests.
You dont need the sub query at all. My query below will get the most recent account record for the user id you choose.
$this->Paginator->settings = array(
'fields' => array('AccountRequest.*'),
'conditions' => array(
'AccountRequest.user_id' => $userID // you need to set the $userID
)
'page' => $page_number,
'order' => array(
'AccountRequest.id DESC' //shows most recent first
),
'limit' => 1 // set however many you want the maximum to be
);
The other thing you cold be meaning is to get multiple entries from multiple users and display them in order of user first and then the order of recent to old for that user. MYSQL lets you order by more than one field, in that case try:
$this->Paginator->settings = array(
'conditions' => array(
'AccountRequest.user_id' => $userID // you need to set the $userID
)
'page' => $page_number,
'order' => array(
'AccountRequest.user_id', //order by the users first
'AccountRequest.id DESC' //then order there requests by recent to old
)
);
If the example data you have added into the question is irrelevant and you are only concerned about how to do nested subqueries it has already been answered here
CakePHP nesting two select queries
However I still think based on the data in the question you can avoid using a nested query.
I am working on a ecommerce with platform cakephp and using google charts for reports.My requirement is to get all records as per all 12 months, so I have used following code for a single month
Query
SELECT COUNT(*) AS count FROM orderproductmasters AS Orderproductmaster
LEFT JOIN ordermasters AS Ordermaster ON
(Orderproductmaster.ordermaster_id = Ordermaster.id) LEFT JOIN productmasters AS Productmaster ON
(Orderproductmaster.productmaster_id = Productmaster.id)
WHERE Ordermaster.orderstatusmaster_id = 1 AND Month(Orderproductmaster.created) = 8
Code
$this->Orderproductmaster->find('count',
array('conditions'=>array('Ordermaster.orderstatusmaster_id'=>1,'
Month(Orderproductmaster.created)'=>8)));
Since, I need records as per Jan, feb,march and all 12 months...,so for 12 months I am using following code
for($i=1;$i<13;$i++)
{
$orderproductmasters[$i] = $this->Orderproductmaster->find('count',
array('conditions'=>array('Ordermaster.orderstatusmaster_id'=>1,
'Month(Orderproductmaster.created)'=>$i)));
}
So question might be silly, but is it possible to get all months record without using for loop i.e, within a single query.
Thanks in advance
I think , your need can be fulfilled by using cursors in stored procedure. And then using stored procedure to cake-php.
Example on db side is here
$options = array();
$options['fields'] = array('COUNT(Orderproductmaster.id)');
$options['conditions'] = array('Ordermaster.orderstatusmaster_id = 1',
'Month(Orderproductmaster.created) BETWEEN 1 AND 12');
$options['joins'] = array(
array(
'table' => 'ordermasters',
'alias' => 'Ordermaster',
'type' => 'left',
'conditions' => array(
'Orderproductmaster.ordermaster_id = Ordermaster.id'
)
),
array(
'table' => 'productmasters',
'alias' => 'Productmaster',
'type' => 'left',
'conditions' => array(
'Orderproductmaster.productmaster_id = Productmaster.id'
)
)
);
$options['group'] => array('Month(Orderproductmaster.created)');
$this->Orderproductmaster->find('all',$options);
What About something like:
$this->Orderproductmaster->find('count',
array(
'fields'=>'DISTINCT(Month(Orderproductmaster.created)),
'conditions'=>array('Ordermaster.orderstatusmaster_id'=>1,'
Month(Orderproductmaster.created)'=>8)));
After quite some trying out and web research I go crazy with this query. I want to build a query for 'Clubs' around a geo point (distance max 500 meters) in php on MongoDB.
But when I run query it ignores the distance limit and shows all clubs in database BUT sorted by distance.
Here is my dataset (2dsphere index geoLoc):
{"_id":ObjectId("547c649e30afe32c23000048"),"name":"Club Ritzz","category":"Club","category_list":[{"id":"191478144212980","name":"Night Club"}],"location":{"city":"Mannheim"},"geoLoc":{"type":"Point","coordinates":[8.473665839156,49.484065272756]}}
{"_id":ObjectId("547c649f30afe32c2300004a"),"name":"Das Zimmer Mannheim","category":"Club","category_list":[{"id":"191478144212980","name":"Night Club"}],"geoLoc":{"type":"Point","coordinates":[8.4709362941178,49.487260552592]}}
{"_id":ObjectId("547c64ab30afe32c23000063"),"name":"Nationaltheater Mannheim","category":"Arts/entertainment/nightlife","category_list":[{"id":"173883042668223","name":"Theatre"}],"geoLoc":{"type":"Point","coordinates":[8.4776534992592,49.48782606969]}}
{"_id":ObjectId("547c64a130afe32c2300004f"),"name":"SOHO Bar Club Lounge","category":"Club","category_list":[{"id":"191478144212980","name":"Night Club"},{"id":"164049010316507","name":"Gastropub"}],"geoLoc":{"type":"Point","coordinates":[8.4630844501277,49.49385193591]}}
{"_id":ObjectId("547c64a730afe32c2300005a"),"name":"Loft Club","category":"Club","category_list":[{"id":"191478144212980","name":"Night Club"},{"id":"176139629103647","name":"Dance Club"}],"geoLoc":{"type":"Point","coordinates":[8.4296300196465,49.484211928258]}}
And here my php code (updated Dec-2):
$qry = $pub->find(
array( '$and' =>
array(
array( 'geoLoc' =>
array('$nearSphere' =>
array('$geometry' =>
array('type'=>'Point',
'coordinates'=>
array(
floatval($sLon), floatval($sLat)
)
),
'maxDistance' => 500
)
)
),
array( '$or' =>
array(
array( 'name' => new MongoRegex("/.*club/i")),
array( 'name' => new MongoRegex("/.*zimm/i"))
)
),
array('$or' =>
array(
array('category_list.name' => 'Night Club'),
array('category_list.name' => 'Dance Club'),
array('category' => 'Club')
)
)
)
),
array('id' => 1, 'name' => 1, '_id' => 0)
);
Anyone know why the results are not limited to the specified maxDistance?
I found a similar issue on StackOverflow which outlines that one has to use radians for the maxDistance parameter.
See https://dba.stackexchange.com/questions/23869/nearsphere-returns-too-many-data-what-am-i-missing-am-i-wrong-is-it-a-bug-d
Also it is probably helpful if you'd test the query in mongo shell without using the PHP APIs first (just to see if the query is generally working and append '.explain()' to it to see what generally happens inside DB).
Sorry for my english, I need help on mongodb indexes. I have a capped collection (size: 10GB) with some fields for my application logs.
Example structure: Logs[_id, userId, sum, type, time, response, request]. I have created compound index: [userId,time,type]. I get two arrays are grouped records by userId for today, where 'type' is "null" and "1". And my two query example:
$group = array(
array(
'$match' => array(
'userId' => $userId,
'time' => array(
'$gt' => date("Y-m-d")
),
'type' => array('$ne' => null)
)
),
array(
'$group' => array(
"_id" => '$userId',
"total" => array('$sum' => '$sum'),
"count" => array('$sum' => 1)
),
)
);
$results = $collections->aggregate($group);
$group = array(
array(
'$match' => array(
'userId' => $userId,
'time' => array(
'$gt' => date("Y-m-d")
),
'type' => 1
)
),
array(
'$group' => array(
"_id" => '$userId',
"count" => array('$sum' => 1)
),
)
);
$results2 = $collections->aggregate($group);
If current user has more 100000 documents on collection for today - the speed of my query is very slow (more 10 sec). Give me some advices on creating the right index, please :) Thanks.
Based on the explain that you posted, the correct index is being used (BtreeCursor), it is using only the index (i.e. it is a covered index query - indexOnly is true) and nothing is being matched (n = 0) in this case. So, that all checks out generally, though $ne as a clause in the first example is not going to be very efficient.
However the main issue based on the explain is likely the fact that the index does not appear to be fully in memory. There are 13 yields listed and the most common reason for a query like this to yield is when it has to fault to disk to page something in. Since, as mentioned previously, it is only using the index, those yields imply faults to disk for the index and hence indicate that the whole index is not in memory.
If you re-run the query immediately after this it should be faster (assuming the index can actually fit into available memory) because the index will have been paged in by the first run. If it is still slow on the second run and showing yields, then you either don't have enough memory to hold the index in memory or something else is evicting it from memory and you essentially have memory contention causing performance problems.
I have a user model which gives me latest users as output. How can I limit the record to just output me 200 records instead of all the users in database?
According to the documentation, the second argument to the find() method is a $params array.
One of the possible values to pass in this array is a limit key. So you could do the following:
$users = $this->User->find('all', array('limit' => 200));
"i have it like array('limit' => 21, 'page' => 1) for paging 21 users in one page.. if i change the limit there to 200 then it paginates 200 users in one page only...in this case how to limit along with proper pagination?? – Anonymous May 14 '09 at 7:22"
yes you can use the cakePHP pagination helper as someone has mentioned. But there may be some cases where you want to do your own pagination or just limit the number of records retrieved per call. For what it's worth here's how I handled one such situation.
Say for example you want to retrieve a certain number of records per page, Then:
$start = 0; -> this is in order to start retrieving records starting from the first one. If you need to say for example, start from the 31st, then $start = 30;
So,
$start = 0;
$length = 20; // we are going to retrieve 20 records starting from the first record
And the code will be something like:
// To retrieve a number of Products per page
$products = $this->Product->find('all', array(
'order' => 'product_number ASC',
'limit' => $start.','.$length,
'recursive' => -1
)
);
Don't paginate with find().
Cake Pagination: http://book.cakephp.org/2.0/en/core-libraries/components/pagination.html
array(
'conditions' => array('Model.field' => $thisValue), //array of conditions
'recursive' => 1, //int
//array of field names
'fields' => array('Model.field1', 'DISTINCT Model.field2'),
//string or array defining order
'order' => array('Model.created', 'Model.field3 DESC'),
'group' => array('Model.field'), //fields to GROUP BY
'limit' => n, //int
'page' => n, //int
)
Limit * page = 200 set your values according to your comfortable view in pages. This might help
You can also try this out
$results = $this->Model->find('all',
array('limit'=>10,
'order'=>array('date DESC')));
open the model file of user and do as follows:
you will need to change the 'limit' property in the relationship variable named
var $hasMany = array( 'Abus' =>
array('className' => 'Abus',
'foreignKey' => 'user_id',
'dependent' => false, 'conditions'
=> '',
'fields' => '', 'order' => '', 'limit' => '200', 'offset'
=> '', 'exclusive' => '', 'finderQuery' => '',
'counterQuery' => '' ) );
OR you can also try this out...
in your users controller set the $paginate to like this.
var $paginate = array('limit' => 200);
The records will be limited to 200 now wherever you use paginate.