I use this code in the MongoDB PHP driver to get all documents in the database
$result = $collection->find();
foreach ($result as $doc) {
print_r($doc);
}
However, when adding a limit to it, it doesn't work anymore: no documents get printed anymore:
$result = $collection->find()->limit(10);
foreach ($result as $doc) {
print_r($doc);
}
There are certainly enough documents in the database. I cannot figure out what the problem with this is.
I have fixed the problem by taking a look at the source of the beta version. The documentation only appeared to be for the legacy mongo extension and not the newer mongodb extension.
The error logs showed this: Call to undefined method MongoDB\\Driver\\Cursor::addOption(). I checked out the documentation and concluded the function should have worked because it said (PECL mongo >=0.9.0). Note the missing db after mongo.
I fixed it by doing:
$collection->find([], [ 'limit' => 2 ]);, providing an empty filters array and adding my options in another array afterwards.
I am trying to describe with example for new php mongodb driver. showing in example skip,limit,Fields slection
require 'vendor/autoload.php'; // include Composer's autoloader
$client = new MongoDB\Client("mongodb://localhost:27017");
// SELECT * FROM YOUR_TABLE_NAME ;
// db.YOUR_COLLECTION_NAME.find({});
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array());
//SELECT * from YOUR_TABLE_NAME WHERE YOUR_COLUMN = "A"
// db.YOUR_COLLECTION_NAME.find({{ YOUR_FIELD: "A" }});
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array('YOUR_FIELD'=>'A'));
//Return the Specified Fields and the _id Field Only
//SELECT _id, item,status YOUR_TABLE_NAME from inventory WHERE status = "A"
//db.YOUR_COLLECTION_NAME.find( { status: "A" }, { item: 1, status: 1 } )
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array('status'=>'A'),array('projection' =>array('item'=>TRUE,'status' => TRUE)));
//Suppress _id Field
//SELECT item, status from YOUR_TABLE_NAME WHERE status = "A"
//db.YOUR_COLLECTION_NAME.find( { status: "A" }, { item: 1, status: 1, _id: 0 } )
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array('status'=>'A'),array('projection' =>array('item'=>TRUE,'status' => TRUE,'_id'=>FALSE)));
//SELECT * FROM YOUR_TABLE_NAME LIMIT 10
//db.YOUR_COLLECTION_NAME.find({}).limit(10);
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array(),array('limit'=>10));
//SELECT * FROM YOUR_TABLE_NAME LIMIT 5,10
//db.YOUR_COLLECTION_NAME.find({}).skip(5).limit(10)
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array(),array('skip'=>5,'limit'=>10));
//Suppress _id Field
//SELECT item, status from YOUR_TABLE_NAME WHERE status = "A" LIMIT 5,10;
//db.YOUR_COLLECTION_NAME.find( { status: "A" }, { item: 1, status: 1, _id: 0 } ).skip(5).limit(10);
$result = $clinet->YOUR_DB_NAME->YOUR_COLLECTION_NAME->find(array('status'=>'A'),array('projection' =>array('item'=>TRUE,'status' => TRUE,'_id'=>FALSE),'skip'=>5,'limit'=>10));
foreach ($result as $entry){
echo "<pre>";
print_r($entry);
echo "</pre>";
}
As a solution to your above mentioned problem please try executing following code snippet.
$result = $collection->find();
$result->limit(10);
foreach ($result as $doc) {
print_r($doc);
}
I had the same issue. There are a lot of code examples using $result = $collection->find()->limit(10);
It turns out, that while this was totally valid for the original version of the MongoDB PHP driver, there is a new version of that very same driver. The original driver is now considered "The Legacy Driver".
Here is one example, how the "old" driver was supposed to be used:
<?php
$m = new MongoClient;
// Select 'demo' database and 'example' collection
$collection = $m->demo->example;
// Create the cursor
$cursor = $collection->find();
At this moment, although a cursor object had been created, the query had not yet executed (i.e. it was not sent to the server). The query would only be executed by starting iteration with foreach ( $cursor as $result ) or calling $cursor->rewind(). This gives you the chance to configure the cursor's query with sort(), limit(), and skip() before it is executed by the server:
// Add sort, and limit
$cursor->sort( [ 'name' => 1 ] )->limit( 40 )
In the new driver, as soon as you have a \MongoDB\Driver\Cursor object, it has already been processed by the server. Because sort (and limit and skip) parameters need to be sent to the server before the query is executed, you can not retroactively call them on an existing Cursor object.
This is the reason, why there is no limit() method anymore, as there used to be. Also, the accepted answer is correct. I want to give a more elaborate example:
$filter = [
'author' => 'rambo',
'views' => [
'$gte' => 100,
],
];
$options = [
/* Return the documents in descending order of searchPage */
'sort' => [
'searchPage' => -1
],
/* Limit to 2 */
'limit' => 2,
/* close the cursor after the first batch */
'singleBatch' => true,
];
$cursor = $collection->find($filter, $options);
Related
I need help with MongoDb with PHP driver.
I have 4 collections:
order_aproved :
{
"order_id" : mongoId ,
"user_id":num ,
"order_date":mongoDate ,
"requset" : string
}
orders_rejected :
{
"order_id" : mongoId,
"user_id" : num ,
"order_date" : mongoDate ,
"requset" : string
}
users :
{
"user_id" : mongoId,
"username" : num ,
"last_order" : mongoDate ,
"num_orders" : num,
"last_order"
}
orders_log :
{
"order_id" : mongoId ,
"order_date" : mongoDate ,
"status" : boolen ,
"user_id" : num
}
Every approved/rejected order, I update the num_orders on user document
that have a new/rejected order. So that number is always changing
and log that order on orders_log.
I need to fetch all orders approved/rejected on orders_log by list of users [array] with condition and get the orders count num_orders and last order date for that user by the order from this user
I am doing it like this:
$cursor = $orders->find()->sort(array("order_date" => -1))->limit(15);
$array = iterator_to_array($cursor,false);
$users_for_aproved = ["123","124","125"];
$users_for_rejcted = ["112","113","114"];
$js = "function() { if ( this.requset ) { return this.requset.length > 0 } }";
$query1 = array( '$and' => array(
array("user_id" => array('$in'=> $users_for_aproved)),
array('$where' => $js )
));
$query1 = array( '$and' => array(
array("user_id" => array('$in'=> $users_for_rejcted)),
array('$where' => $js )
));
$query_or = array('$or' => array($query,$query1);
$cursor = $orders_log->find($query_or)->sort(array("order_date" => -1))->limit(15);
$array = iterator_to_array($cursor,false);
for ( $x=0; $x < count($array) ; $x++ ) {
$query = array( "user_id" => $order["user_id"] );
$cursor = $orders->find($query)->limit(1);
$array = iterator_to_array($cursor,false);
$order_count = $array[0]["num_orders"];
$array[$x]["order_count"] = $order_count;
}
return $array;
It's working but its not very efficient , i need a way to fetch data from another collection and add the num_orders to the doc that i have find without a use form anther query
like SQL JOIN but on mongo and php driver
Thanks!
There are two ways to achieve gain in performance in this case:
create an index on MongoDB:
db.order_aproved.createIndex( { user_id: 1 } )
You may create the above index either in the above way, or in the background:
db.order_aproved.createIndex( { user_id: 1 }, { background: true } )
In the last case, the creation will be slower, but it will not bother the currently ongoing operations on the database. If you may afford it, I think you should better create an index not in the background, esp. if are not running this script on the Production Database
re-design the collections, so that instead of the different collections, joined by some ID, you should create embedded documents inside the main document, thus eliminating the need to perform any operations, similar to JOINs in RDBMSs.
Of the above, simplest and more straight forward solution in your case, seems to me the first one. Choosing it, you will also avoid performance losses in updates for embedded documents
I used to group on mongoDB via PHP to get the max date of my items.
As i have too many items (more than 10 000), i read i must use MapReduce.
Here's my past group function :
$keys = array('ItemDate'=> true);
$initial = array('myLastDate' => 0);
$reduce = "function(obj, prev) {
if (myLastDate < obj.ItemDate) {
myLastDate = ItemDate;
}
}";
$conds = array( 'ItemID' => (int) $id );
$results = $db->items->group($keys, $initial, $reduce,
array('condition'=> $conds ) );
I've tried something but seems not to work ...
$map = new MongoCode("function() {
emit(this.ItemID,this.ItemDate);
}");
$reduce = new MongoCode("function(obj, prev) {
if(prev.myLastDate < obj.ItemDate) {
prev.myLastDate = obj.ItemDate;
}
}");
$items = $db->command(array(
"mapreduce" => "items",
"map" => $map,
"reduce" => $reduce,
"query" => array("ItemID" => $id);
$results = $db->selectCollection($items['result'])->find();
Can you please help ?
Solution
You don't need to use map/reduce for that. Provided your date field contains an ISODate, a simple query does the trick:
db.yourColl.find({},{_id:0,ItemDate:1}).sort({ItemDate:-1}).limit(1)
In order to have this query done efficiently, you need to set an index on ItemDate
db.yourColl.createIndex({ItemDate:-1})
Explanation
The query
Let us dissect the query. db.yourColl...
.find({} The default query
,{_id:0,ItemDate:1} We want only ItemDate to be returned. This is called a projection.
.sort({ItemDate:-1}) The documents returned should be sorted in descending order on ItemDate, making the document with the newest date the first to be returned.
.limit(1) And since we only want the newest, we limit the result set to it.
The index
We create the index in descending order, since this is the way you are going to use it. However, if you need to change the default query to something else, the index you create should include all fields you inspect in the query, in the exact order.
I'm trying to write a bit of code to inventory our OpenStack deployment, and I've run into an issue where serverList() only ever returns 100 results instead of the 600+ I'm expecting. I've reviewed the documentation and a bit of the source, and as far as I can tell there's no reason that this should be happening as the PaginatedIterator should be doing its pagination transparently.
There are no errors or warning either generated in my code, or logged on my controller [that I can find]. I am using php-opencloud v1.12 via composer.
use OpenCloud\OpenStack;
$client = new OpenStack('http://1.2.3.4:5000/v2.0/', array(
'username' => 'admin',
'password' => 'hunter2',
'tenantName'=> 'admin',
));
$service = $client->computeService('nova', 'RegionOne');
$stmt = $dbh->prepare('INSERT INTO servers VALUES (?,?)');
/* foreach($service->serverList() as $server) {
$stmt->execute([$server->id, $server->name]);
} // neither method works */
$list = $service->serverList();
while( $list->valid() ) {
$server = $list->current();
$stmt->execute([$server->id, $server->name]);
$list->next();
}
echo "\n";
var_dump($dbh->query('SELECT * FROM servers')->fetchAll(PDO::FETCH_ASSOC));
The default limit for pagination is 100. It is possible to override this with a higher limit like so:
$list = $service->serverList(null, array('limit' => 700));
I am using this https://api.twitter.com/1.1/followers/ids.json?cursor=-1&screen_name=sitestreams&count=5000 to list the Twitter followers list, But I got only list of 200 followers. How to increase the list of Twitter followers using the new API 1.1?
You must first setup you application
<?php
$consumerKey = 'Consumer-Key';
$consumerSecret = 'Consumer-Secret';
$oAuthToken = 'OAuthToken';
$oAuthSecret = 'OAuth Secret';
# API OAuth
require_once('twitteroauth.php');
$tweet = new TwitterOAuth($consumerKey, $consumerSecret, $oAuthToken, $oAuthSecret);
You can download the twitteroauth.php from here: https://github.com/elpeter/pv-auto-tweets/blob/master/twitteroauth.php
Then
You can retrieve your followers like this:
$tweet->get('followers/ids', array('screen_name' => 'YOUR-SCREEN-NAME-USER'));
If you want to retrieve the next group of 5000 followers you must add the cursor value from first call.
$tweet->get('followers/ids', array('screen_name' => 'YOUR-SCREEN-NAME-USER', 'cursor' => 9999999999));
You can read about: Using cursors to navigate collections in this link: https://dev.twitter.com/docs/misc/cursoring
You can't fetch more than 200 at once... It was clearly stated on the documentation where count:
The number of users to return per page, up to a maximum of 200. Defaults to 20.
you can somehow make it via pagination using
"cursor=-1" #means page 1, "If no cursor is provided, a value of -1 will be assumed, which is the first “page."
Here's how I run/update full list of follower ids on my platform. I'd avoid using sleep() like #aphoe script. Really bad to keep a connection open that long - and what happens if your user has 1MILL followers? You going to keep that connection open for a week? lol If you must, run cron or save to redis/memcache. Rinse and repeat until you get all the followers.
Note, my code below is a class that's run through a cron command every minute. I'm using Laravel 5.1. So you can probably ignore a lot of this code, as it's unique to my platform. Such as the TwitterOAuth (which gets all oAuths I have on db), TwitterFollowerList is another table and I check if an entry already exists, TwitterFollowersDaily is another table where I store/update total amount for the day for the user, and TwitterApi is the Abraham\TwitterOAuth package. You can use whatever library though.
This might give you a good sense of what you might do the same or even figure out a better way. I won't explain all the code, as there's a lot happening, but you should be able to guide through it. Let me know if you have any questions.
/**
* Update follower list for each oAuth
*
* #return response
*/
public function updateFollowers()
{
TwitterOAuth::chunk(200, function ($oauths)
{
foreach ($oauths as $oauth)
{
$page_id = $oauth->page_id;
$follower_list = TwitterFollowerList::where('page_id', $page_id)->first();
if (!$follower_list || $follower_list->updated_at < Carbon::now()->subMinutes(15))
{
$next_cursor = isset($follower_list->next_cursor) ? $follower_list->next_cursor : -1;
$ids = isset($follower_list->follower_ids) ? $follower_list->follower_ids : [];
$twitter = new TwitterApi($oauth->oauth_token, $oauth->oauth_token_secret);
$results = $twitter->get("followers/ids", ["user_id" => $page_id, "cursor" => $next_cursor]);
if (isset($results->errors)) continue;
$ids = $results->ids;
if ($results->next_cursor !== 0)
{
$ticks = 0;
do
{
if ($ticks === 13)
{
$ticks = 0;
break;
}
$ticks++;
$results = $twitter->get("followers/ids", ["user_id" => $page_id, "cursor" => $results->next_cursor]);
if (!$results) break;
$more_ids = $results->ids;
$ids = array_merge($ids, $more_ids);
}
while ($results->next_cursor > 0);
}
$stats = [
'page_id' => $page_id,
'follower_count' => count($ids),
'follower_ids' => $ids,
'next_cursor' => ($results->next_cursor > 0) ? $results->next_cursor : null,
'updated_at' => Carbon::now()
];
TwitterFollowerList::updateOrCreate(['page_id' => $page_id], $stats);
TwitterFollowersDaily::updateOrCreate([
'page_id' => $page_id,
'date' => Carbon::now()->toDateString()
],
[
'page_id' => $page_id,
'date' => Carbon::now()->toDateString(),
'follower_count' => count($ids),
]
);
continue;
}
}
});
}
I have lots of records in a collection. They are indexed with a binary uuid :
db.users.find().limit(1);
[{ "_id": ..., "guid": BinData(2,"EAAAANR56IodpE3xhYLtfugc7SY="), otherdata }]
If I query from the CLI, I can retrieve the records:
db.users.find({ "guid": BinData(2,"EAAAANR56IodpE3xhYLtfugc7SY=") });
[{ "_id": ..., "guid": BinData(2,"EAAAANR56IodpE3xhYLtfugc7SY="), otherdata }]
If I want to do the same thing from PHP, the query returns nothing:
$client->db->setProfilingLevel(2);
$res = $client->db->users->find(array('guid' => new MongoBinData($bin_data, 2)));
$res->next(); // perform the query
echo $res->count(); // display 0
Of course I have tested the $bin_data variable hold the right value.
If I look at mongo's logs,
Thu Jan 10 18:05:06 [conn1] query db.users query: { guid: BinData } ntoreturn:0 ntoskip:0 nscanned:0 keyUpdates:0 locks(micros) r:14807 nreturned:0 reslen:20 14ms
the scanned value is 0 ! This means it does not even scan the collection for results ?
Any clue ?
Edit I have set the profiling level to 2, it does change anything. I can see authentication in the logs but still no query.
Edit2 Added log line.
I must admit I am confused myself. I ran:
$mongo = new Mongo();
$db = $mongo->mydb;
$col = $db->gjgjgj->insert(
array( "guid" => new MongoBinData("EAAAANR56IodpE3xhYLtfugc7SY=", 2) )
);
var_dump($col);
$cur = $db->gjgjgj->find(
array( "guid" => new MongoBinData("EAAAANR56IodpE3xhYLtfugc7SY=", 2) )
);
var_dump(iterator_to_array($cur));
And my output was:
boolean true
array
'50ef0bc96803fa0b04000000' =>
array
'_id' =>
object(MongoId)[7]
public '$id' => string '50ef0bc96803fa0b04000000' (length=24)
'guid' =>
object(MongoBinData)[9]
public 'bin' => string 'EAAAANR56IodpE3xhYLtfugc7SY=' (length=28)
public 'type' => int 2
Can you tell us what PHP driver version this is?
Can you also post your entire script that can reproduce this?
Edit
Of course running count() after running a next() on the cursor works for me too.
Mongo doesn't log all queries by default. Only slow queries (usually taking more than 100 milliseconds) are logged. Also check out docs on profiling.
You can use mongoDB profiling from php to see what is happining there
http://php.net/manual/en/mongodb.setprofilinglevel.php
$this->command(array('profile' => 2))