Can somebody tell me please if is possible to force Mongodb to use specific index from PHP script? I have something like this:
$mongoPipeline = [
[
'$match' => [
'global_campaign_id' => ['$in' => $campaigns_ids]
]
],
[
'$group' => [
'_id' => [
'global_campaign_id' => '$global_campaign_id',
'device_id' => '$device_id',
'partner_id' => '$partner_id',
],
'date_last' => ['$max' => '$date_created'],
'partner_id' => ['$first' => '$server'],
'campaign_id' => ['$first' => '$global_campaign_id'],
'device_id' => ['$first' => '$device_id']
]
]
];
$options = [
'allowDiskUse' => true,
'maxTimeMS' => 1000 * 60 * 1,
'explain' => true,
];
$cursor = $this->mongoDb->{$collection}->aggregate($pipeline, $options);
Is it possible to add index to options object? If I use explain it seems it does not use an index prepared for this query. It is compoud index:
global_campaign_id: 1, device_id: 1, partner_id: 1, date_created: 1
Related
I need to iterate over large result set of Mongodb collection in PHP. But I am getting timeout error. How to use it like a cursor till result is not finished and parse returned documents in chunks? The pipeline looks like
$mongoPipeline = [
[
'$match' => [
'global_campaign_id' => ['$in' => $campaigns_ids]
]
],
[
'$group' => [
'_id' => [
'global_campaign_id' => '$global_campaign_id',
'device_id' => '$device_id',
'partner_id' => '$partner_id',
],
'date_last' => ['$max' => '$date_created'],
'partner_id' => ['$first' => '$server'],
'campaign_id' => ['$first' => '$global_campaign_id'],
'device_id' => ['$first' => '$device_id']
]
],
[
'$limit' => 5,
],
];
$options = [
'allowDiskUse' => true,
//'maxTimeMS' => 1000 * 60 * 1,
'useCursor' => true,
//'explain' => true,
];
$result = [];
/** #var Cursor $cursor */
$cursor = $this->mongoDb->{$collection}->aggregate($pipeline, $options);
$cursor->setTypeMap(['root' => 'array', 'document' => 'array', 'array' => 'array']);
foreach ($result as $doc) $result[] = $this->parseDocument($doc);
return $result;
If I use $cursor->toArray() if fall down on timeoutError.
How this script should look like?
I'm struggling to update a document and add new elements (fields) into it's existing array without losing the array elements on update.
This is my code which inserts a new document into the collection if it doesn't already exist (upsert):
$updateResult = $collection->findOneAndUpdate(
[
'recording-id' => $out['recording']->id
],
['$set' => [
'release' => [
'id' => $out['release']->id,
'title' => $out['release']->title,
'date' => $out['release']->date,
'country' => $out['release']->country
],
'artist' => [
'id' => $out['artist']->id,
'name' => $out['artist']->name,
],
'recording' => [
'id' => $out['recording']->id,
'title' => $out['recording']->title,
'score' => $out['recording']->score,
'length' => $out['recording']->length,
'release-count' => count($out['recording']->releases),
],
'release-group' => [
'id' => $out['release-group']['id'],
'title' => $out['release-group']['title'],
'first-release-date'=>$out['release-group']['first-release-date'],
'primary-type' => $out['release-group']['primary-type'],
'musicbrainz' => $out['release-group']['musicbrainz'],
'url-rels' => $out['release-group']['url-rels'],
'coverart' => $out['release-group']['coverart']
],
'execution' => [
'firstfind' => $out['execution']->time
]
]
],
['upsert' => true,
'projection' =>
[
'_id' => 0,
'release' => 1,
'artist' => 1,
'recording' => 1,
'release-group' => 1,
'execution' => 1
],
'returnDocument' => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER,
]
);
So now I have an existing document in a collection:
{
"_id" : ObjectId("5d1a6aaf5ecc8001ee858f6c"),
"recording-id" : "d0d439f9-5324-4728-8706-2da39adb89c5",
"artist" : {
"id" : "9d97b077-b28d-4ba8-a3d9-c71926e3b2b6",
"name" : "Gordon Lightfoot"
},
"recording" : {
"id" : "d0d439f9-5324-4728-8706-2da39adb89c5",
"title" : "Sundown",
"score" : 100,
"length" : 184000,
"release-count" : 2
},
"release" : {
"id" : "0c008d76-2bc9-44a3-854b-0a08cde89337",
"title" : "All Live",
"date" : "2012-04-24",
"country" : "CA"
},
"release-group" : {
"id" : "0a5d5f33-8e9d-4fa4-b622-a95e4218a3c4",
"title" : "All Live",
"first-release-date" : "2012-04-24",
"primary-type" : "Album",
"musicbrainz" : "https://musicbrainz.org/release-group/0a5d5f33-8e9d-4fa4-b622-a95e4218a3c4",
"url-rels" : "https://musicbrainz.org/ws/2/release-group/0a5d5f33-8e9d-4fa4-b622-a95e4218a3c4?inc=url-rels&fmt=json",
"coverart" : null
}
}
Now, I would like to update this document, and add new fields into the arrays. The new fields are to be added to certain fields.
Here is the code doing that:
$collection = (new MongoDB\Client)->stream->musicbrainz;
$updateResult = $collection->updateOne(
[
'recording-id' => $out['recording']['id']
],
['$addToSet' => [
'artist' => [
'wikiQiD' => $out['artist']['qid'],
'wiki-extract' => $out['artist']['wiki-extract'],
'wiki-pageid' => $out['artist']['pageid'],
],
'release-group' => [
'wikiQiD' => $out['release-group']['qid'],
'wiki-extract' => $out['release-group']['wiki-extract']
]
]
],
[
'upsert' => true,
'returnDocument' => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER,
]
);
I've noticed there's "$addToSet" and "$push" commands, and could use assistance with what the difference is between these two commands.
If the field is absent in the document to update, $push adds the array
field with the value as its element.
The $addToSet operator adds a value to an array unless the value is
already present, in which case $addToSet does nothing to that array.
I did some googling, and reading of the MongoDB/Client UpdateOne function, but can't seem to find a way to append these fields to the existing arrays.
The error I'm getting is:
Fatal error: Uncaught MongoDB\Driver\Exception\BulkWriteException: The field 'artist' must be an array but is of type object in document {_id: ObjectId('5d1a6aaf5ecc8001ee858f6c')} in ...
I know the following:
It could be my document, as it's not a proper array that Fatal error is complaining about.
It could be my `findOneAndUpdate' formatting, and I'm not doing that correctly.
It could be both and I have it all wrong from the very start.
Any insight or constructive criticism is appreciated, just refrain from flames, pls.
Here's the working code I finally use to get it do what I want it to.
It's a simple matter of just setting your field names and values and mongo will update the found record replacing it with the array sent to it. No need to push or anything.
function firstFindMongoUpdate($out) {
$collection = (new MongoDB\Client)->stream->musicbrainz;
$updateResult = $collection->findOneAndUpdate(
[
'recording-id' => $out['recording']->id
],
['$set' => [
'query' => $out['query'],
'release' => [
'id' => $out['release']->id,
'title' => $out['release']->title,
'date' => $out['release']->date,
'country' => $out['release']->country,
'label' => $out['release']->label,
],
'artist' => [
'id' => $out['artist']->id,
'name' => $out['artist']->name,
'wiki' => $out['artist']->wiki,
],
'recording' => [
'id' => $out['recording']->id,
'title' => $out['recording']->title, // sometimes contains apostophe ie; Bill‘s Love (option ] key)
'score' => $out['recording']->score,
'length' => $out['recording']->length,
'release-count' => count($out['recording']->releases)
],
'release-group' => [
'id' => $out['release-group']['id'],
'title' => $out['release-group']['title'],
'first-release-date'=>$out['release-group']['first-release-date'],
'primary-type' => $out['release-group']['primary-type'],
'musicbrainz' => $out['release-group']['musicbrainz'],
'url-rels' => $out['release-group']['url-rels'],
'coverart' => $out['release-group']['coverart'],
'wiki' => $out['release-group']['wiki']
],
'execution' => [
'artistQuery' => $out['execution']->artistQuery,
'recordingQuery'=> $out['execution']->recordingQuery,
'time' => $out['execution']->time
]
]
],
['upsert' => true,
'projection' =>
[
'_id' => 1,
'query' => 1,
'release' => 1,
'artist' => 1,
'recording' => 1,
'release-group' => 1,
'execution' => 1
],
'returnDocument' => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER,
]
);
I have a data of stores, report_type is every type of data
- type 3 stored how much money that clients buy and how many transaction.
- type 5 stored how many people and average age of them.
I need an array like this:
"report_type3"
(
[_id] => 2441
[amountclient] => 3749078452,
[trans] => 1000
),
(
[_id] => 2442
[amountclient] => 15533754,
[trans] => 900
)
,
"report_type5"
(
[_id] => 2441
[people] => 120,
[avgage] => 23,
),
(
[_id] => 2441
[people] => 120,
[avgage] => 23,
)
My query looks like this, but it's not working
$cursor = $collection->aggregate([
['$match' => [ 'id_station' => ['$in' => $store ], 'report_type' => 3,
'date' => ['$gte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-07-01")*1000), '$lte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-08-05")*1000)]
]
],
['$group' => [
'_id' => [ 'id_station' => '$id_station', 'receiptid' => '$receiptid' ],
'amount' => [ '$sum' => '$amount' ]
]], .... *I group by receiptid because there are many items in one clients.*
[ '$group' => [
'_id' => '$_id.id_station',
'amount' => [ '$sum' => '$amount' ],
'trans' => [ '$sum' => '$trans' ]
]],
[
'$project' => [
'report_type3' => ['_id','$amountclient','$trans']
]
],
['$match' => [ 'id_station' => ['$in' => $store ], 'report_type' => 5,
'date' => ['$gte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-07-01")*1000), '$lte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-08-05")*1000)]
]
],
[ '$group' => [
'_id' => ['$_id.id_station', 'area_type' => '$area_type'],
'people' => [ '$sum' => '$people' ],
'avgage' => [ '$avg' => '$age' ]
]],
[
'$project' => [
'report_type5' => [ '_id', 'people', 'avgage' ]
]
]
]);
Could you please help me on this?
I am trying to find a document using ElasticSearch and update a field in it. The code is:
require_once 'vendor/autoload.php';
$client = Elasticsearch\ClientBuilder::create()->build();
$params = [
'index' => 'myIndex',
'type' => 'myCollection',
'body' => [
'query'=> [
'match'=> [
'accessToken' => $accessToken
]
] ,
'script' => 'ctx._source.deviceId= deviceId',
'params' => [
'deviceId' => $deviceId
],
'upsert' => [
'counter' => 1
]
]
];
The code is giving 500 Internal Error. Any idea what am I missing?
If you want to increase the value of deviceid by 1 (mean if value of deviceid is 4 then u can increase it to 5 by adding 1 to it ),then follow below code
'script' => 'ctx._source.deviceId= deviceId',
replace with
'script' => 'ctx._source.counter += deviceId',
Example:
$params = [
'index' => 'my_index',
'type' => 'my_type',
'id' => 'my_id',
'body' => [
'script' => 'ctx._source.counter += count',
'params' => [
'count' => 4
],
'upsert' => [
'counter' => 1
]
]
];
$response = $client->update($params);
Check i Elasticsearch php
I am not able to highlight my result, which part of my query is wrong?
PHPClient for elasticsearch throws exception on execution.
$query = [
"query" => [
"filtered" => [
"query" => [
"bool" => [
"should" => [
[
'query_string' => [
'fields' => [
'Title.title^4',
'Title.ngrams_front^2',
'Title.ngrams_back'
],
'defaultOperator' => 'or',
'query' => $paramsObj->q
]
],
[
'query_string' => [
'auto_generate_phrase_queries' => 0,
'enable_position_increments' => false,
'fields' => [
'Title.title',
'Address',
'keys'
],
'query' => $paramsObj->q,
'use_dis_max' => false,
'boost' => 2
]
],
[
'fuzzy' => [
'Title.title' => [
'value' => $paramsObj->q,
'boost' => 1,
'min_similarity' => 0.5,
'max_expansions' => 20,
'prefix_length' => 0
]
]
]
]
]
],
"filter" => $filters
]
],
"highlight" => [
"fields" => [
'Title.title' => [ "pre_tags" => "<em>", "post_tags" => "</em>" ]
]
]
];
First i tried highlighting at filtered level, then i googled and found out i need to do at query level at top of filtered level, so i did but still it throws exception.
Fatal error: Uncaught exception 'Guzzle\Http\Exception\ClientErrorResponseException'
If at all anyone can help, kindly help.
Try something like this:
$query = array(
'query' => array(
'bool' => array(
'should' => array(
'fuzzy' => array(
'name' => array(
'value' => $serachstring,
'boost' => 1,
'min_similarity' => 0.5,
'max_expansions' => 20,
'prefix_length' => 0
),
),
// ...
)
),
),
'highlight' => array(
"pre_tags" => "<em>",
"post_tags" => "</em>",
'fields' => array(
'name' => (object) array()
)
),
);