I'm struggling to update a document and add new elements (fields) into it's existing array without losing the array elements on update.
This is my code which inserts a new document into the collection if it doesn't already exist (upsert):
$updateResult = $collection->findOneAndUpdate(
[
'recording-id' => $out['recording']->id
],
['$set' => [
'release' => [
'id' => $out['release']->id,
'title' => $out['release']->title,
'date' => $out['release']->date,
'country' => $out['release']->country
],
'artist' => [
'id' => $out['artist']->id,
'name' => $out['artist']->name,
],
'recording' => [
'id' => $out['recording']->id,
'title' => $out['recording']->title,
'score' => $out['recording']->score,
'length' => $out['recording']->length,
'release-count' => count($out['recording']->releases),
],
'release-group' => [
'id' => $out['release-group']['id'],
'title' => $out['release-group']['title'],
'first-release-date'=>$out['release-group']['first-release-date'],
'primary-type' => $out['release-group']['primary-type'],
'musicbrainz' => $out['release-group']['musicbrainz'],
'url-rels' => $out['release-group']['url-rels'],
'coverart' => $out['release-group']['coverart']
],
'execution' => [
'firstfind' => $out['execution']->time
]
]
],
['upsert' => true,
'projection' =>
[
'_id' => 0,
'release' => 1,
'artist' => 1,
'recording' => 1,
'release-group' => 1,
'execution' => 1
],
'returnDocument' => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER,
]
);
So now I have an existing document in a collection:
{
"_id" : ObjectId("5d1a6aaf5ecc8001ee858f6c"),
"recording-id" : "d0d439f9-5324-4728-8706-2da39adb89c5",
"artist" : {
"id" : "9d97b077-b28d-4ba8-a3d9-c71926e3b2b6",
"name" : "Gordon Lightfoot"
},
"recording" : {
"id" : "d0d439f9-5324-4728-8706-2da39adb89c5",
"title" : "Sundown",
"score" : 100,
"length" : 184000,
"release-count" : 2
},
"release" : {
"id" : "0c008d76-2bc9-44a3-854b-0a08cde89337",
"title" : "All Live",
"date" : "2012-04-24",
"country" : "CA"
},
"release-group" : {
"id" : "0a5d5f33-8e9d-4fa4-b622-a95e4218a3c4",
"title" : "All Live",
"first-release-date" : "2012-04-24",
"primary-type" : "Album",
"musicbrainz" : "https://musicbrainz.org/release-group/0a5d5f33-8e9d-4fa4-b622-a95e4218a3c4",
"url-rels" : "https://musicbrainz.org/ws/2/release-group/0a5d5f33-8e9d-4fa4-b622-a95e4218a3c4?inc=url-rels&fmt=json",
"coverart" : null
}
}
Now, I would like to update this document, and add new fields into the arrays. The new fields are to be added to certain fields.
Here is the code doing that:
$collection = (new MongoDB\Client)->stream->musicbrainz;
$updateResult = $collection->updateOne(
[
'recording-id' => $out['recording']['id']
],
['$addToSet' => [
'artist' => [
'wikiQiD' => $out['artist']['qid'],
'wiki-extract' => $out['artist']['wiki-extract'],
'wiki-pageid' => $out['artist']['pageid'],
],
'release-group' => [
'wikiQiD' => $out['release-group']['qid'],
'wiki-extract' => $out['release-group']['wiki-extract']
]
]
],
[
'upsert' => true,
'returnDocument' => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER,
]
);
I've noticed there's "$addToSet" and "$push" commands, and could use assistance with what the difference is between these two commands.
If the field is absent in the document to update, $push adds the array
field with the value as its element.
The $addToSet operator adds a value to an array unless the value is
already present, in which case $addToSet does nothing to that array.
I did some googling, and reading of the MongoDB/Client UpdateOne function, but can't seem to find a way to append these fields to the existing arrays.
The error I'm getting is:
Fatal error: Uncaught MongoDB\Driver\Exception\BulkWriteException: The field 'artist' must be an array but is of type object in document {_id: ObjectId('5d1a6aaf5ecc8001ee858f6c')} in ...
I know the following:
It could be my document, as it's not a proper array that Fatal error is complaining about.
It could be my `findOneAndUpdate' formatting, and I'm not doing that correctly.
It could be both and I have it all wrong from the very start.
Any insight or constructive criticism is appreciated, just refrain from flames, pls.
Here's the working code I finally use to get it do what I want it to.
It's a simple matter of just setting your field names and values and mongo will update the found record replacing it with the array sent to it. No need to push or anything.
function firstFindMongoUpdate($out) {
$collection = (new MongoDB\Client)->stream->musicbrainz;
$updateResult = $collection->findOneAndUpdate(
[
'recording-id' => $out['recording']->id
],
['$set' => [
'query' => $out['query'],
'release' => [
'id' => $out['release']->id,
'title' => $out['release']->title,
'date' => $out['release']->date,
'country' => $out['release']->country,
'label' => $out['release']->label,
],
'artist' => [
'id' => $out['artist']->id,
'name' => $out['artist']->name,
'wiki' => $out['artist']->wiki,
],
'recording' => [
'id' => $out['recording']->id,
'title' => $out['recording']->title, // sometimes contains apostophe ie; Bill‘s Love (option ] key)
'score' => $out['recording']->score,
'length' => $out['recording']->length,
'release-count' => count($out['recording']->releases)
],
'release-group' => [
'id' => $out['release-group']['id'],
'title' => $out['release-group']['title'],
'first-release-date'=>$out['release-group']['first-release-date'],
'primary-type' => $out['release-group']['primary-type'],
'musicbrainz' => $out['release-group']['musicbrainz'],
'url-rels' => $out['release-group']['url-rels'],
'coverart' => $out['release-group']['coverart'],
'wiki' => $out['release-group']['wiki']
],
'execution' => [
'artistQuery' => $out['execution']->artistQuery,
'recordingQuery'=> $out['execution']->recordingQuery,
'time' => $out['execution']->time
]
]
],
['upsert' => true,
'projection' =>
[
'_id' => 1,
'query' => 1,
'release' => 1,
'artist' => 1,
'recording' => 1,
'release-group' => 1,
'execution' => 1
],
'returnDocument' => MongoDB\Operation\FindOneAndUpdate::RETURN_DOCUMENT_AFTER,
]
);
I'm triying to get all the logs around a expecific geopoint, and group them by a subfield (context.id) but I'm having problems. I tried the nested aggregation, etc.. but I'm having no luck. I'm using the PHP library so I wrote the query as a php array. All is working until addign the aggregation query.
The exception throw is :
illegal_state_exception: Field data loading is forbidden on [context.id]
$params = [
'index' => 'logstash-*',
'type' => 'INFO',
'body' => [
'query' => [
'bool' => [
"must" => [
["term" => ["tags" => "producer"]],
["term" => ["tags" => "statistics"]],
["term" => ["message" => "view"]],
],
"filter" => [
"geo_distance" => [
"distance" => "10km",
"distance_type" => "plane",
"geoip.location" => [
"lat" => 40.4326058,
"lon" => -3.6996032
]
]
]
]
],
"aggs" => [
"context" => [
"nested" => [
"path" => "context"
],
"aggs" => [
"group_by_id" => [
"terms" => ["field" => "context.id"]
]
]
]
],
]
];
Can someone point me to the right query?
Finally, the problem here was the field. It was a indexed field I need to use the "context.id.raw" field instead.
I'm new to ElasticSearch so sorry if it's noob question. I'm trying to get users who has salary less than some value but I'm getting this error:
query_parsing_exception: No query registered for [salary]
My other queries works fine, only range query is failing, this is my code:
$items = $this->client->search([
'index' => 'offerprofiles',
'type' => 'profile',
'body' => [
'query' => [
'bool' => [
"must" => [
"match" => [
"jobcategories.name" => [
"query" => $query['category']
]
],
"range" => [
"salary" => [
"lt" => 20
]
]
],
"should" => [
"match" => [
"skills.name" => [
"query" => $query['skills']
]
]
],
"minimum_should_match" => 1
]
],
'size' => 50,
]
]);
If I remove range query then everything works fine, also I checked indexed values and salary is there (integer).
Thanks
The query is not a valid DSL. In the particular you are missing a bunch of brackets in the must clause. The must in the bool query should be an array of clauses instead in the above it is an object with key match and range.
Example :
$items = $this->client->search([
'index' => 'offerprofiles',
'type' => 'profile',
'body' => [
'query' => [
'bool' => [
"must" => [
[
"match" => [
"jobcategories.name" => [
"query" => $query['category']
]
]
],
[
"range" => [
"salary" => [
"lt" => 20
]
]
]
],
"should" => [
"match" => [
"skills.name" => [
"query" => $query['skills']
]
]
],
"minimum_should_match" => 1
]
],
'size' => 50,
]
]);
I'm using Elastic search in Laravel.
So I have query that returns correct result:
$results = Es::search(array(
'index' => 'testindex',
'type' => $type,
'body' => [
'query' => [
'filtered' => [
'query' => [
'match' => [
'_all' => '2015-02'
]
],
'filter' => [
'bool' => [
'must' => [
'term' => [
'type' => 11
]
],
]
]
]
],
'size' => 5,
'from' => 0
]
));
What it does is:
searches for $query in all fields AND
field "type" must be 11.
Here is output: http://pastebin.com/icWniix4
Total 9 results which is correct.
But when I add another must term then it returns invalid results
$results = Es::search(array(
'index' => 'testindex',
'type' => $type,
'body' => [
'query' => [
'filtered' => [
'query' => [
'match' => [
'_all' => '2015-02'
]
],
'filter' => [
'bool' => [
'must' => [
'term' => [
'type' => 11
],
'term' => [
'public' => 1
]
],
]
]
]
],
'size' => 5,
'from' => 0
]
));
So this only add term for "public".
What it does is:
searches for $query in all fields AND
field "type" must be 11 AND
field "public" must be 1
So now results are in total 429. It ignores "type" term and returns everything with "public" = 1. But as per docs if I use MUST then it should match all of them. Search result http://pastebin.com/cVcatcyi
So how can I write query that I need? $query + type + public
Official doc's fail to answer my questions.
Any advice?
This is your trouble spot:
'must' => [
'term' => [
'type' => 11
],
'term' => [
'public' => 1
]
]
Here, you're assigning the value of must as an associative array that has only one distinct key - term actually gets assigned twice, so presumably only one of the assignments will "survive" (presumably public survives because it appears last in the definition). The end result is that must ends up pointing at an associative array with only one key-value pair.
What I suspect you have to do is this:
'must' => [
[
'term' => [
'type' => 11
]
],
[
'term' => [
'public' => 1
]
]
]
Now must is actually pointing at an array with two items.
I am using ES for my Laravel app.
What I want to do is a search filtering.
I do a fulltext search on the title field and then check that the price is between 0 - 9999 and that active is set to 1.
But both these queries seems to work fine on my test data. But what is the difference between them? Does the order "query" comes in play any diffrence?
Ignore the syntax, just take a look at the query structure.
First query
'filtered' => [
'query' => [
'match' => ['title' => Input::get('query')]
],
'filter'=> [
'bool' => [
'must' => [
['term' => [ 'active' => 1] ],
[ 'range' => [
'price' => [
'gte' => 1,
'lte' => 99999,
]
]
]
]
]
],
],
Second query
'filtered' => [
'filter' => [
'bool' => [
'must' => [
['term' => [ 'status' => 1] ],
[
'range' => [
'price' => [
'gte' => 1,
'lte' => 99999,
]
]
]
]
]
],
'query' => [
'match' => [
'title' => Input::get('query', '')
]
]
]
Thanks in advance.
It makes no difference at all in which order filter and query are mentioned in a filtered query. What dictates if query or filter is executed first for a document depends on an expert-level optional field called strategy of filtered query. For more information, read this.