I have severals doc and each doc contains nested fields and I want to order by position depending on thematic id
Document1 :
{
...
"thematics":[
{
"id": 1,
"position": 100
},
{
"id": 2,
"position": 1
}
]
}
Document2:
{
...
"thematics":[
{
"id": 2,
"position": 3
}
]
}
Document3:
{
...
"thematics":[
{
"id": 1,
"position": 40
}
]
}
For example, I would like to get only documents with thematics which contains id = 2
So I did something like that
$filter = BoolQuery();
...
$filter->addMust(new Query\Term(["thematics.id" => 2]));
And then when I want to apply sort method on position where thematic id = 2 and not something else.
I tried something like that :
case 'atp': // asc thematic position
$sort = [
"_score",
[
"thematics.position" => [
"order" => "asc",
"missing" => 0,
],
],
];
break;
...
$this->setSort($sort); // call parent method setSort(array()) of elastica-ruflin
Response example :
First case expected :
If I want to display all documents from thematic 1 the order must be :
Document3 then Document1.
Second case expected :
If I want to display all documents from thematic 2 the order must be :
Document1 then Document2.
But for now what I've got is :
- First case : Document3, Document1
- Second case : Document2, Document1
I'm guessing that it takes the first thematic's position of document1 to sort in both cases.
Edit :
I tried to change de mapping with a nested type
thematics:
type: nested
properties:
label: { type: string, index: not_analyzed }
slug: { index: not_analyzed }
name: { type: string, index: not_analyzed }
position: { type: integer }
id: { type: integer }
And the query but still not working
{
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"match_all": {}
}
]
}
}
],
"filter": [
{
"bool": {
"must": [
{
"term": {
"is_searchable": true
}
},
{
"nested": {
"path": "thematics",
"query": {
"term": {
"thematics.id": {
"value": 2
}
}
}
}
},
{
"exists": {
"field": "valuation"
}
},
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "valuation.translations.fr.title"
}
}
]
}
},
{
"bool": {
"must": [
{
"exists": {
"field": "valuation.translations.en.title"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"commercial_subcategory.category.id": 33
}
}
]
}
}
]
}
}
]
}
}
]
}
},
"boost_mode": "multiply",
"functions": [
{
"field_value_factor": {
"field": "booster",
"modifier": "square"
}
}
]
}
},
"sort": [
"_score",
{
"thematics.position": {
"order": "asc",
"missing": 0,
"mode": "min",
"nested_filter": {
"term": {
"thematics.id": {
"value": 2
}
}
}
}
}
]
}
Edit2: I get around the problem.
I changed my mapping so each document looks like that :
{
...
"thematics":[
"1": {
"id": 1,
"position": 100
},
"2": {
"id": 2,
"position": 1
}
]
}
And then I apply a Bool Query must / Exists Filter on "thematics.".$thematicId
And finally, my sort method looks like this :
case 'atp': // asc thematic position
$sort = [
"_score",
[
"thematics." . $thematicId . ".position" => [
"order" => "asc",
"missing" => 0,
],
],
];
break;
...
$this->setSort($sort); // call parent method setSort(array()) of elastica-ruflin
Related
This was working on 7.6.2 but since upgrading to 7.7 it has stopped working and do not know why?
I am doing a query with a nested or with a nested must so it has to be 5 5 5 or 6 6 6 on three columns.
I am using the laravel scout driver for elastic search babenkoivan/scout-elasticsearch-driver
Thanks :)!
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"bool": {
"must": [
[
{
"term": {
"section": "205"
}
},
{
"term": {
"profile": "40"
}
},
{
"term": {
"rim_size": "17"
}
}
]
]
}
}
]
}
},
{
"bool": {
"should": [
[
{
"term": {
"supplier_id": 3
}
}
]
]
}
}
]
}
},
Error:
{
"error":{
"root_cause":[
{
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [must]"
}
],
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [must]",
"caused_by":{
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [should]",
"caused_by":{
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [must]",
"caused_by":{
"type":"illegal_state_exception",
"reason":"expected value but got [START_ARRAY]"
}
}
}
},
"status":400
}
You have two nested arrays in your bool/must, you need to remove one:
"must": [
>>> [
{
"term": {
"section": "205"
}
},
{
"term": {
"profile": "40"
}
},
{
"term": {
"rim_size": "17"
}
}
>>> ]
]
It should look like this instead:
"must": [
{
"term": {
"section": "205"
}
},
{
"term": {
"profile": "40"
}
},
{
"term": {
"rim_size": "17"
}
}
]
I am saving id's in the database as comma separated and indexing the same to ElasticSearch. Now I need to retrieve if the user_id matches with the value.
For example it it saving like this in the indexing for the column user_ids (database type is varchar(500) in elasticsearch it is text)
8938,8936,8937
$userId = 8936; // For example expecting to return that row
$whereCondition = [];
$whereCondition[] = [
"query_string" => [
"query"=> $userId,
"default_field" => "user_ids",
"default_operator" => "OR"
]
];
$searchParams = [
'query' => [
'bool' => [
'must' => [
$whereCondition
],
'must_not' => [
['exists' => ['field' => 'deleted_at']]
]
]
],
"size" => 10000
];
User::search($searchParams);
Json Query
{
"query": {
"bool": {
"must": [
[{
"query_string": {
"query": 8936,
"default_field": "user_ids",
"default_operator": "OR"
}
}]
],
"must_not": [
[{
"exists": {
"field": "deleted_at"
}
}]
]
}
},
"size": 10000
}
Mapping details
{
"user_details_index": {
"aliases": {},
"mappings": {
"test_type": {
"properties": {
"created_at": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
"deleted_at": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
"updated_at": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
},
"user_ids": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"creation_date": "1546404165500",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "krpph26NTv2ykt6xE05klQ",
"version": {
"created": "6020299"
},
"provided_name": "user_details_index"
}
}
}
}
I am trying with above logic, but not unable to retrieve. Can someone help on this.
Since the field user_ids is of type text any no analyzer is specified for it by default it will use standard analyzer which won't break 8938,8936,8937 into terms 8938, 8936 and 8937 and hence the id can't match.
To solve this I would suggest you to store array of ids to user_ids field instead of csv. So while indexing you json input should look as below:
{
...
"user_ids": [
8938,
8936,
8937
]
...
}
Since user ids are integer values following changes should be done in mapping:
{
"user_ids": {
"type": "integer"
}
}
The query will be now as follow:
{
"query": {
"bool": {
"filter": [
[
{
"terms": {
"userIds": [
8936
]
}
}
]
],
"must_not": [
[
{
"exists": {
"field": "deleted_at"
}
}
]
]
}
},
"size": 10000
}
can you help me please? I have a ecommerce website with 1000+ products. Each product has a bunch of options like "color", "size", and other specs... but i don't know all the attributes. so i define a document with this mapping:
"mappings" : {
"article" : {
"properties": {
"options": {
"type": "nested",
"include_in_parent":"true",
"properties": {
"id": {"type": "string"},
"name": {"type": "string"},
"values": {"type": "string"}
}
}
}
}
And this is my Query to get the Bucket list:
{
"query": {
"bool": {
"must": [
{
"term": {
"categorie_id": "f52330ce2669dfab884c2d60468b8466"
}
}
],
"must_not": [],
"should": []
}
},
"from": 0,
"size": 1,
"sort": [
{
"sorttype": {
"order": "desc"
}
},
"_score"
],
"aggs": {
"baked_goods": {
"nested": {
"path": "options"
},
"aggs": {
"name": {
"terms": {
"field": "id"
},
"aggs": {
"name": {
"terms": {
"field": "values"
}
}
}
}
}
}
}
}
I get Documents, but the Result of the Buckets is Empty...
"aggregations": {
"baked_goods": {
"doc_count": 3331,
"name": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [ ]
}
}
}
i want something like:
"color" => "red" (4)
"color" => "blue" (2)
"size" => "X" (11)
..
Can you please help me??
i found a solution.
Mapping:
"options": {
"type": "nested",
"include_in_parent": true,
"properties": {
"name": { "type": "text" , "analyzer": "whitespace", "fielddata": true},
"values": { "type": "text" , "analyzer": "whitespace", "fielddata": true}
}
}
Query:
"aggs": {
"facets": {
"nested": {
"path": "options"
},
"aggs": {
"name": {
"terms": {
"field": "options.name"
},
"aggs": {
"name": {
"terms": {
"field": "options.values"
}
}
}
}
}
} }
All would like to use the filtered query where results should contain data from the "query_string" and also from the "term - filter" applied.
GET blog/_search
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "description" ],
"query": "a" // or just ""
}
},
"filter": {
"terms": {
"topic_id": [
10
]
}
}
}
}
}
The expected result is:
all blog records having letter "a" or "" in it with topic_id is 10.
also rest of the records where topic_id is 10 even if the description is blank/empty.
So final result should be - the matching records with higher score and should come at the top, then the records just matching the "topic_id" from the filter.
One way to achieve this is use muti_fields mapping for description field. One of the fields in multi-field should be non-analyzed.
Once the data has been reindexed you can use a simple bool query to achieve what you want :
Example
Create Index:
put test
{
"mappings": {
"data" : {
"properties": {
"description" : {
"type": "string",
"fields": {
"raw" : {"type": "string","index": "not_analyzed"}
}
}
}
}
}
}
Index Data:
put test/data/1
{
"description" : "a",
"test_id" : 10
}
put test/data/2
{
"description" : "",
"test_id" : 10
}
put test/data/3
{
"description" : "hello",
"test_id" : 10
}
put test/data/4
{
"description": "a",
"test_id" : 20
}
Query:
post test/data/_search
{
"query": {
"filtered": {
"query": {
"bool": {
"disable_coord": "true",
"should": [
{
"query_string": {
"fields": [
"description"
],
"query": "a"
}
},
{
"constant_score": {
"filter": {
"term": {
"description.raw": ""
}
},
"boost": 0.2
}
},
{
"constant_score": {
"filter": {
"exists": {
"field": "description"
}
},
"boost": 0.1
}
}
]
}
},
"filter": {
"terms": {
"test_id": [
10
]
}
}
}
}
}
Results :
"hits": [
{
"_index": "test",
"_type": "data",
"_id": "1",
"_score": 0.5113713,
"_source": {
"description": "a",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "2",
"_score": 0.29277003,
"_source": {
"description": "",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "3",
"_score": 0.097590014,
"_source": {
"description": "hello",
"test_id": 10
}
}
]
Query Empty string:
{
"query": {
"filtered": {
"query": {
"bool": {
"disable_coord": "true",
"should": [
{
"query_string": {
"fields": [
"description"
],
"query": ""
}
},
{
"constant_score": {
"filter": {
"term": {
"description.raw": ""
}
},
"boost": 0.2
}
},
{
"constant_score": {
"filter": {
"exists": {
"field": "description"
}
},
"boost": 0.1
}
}
]
}
},
"filter": {
"terms": {
"test_id": [
10
]
}
}
}
}
}
Result :
"hits": [
{
"_index": "test",
"_type": "data",
"_id": "2",
"_score": 1.3416407,
"_source": {
"description": "",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "1",
"_score": 0.44721356,
"_source": {
"description": "a",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "3",
"_score": 0.44721356,
"_source": {
"description": "hello",
"test_id": 10
}
}
]
Have you considered using wildcard query? Check this query it will work fine for you.
all blog records having letter "a" in it with topic_id is 10.
{
"filter": {
"and": [
{
"in": {
"topic_id": [
"10"
]
}
},
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"query": {
"wildcard": {
"description": {
"value": "*a*"
}
}
}
}
]
}
}
}
}
}
]
}
}
Also rest of the records where topic_id is 10 even if the description is blank/empty. This will return all the other records that doesn't match the wildcard.
{
"filter": {
"and": [
{
"in": {
"topic_id": [
"10"
]
}
},
{
"not": {
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"query": {
"wildcard": {
"description": {
"value": "*a*"
}
}
}
}
]
}
}
}
}
}
}
]
}
}
To find only the empty " " description fields with topic_id 10. try this,
{
"filter": {
"and": [
{
"in": {
"topic_id": [
"10"
]
}
},
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "_source.description.length() == 0"
}
}
}
}
}
]
}
}
For ES 2.x
Using a bool query should do the trick.
Here's the query I will use:
GET blog/_search
{
"query": {
"bool": {
"should": [
{
"query_string": {
"fields": [ "description" ],
"query": "a"
}
}
],
"must": [
{
"terms": {
"topic_id": [
10
]
}
}
]
}
}
}
Here, the should clause of the bool query will tell Elassticsearch that document matching the query_string should be returned. In the query_string consider using wildcards if you want to match any document containing a.
For example "query_string": { "query": "*a*" }
The must clause in the other hand will tell that, for considering the document a valid match, it must contain 10 in the topic_id field. Wether the should clause could or could not match.
Bool filter
I hope this could help you.
I need to sort result in next order:
Users, that I following
Users, that follows me
All other users
I have users which look like this:
{
"username": "admin"
"followers": [
{
"id": 2,
"username": "kiehn.nicola2"
},
{
"id": 3,
"username": "adaline253"
},
{
"id": 4,
"username": "skuhic4"
}
],
"following": [
{
"id": 2,
"username": "kiehn.nicola2"
},
{
"id": 3,
"username": "adaline253"
},
{
"id": 4,
"username": "skuhic4"
},
{
"id": 5,
"username": "heaney.garth5"
}
]
}
Is it possible?
Of course, I know current user id and username.
I write this query, but it doesn't work (for example, user id is 1):
{
"query": {
"bool": {
"must": [
{
"wildcard": {
"username": {
"value": "*a*",
"boost": 1
}
}
}
]
}
},
"sort": [
{
"following.username": {
"order": "asc",
"nested_path": "following",
"nested_filter": {
"term": {
"following.id": 1
}
}
},
"followers.username": {
"order": "asc",
"nested_path": "followers",
"nested_filter": {
"term": {
"followers.id": 1
}
}
}
}
],
"size": 40
}
I would do this by boosting; boost the hits that have the searchers id in their followers by an amount, then boost by a lower value the hits that have the searcher in their 'following' field:
NOTE: the searcher's id is 55 in this example
"query": {
"bool": {
"should": [
{
"nested": {
"path": "followers",
"query": {
"term" : { "followers.id": { "value": 55, "boost": 3.0 } }
}
}
},
{
"nested": {
"path": "following",
"query": {
"term" : { "following.id": { "value": 55, "boost": 2.0 } }
}
}
},
{
"match_all": { "boost": 1.0 }
}
]
}
}
If the searcher is in the hit's followers field, then the searcher is following that hit and so the boost is highest, etc...
You said you wanted all other users, hence the "match_all: {} query at the end.