Search in elasticsearch using php curl - php

I have make the application with elasticsearch and everything is running perfectly except the search using the php curl; the error is below
[match] query parsed in simplified form, with direct field name, but included more options than just the field name, possibly use its 'options' form, with 'query' element?]
but the same query is running perfectly in the command-line.
When I do some changes, I found this is occurring by the curl POST.
I am using the following code to run the php curl and tried the GET method too
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 200);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, strtoupper($method));
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
$response = curl_exec($ch);
curl_close ($ch);
and the query is
{"query":{"filtered":{"query":{"bool":{"should":[{"bool":{"should":[{"match_phrase":{"name":"india"}}],"boost":16}},{"bool":{"should":[{"match_phrase":{"description":"india"}}],"boost":8}},{"bool":{"should":[{"match":{"name":{"query":"india","analyzer":"standard"}}},{"match":{"description":{"query":"india","analyzer":"standard"}}}],"boost":4}},{"match":{"name.ngram":[{"query":"india","analyzer":"standard"}]}},{"match":{"description":[{"query":"india","analyzer":"standard"}]}},{"match":{"name.ngram":[{"query":"india","analyzer":"standard","fuzziness":"auto"}]}},{"match":{"description":[{"query":"india","analyzer":"standard","fuzziness":"auto"}]}}],"boost":2}},"filter":{"and":[{"terms":{"type":["book"]}},{"range":{"price":{"from":"1","to":"100"}}}]}}},"from":0,"size":20,"filter":{"and":[]},"sort":[{"popularity":{"order":"desc","missing":"_last"}}]}

Problem might be in you are using blank filters in your query dsl.
"filter": {
"and":[]
}
Try without blank filters. Also top level filters are renamed in Elasticsearch 1.0 + version.
Update Query DSL:
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"bool": {
"should": [
{
"match_phrase": {
"name": "india"
}
}
],
"boost": 16
}
},
{
"bool": {
"should": [
{
"match_phrase": {
"description": "india"
}
}
],
"boost": 8
}
},
{
"bool": {
"should": [
{
"match": {
"name": {
"query": "india",
"analyzer": "standard"
}
}
},
{
"match": {
"description": {
"query": "india",
"analyzer": "standard"
}
}
}
],
"boost": 4
}
},
{
"match": {
"name.ngram": [
{
"query": "india",
"analyzer": "standard"
}
]
}
},
{
"match": {
"description": [
{
"query": "india",
"analyzer": "standard"
}
]
}
},
{
"match": {
"name.ngram": [
{
"query": "india",
"analyzer": "standard",
"fuzziness": "auto"
}
]
}
},
{
"match": {
"description": [
{
"query": "india",
"analyzer": "standard",
"fuzziness": "auto"
}
]
}
}
],
"boost": 2
}
},
"filter": {
"and": [
{
"terms": {
"type": [
"book"
]
}
},
{
"range": {
"price": {
"from": "1",
"to": "100"
}
}
}
]
}
}
},
"from": 0,
"size": 20,
"sort": [
{
"popularity": {
"order": "desc",
"missing": "_last"
}
}
]
}

I found the solution; below is the correct json
{"query":{"filtered":{"query":{"bool":{"should":[{"bool":{"should":[{"match_phrase":{"name":"india"}}],"boost":16}},{"bool":{"should":[{"match_phrase":{"description":"india"}}],"boost":8}},{"bool":{"should":[{"match":{"name":{"query":"india","analyzer":"standard"}}},{"match":{"description":{"query":"india","analyzer":"standard"}}}],"boost":4}},{"match":{"name.ngram":{"query":"india","analyzer":"standard"}}},{"match":{"description":{"query":"india","analyzer":"standard"}}},{"match":{"name.ngram":{"query":"india","analyzer":"standard","fuzziness":"auto"}}},{"match":{"description":{"query":"india","analyzer":"standard","fuzziness":"auto"}}}],"boost":2}},"filter":[]}},"from":0,"size":20,"sort":[{"popularity":{"order":"desc","missing":"_last"}}]}
the problem is in the match query phase; I was appending the parameters in match query as nested array.
Now I have removed the nested array and append the options in match query phase as sequential array
Thanks guys for your co-opration

Related

Elasticsearch 7.7 broken query

This was working on 7.6.2 but since upgrading to 7.7 it has stopped working and do not know why?
I am doing a query with a nested or with a nested must so it has to be 5 5 5 or 6 6 6 on three columns.
I am using the laravel scout driver for elastic search babenkoivan/scout-elasticsearch-driver
Thanks :)!
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"bool": {
"must": [
[
{
"term": {
"section": "205"
}
},
{
"term": {
"profile": "40"
}
},
{
"term": {
"rim_size": "17"
}
}
]
]
}
}
]
}
},
{
"bool": {
"should": [
[
{
"term": {
"supplier_id": 3
}
}
]
]
}
}
]
}
},
Error:
{
"error":{
"root_cause":[
{
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [must]"
}
],
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [must]",
"caused_by":{
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [should]",
"caused_by":{
"type":"x_content_parse_exception",
"reason":"[1:106] [bool] failed to parse field [must]",
"caused_by":{
"type":"illegal_state_exception",
"reason":"expected value but got [START_ARRAY]"
}
}
}
},
"status":400
}
You have two nested arrays in your bool/must, you need to remove one:
"must": [
>>> [
{
"term": {
"section": "205"
}
},
{
"term": {
"profile": "40"
}
},
{
"term": {
"rim_size": "17"
}
}
>>> ]
]
It should look like this instead:
"must": [
{
"term": {
"section": "205"
}
},
{
"term": {
"profile": "40"
}
},
{
"term": {
"rim_size": "17"
}
}
]

Elasticsearch distance query exception throwing

I have written the following query to sort by nearest user first, and then by remaining users but when I run this query on kibana, it throws an exception and I don't know what the mistake is?
Query
{
"query": {
"bool": {
"must": [
{
"term": {
"type": "user"
}
}
],
"filter": {
"geo_distance": {
"distance": "1000km",
"location": {
"lat": 24.71532,
"lon": 46.66479
}
}
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 24.71532,
"lon": 46.66479
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}
]
}
}
Exception
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 21,
"col": 5
}
],
"type": "parsing_exception",
"reason": "[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 21,
"col": 5
},
"status": 400
}
I found this code on the elastic search official website. when I run this query without the sort filter this query works, but when I add the sort filter U get an exception.
Kindly guide me to solve this query issue.
The sort section needs to go at the same level as the query section, not inside it:
{
"sort": [
{
"_geo_distance": {
"location": {
"lat": 24.71532,
"lon": 46.66479
},
"order": "asc",
"unit": "km",
"distance_type": "plane"
}
}
],
"query": {
"bool": {
"must": [
{
"term": {
"type": "user"
}
}
],
"filter": {
"geo_distance": {
"distance": "1000km",
"location": {
"lat": 24.71532,
"lon": 46.66479
}
}
}
}
}
}

Elastic Search bucket list for a ECommerce Shop / Neast Attributes

can you help me please? I have a ecommerce website with 1000+ products. Each product has a bunch of options like "color", "size", and other specs... but i don't know all the attributes. so i define a document with this mapping:
"mappings" : {
"article" : {
"properties": {
"options": {
"type": "nested",
"include_in_parent":"true",
"properties": {
"id": {"type": "string"},
"name": {"type": "string"},
"values": {"type": "string"}
}
}
}
}
And this is my Query to get the Bucket list:
{
"query": {
"bool": {
"must": [
{
"term": {
"categorie_id": "f52330ce2669dfab884c2d60468b8466"
}
}
],
"must_not": [],
"should": []
}
},
"from": 0,
"size": 1,
"sort": [
{
"sorttype": {
"order": "desc"
}
},
"_score"
],
"aggs": {
"baked_goods": {
"nested": {
"path": "options"
},
"aggs": {
"name": {
"terms": {
"field": "id"
},
"aggs": {
"name": {
"terms": {
"field": "values"
}
}
}
}
}
}
}
}
I get Documents, but the Result of the Buckets is Empty...
"aggregations": {
"baked_goods": {
"doc_count": 3331,
"name": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [ ]
}
}
}
i want something like:
"color" => "red" (4)
"color" => "blue" (2)
"size" => "X" (11)
..
Can you please help me??
i found a solution.
Mapping:
"options": {
"type": "nested",
"include_in_parent": true,
"properties": {
"name": { "type": "text" , "analyzer": "whitespace", "fielddata": true},
"values": { "type": "text" , "analyzer": "whitespace", "fielddata": true}
}
}
Query:
"aggs": {
"facets": {
"nested": {
"path": "options"
},
"aggs": {
"name": {
"terms": {
"field": "options.name"
},
"aggs": {
"name": {
"terms": {
"field": "options.values"
}
}
}
}
}
} }

PHP Elastic Search Filtered Query String Search

All would like to use the filtered query where results should contain data from the "query_string" and also from the "term - filter" applied.
GET blog/_search
{
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [ "description" ],
"query": "a" // or just ""
}
},
"filter": {
"terms": {
"topic_id": [
10
]
}
}
}
}
}
The expected result is:
all blog records having letter "a" or "" in it with topic_id is 10.
also rest of the records where topic_id is 10 even if the description is blank/empty.
So final result should be - the matching records with higher score and should come at the top, then the records just matching the "topic_id" from the filter.
One way to achieve this is use muti_fields mapping for description field. One of the fields in multi-field should be non-analyzed.
Once the data has been reindexed you can use a simple bool query to achieve what you want :
Example
Create Index:
put test
{
"mappings": {
"data" : {
"properties": {
"description" : {
"type": "string",
"fields": {
"raw" : {"type": "string","index": "not_analyzed"}
}
}
}
}
}
}
Index Data:
put test/data/1
{
"description" : "a",
"test_id" : 10
}
put test/data/2
{
"description" : "",
"test_id" : 10
}
put test/data/3
{
"description" : "hello",
"test_id" : 10
}
put test/data/4
{
"description": "a",
"test_id" : 20
}
Query:
post test/data/_search
{
"query": {
"filtered": {
"query": {
"bool": {
"disable_coord": "true",
"should": [
{
"query_string": {
"fields": [
"description"
],
"query": "a"
}
},
{
"constant_score": {
"filter": {
"term": {
"description.raw": ""
}
},
"boost": 0.2
}
},
{
"constant_score": {
"filter": {
"exists": {
"field": "description"
}
},
"boost": 0.1
}
}
]
}
},
"filter": {
"terms": {
"test_id": [
10
]
}
}
}
}
}
Results :
"hits": [
{
"_index": "test",
"_type": "data",
"_id": "1",
"_score": 0.5113713,
"_source": {
"description": "a",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "2",
"_score": 0.29277003,
"_source": {
"description": "",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "3",
"_score": 0.097590014,
"_source": {
"description": "hello",
"test_id": 10
}
}
]
Query Empty string:
{
"query": {
"filtered": {
"query": {
"bool": {
"disable_coord": "true",
"should": [
{
"query_string": {
"fields": [
"description"
],
"query": ""
}
},
{
"constant_score": {
"filter": {
"term": {
"description.raw": ""
}
},
"boost": 0.2
}
},
{
"constant_score": {
"filter": {
"exists": {
"field": "description"
}
},
"boost": 0.1
}
}
]
}
},
"filter": {
"terms": {
"test_id": [
10
]
}
}
}
}
}
Result :
"hits": [
{
"_index": "test",
"_type": "data",
"_id": "2",
"_score": 1.3416407,
"_source": {
"description": "",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "1",
"_score": 0.44721356,
"_source": {
"description": "a",
"test_id": 10
}
},
{
"_index": "test",
"_type": "data",
"_id": "3",
"_score": 0.44721356,
"_source": {
"description": "hello",
"test_id": 10
}
}
]
Have you considered using wildcard query? Check this query it will work fine for you.
all blog records having letter "a" in it with topic_id is 10.
{
"filter": {
"and": [
{
"in": {
"topic_id": [
"10"
]
}
},
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"query": {
"wildcard": {
"description": {
"value": "*a*"
}
}
}
}
]
}
}
}
}
}
]
}
}
Also rest of the records where topic_id is 10 even if the description is blank/empty. This will return all the other records that doesn't match the wildcard.
{
"filter": {
"and": [
{
"in": {
"topic_id": [
"10"
]
}
},
{
"not": {
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"query": {
"wildcard": {
"description": {
"value": "*a*"
}
}
}
}
]
}
}
}
}
}
}
]
}
}
To find only the empty " " description fields with topic_id 10. try this,
{
"filter": {
"and": [
{
"in": {
"topic_id": [
"10"
]
}
},
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "_source.description.length() == 0"
}
}
}
}
}
]
}
}
For ES 2.x
Using a bool query should do the trick.
Here's the query I will use:
GET blog/_search
{
"query": {
"bool": {
"should": [
{
"query_string": {
"fields": [ "description" ],
"query": "a"
}
}
],
"must": [
{
"terms": {
"topic_id": [
10
]
}
}
]
}
}
}
Here, the should clause of the bool query will tell Elassticsearch that document matching the query_string should be returned. In the query_string consider using wildcards if you want to match any document containing a.
For example "query_string": { "query": "*a*" }
The must clause in the other hand will tell that, for considering the document a valid match, it must contain 10 in the topic_id field. Wether the should clause could or could not match.
Bool filter
I hope this could help you.

ElasticSearch sorting by nested field value

I need to sort result in next order:
Users, that I following
Users, that follows me
All other users
I have users which look like this:
{
"username": "admin"
"followers": [
{
"id": 2,
"username": "kiehn.nicola2"
},
{
"id": 3,
"username": "adaline253"
},
{
"id": 4,
"username": "skuhic4"
}
],
"following": [
{
"id": 2,
"username": "kiehn.nicola2"
},
{
"id": 3,
"username": "adaline253"
},
{
"id": 4,
"username": "skuhic4"
},
{
"id": 5,
"username": "heaney.garth5"
}
]
}
Is it possible?
Of course, I know current user id and username.
I write this query, but it doesn't work (for example, user id is 1):
{
"query": {
"bool": {
"must": [
{
"wildcard": {
"username": {
"value": "*a*",
"boost": 1
}
}
}
]
}
},
"sort": [
{
"following.username": {
"order": "asc",
"nested_path": "following",
"nested_filter": {
"term": {
"following.id": 1
}
}
},
"followers.username": {
"order": "asc",
"nested_path": "followers",
"nested_filter": {
"term": {
"followers.id": 1
}
}
}
}
],
"size": 40
}
I would do this by boosting; boost the hits that have the searchers id in their followers by an amount, then boost by a lower value the hits that have the searcher in their 'following' field:
NOTE: the searcher's id is 55 in this example
"query": {
"bool": {
"should": [
{
"nested": {
"path": "followers",
"query": {
"term" : { "followers.id": { "value": 55, "boost": 3.0 } }
}
}
},
{
"nested": {
"path": "following",
"query": {
"term" : { "following.id": { "value": 55, "boost": 2.0 } }
}
}
},
{
"match_all": { "boost": 1.0 }
}
]
}
}
If the searcher is in the hit's followers field, then the searcher is following that hit and so the boost is highest, etc...
You said you wanted all other users, hence the "match_all: {} query at the end.

Categories