Using match_all with filters - php

I have a page which allows users to query datasets and apply filters. They can also apply filters without querying with a string. To do so I'm attempting to use match_all with filters but getting the following error
"{"error":{"root_cause":[{"type":"parsing_exception","reason":"[match_all]
malformed query, expected [END_OBJECT] but found
[FIELD_NAME]","line":1,"col":26}],"type":"parsing_exception","reason":"[match_all]
malformed query, expected [END_OBJECT] but found
[FIELD_NAME]","line":1,"col":26},"status":400}",
This is an example of the search parameters that I'm building and sending to the elastic client.
[
"type" => "events"
"index" => "events"
"body" => [
"query" => [
"match_all" => {}
"bool" => [
"filter" => [
"range" => [
"start_date.date" => [
"gte" => "01/05/2019"
"lte" => "05/2019"
"format" => "dd/MM/yyyy||MM/yyyy"
]
]
]
]
]
"from" => 0
"size" => 30
]
]
I can't seem to figure out how to use both of them. Any pointers? Thank you.

You will need to wrap your query in a bool query like this:
"query": {
"bool" : {
"must" : {
"match_all": {}
},
"filter": {
"range" : { /* your filter here*/ }
}
}
}
Just wrap the bool and a must query around your match_all and it should work.
I don't know the exact PHP syntax but it should be something like this:
[
"type" => "events"
"index" => "events"
"body" => [
"query" => [
"bool" => [
"must" => [ "match_all" => {}]
"filter" => [
"range" => [
"start_date.date" => [
"gte" => "01/05/2019"
"lte" => "05/2019"
"format" => "dd/MM/yyyy||MM/yyyy"
]
]
]
]
]
"from" => 0
"size" => 30
]
]
For reference see the docs Elasticsearch Reference [7.0] » Query DSL » Compound queries » Bool Query, it contains an example like yours with match_all combined with filters.

Related

How to add "runtime_mappings" to the query for FOS/Elastica in PHP?

Need to add the next json query to php code with using FOSElasticaBundle:
"runtime_mappings": {
"Agreement": {
"type": "keyword",
"script": {
"source": "if(doc['winningBidder.edrpou'].size()>0 && doc['seller'].size()>0)\r\n{\r\nemit(\r\n doc['seller'].value+\":\"+\r\n doc['trading.id'].value+\":\"+\r\n doc['winningBidder.edrpou'].value+\":\"\r\n )\r\n}"
}
}
}
If I'm set this in simple method (\Elastica\Query)->addParam():
->addParam('runtime_mappings', [
'Agreement' => [
'type' => 'keyword',
'script' => [
'source' => "if(doc['winningBidder.edrpou'].size()>0 && doc['seller'].size()>0)\r\n{\r\nemit(\r\n doc['seller'].value+\":\"+\r\n doc['trading.id'].value+\":\"+\r\n doc['winningBidder.edrpou'].value+\":\"\r\n )\r\n}"
]
]
])
Then I get an error when I try to collect the query:
Unknown key for a START_ARRAY in [runtime_mappings].
Alternatively, you can create a query by starting with the "runtime_mappings" array and adding all other parameters after:
$query = \Elastica\Query::create(['runtime_mappings' => [
'Agreement' => [
'type' => 'keyword',
'script' => [
'source' => "if(doc['winningBidder.edrpou'].size()>0 && doc['seller'].size()>0)\r\n{\r\nemit(\r\n doc['seller'].value+\":\"+\r\n doc['trading.id'].value+\":\"+\r\n doc['winningBidder.edrpou'].value+\":\"\r\n )\r\n}"
]
],
]]);
// add other params to query...
$query->addParam();

(Elasticsearch) Convert Unix formatted data into timestamp (without changing the mappings)

We're executing an Elasticsearch query like this using PHP API:
$params = [
//please ignore the variables below,
//we made it in dynamic parameter-based in our function,
//that's why they're variables
'index' => $ourIndex,
'type' => $ourType,
'from' => $from,
'size' => $page_size,
'body' => [
"query" => [
'bool' => [
'must' => [
[
"query_string" => [
"default_field" => $content,
"query" => "$keywords"
]
],
[
"range" => [
"#timestamp" => [
"from" => $parseParams['pub_date_start'],
"to" => $parseParams['pub_date_end'],
'format' => "yy-MMM-dd'T'HH:mm:ss.SSS'Z'",
]
]
]
]
]
]
]
];
The query above works with our #timestamp field because its type is on date
"#timestamp" : {
"type" : "date"
}
And a sample value is like this:
"#timestamp" : "2019-06-17T16:53:55.778Z"
However, we want to target our pub_date field in our index, and in its mapping, the field has a type of long
"pub_date" : {
"type" : "long"
},
so it has this kind of values when we're displaying the documents:
"pub_date" : 1510358400
When we changed the query above to target instead of #timestamp to pub_date, it now displays an error like this:
Tried Solutions
I tried to add an additional format epoch_millis in the format property:
[
"range" => [
"pub_date" => [
"from" => $parseParams['pub_date_start'],
"to" => $parseParams['pub_date_end'],
'format' => "yyyy-MM-dd||yy-MMM-dd'T'HH:mm:ss.SSS'Z'||epoch_millis",
]
]
]
But still fails
Main Question
I feel that the Unix formatted values cant be recognized by the range query of Elasticsearch so that's why the query fails. Is there a work-around for this without changing the MAPPINGS of the index?
Because the other possible solutions suggested to change the mapping, but we already have around 25 million documents in the index, so we thought formatting it in PHP would be a nicer approach
Since the field is of type long and stores the unix timestamp, simply convert the date in $parseParams['pub_date_start'] and $parseParams['pub_date_end'] to unix timestamp using strtotime. Update the range query as below:
"range" => [
"pub_date" => [
"from" => strtotime($parseParams['pub_date_start']),
"to" => strtotime($parseParams['pub_date_end']),
]
]

Combine two queries in one request

I want to execute multiple queries on elasticsearch server with one request. Specifically I have the following query (is on elastcisearch-php-client)
$params = [
"index" => "bookydate",
"type" => "vendor_service",
"body" => [
"query" => [
"bool" => [
"must" => [
"term" => [
"sys_service_id" => $request->input("sys_service_id")
]
],
"should" => [
"geo_shape" => [
"served_location" => [
"shape" => [
"type" => "point",
"coordinates" => [
"{$request->input('loc_lon')}",
"{$request->input('loc_lat')}"]
]
]
]
]
]
]
]
];
What I want to do is the fetch also all the documents that have the "hole_country" field to true.
What I have tried already is to make another request to Elasticsearch server and with array_merge combine the two results, but did not work because of PHP restrictions on arrays with multiple same keys.
UPDATE
Elastcisearch supports a feature called Multisearch that is exactly what im looking for. The problem is that php-client does not support multisearch so I have to use Guzzle in order to send the requests.
Guzzle docs does not have a full info about how to construct a correct request body. Any info is welcome
Already i have the following body but elastcisearch is returing bad request error
$body = [
["index"=>"bookydate"],
["query"=>["bool"=> ["must"=>[["term"=>["sys_service_id"=>"1"]],["geo_shape"=>["served_location"=>["shape"=>["type"=>"circle","coordinates"=>[25,3],"radius"=>"90km"]]]]]]]],
["index"=>"bookydate"],
["query"=>["bool"=>["must"=>["term"=>["hole_country"=>true]]]]]
];
You can use the multisearch API of Elasticsearch. This is more or less appending all your queries as JSON format in a single POST request. I hope the PHP client supports this, otherwise you might have to manually do the POST request.
Multi-search API
Although it's not documented the Multi Search API is supported by the elasticsearch php client.
Instead of search call msearch, and group your queries like this:
$params = [
'body' => [
["index" => "bookydate", "type" => "vendor_service"],
["query" => [
"bool" => [
"must" => [
"term" => [
"sys_service_id" => $request - > input("sys_service_id")
]
],
"should" => [
"geo_shape" => [
"served_location" => [
"shape" => [
"type" => "point",
"coordinates" => [
"{$request->input('loc_lon')}",
"{$request->input('loc_lat')}"
]
]
]
]
]
]
]]
];
So using your updated syntax is correct. You must just call msearch.

Elastic search 2.3 wildcard query not returning results for exact match

I want to use wildcard search using elastic search 2.3 using its official PHP client.
I am facing a issue which is like this:
Case 1. When i search for word wood, it returns the words which are having woodman, hollywood and hollywoodbolly.
Case 2. But when i search for hollywood, it does not return the words which are having hollywood in them.
However, everything is working fine when done in query string like this:
"query" => [
"query_string" => [
"query" => "*$keyword*",
"analyze_wildcard" => true,
"fields" => $fields
]
],
But when used like follwing, Case 2 is not working:
"query" => [
"bool" => [
"must" => [
[
"wildcard" => [
'name' => "*$keyword*",
]
],
[
"nested" => [
"path" => "address",
"score_mode" => "max",
"query" => [
"bool" => [
"must" => [..match[] parameters..]
]
]
]
]
]
]
I am not sure what I am doing wrong. Please help.
EDIT:
NOTE: I have made the field as not_analysed.
My query is returning cardboard when searching for card but not returning cardboard when searching for cardboard
Thanks.
Elasticsearch supports wildcard queries only on not_analyzed fields
So if you would like to use the wildcard capability you could either use it under the query_string object, or change the mapping for that field to index: not_analyzed and then you would be able to do a wildcard search.

Is it possible to only use filters without text search in elasticsearch

I am using ES for my Laravel app, and I need to do a search query that only contains filters and no "text search" but I am not sure on how to write it.
Must I use match_all eg:
$query = [
'filtered' => [
'query' => [
'match_all' => []
],
'filter'=> [
'bool' => [
'must' => [
[ 'range' => [
'price' => [
'lte' => 9000
]
]
],
],
]
],
],
];
Or like this:
$query = [
'filtered' => [
'filter'=> [
'bool' => [
'must' => [
[ 'range' => [
'price' => [
'lte' => 9000
]
]
],
],
]
],
],
];
What I want is to only use a filtered bool query without text search.
In fact, if you don't specify the query part in your filtered query, a match_all query is used by default. Quoting the doc :
If a query is not specified, it defaults to the match_all query. This
means that the filtered query can be used to wrap just a filter, so
that it can be used wherever a query is expected.
Your second query should do the job : filters must be wrapped either in filtered (doc) or constant_score (doc) queries to be used.
If the scoring part isn't useful for you, you can stick to the filtered query.
Last thing : you don't have to nest your filter in a bool filter, unless you want to combine it with other(s) filter(s). In your demo case, you can write directly :
$query = [
'filtered' => [
'filter'=> [
'range' => [
'price' => [
'lte' => 9000
]
]
]
]
];
Hope this will be helpful :)
It's actually exactly the same thing since if a query is not specified in the clause it defaults to using the match_all query.
While in query context, if you need to use a filter without a query (for instance, to match all emails in the inbox), you can just omit the query:
GET /_search
{
"query": {
"filtered": {
"filter": { "term": { "folder": "inbox" }}
}
}
}
If a query is not specified it defaults to using the match_all query, so the preceding query is equivalent to the following:
GET /_search
{
"query": {
"filtered": {
"query": { "match_all": {}},
"filter": { "term": { "folder": "inbox" }}
}
}
}
Check here the official documentation: http://www.elastic.co/guide/en/elasticsearch/guide/current/_combining_queries_with_filters.html

Categories