We're executing an Elasticsearch query like this using PHP API:
$params = [
//please ignore the variables below,
//we made it in dynamic parameter-based in our function,
//that's why they're variables
'index' => $ourIndex,
'type' => $ourType,
'from' => $from,
'size' => $page_size,
'body' => [
"query" => [
'bool' => [
'must' => [
[
"query_string" => [
"default_field" => $content,
"query" => "$keywords"
]
],
[
"range" => [
"#timestamp" => [
"from" => $parseParams['pub_date_start'],
"to" => $parseParams['pub_date_end'],
'format' => "yy-MMM-dd'T'HH:mm:ss.SSS'Z'",
]
]
]
]
]
]
]
];
The query above works with our #timestamp field because its type is on date
"#timestamp" : {
"type" : "date"
}
And a sample value is like this:
"#timestamp" : "2019-06-17T16:53:55.778Z"
However, we want to target our pub_date field in our index, and in its mapping, the field has a type of long
"pub_date" : {
"type" : "long"
},
so it has this kind of values when we're displaying the documents:
"pub_date" : 1510358400
When we changed the query above to target instead of #timestamp to pub_date, it now displays an error like this:
Tried Solutions
I tried to add an additional format epoch_millis in the format property:
[
"range" => [
"pub_date" => [
"from" => $parseParams['pub_date_start'],
"to" => $parseParams['pub_date_end'],
'format' => "yyyy-MM-dd||yy-MMM-dd'T'HH:mm:ss.SSS'Z'||epoch_millis",
]
]
]
But still fails
Main Question
I feel that the Unix formatted values cant be recognized by the range query of Elasticsearch so that's why the query fails. Is there a work-around for this without changing the MAPPINGS of the index?
Because the other possible solutions suggested to change the mapping, but we already have around 25 million documents in the index, so we thought formatting it in PHP would be a nicer approach
Since the field is of type long and stores the unix timestamp, simply convert the date in $parseParams['pub_date_start'] and $parseParams['pub_date_end'] to unix timestamp using strtotime. Update the range query as below:
"range" => [
"pub_date" => [
"from" => strtotime($parseParams['pub_date_start']),
"to" => strtotime($parseParams['pub_date_end']),
]
]
Related
I have a page which allows users to query datasets and apply filters. They can also apply filters without querying with a string. To do so I'm attempting to use match_all with filters but getting the following error
"{"error":{"root_cause":[{"type":"parsing_exception","reason":"[match_all]
malformed query, expected [END_OBJECT] but found
[FIELD_NAME]","line":1,"col":26}],"type":"parsing_exception","reason":"[match_all]
malformed query, expected [END_OBJECT] but found
[FIELD_NAME]","line":1,"col":26},"status":400}",
This is an example of the search parameters that I'm building and sending to the elastic client.
[
"type" => "events"
"index" => "events"
"body" => [
"query" => [
"match_all" => {}
"bool" => [
"filter" => [
"range" => [
"start_date.date" => [
"gte" => "01/05/2019"
"lte" => "05/2019"
"format" => "dd/MM/yyyy||MM/yyyy"
]
]
]
]
]
"from" => 0
"size" => 30
]
]
I can't seem to figure out how to use both of them. Any pointers? Thank you.
You will need to wrap your query in a bool query like this:
"query": {
"bool" : {
"must" : {
"match_all": {}
},
"filter": {
"range" : { /* your filter here*/ }
}
}
}
Just wrap the bool and a must query around your match_all and it should work.
I don't know the exact PHP syntax but it should be something like this:
[
"type" => "events"
"index" => "events"
"body" => [
"query" => [
"bool" => [
"must" => [ "match_all" => {}]
"filter" => [
"range" => [
"start_date.date" => [
"gte" => "01/05/2019"
"lte" => "05/2019"
"format" => "dd/MM/yyyy||MM/yyyy"
]
]
]
]
]
"from" => 0
"size" => 30
]
]
For reference see the docs Elasticsearch Reference [7.0] » Query DSL » Compound queries » Bool Query, it contains an example like yours with match_all combined with filters.
I am trying to push data to a firestore DB using PHP and the Google apis.
Inside the documentation and examples I have seen around the web, I am able to use mapValue and arrayValue when sending data.
The example I am using is as follows:-
[
"orderName" => [
"stringValue" => "Gbeila Aliu Wahab"
],
"orderLocationName" => [
"stringValue" => "Accra Mall Limited"
],
"orderTotalAmount" => [
"doubleValue" => 150.5
],
"orderDescription" => [
"stringValue" => "Lorem Ipsum is simply dummy text of the printing and typesetting industry"
],
"orderLocationGeoPoints" => [
"geoPointValue" => (object) [
"latitude" => 5.5557,
"longitude" => -0.1963
]
],
"orderStatus" => [
"stringValue" => "NotAssigned"
],
]
This works perfectly fine, but when I attempt to send an object or an array I get the following error returned to me:-
"message": "Invalid JSON payload received. Unknown name \"map_value\" at 'document.fields[0].value': Proto field is not repeating, cannot start list.",
when attempting to map the value using the following code:-
"orderName" => [
"mapValue" => ["Gbeila Aliu Wahab", 123]
]
// or
"orderName" => [
"arrayValue" => [
"first" => [
"stringValue" => "test"
],
"second" => [
"stringValue" => "test123"
]
]
]
I have tried many variations to try to get this to work.
How am I supposed to be using the mapValue and arrayValue I can see a lot of mentions regarding the value option but I cannot see any examples on how to use the.
Any help would be greatly appreciated.
Payload to your array or map you're generating is incorrect as per the documentation. You need to wrap your actual data (to store) under values key, your final array should be:
["orderName" => ["arrayValue" => ["values" => [["stringValue" => "test"], ["stringValue" => "test123"]]]]]
Similarly your mapValue should be
["orderName" => ["mapValue" => ["fields" => ["field1" => ["stringValue" => "Gbeila Aliu Wahab"]]]]]
Also, you can play with other data mapper via this package.
I have MongoDB document which contains following data(xxx)
{
"_id" : "48e5f6b1-026f-48b8-98cb-6572cfa0eaa6",
"esti_delivery_time" : "2015-1-1",
"original_price" : NumberInt(4060),
"price":109.2,
"code" : "JS1709137",
"updated_at" : ISODate("2017-09-13T06:01:18.000+0000"),
"created_at" : ISODate("2017-09-13T03:45:54.000+0000")
}
I want to
SELECT "xxx" with SUM "price" and SUM "original_price"
Because you need the sums of the price and original_price elements, it is means that you need to group the documents by some data (one or more elements). For this task you need to use the $group aggregator.
MongoDB PHP Driver documentation
Example:
$con = new MongoDB\Client('mongodb://localhost:27017', [], [
'typeMap' => [
'root' => 'array',
'document' => 'array',
'array' => 'array'
]
]);
$collection = $con->selectDatabase("db_name")->selectCollection("collection_name");
$cursor = $collection->aggregate([
[
'$group' => [
"_id" => '$code',
"sum_price" => ['$sum' => '$price'],
"sum_original_price" => ['$sum' => '$original_price']
]
]
]);
My answer:
enter image description here
Thanks for your reply
I want to execute multiple queries on elasticsearch server with one request. Specifically I have the following query (is on elastcisearch-php-client)
$params = [
"index" => "bookydate",
"type" => "vendor_service",
"body" => [
"query" => [
"bool" => [
"must" => [
"term" => [
"sys_service_id" => $request->input("sys_service_id")
]
],
"should" => [
"geo_shape" => [
"served_location" => [
"shape" => [
"type" => "point",
"coordinates" => [
"{$request->input('loc_lon')}",
"{$request->input('loc_lat')}"]
]
]
]
]
]
]
]
];
What I want to do is the fetch also all the documents that have the "hole_country" field to true.
What I have tried already is to make another request to Elasticsearch server and with array_merge combine the two results, but did not work because of PHP restrictions on arrays with multiple same keys.
UPDATE
Elastcisearch supports a feature called Multisearch that is exactly what im looking for. The problem is that php-client does not support multisearch so I have to use Guzzle in order to send the requests.
Guzzle docs does not have a full info about how to construct a correct request body. Any info is welcome
Already i have the following body but elastcisearch is returing bad request error
$body = [
["index"=>"bookydate"],
["query"=>["bool"=> ["must"=>[["term"=>["sys_service_id"=>"1"]],["geo_shape"=>["served_location"=>["shape"=>["type"=>"circle","coordinates"=>[25,3],"radius"=>"90km"]]]]]]]],
["index"=>"bookydate"],
["query"=>["bool"=>["must"=>["term"=>["hole_country"=>true]]]]]
];
You can use the multisearch API of Elasticsearch. This is more or less appending all your queries as JSON format in a single POST request. I hope the PHP client supports this, otherwise you might have to manually do the POST request.
Multi-search API
Although it's not documented the Multi Search API is supported by the elasticsearch php client.
Instead of search call msearch, and group your queries like this:
$params = [
'body' => [
["index" => "bookydate", "type" => "vendor_service"],
["query" => [
"bool" => [
"must" => [
"term" => [
"sys_service_id" => $request - > input("sys_service_id")
]
],
"should" => [
"geo_shape" => [
"served_location" => [
"shape" => [
"type" => "point",
"coordinates" => [
"{$request->input('loc_lon')}",
"{$request->input('loc_lat')}"
]
]
]
]
]
]
]]
];
So using your updated syntax is correct. You must just call msearch.
I want to use wildcard search using elastic search 2.3 using its official PHP client.
I am facing a issue which is like this:
Case 1. When i search for word wood, it returns the words which are having woodman, hollywood and hollywoodbolly.
Case 2. But when i search for hollywood, it does not return the words which are having hollywood in them.
However, everything is working fine when done in query string like this:
"query" => [
"query_string" => [
"query" => "*$keyword*",
"analyze_wildcard" => true,
"fields" => $fields
]
],
But when used like follwing, Case 2 is not working:
"query" => [
"bool" => [
"must" => [
[
"wildcard" => [
'name' => "*$keyword*",
]
],
[
"nested" => [
"path" => "address",
"score_mode" => "max",
"query" => [
"bool" => [
"must" => [..match[] parameters..]
]
]
]
]
]
]
I am not sure what I am doing wrong. Please help.
EDIT:
NOTE: I have made the field as not_analysed.
My query is returning cardboard when searching for card but not returning cardboard when searching for cardboard
Thanks.
Elasticsearch supports wildcard queries only on not_analyzed fields
So if you would like to use the wildcard capability you could either use it under the query_string object, or change the mapping for that field to index: not_analyzed and then you would be able to do a wildcard search.