I need to run one ES bool query which is looks something like this
{
"aggs": {
"column1": {
"terms": {
"field": "column1.raw",
"size": 5
}
}
},
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"range": {
"time": {
"gt": "2015-02-19 00:00:00",
"lt": "2015-02-20 00:00:00"
}
}
}
],
"should": [
{
"term": {
"column1.raw": "value1"
}
},
{
"term": {
"column1.raw": "value2"
}
}
]
}
}
}
}
}
I have done the things upto should and now need to formulate terms part. As it is having the same index value multiple times , how can I formulate the array for it ?
I am using the following PHP code now :
foreach ($terms as $term) {
$termFIlter = array(
"term" => array(
"column1.raw" => $term
)
);
}
$options['body']['query'] = array(
"filtered" => array(
"filter" => array(
"bool" => array(
"must" => array(
"range" => array(
"time" => array(
"gt" => $start,
"lt" => $end
)
)
),
"should" => $termFIlter
)
)
)
);
take this example please change fields name.set what are fields your are using .
$options = array(
'fields' => array('title', 'content', 'profile_id', 'type', 'name', 'description', 'date', 'url'),
'from' => 0,
'size' => 10,
'query' => array(
($type ?
array(
'bool' => array(
'must' => array(
array('term' => array('_all' => $term)),
array('term' => array('type' => $type))
)
)
) :
array('match' => array('_all' => $term))
)
)
);
In elasticsearch php client documentation this is how you handle needing to put multiple index values of match:
$params = [
'index' => 'my_index',
'type' => 'my_type',
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'testField' => 'abc' ] ],
[ 'match' => [ 'testField2' => 'xyz' ] ],
]
]
]
]
];
So I think it would be the same for your query but with term. So like you showed in your edit or like this:
$options['body']['query'] = array(
"filtered" => array(
"filter" => array(
"bool" => array(
"must" => array(
"range" => array(
"time" => array(
"gt" => $start,
"lt" => $end
)
)
),
"should" => [
[
"term" => [
"column1.raw" => "value1"
]
],
[
"term" => [
"column1.raw" => "value2"
]
]
]
)
)
)
);
Related
Starting to work on ES. I would like to be able to make a multi_match with a filter in PHP. I followed the official ES documentation but I don't understand my mistake.
Here is the code:
public function search_data_into_index($array)
{
$params = [
'index' => 'skills',
'type' => 'people',
'body' => [
'query' => [
'multi_match' => [
'query' => 'react',
'fields' => [$array[2]],
'fuzziness' => 'AUTO',
],
'filter' => [
'geo_distance' => [
'distance' => '300m',
'location' => '-25, -49'
]
]
]
]
];
$response = $this->client->search($params);
print_r($response);
}
Here my error :
{"error":{"root_cause":[{"type":"parsing_exception","reason":"[multi_match] malformed query, expected [END_OBJECT] but found [FIELD_NAME]","line":1,"col":94}],"type":"parsing_exception","reason":"[multi_match] malformed query, expected [END_OBJECT] but found [FIELD_NAME]
The multi_match query must be located inside bool/must:
public function search_data_into_index($array)
{
$params = [
'index' => 'skills',
'type' => 'people',
'body' => [
'query' => [
'bool' => [
'must' => [
'multi_match' => [
'query' => 'react',
'fields' => [$array[2]],
'fuzziness' => 'AUTO',
]
],
'filter' => [
'geo_distance' => [
'distance' => '300m',
'location' => '-25, -49'
]
]
]
]
]
];
$response = $this->client->search($params);
print_r($response);
}
You need to combine multiple queries using boolean query
In JSON format your query will look like
{
"query": {
"bool": {
"must": {
"multi_match": {
"query": "react",
"fields": [$array[2]]
}
},
"filter": {
"geo_distance": {
"distance": "300m",
"location": [
"-25, -49"
]
}
}
}
}
}
I have an elasticsearch that is working fine and so is my php client used to query the index. However, my client works only with one index. When I try adding multiple indices in a string as shown here my code breaks.
Here is a snippet of my code:
if ($q2 == '') {
$query = $client->search([
'index' => 'trial2',
'body' => [
'from' => $from,
'size' => $size,
'query' => [
'multi_match' => [
'query' => $q,
'operator' => $op,
'fields' => ['content','file','file.extension','meta', 'path']
]
]
]
]);
$data['q'] = $q;
}
elseif ($q2 == "docx") {
$params = [
'index' => 'trial2',
'type' => '_doc',
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'terms' => [ 'file.extension' => ["docx", "doc", "txt", "rtf"] ] ],
[ 'match' => [ 'content' => $q ] ],
]
]
]
]
];
$query = $client->search($params);
$data['q'] = $q;
}
I would like to query several indexes (trial2, trial3 and trial4). How do I do it?
I have this code what executes an query.
I am using the official php elastic search library.
https://github.com/elastic/elasticsearch-php
The field "Tijdsperiode" is in this format : ( 2016-01-30 00:00:00 ) ( YY-MM-DD HH:MM:SS )
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'body' => [
'query' => [
'match_all' => [],
'filter' => [
'range' => [
'Tijdsperiode' => [
'gte' => '2016-01-30 01:00:00',
'lte' => '2016-01-30 08:00:00'
]
]
]
],
],
];
$response = $client->search($params);
var_dump($response);
I just wonder what the format need to be to get results between the 2 dates.
When i do this :
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'body' => [
'query' => [
'match_all' => [],
],
],
];
It's working fine but i need the results between the 2 dates!
I also tried this but with no result :
$json = '{
"query": {
"bool": {
"must": [
{
"range": {
"Tijdsperiode": {
"gt": "2016-01-30 07:00:00",
"lt": "2016-01-30 09:00:00"
}
}
}
]
}
}
}';
$client = ckan_graphmapper_client();
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'body' => $json
];
$response = $client->search($params);
I also tried it with a mapping :
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'size' => $size,
'body' => [
'query' => [
"match_all" => [],
'filter' => [
'range' => [
'Tijdsperiode' => [
'gte' => '2016-01-30 01:00:00',
'lte' => '2016-01-30 08:00:00'
]
]
]
],
'mappings' => [
'_default_' => [
'properties' => [
'Tijdsperiode' => [
'type' => 'date',
'format' => 'yyyy-MM-dd HH:mm:ss'
]
]
]
]
]
];
How to do the right syntax for choosing between 2 dates?. Both string formats ( 2016-01-30 00:00:00 ) ( YY-MM-DD HH:MM:SS )
Thanks!
So it turns out my import of data was not mapping the correct way.
The date field was recognised as a string value.
When i finnaly had the correct mapping in elastic i could do this query :
$query = [
'query' => [
'filtered' => [
'query' => [
'match_all' => []
],
'filter' => [
'range' => [
'Tijdsperiode' => [
'gte' => $start,
'lte' => $end
]
]
]
]
]
];
I have a user collection with following data
[
{
"user_id": "5625c95ac2d34f27148b64fa",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "562744ccc2d34f27148b6eb7",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "56248eb9c2d34f2f148b5a18",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "561f40bac2d34f17148b462c",
"friend_id": "561f3e06c2d34f27148b45f6"
},
{
"user_id": "561f40bac2d34f17148b462c",
"friend_id": "5620de97c2d34f2f148b578f"
},
{
"user_id": "56276b52c2d34f27148b7128",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "561f40bac2d34f17148b462c",
"friend_id": "56276b52c2d34f27148b7128"
}
]
i need to fetch the documents in which combination of user_id and friend_id not repeated. i.e in the above example last two documents user_id repeated in friend_id of next document.
I tried with mongo aggrigate and group by but could not reduce it.
In order to do this you basically need to combine both user_id and friend_id values in a uniquely sorted combination. This means creating an array for each document with those members and sorting that array so that the order is always the same.
Then you can $group on that sorted array content to see which documents contain that same combination and then only return those that do not share that same combination.
This leads to this aggregate statement:
db.collection.aggregate([
{ "$project": {
"user_id": 1,
"friend_id": 1,
"combined": {
"$map": {
"input": ["A","B"],
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el", "A" ] },
"$user_id",
"$friend_id"
]
}
}
}
}},
{ "$unwind": "$combined" },
{ "$sort": { "combined": 1 } },
{ "$group": {
"_id": "$_id",
"combined": { "$push": "$combined" },
"user_id": { "$first": "$user_id" },
"friend_id": { "$first": "$friend_id" }
}},
{ "$group": {
"_id": "$combined",
"docs": { "$push": {
"_id": "$_id",
"user_id": "$user_id",
"friend_id": "$friend_id"
}}
}},
{ "$redact": {
"$cond": {
"if": { "$ne": [{ "$size": "$docs" }, 1] },
"then": "$$PRUNE",
"else": "$$KEEP"
}
}}
])
The PHP translation for laravel means to need to access the raw collection object from the manager, where "collection" is the actual name of the collection in MongoDB:
$result = DB::collection("collection")->raw(function($collection) {
return $collection->aggregate(
array(
array(
'$project' => array(
'user_id' => 1,
'friend_id' => 1,
'combined' => array(
'$map' => array(
'input' => array("A","B"),
'as' => 'el',
'in' => array(
'$cond' => array(
array( '$eq' => array( '$el', 'A' ) ),
'$user_id',
'$friend_id'
)
)
)
)
)
),
array( '$unwind' =>'$combined' ),
array( '$sort' => array( 'combined' => 1 ) ),
array(
'$group' => array(
'_id' => '$_id',
'combined' => array( '$push' => '$combined' ),
'user_id' => array( '$first' => '$user_id' ),
'friend_id' => array( '$first' => '$friend_id' )
)
),
array(
'$group' => array(
'_id' => '$combined',
'docs' => array(
'$push' => array(
'_id' => '$_id',
'user_id' => '$user_id',
'friend_id' => 'friend_id'
)
)
)
),
array(
'$redact' => array(
'$cond' => array(
'if' => array( '$ne' => array( array( '$size' => '$docs'), 1) ),
'then' => '$$PRUNE',
'else' => '$$KEEP'
)
)
)
)
);
});
Or if your MongoDB version is less than 2.6, and you lack operators like $map and $redact, then you can still do this, but not as efficiently:
$result = DB::collection("collection")->raw(function($collection) {
return $collection->aggregate(
array(
array(
'$project' => array(
'user_id' => 1,
'friend_id' => 1,
'type' => array( '$const' => array( 'A', 'B' ) )
)
),
array( '$unwind' => '$type' ),
array(
'$group' => array(
'_id' => '$_id',
'user_id' => array( '$first' => '$user_id' ),
'friend_id' => array( '$first' => '$friend_id' ),
'combined' => array(
'$push' => array(
'$cond' => array(
array( '$eq' => array( '$type', 'A' ) ),
'$user_id',
'$friend_id'
)
)
)
)
)
array( '$unwind' =>'$combined' ),
array( '$sort' => array( 'combined' => 1 ) ),
array(
'$group' => array(
'_id' => '$_id',
'combined' => array( '$push' => '$combined' ),
'user_id' => array( '$first' => '$user_id' ),
'friend_id' => array( '$first' => '$friend_id' )
)
),
array(
'$group' => array(
'_id' => '$combined',
'docs' => array(
'$push' => array(
'_id' => '$_id',
'user_id' => '$user_id',
'friend_id' => 'friend_id'
)
),
'count' => array( '$sum' => 1 )
)
),
array( '$match' => array( 'count' => 1 ) )
)
);
});
Where the first three stages mimic what the first stage is doing in the first example listing by putting both values in a single array. Of course the last two stages by "counting" the array members while grouping and then filtering out anything that does not have a "count" of 1.
In either case this leaves you with output that only lists the documents where that combination does not occur in either order:
{
"_id" : [ "561f40bac2d34f17148b462c", "5625c95ac2d34f27148b64fa" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0cc9"),
"user_id" : "5625c95ac2d34f27148b64fa",
"friend_id" : "561f40bac2d34f17148b462c"
}
]
}
{
"_id" : [ "561f3e06c2d34f27148b45f6", "561f40bac2d34f17148b462c" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0ccc"),
"user_id" : "561f40bac2d34f17148b462c",
"friend_id" : "561f3e06c2d34f27148b45f6"
}
]
}
{
"_id" : [ "561f40bac2d34f17148b462c", "56248eb9c2d34f2f148b5a18" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0ccb"),
"user_id" : "56248eb9c2d34f2f148b5a18",
"friend_id" : "561f40bac2d34f17148b462c"
}
]
}
{
"_id" : [ "561f40bac2d34f17148b462c", "5620de97c2d34f2f148b578f" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0ccd"),
"user_id" : "561f40bac2d34f17148b462c",
"friend_id" : "5620de97c2d34f2f148b578f"
}
]
}
{
"_id" : [ "561f40bac2d34f17148b462c", "562744ccc2d34f27148b6eb7" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0cca"),
"user_id" : "562744ccc2d34f27148b6eb7",
"friend_id" : "561f40bac2d34f17148b462c"
}
]
}
You can pretty up the output, but this serves the purpose of showing the ordered combination used along with the original document data.
I've been trying to get the following JSON working in PHP Arrays but I don't seem to get any hits.
The JSON is as follows:
{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "search"
}
}
}
},
"fields": [
"body",
"title",
"postDate",
"user",
"name"
],
"from": 0,
"size": 50,
"sort": {
"_score": {
"order": "asc"
}
},
"explain": true
}
And the PHP I managed to create is like this:
$docs = $client->search([
'index' => 'blog',
'type' => 'posts',
'body' => [
'query' => [
'filtered' => [
'query' => [
'query_string' => [
'query' => $search_query
]
]
]
],
'fields' => [
'body',
'title',
'postDate',
'user',
'name'
],
'from' => 0,
'size' => 50,
'sort' => [
'_score' => [
'order' => 'asc'
]
]
]
]);
It returns an response but no hits, even though it should (and it does in case of the JSON request)
What is going on here?
The post type wasn't required at all... I somehow thought it was. I used a tool called ElasticHQ to generate the JSON and i didn't realize it wasnt using Posts as a type.
Changed it to
$docs = $client->search([
'index' => 'blog',
'body' => [
'query' => [
'filtered' => [
'query' => [
'query_string' => [
'query' => $search_query
]
]
]
],
'fields' => [
'body',
'title',
'postDate',
'user',
'name'
],
'from' => 0,
'size' => 50,
'sort' => [
'_score' => [
'order' => 'asc'
]
]
]
]);