I did the following rest request and it's working
{
"aggs": {
"gender": {
"terms": {
"field": "gender"
}
}
},
"size": 0
}
But when i do it in PHP with array like this:
['aggs' => [
'gender' => [
'terms' => [
'field' => 'gender'
],
],
],
];
I'm getting the following error :
{
code: 500,
message: ""aggs" is not a valid parameter. Allowed parameters are: "analyzer", "analyze_wildcard", "default_operator", "df", "explain", "fields", "from", "ignore_unavailable", "allow_no_indices", "expand_wildcards", "indices_boost", "lenient", "lowercase_expanded_terms", "preference", "q", "query_cache", "request_cache", "routing", "scroll", "search_type", "size", "sort", "source", "_source", "_source_exclude", "_source_include", "stats", "suggest_field", "suggest_mode", "suggest_size", "suggest_text", "timeout", "version", "fielddata_fields", "filter_path", "client", "custom", "filter_path"",
errors: [ ]
}
I found the solution, it was a problem in the code PHP
We are using a Framework Majora, and we are only returning Queries from ElasticSearch to a response.
So i will fix that for the Aggregation to return aggs in the response.
Thanks
Related
I'm trying to use aggregate to return token and kyc_status in an API call.
I have below data in tokens collection
{
"_id": {
"$oid": "5fca628feb1f03d82c76e85f"
},
"user_id": "5fca62d0eb1f03d82c76e860",
"token": "token_here",
"valid_till": "2021-01-03",
"created": "2020-12-04 20:27:05",
"modified": "2020-12-04 20:27:05"
}
then I have below data in users collection
{
"_id": {
"$oid": "5fca62d0eb1f03d82c76e860"
},
"seller_type_id": "",
"status": {
"$numberInt": "1"
},
"kyc_status": "pending",
"username": "user_name",
"password": "password_here",
"email": "email_here",
"primary_contact": "primary",
"secondary_contact": "secondary"
}
Now i have id 5fca62d0eb1f03d82c76e860 in $user_id PHP variable
I'm using the below code to achieve to show kyc_status and token
$result = $collection->aggregate( [
[ '$lookup' => [
'from' => 'users',
'localField' => '_id',
'foreignField' => 'user_id',
'as' => 'user_role'
]
],
[ '$unwind' => '$user_info' ],
[
'$project' => [ 'kyc_staus' => '$user_info.kyc_status' ]
]
]
);
I do not get any out put when I print the $result variable it shows me a whole lot of data but does not show what I want, I searched and tried most of StackOverflow answers to the same but did not get any.
Can anyone let me know where i am goring wrong / what should i do to get the desired output?
I have a laravel collection that has this format:
{
"24385528032901": [
{
"time": "2020-06-30T22:30:00.000000Z",
"conso_prod": "Prod",
"meter_id": "24385528032901",
"delta": "0",
},
{
"time": "2020-06-30T23:00:00.000000Z",
"conso_prod": "Prod",
"meter_id": "24385528032901",
"delta": "2",
}
],
"24385528032777": [
{
"time": "2020-06-30T22:30:00.000000Z",
"conso_prod": "Prod",
"meter_id": "24385528032777",
"delta": "0",
},
{
"time": "2020-06-30T23:00:00.000000Z",
"conso_prod": "Prod",
"meter_id": "24385528032777",
"delta": "5",
}
], etc.
}
I'm having an hard time converting it to a chartJS linechart graph format:
[
[
'label' => '24385528032901',
'data' => $measures->map->delta,
], [
'label' => '24385528032777',
'data' => $measures->map->delta,
],
]
I know there is a collection method to do this, but can't find it anymore. Anyone ?
If i got your question right and you want to get meter_id and delta as label and data you can get meter_id and delta fields only from your collection then use collection's map helper to map these fields to your new field names like so
Model::get(['meter_id', 'delta'])->map(function (Model $model) {
return [
'label' => $model->meter_id,
'data' => $model->delta,
];
})->toArray()
I'm trying to return records that fall under a specific range but it seems that the wrong records are returned, this issue does not occur when I opt to use the $lt condition alone.
Below is the returned json
{
"_id": {
"$oid": "5a5528c471872a00ee77731d"
},
"histories": [
{
"created": "2017-09-21T08:24:18.716+0100",
"items": [
{
"field": "status",
"fieldtype": "jira",
"fieldId": "status",
"from": "10007",
"fromString": "With Testing",
"to": "10012",
"toString": "Completed"
}
]
},
{
"created": "2017-10-27T15:10:58.179+0100",
"items": [
{
"field": "status",
"fieldtype": "jira",
"fieldId": "status",
"from": "10007",
"fromString": "With Testing",
"to": "10012",
"toString": "Completed"
}
]
}
]
}
Below is the aggregate pipeline being used with commented out attempts.
'pipeline' => [
[
'$match' => [
'histories.created' => [
'$gte' => '2017-09-28T00:00:00Z+0100',
'$lt' => '2017-10-04T00:00:00Z+0100'
//'$gte' => '2017-09-28T00:00:00Z',
//'$lt' => '2017-10-04T00:00:00Z'
//'$gte' => '2017-09-28',
//'$lt' => '2017-10-04'
]
]
]
]
The mongo collection I'm querying has no histories.created with a value between 2017-09-28 to 2017-10-04 so I should an empty json from the above query instead of one record
{}
I'm not sure what I'm doing wrong, I tried converting the date strings in to UTCDateTime objects but I still get the same results
I'm using mongodb with php
Sample records in the collection,
(doc 1)
[{
"_id": ObjectId("567941aaf0058ed6755ab3dc"),
"hash_count": NumberInt(7),
"time": [
NumberInt(1450787170),
NumberInt(1450787292),
NumberInt(1450787307),
NumberInt(1450787333),
NumberInt(1450787615)
],
"word": "batman"
},
(doc 2)
{
"_id": ObjectId("567941aaf0058ed6755ab3dc"),
"hash_count": NumberInt(7),
"time": [
NumberInt(1450787170),
NumberInt(1450787292),
NumberInt(1450787307),
NumberInt(1450787333),
NumberInt(1450787354),
NumberInt(1450787526),
NumberInt(1450787615)
],
"word": "apple"
}]
Have stored using PHP,
I want to find the number of records in between time (1450787307) and (1450787615)
Answer:
apple=5
batman=3
What should be query for it?
I ran this command
{
aggregate : "hashtags",
pipeline:
[
{$match:{"time":{$gte:NumberInt(1450787307), $lte:NumberInt(1450787615)}}},
{$unwind:"$time"},
{$match:{"time":{$gte:NumberInt(1450787307), $lte:NumberInt(1450787615)}}},
{$group:{"_id":"$word","count":{$sum:1}}}
]
}
which gave this result
Response from server:
{
"result": [
],
"ok": 1
}
Since you are stuck with an older version of mongoDB, you cannot leverage the power of the array aggregation operators introduced in 3.2.
You would have to aggregate as below:
db.collection.aggregate([
{$match:{"time":{$gte:NumberInt(1450787307), $lte:NumberInt(1450787615)}}},
{$unwind:"$time"},
{$match:{"time":{$gte:NumberInt(1450787307), $lte:NumberInt(1450787615)}}},
{$group:{"_id":"$word","count":{$sum:1}}}
])
translated to PHP,
$result = $c->aggregate([
[ '$match' => [ 'time' => [ '$gte' => NumberInt(1450787307),
'$lte' => NumberInt(1450787615) ] ] ],
[ '$unwind' => '$time' ],
[ '$match' => [ 'time' => [ '$gte' => NumberInt(1450787307),
'$lte' => NumberInt(1450787615) ] ] ],
[ '$group' => [ '_id' => '$word', 'count' => [ '$sum' => 1 ] ] ]
]);
In version 3.2, you could use the combination of $filter and $size to acheive the same result and with less expensive operations.
db.collection.aggregate([
{$match:{"time":{$gte:NumberInt(1450787307),
$lte:NumberInt(1450787615)}}},
{$project:{"_id":0,"word":1,
"count":{$size:{$filter:
{"input":"$time",
"as":"t",
"cond":{$and:[
{$gte:["$$t",NumberInt(1450787307)]},
{$lte:["$$t",NumberInt(1450787615)]}]}
}
}
}
}}
])
ok , after trying a lot I have come with this answer and is correct
for
1450787615- lower limit
1450855155- upper limit
db.hashtags.aggregate([
{
"$match": {
"time": {
"$gte": 1450787615, "$lte": 1450855155
}
}
},
{ "$unwind": "$time" },
{
"$match": {
"time": {
"$gte": 1450787615, "$lte": 1450855155
}
}
},
{
"$group": {
"_id": "$word",
"count": {
"$sum": 1
}
}
}
])
answer is like
{
"result" : [
{
"_id" : "batman",
"count" : 3
},
{
"_id" : "dear",
"count" : 1
},
{
"_id" : "ghost",
"count" : 1
}
],
"ok" : 1
}
db.collection.find({time:{$gt: 1450787307, $lt: 1450787615}});
This will first give you a cursor of all docs that fit within your given time range. Once you have that you can iterate through the cursor and print out the name as well as some loop logic to find the number of occurences for each one. I've only lightly worked with mongodb so there may be a more efficient way to do this.
reference:
https://docs.mongodb.org/v3.0/reference/method/db.collection.find/
I Have been using Elastic Search for a project, but I find the result of Snowball Analyzer a bit strange.
Below is my example of Mapping used.
$myTypeMapping = array(
'_source' => array(
'enabled' => true
),
'properties' => array(
'id' => array(
'type' => 'integer',
'index' => 'not_analyzed'
),
'name' => array(
'type' => 'string',
'analyzer' => 'snowball',
'boost' => 2.0
),
'food_types' => array(
'type' => 'string',
'analyzer' => 'keyword'
),
'location' => array(
'type' => 'geo_point',
"geohash_precision"=> 4
),
'city' => array(
'type' => 'string',
'analyzer' => 'keyword'
)
)
);
$indexParams['body']['mappings']['online_pizza'] = $myTypeMapping;
// Create the index
$elastic_client->indices()->create($indexParams);
On quering the http://localhost:9200/online_pizza/online_pizza/_mapping I get the following results,
{
"online_pizza": {
"properties": {
"city": {
"type": "string",
"analyzer": "keyword"
},
"food_types": {
"type": "string",
"analyzer": "keyword"
},
"id": {
"type": "integer"
},
"location": {
"type": "geo_point",
"geohash_precision": 4
},
"name": {
"type": "string",
"boost": 2,
"analyzer": "snowball"
}
}
}
}
My Question is, I have data, which has Name field as "Milano". On querying for "Milano" I get the desired result, but if I query for "Milan" or "Mil" I get no result found.
{
"query": {
"query_string": {
"default_field": "name",
"query": "Milan"
}
}
}
I've also tried to snowball analyzer during querying, no help.
{
"query": {
"query_string": {
"default_field": "name",
"query": "Milan",
"analyzer": "snowball"
}
}
}
Second Question is Keyword Search is case sensitive, eg, Pizza != pizza, how do i get away with this ?
Thanks,
The snowball stemmer doesn't want exact words. If you try it with jumping, it outputs jump as expected.
However, depending on the case, you word may be understemmed as it doesn't match any stemmer rule.
If you use the analyze API endpoint (more info here), you will see that analyzing Milano with snowball analyzer gives you the token milano :
GET _analyze?analyzer=snowball&text=Milano
Output :
{
"tokens": [
{
"token": "milano",
"start_offset": 0,
"end_offset": 6,
"type": "<ALPHANUM>",
"position": 1
}
]
}
Then, using same snowball analyzer on Mil like this :
GET _analyze?analyzer=snowball&text=Mil
gives you this token :
{
"tokens": [
{
"token": "mil",
"start_offset": 0,
"end_offset": 3,
"type": "<ALPHANUM>",
"position": 1
}
]
}
That's why searching for 'milan' or 'mil' won't match 'Milano' documents : it doesn't match the milano term stored in index.
For your second question, you can prepare a custom analyzer combining keyword tokenizer and a lowercase tokenfilter in order to have your keyword search case-insensitive (if you use the same analyzer at search time) :
POST index_name
{
"analysis": {
"analyzer": {
"case_insensitive_keyword": {
"type": "custom",
"tokenizer": "keyword",
"filter": ["lowercase"]
}
}
}
}
Test :
GET analyse/_analyze?analyzer=case_insensitive_keyword&text=Choo Choo
Output :
{
"tokens": [
{
"token": "choo choo",
"start_offset": 0,
"end_offset": 9,
"type": "word",
"position": 1
}
]
}
I hope I'm clear enough in my explainations :)