I have a field "location_facet", which is a strig with mapping
"location_facet": {
"type": "string",
"index": "not_analyzed",
"include_in_all": true
},
In this field I have location_ids and I would like to aggregate on them by the query. Also I want to become some information by the aggregation, that's why I would like to execute a script, for example I want to become the name of the city, but there is always an error coming:
"aggs": {
"location_radius": {
"terms": {
"field": "location_facet",
"size": 10,
"script": "doc[\"location_name\"].value"
}
}
},
"type": "script_exception",
"reason": "failed to run inline script [doc[\"location_name\"].value] using lang [groovy]"
I have actually this implementation with Facets:
$tagFacet = new \Elastica\Facet\Terms("location_radius");
$tagFacet->setField('location_name');
$tagFacet->setAllTerms(true);
$tagFacet->setSize(100);
$tagFacet->setScript(
"doc['location'].empty ? null : ceil(doc['location'].arcDistanceIn".$unit."(".
$entries->getLocationLatitude().", ".
$entries->getLocationLongitude().")) + '|' + doc['location_name'].value
+ '|' + doc['location_id'].value"
);
What I am doing wrong?
If I test this one:
"aggs": {
"location_radius": {
"terms": {
"field": "location_facet",
"size": 10,
"script": "doc['location'].empty ? null : ceil(doc['location'].arcDistanceInKm(51.2249429, 6.7756524))"
}
}
},
I get the error:
"type": "script_exception",
"reason": "failed to run inline script [doc['location'].empty ? null : ceil(doc['location'].arcDistanceInKm(51.2249429, 6.7756524))] using lang [groovy]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Negative position"
}
Solution: All the doc fields I can use need to be also "index": "not_analyzed".
Related
I've building out a small app that connects to a Quickbooks API via an SDK. The SDK provides batch operations to help reduce the number of API requests needed.
However, I'm hoping to make a large amount of requests (ie: bulk deletes, uploads in the 100s/1000s). I've gotten the deletes to work, however, now I'm hoping to integrate Laravel's Queue system so that any items in the $batch that fail (due to these business-rules or other reasons) are sent to a worker who will reattempt them after waiting a minute .
Below is an example of a delete request.
class QuickBooksAPIController extends Controller
{
public function batchDelete(Request $request, $category)
{
$chunks = array_chunk($request->data, 30);
foreach ($chunks as $key => $value) {
$batch[$key] = $this->dataService()->CreateNewBatch();
foreach ($value as $id) {
$item = $this->dataService()->FindById($category, $id);
$batch[$key]->AddEntity($item, $id, "delete");
}
$batch[$key]->Execute();
}
return response()->json(['message' => 'Items Deleted'], 200);
}
}
The documentations are a bit sparse for my scenario though. How can I get the failed batch items on order to try again?
Is using batches even the right choice here? Because I have to hit the API anyway to get the $item... which doesn't make sense to me (I think I'm doing something wrong there).
EDIT:
I intentionally sent out a request with more then 30 items and this is the failure message. Which doesn't have the values that didn't make the cut.
EDIT#2:
Ended up using array_chunk to separate the payload into 30 items (which is the limit of the API). Doing so helps process many requests. I've adjusted my code above to represent my current code.
How can I get the failed batch items on order to try again?
If you look at Intuit's documentation, you can see that the HTTP response the API returns contains this information. Here's the example request they show:
{
"BatchItemRequest": [
{
"bId": "bid1",
"Vendor": {
"DisplayName": "Smith Family Store"
},
"operation": "create"
},
{
"bId": "bid2",
"operation": "delete",
"Invoice": {
"SyncToken": "0",
"Id": "129"
}
},
{
"SalesReceipt": {
"PrivateNote": "A private note.",
"SyncToken": "0",
"domain": "QBO",
"Id": "11",
"sparse": true
},
"bId": "bid3",
"operation": "update"
},
{
"Query": "select * from SalesReceipt where TotalAmt > '300.00'",
"bId": "bid4"
}
]
}
And the corresponding response:
{
"BatchItemResponse": [
{
"Fault": {
"type": "ValidationFault",
"Error": [
{
"Message": "Duplicate Name Exists Error",
"code": "6240",
"Detail": "The name supplied already exists. : Another customer, vendor or employee is already using this \nname. Please use a different name.",
"element": ""
}
]
},
"bId": "bid1"
},
{
"Fault": {
"type": "ValidationFault",
"Error": [
{
"Message": "Object Not Found",
"code": "610",
"Detail": "Object Not Found : Something you're trying to use has been made inactive. Check the fields with accounts, customers, items, vendors or employees.",
"element": ""
}
]
},
"bId": "bid2"
},
{
"Fault": {
"type": "ValidationFault",
"Error": [
{
"Message": "Stale Object Error",
"code": "5010",
"Detail": "Stale Object Error : You and root were working on this at the same time. root finished before you did, so your work was not saved.",
"element": ""
}
]
},
"bId": "bid3"
},
{
"bId": "bid4",
"QueryResponse": {
"SalesReceipt": [
{
"TxnDate": "2015-08-25",
"domain": "QBO",
"CurrencyRef": {
"name": "United States Dollar",
"value": "USD"
},
"PrintStatus": "NotSet",
"PaymentRefNum": "10264",
"TotalAmt": 337.5,
"Line": [
{
"Description": "Custom Design",
"DetailType": "SalesItemLineDetail",
"SalesItemLineDetail": {
"TaxCodeRef": {
"value": "NON"
},
"Qty": 4.5,
"UnitPrice": 75,
"ItemRef": {
"name": "Design",
"value": "4"
}
},
"LineNum": 1,
"Amount": 337.5,
"Id": "1"
},
{
"DetailType": "SubTotalLineDetail",
"Amount": 337.5,
"SubTotalLineDetail": {}
}
],
"ApplyTaxAfterDiscount": false,
"DocNumber": "1003",
"PrivateNote": "A private note.",
"sparse": false,
"DepositToAccountRef": {
"name": "Checking",
"value": "35"
},
"CustomerMemo": {
"value": "Thank you for your business and have a great day!"
},
"Balance": 0,
"CustomerRef": {
"name": "Dylan Sollfrank",
"value": "6"
},
"TxnTaxDetail": {
"TotalTax": 0
},
"SyncToken": "1",
"PaymentMethodRef": {
"name": "Check",
"value": "2"
},
"EmailStatus": "NotSet",
"BillAddr": {
"Lat": "INVALID",
"Long": "INVALID",
"Id": "49",
"Line1": "Dylan Sollfrank"
},
"MetaData": {
"CreateTime": "2015-08-27T14:59:48-07:00",
"LastUpdatedTime": "2016-04-15T09:01:10-07:00"
},
"CustomField": [
{
"DefinitionId": "1",
"Type": "StringType",
"Name": "Crew #"
}
],
"Id": "11"
}
],
"startPosition": 1,
"maxResults": 1
}
}
],
"time": "2016-04-15T09:01:18.141-07:00"
}
Notice the separate response object for each request.
The bId value is a unique value you send in the request, which is then echo'd back to you in the response, so you can match up the requests you send with the responses you get back.
Here's the docs:
https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/batch#sample-batch-request
Is using batches even the right choice here?
Batches make a lot of sense when you are doing a lot of things all at once.
The way you're trying to use them is... weird. What you should probably be doing is:
Batch 1
- go find all your items
Batch 2
- delete all the items
Your existing code doesn't make sense because you're trying to both find the item and delete the item in the exact same batch HTTP request, which isn't possible via the API.
I intentionally sent out a request with more then 30 items and this is the failure message.
No, it's not. That's a PHP error message - you have an error in your code.
You need to fix the PHP error, and then look at the actual response you're getting back from the API.
I am trying to use geo_point for distance but it always shows location type double not geo_point how can I set location mapped to geo_point .
Actually I have to find all records within 5km sorted.
"pin" : {
"properties" : {
"location" : {
"properties" : {
"lat" : {
"type" : "double"
},
"lon" : {
"type" : "double"
}
}
},
"type" : {
"type" : "string"
}
}
},
and when I am trying searches with this query below to find result within 5km of delhi lat long :
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"pin": {
"distance": "5",
"distance_unit": "km",
"coordinate": {
"lat": 28.5402707,
"lon": 77.2289643
}
}
}
}
}
}
It shows me error query_parsing_exception and No query registered for [pin]. I cannot figure out the problem. It always throws this exception
{
"error": {
"root_cause": [
{
"type": "query_parsing_exception",
"reason": "No query registered for [pin]",
"index": "find_index",
"line": 1,
"col": 58
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "find_index",
"node": "DtpkEdLCSZCr8r2bTd8p5w",
"reason": {
"type": "query_parsing_exception",
"reason": "No query registered for [pin]",
"index": "find_index",
"line": 1,
"col": 58
}
}
]
},
"status": 400
}
Please help me to figure out this problem. how can I set geo_point and solve this exception error and status 400 and all_shards_failed error
You simply need to map your pin type like this, i.e. using the geo_point data type:
# 1. first delete your index
DELETE shouut_find_index
# 2. create a new one with the proper mapping
PUT shouut_find_index
{
"mappings": {
"search_shouut": {
"properties": {
"location": {
"type": "geo_point"
},
"type": {
"type": "string"
}
}
}
}
}
Then you can index a document like this
PUT shouut_find_index/search_shouut/1
{
"location": {
"lat": 28.5402707,
"lon": 77.2289643
},
"type": "dummy"
}
And finally your query can look like this
POST shouut_find_index/search_shouut/_search
{
"query": {
"filtered": {
"filter": {
"geo_distance": {
"distance": "5km",
"location": {
"lat": 28.5402707,
"lon": 77.2289643
}
}
}
}
}
}
I am having problem with sorting in PHP, here is my mapping:
{
"jdbc": {
"mappings": {
"jdbc": {
"properties": {
"admitted_date": {
"type": "date",
"format": "dateOptionalTime"
},
"et_tax": {
"type": "string"
},
"jt_tax": {
"type": "string"
},
"loc_cityname": {
"type": "string"
},
"location_countryname": {
"type": "string"
},
"location_primary": {
"type": "string"
},
"pd_firstName": {
"type": "string"
}
}
}
}
}
}
When I use order the result by sort, it will order the results with alphanumeric, it will load the results with numeric as first. I need to order the results only starting letter alphabets. Now it orders like this:
http://localhost:9200/jdbc/_search?pretty=true&sort=pd_lawFirmName:asc
BM&A
Gomez-Acebo & Pombo
Addleshaw Goddard
How to order the results like this?
Addleshaw Goddard
BM&A
Gomez-Acebo & Pombo
Here is my query i using for indexing
{
"type" : "jdbc",
"jdbc" : {
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://localhost:3306/dbname",
"user" : "user",
"password" : "pass",
"sql" : "SQL QUERY",
"poll" : "24h",
"strategy" : "simple",
"scale" : 0,
"autocommit" : true,
"bulk_size" : 5000,
"max_bulk_requests" : 30,
"bulk_flush_interval" : "5s",
"fetchsize" : 100,
"max_rows" : 149669,
"max_retries" : 3,
"max_retries_wait" : "10s",
"locale" : "in",
"digesting" : true,
"mappings": {
"sorting": {
"properties": {
"pd_lawFirmName": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
}
This is like that because Elasticsearch will tokenize the text using the default analyzer, which is standard. For example, McDermott Will Amery is indexed like:
"amery",
"mcdermott",
"will"
If you want to sort like that, I would suggest to change the mapping of your pd_lawFirmName in something like this:
"pd_lawFirmName": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
and sort by the raw subfield:
http://localhost:9200/jdbc/_search?pretty=true&sort=pd_lawFirmName.raw:asc
this is my first question here, I am having a problem with a simple elasticsearch query made throught the php sdk, json example:
{
"_id": "event:5569fbbdddc85",
"_type": "event",
"videos": {},
"status": "published",
"owner": {
"firstname": "Patricio",
"lastname": "",
"profilepicture": "http://pbs.twimg.com/profile_images/581193413426544640/Q5aqMmPk_normal.jpg",
"_id": "twitter:2383339241",
"_type": "user",
"updated": 1433008088365,
"created": 1428439794713
},
"max_age": "18",
"min_age": "18",
"max_invites": "5",
"min_invites": "2",
"updated": 1433009134942,
"created": 1433009134942
}
What I need to do is a filter by owner._id and I am doing this:
$params['index'] = 'default';
$params['type'] = 'event';
$params['size'] = $limit;
$params['from'] = $from;
$params['body']['query']['match']['owner._id'] = $userId;
// elasticsearch search query
$res = \Es::search($params);
the result is no filter. All the events in database are comming back.
I am following exactly the docs, but with no results, obviously I am missing something
Thanks!
You need your _id field to be not_analyzed or analyzed with the keyword analyzer so that, when indexed by ES, to stay unchanged.
Also, for a query like yours, for _id it is best to use a filter of type term. I am no php developer, but from ES point of view it should look like this:
"_id": {
"type": "string",
"index": "not_analyzed"
}
And the query should be of this form, for _id:
"query": {
"filtered": {
"filter": {
"term": {
"owner._id": "twitter:2383339242"
}
}
}
}
Following is my elastic search query, I am using function_score to order my result in following manner:
1. First query should match to some fields (Fulltext search)
2. order result as per user current location (using gauss function)
3. give more weight to those service providers who has some recommendations (using gauss function)
4. give more preference to those service providers which has been recently reviewed (using script score)
Now (2,3,4) point ordering will be done on resulting set, but problem is whenever i am using geo location function exact matched service provider reordered and down in the listing and my query shows result those are near to users location irrespective its less matching to other documents.
Following is my query, please help me resolve this issue. Please also suggest to optimize my this scenario what is the best way to solve this issue.
{
"from": 0,
"size": 15,
"sort": {
"_score": {
"order": "desc"
}
},
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"term": {
"status": "1"
}
},
{
"query_string": {
"default_field": "_all",
"query": "Parag Gadhia Parenting classes",
"fields": [
"service_prrovider_name^3",
"location^2",
"category_name^5",
"keyword"
],
"use_dis_max": true
}
},
{
"term": {
"city_id": "1"
}
}
],
"should": [
{
"term": {
"category.category_name": "parenting classes"
}
}
]
}
},
"functions": [
{
"gauss": {
"geo_location": {
"origin": {
"lat": "19.451624199999998",
"lon": "72.7966481"
},
"offset": "20km",
"scale": "3km"
}
}
},
{
"gauss": {
"likes_count": {
"origin": 3,
"offset": "5",
"scale": "20"
}
},
"weight": 2
},
{
"script_score": {
"script": "(0.08 / ((3.16*pow(10,-11)) * abs(1426072330 - doc[\"reviews.created_time\"].value) + 0.05)) + 1.0"
}
}
]
}
}
}
Yes this is "normal", script_score will override the previous score.
You can use _score variable inside script to use it.
(Such as "script": "_score * (0.08 / ((3.16*pow(10,-11)) * abs(1426072330 - doc[\"reviews.created_time\"].value) + 0.05)) + 1.0")