I have been reading through elastic search docs over the last few months and have continued to optimize my query, but I can't seem to get a search query below 500-600ms. Locally with less data I can get responses in ~80-200ms.
To outline what I am trying to accomplish:
I have 12 different models in Laravel that are searchable from a single search bar. As someone types it is searched and returned in a list of results.
Currently, I have this for my search query. Are there any references for how I can improve this? I looked into multi_match, but I was having issues with partial matches and specifying all fields.
$results = $this->elastic->search([
'index' => config('scout.elasticsearch.index'),
'type' => $type ?? implode(',', array_keys($this->permissions, true, true)),
'body' => [
'query' => [
'bool' => [
'must' => [
[
'query_string' => [
'query' => "$searchQuery*",
],
],
],
'filter' => [
[
'term' => [
'account_id' => $accountId,
],
],
],
'should' => [
[
'term' => [
'_type' => [
'value' => 'customers',
'boost' => 1.3,
],
],
],
[
'term' => [
'_type' => [
'value' => 'contacts',
'boost' => 1.3,
],
],
],
[
'term' => [
'_type' => [
'value' => 'users',
'boost' => 1.3,
],
],
],
[
'term' => [
'_type' => [
'value' => 'chart_accounts',
'boost' => 1.2,
],
],
],
],
],
],
'from' => $from,
'size' => $size,
],
]);
Related
The query returns this error.
Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"parsing_exception","reason":"unknown query [query]","line":1,"col":62}],"type":"x_content_parse_exception","reason":"[1:62] [bool] failed to parse field [should]","caused_by":{"type":"parsing_exception","reason":"unknown query [query]","line":1,"col":62,"caused_by":{"type":"named_object_not_found_exception","reason":"[1:62] unknown field [query]"}}},"status":400} in
How can i write this query ? . In array first one is (manufacturer_code,ean) parent, other objects are nested.
$query = [
'bool' => [
'should' => [
['query' => [ //parent root
'query_string' => [
'fields' => ['manufacturer_code', 'ean'],
'query' => $search,
],
],
],
['nested' => [ //nested parent.lang
'path' => 'lang',
'query' => [
'query_string' => [
'fields' => 'lang.name',
'query' => $search ,
],
],
]],
['nested' => [ //nested parent.product_base.lang
'path' => 'product_base.lang',
'query' => [
'query_string' => [
'fields' => 'product_base.lang.meta_keywords',
'query' => $search ,
],
],
]],
],
'minimum_should_match' => 1,
],
];
You're almost there, in the first should clause you need to remove the first query, i.e. query_string should be at top level
$query = [
'bool' => [
'should' => [
['query_string' => [ //parent root <-- modify this clause
'fields' => ['manufacturer_code', 'ean'],
'query' => $search,
],
],
['nested' => [ //nested parent.lang
'path' => 'lang',
'query' => [
'query_string' => [
'fields' => 'lang.name',
'query' => $search ,
],
],
]],
['nested' => [ //nested parent.product_base.lang
'path' => 'product_base.lang',
'query' => [
'query_string' => [
'fields' => 'product_base.lang.meta_keywords',
'query' => $search ,
],
],
]],
],
'minimum_should_match' => 1,
],
];
I'm trying to give a negative boost to push results down in the ranking if they have 'b-stock' in the title.
Here is my code:
'body' => [
'size' => 15,
'query' => [
'boosting' => [
'positive' =>[
'bool' => [
'should' => [
['query_string' => [
'default_field' => 'title_tag',
'query' => $term
]],
['query_string' => [
'default_field' => 'name',
'query' => $term
]],
['query_string' => [
'default_field' => 'description',
'query' => $term
]],
]
],
],
'negative' => [
'term' => [
'name' => 'B-Stock'
]
],
'negative_boost' => 2
]
]
]
However this seems to have no affect on the results even if I remove the term array from the 'negative' array the same results set is returned.
I am new to Elasticsearch and I am using REST API for PHP to play around with data returned. I am using following code to retrieve data.
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
Above code returns data perfectly matching "validated=>true" and "image=>true".
Further I want to add open text search like we use /_search/?q=Apple macbook. I have tried to use match, multi_match, query_string options, but couldn't get success.
So, I want to retrieve results from ES that have "validated=>true", "image=>true" and matches with text "Apple macbook".
Thanks in advance.
You can try with query_string or simple_query_string
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ],
[ 'query_string' => [ 'query' => 'Apple macbook' ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ],
[ 'simple_query_string' => [ 'query' => 'Apple macbook' ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
you can also do this by
enabling all_field mapping for your index, you can do that by following the below URL
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-all-field.html
and then use the below ES query:
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ '_all' => 'Apple macbook' ] ],
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
Trying to query in Elasticsearch w/ the PHP client and give priority to partial words matches but still include fuzzy matches. If I remove the address.company match block, the query works as expected, but is broken with it present no matter how I seem to frame it. I am lost on the formatting to also include the fuzzy searches with a lower priority?
$search_data = [
"from" => (int) $start, "size" => (int) $count,
'query' => [
'bool' => [
'filter' => [
['term' => ['active' => 1]],
['term' => ['type' => 2]],
],
'must' => [
'wildcard' => [
'address.company' => '*' . $search_query . '*'
],
'match' => [
'address.company' => [
'query' => $search_query,
'operator' => 'and',
'fuzziness' => 'AUTO',
],
],
],
],
],
];
While I am still new to ES as likely is apparent, this solution seems to get the data I'm after. I only mention that because there may be a more ideal way if someone views this in the future. Switching from must to should and wrapping the arrays a bit differently did the trick.
$search_data = [
"from" => (int) $start, "size" => (int) $count,
'query' => [
'bool' => [
'filter' => [
['term' => ['active' => 1]],
['term' => ['type' => 2]],
],
'should' => [
[
'match' => ['address.company' => ['query'=>$search_query,'boost'=>10]],
],
[
'match' =>
[
'address.company' =>
[
'query' => $search_query,
'fuzziness' => 'AUTO',
],
],
],
],
'minimum_should_match'=>1,
],
],
];
I'm trying to learn how to use elasticsearch (using elasticsearch-php for queries). I have inserted a few data, which look something like this:
['id' => 1, 'name' => 'butter', 'category' => 'food'],
['id' => 2,'name' => 'buttercup', 'category' => 'food'],
['id' => 3,'name' => 'something else', 'category' => 'butter']
Now I created a search query which looks like this:
$query = [
'filtered' => [
'query' => [
'bool' => [
'should' => [
['match' => [
'name' => [
'query' => $val,
'boost' => 7
]
]],
['match' => [
'category' => [
'query' => $val,
'boost' => 5
]
]],
],
]
]
]
];
where $val is the search term. This works nicely, the only problem I have: when I search for "butter", I find ids 1 and 3, but not 2, because the searchterm seems to match exact words only. Is there a way to search "within words", or, in mysql terms, to do something like WHERE name LIKE '%val%' ?
You can try the wildcard query
$query = [
'filtered' => [
'query' => [
'bool' => [
'should' => [
['wildcard' => [
'name' => [
'query' => '*'.$val.'*',
'boost' => 7
]
]],
['wildcard' => [
'category' => [
'query' => '*'.$val.'*',
'boost' => 5
]
]],
],
]
]
]
];
or the query_string query.
$query = [
'filtered' => [
'query' => [
'bool' => [
'should' => [
['query_string' => [
'default_field' => 'name',
'query' => '*'.$val.'*',
'boost' => 7
]],
['query_string' => [
'default_field' => 'category',
'query' => '*'.$val.'*',
'boost' => 7
]],
],
]
]
]
];
Both will work but are not really performant if you have lots of data.
The correct way of doing this is to use a custom analyzer with a standard tokenizer and an ngram token filter in order to slice and dice each of your tokens into small ones.