I'm having trouble duplicating my MySQL delete query in elastic search, I am using this documentation: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-delete-by-query.html using the PHP wrapper for Laravel.
I'm trying this:
$this->es->deleteByQuery([
'index' => 'users',
'type' => 'user',
'body' => [
'query' => [
'term' => ['field1' => $this->field1],
'term' => ['field2' => $this->field2],
'term' => ['temp' => 0]
]
]
]);
Its suppose to be a DELETE FROM users WHERE field1 = $this->field1 AND field2 = $this->field2...
I'm having trouble translating the WHERE AND syntax to Elastic Search.
Any help?
Your second comment was mostly correct:
I think I have have gone overboard right now. I have body => query =>
filter => filtered => bool => must => term, term, term. Do I need the
filter => filtered arrays?
The bool filter is preferable over the bool query, since filtering is often much faster than querying. In your case, you are simply filtering documents that have the various terms, and don't want them contributing to the score, so filtering is the correct approach.
This should be done though the query clause, however, since the top-level filter clause is used for a different purpose (filtering facets/aggregations...it was in-fact renamed to post_filter in 1.0 to signify that it is a "post filtering" operation).
Your query should look something like this:
$this->es->deleteByQuery([
'index' => 'users',
'type' => 'user',
'body' => [
'query' => [
'filtered' => [
'filter' => [
'must' => [
['term' => ['field1' => $this->field1]],
['term' => ['field2' => $this->field2]],
['term' => ['temp' => 0]]
]
]
]
]
]
]);
Related
$params = [
'from' => $from,
'size' => config('app.pagination'),
'index' => $index,
//'type' => $this->type,
'body' => [
'query' => [
'bool' => [
'filter' => [
'term' => $where
],
'must' => [
'multi_match' => [
'query' => $match,
'fields'=>$fields,
'fuzziness' => "AUTO:1,5",
]
]
],
]
]
];
Hello, I don't have a problem with my query above.
For example, I am looking for a pizza. I am writing pizaz again.
These queries return the correct records to me.
But I have a problem.
It doesn't return anything when you type piz.
How can I solve this problem? I want it to work when I write it missing.
Add minimum_should_match parameter to your query.
Using minimum_should_match sets a threshold (absolute number, percentage, or combination of these) for matching clauses in boolean queries.
As you know from docs
If the bool query includes at least one should clause and no must or
filter clauses, the default value is 1. Otherwise, the default value
is 0.
I am currently developing a skill for Amazon's echo dot which requires the use of persistent data. I ran into an issue when developing a web interface for my skill where I was not able to easily update the mapAttr column of the DynamoDB table used by the skill.
I've been trying to work this out for the last 2 days, I've looked everywhere including the documentation but can't seem to find anything that'll help me.
This is the code I am using:
$result = $client->updateItem([
'TableName' => 'rememberThisDBNemo',
'Key' => [
'userId' => [ 'S' => $_SESSION['userDataAsk'] ]
],
'ExpressionAttributeNames' => [
'#attr' => 'mapAttr.ReminderJSON'
],
'ExpressionAttributeValues' => [
':val1' => json_encode($value)
],
'UpdateExpression' => 'SET #attr = :val1'
]);
I have tried many different things so this might be just absolutely wrong, but nothing that I have found has worked.
The table has 2 columns, userId and mapAttr, userId is a string and mapAttr is a map. Originally I thought it was simply a JSON string but it was not like that as when I tried to update it with a JSON string directly it would stop working when read by Alexa.
I am only trying to update 1 out of the 2 attributes of mapAttr. That is ReminderJSON which is a string.
Any help would be appreciated. Thanks.
Try calling updateItem like this
$result = $client->updateItem([
'TableName' => 'rememberThisDBNemo',
'Key' => [
'userId' => [ 'S' => $_SESSION['userDataAsk'] ]
],
'ExpressionAttributeNames' => [
'#mapAttr' => 'mapAttr',
'#attr' => 'ReminderJSON'
],
'ExpressionAttributeValues' => [
':val1' => ['S' => json_encode($value)]
],
'UpdateExpression' => 'SET #mapAttr.#attr = :val1'
]);
However, please be aware that in order for this to work, attribute mapAttr must already exist. If it doesn't, you'll get ValidationException saying The document path provided in the update expression is invalid for update...
As a workaround, you may want to add a ConditionExpression => 'attribute_exists(mapAttr)' to your params, catch possible exception, and then perform another update adding a new attribute mapAttr:
try {
$result = $client->updateItem([
'TableName' => 'rememberThisDBNemo',
'Key' => [
'userId' => [ 'S' => $_SESSION['userDataAsk'] ]
],
'ExpressionAttributeNames' => [
'#mapAttr' => 'mapAttr'
'#attr' => 'ReminderJSON'
],
'ExpressionAttributeValues' => [
':val1' => ['S' => json_encode($value)]
],
'UpdateExpression' => 'SET #mapAttr.#attr = :val1'
'ConditionExpression' => 'attribute_exists(#mapAttr)'
]);
} catch (\Aws\Exception\AwsException $e) {
if ($e->getAwsErrorCode() == "ConditionalCheckFailedException") {
$result = $client->updateItem([
'TableName' => 'rememberThisDBNemo',
'Key' => [
'userId' => [ 'S' => $_SESSION['userDataAsk'] ]
],
'ExpressionAttributeNames' => [
'#mapAttr' => 'mapAttr'
],
'ExpressionAttributeValues' => [
':mapValue' => ['M' => ['ReminderJSON' => ['S' => json_encode($value)]]]
],
'UpdateExpression' => 'SET #mapAttr = :mapValue'
'ConditionExpression' => 'attribute_not_exists(#mapAttr)'
]);
}
}
I've just downloaded and installed the last version of Elasticsearch on my Windows machine. I did my first search queries and everything seemed to work ok. However. when I try to highlight the search results, I fail. So, this is how my query looks like:
$params = [
'index' => 'test_index',
'type' => 'test_index_type',
'body' => [
'query' => [
'bool' => [
'should' => [ 'match' => [ 'field1' => '23' ] ]
]
],
'highlight' => [
'pre_tags' => "<em>",
'post_tags' => "</em>",
'fields' => (object)Array('field1' => new stdClass),
'require_field_match' => false
]
]
]
$res = $client->search($params);
On the whole the query itself works nice - the results are filtered. In the console I see, that all documents indeed contain "23" value in their field1 field. However, these tags - <em></em> are simply not added to the result. What I see is just the raw value in field1 like "some text 23", "23 another text". It is not what I expect to see - "some text <em>23</em>", "<em>23</em> another text". So, what is wrong with that and how can I fix it?
From the manual:
The value of pre_tags and post_tags should be an array (however if you don't want to change the em tags you can ignore them, they already set as default).
The fields value should be an array, key is the field name and the value is an array with the field options.
Try this fix:
$params = [
'index' => 'test_index',
'type' => 'test_index_type',
'body' => [
'query' => [
'bool' => [
'should' => [ 'match' => [ 'field1' => '23' ] ]
]
],
'highlight' => [
// 'pre_tags' => ["<em>"], // not required
// 'post_tags' => ["</em>"], // not required
'fields' => [
'field1' => new \stdClass()
],
'require_field_match' => false
]
]
];
$res = $client->search($params);
var_dump($res['hits']['hits'][0]['highlight']);
update
Did a double check, the value of the field in the fields array should be an object (which is a requirement, not exactly the same as other options).
The pre/post_tags can also be strings (and not array).
Did you check the correct response? $res['hits']['hits'][0]['highlight']
The important thing to notice is that the highligted results goes into the highlight array - $res['hits']['hits'][0]['highlight'].
Consider this PHP array for defining a search query on elasticsearch
$searchParams = [
'index' => $index_name,
'type' => 'nginx-access',
'size' => 1000,
'sort' => ['_score'],
'fields' => ["remote_addr", "#timestamp", "request", "method", "status"],
'body' => [
// 'min_score' => 4,
'query' => [
'bool' => [
'must' => [
['match' => ['status' => '200']],
['match' => ['method' => 'POST']],
['match' => ['request' => '/wp-login.php']],
],
]
]
]
];
$results = $this->client->search($searchParams);
var_dump($results);
Notice that 'min_score' is commented out. The array of results returns everything as expected, however I only want results that match all of the criteria. I thought this is what query->bool->must is supposed to do, however near the end of the dataset I am getting matches for other seemingly random things as well.
If I run it with min_score then I'm always going to get the results I want, but this does not seem the right way to do it.
Am I missing something obvious here?
I am using Elasticsearch for the first time and have the indexing and basic searching down, but I am looking todo some complex searching.
With the PHP client how do you do partial searches and field boosting / relevance? Ultimately, I want to search multiple fields for partial matches, exact matches, and boost some of the fields.
Here is what I have so far, but I can't get it working. The Elasticsearch documentation is no good.
$show_params = [
'index' => env('ES_INDEX'),
'type' => 'show',
'size' => 6,
'body' => [
'query' => [
'bool' => [
'should' => [
[
'match' => [
'title' => [
'query' => '*' . $q . '*',
'boost' => 2
]
]
],
[
'match' => [
'synopsis' => '*' . $q . '*'
]
]
]
]
]
]
];
$client = \Elasticsearch\ClientBuilder::create()->build();
$show_raw_results = $client->search($show_params);
The basic match query doesn't support wildcards, which is why your queries are not working (the boosting syntax is, however, correct).
You can try using the wildcard query, but it has some limitations (in particular, it is a not_analyzed query, which means your input text needs to be "pre-analyzed"):
$show_params = [
'index' => env('ES_INDEX'),
'type' => 'show',
'size' => 6,
'body' => [
'query' => [
'bool' => [
'should' => [
[
'wildcard' => [
'title' => [
'value' => '*' . $q . '*',
'boost' => 2
]
]
],
[
'wildcard' => [
'synopsis' => '*' . $q . '*'
]
]
]
]
]
]
];
$client = \Elasticsearch\ClientBuilder::create()->build();
$show_raw_results = $client->search($show_params);
However, this isn't necessarily the best approach. Wildcards are slower, and will pull back many documents which may frustrate your users (since there are many low-scoring, less-relevant hits).
And most importantly, wildcards are not_analyzed queries, meaning the query text won't go through analysis. If you search for "quick brown fox", the above query will search your index for "*quick brown fox*" exactly as if it were a single token, instead of breaking it into multiple tokens (["quick","brown","fox"]) and searching with those.
I would highly suggest reading through the section in the Definitive Guide on Partial Matching, or just start at the beginning of the Fulltext Search chapter and work your way through it. You'll need a good understanding of analysis and tokenization to get decent results with partial/fuzzy matching.