I am using ES for my Laravel app in order to search a table/type.
My users can search a total of 5 columns which means that there can be a total of 31 query combinations.
So my question is now if I can use the same query but dont provide ES with all the seach params. Or somehow write dynamic queries.
Eg:
'filtered' => [
'query' => [
'match' => ['title' => Input::get('query')]
],
'filter'=> [
'bool' => [
'must' => [
['term' => [ 'type' => 1] ],
['term' => [ 'state' => 22] ],
['term' => [ 'city' => ] ], (empty)
[
'range' => [
'price' => [
'gte' => , (empty)
'lte' => , (empty)
]
]
]
]
]
],
],
Otherwise I have to write 31 different combinations of this query - If ES dont have anything that can help me. And I can use Laravels eloquent ORM for this.
Thanks in advance
You can use Elasticquent
Elasticquent makes working with Elasticsearch and Eloquent models easier by mapping them to Elasticsearch types. You can use the default settings or define how Elasticsearch should index and search your Eloquent models right in the model.
Elasticquent uses the official Elasticsearch PHP API. To get started, you should have a basic knowledge of how Elasticsearch works (indexes, types, mappings, etc).
https://github.com/adamfairholm/Elasticquent
Related
I used mongodb db.version() 5.0.9 with PHP 7.4.30
I worked with queue based on mongo db and in architecure present possibility to scaling, so could be several workers. My question - how I can protect my record from the concurence query, when two workers try to execute findOneAndUpdate in the same time ? Or maybe findOneAndUpdate has this opportunity by default ?
$document = $this->collection->findOneAndUpdate(
[
'$and' => [
[
'job_type' => 'metrics'
],
$conditionDelivered,
[
'id.pub_key_digest' => [
'$exists' => true
]
],
[
'timestamp' => [
'$gt' => $timestampOffset
]
],
],
],
[
'$set' => $setSection,
],
[
'sort' => [
'timestamp' => 1,
],
'writeConcern' => new WriteConcern(WriteConcern::MAJORITY),
'returnDocument' => FindOneAndUpdate::RETURN_DOCUMENT_AFTER
]
);
$filter
{"$and":[{"job_type":"metrics"},{"alert_delivered_at":{"$exists":false},"timescale_delivered_at":{"$exists":false}},{"id.pub_key_digest":{"$exists":true}},{"timestamp":{"$gt":1291852800000000000}}]}
$update
{"$set":{"alert_consumer_id":"consumer_62b2d4e4520339.76461865","alert_delivered_at":{"$date":{"$numberLong":"1655887076643"}},"timescale_delivered_at":{"$date":{"$numberLong":"1655887076643"}}}}
and $options
{"sort":{"timestamp":1},"writeConcern":{"w":"majority"},"returnDocument":2}
Could you explain please best way for find and update data with guarantee which will be executed by only one worker, some lock, the other workers will be work with another records ?
I am building an E-commerce project with demo database of products, brands and vendors.I have implemented auto suggest in search bar,it works.
what I am unable to do is fix for spelling errors. For eg-if a user search for Motorola as motoroal no results is given.
Can anybody point me to right direction?
That's not a easy task.
I use the following setup:
'analysis' => [
'filter' => [
'nGram_filter' => [
'type' => 'edgeNGram',
'min_gram' => '1',
'max_gram' => '20',
'token_chars' => [
'letter',
'digit',
'punctuation',
'symbol',
],
],
],
'analyzer' => [
'nGram_analyzer' => [
'type' => 'custom',
'tokenizer' => 'whitespace',
'filter' => [
'lowercase',
'asciifolding',
'nGram_filter',
],
],
]
]
Then I set my fields map to use the nGram_analyzer as analyzer.
The edgeNGram tokenizer will break your words in this way:
motorola will be break into: m mo mot moto motor motoro motorol motorola
This setup will act like an auto complete, predicting what your user is trying to search.
To help with spelling errors, you will need to apply some similar analyzer to the search_analyzer on your field mapping.
I am using Elasticsearch for the first time and have the indexing and basic searching down, but I am looking todo some complex searching.
With the PHP client how do you do partial searches and field boosting / relevance? Ultimately, I want to search multiple fields for partial matches, exact matches, and boost some of the fields.
Here is what I have so far, but I can't get it working. The Elasticsearch documentation is no good.
$show_params = [
'index' => env('ES_INDEX'),
'type' => 'show',
'size' => 6,
'body' => [
'query' => [
'bool' => [
'should' => [
[
'match' => [
'title' => [
'query' => '*' . $q . '*',
'boost' => 2
]
]
],
[
'match' => [
'synopsis' => '*' . $q . '*'
]
]
]
]
]
]
];
$client = \Elasticsearch\ClientBuilder::create()->build();
$show_raw_results = $client->search($show_params);
The basic match query doesn't support wildcards, which is why your queries are not working (the boosting syntax is, however, correct).
You can try using the wildcard query, but it has some limitations (in particular, it is a not_analyzed query, which means your input text needs to be "pre-analyzed"):
$show_params = [
'index' => env('ES_INDEX'),
'type' => 'show',
'size' => 6,
'body' => [
'query' => [
'bool' => [
'should' => [
[
'wildcard' => [
'title' => [
'value' => '*' . $q . '*',
'boost' => 2
]
]
],
[
'wildcard' => [
'synopsis' => '*' . $q . '*'
]
]
]
]
]
]
];
$client = \Elasticsearch\ClientBuilder::create()->build();
$show_raw_results = $client->search($show_params);
However, this isn't necessarily the best approach. Wildcards are slower, and will pull back many documents which may frustrate your users (since there are many low-scoring, less-relevant hits).
And most importantly, wildcards are not_analyzed queries, meaning the query text won't go through analysis. If you search for "quick brown fox", the above query will search your index for "*quick brown fox*" exactly as if it were a single token, instead of breaking it into multiple tokens (["quick","brown","fox"]) and searching with those.
I would highly suggest reading through the section in the Definitive Guide on Partial Matching, or just start at the beginning of the Fulltext Search chapter and work your way through it. You'll need a good understanding of analysis and tokenization to get decent results with partial/fuzzy matching.
I am currently building a second system with yii2. It will use some tables from a Yii1 database for translation. The Yii1 project was originally translated using files but this has now been moved to the db. All the translations of the Yii1 system use one of three categories app,flash,email of which the vast majority use app.
In the Yii2 project I have the following in the web.php config file.
'i18n' => [
'translations' => [
'*' => [
'class' => 'yii\i18n\DbMessageSource',
'db' => 'cdb',
'sourceMessageTable' => 'translation_source',
'messageTable' => 'translation',
'forceTranslation'=>true,
],
],
],
All of the translations on the system that use app are not translated, however, all other categories are. If I change the above code to
'i18n' => [
'translations' => [
'app*' => [
Then I get an error for other categories but not app and the app strings are translated as expected. The error I get is
Unable to locate message source for category 'flash'.
If however I change my config to the following, this works for all translations.
'i18n' => [
'translations' => [
'*' => [
'class' => 'yii\i18n\DbMessageSource',
'db' => 'cdb',
'sourceMessageTable' => 'translation_source',
'messageTable' => 'translation',
'forceTranslation' => true,
],
'app*' => [
'class' => 'yii\i18n\DbMessageSource',
'db' => 'cdb',
'sourceMessageTable'=>'translation_source',
'messageTable' => 'translation',
'forceTranslation' => true,
],
],
],
It just seems odd that I am having to include effectively the same code block twice. Can anyone tell me how I can achieve this in one array or is this the expected behaviour?
** Update **
I do see some mention of * in the docs Docs. I also see some mention of this in the Forum
I'm having trouble duplicating my MySQL delete query in elastic search, I am using this documentation: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-delete-by-query.html using the PHP wrapper for Laravel.
I'm trying this:
$this->es->deleteByQuery([
'index' => 'users',
'type' => 'user',
'body' => [
'query' => [
'term' => ['field1' => $this->field1],
'term' => ['field2' => $this->field2],
'term' => ['temp' => 0]
]
]
]);
Its suppose to be a DELETE FROM users WHERE field1 = $this->field1 AND field2 = $this->field2...
I'm having trouble translating the WHERE AND syntax to Elastic Search.
Any help?
Your second comment was mostly correct:
I think I have have gone overboard right now. I have body => query =>
filter => filtered => bool => must => term, term, term. Do I need the
filter => filtered arrays?
The bool filter is preferable over the bool query, since filtering is often much faster than querying. In your case, you are simply filtering documents that have the various terms, and don't want them contributing to the score, so filtering is the correct approach.
This should be done though the query clause, however, since the top-level filter clause is used for a different purpose (filtering facets/aggregations...it was in-fact renamed to post_filter in 1.0 to signify that it is a "post filtering" operation).
Your query should look something like this:
$this->es->deleteByQuery([
'index' => 'users',
'type' => 'user',
'body' => [
'query' => [
'filtered' => [
'filter' => [
'must' => [
['term' => ['field1' => $this->field1]],
['term' => ['field2' => $this->field2]],
['term' => ['temp' => 0]]
]
]
]
]
]
]);