PhpUnit testing elasticsearch results are empty - php

I am writing testing methods of my app and in my app I use elasticsearch. When I run a test method which should return values using elasticsearch, the response is always empty. How can I solve the problem? Here is the code I send.
public function testGetPosts()
{
$brand = factory(Brand::class)->create();
$account = factory(Account::class)->create();
$post = factory(Post::class)->create();
$response = $this->actingAs($this->owner)->json(
'GET',
('/api/publish/posts'),
['account_id' => [(string) $account->id],
'skip' => 0]
);
$response->assertStatus(200);
}

I know this post is old, but I add there the answer I found for this problem.
All you need to ensure your data is indexed before querying is to call a refresh on index you just wrote on.
It forces ES to index data, so you are sure data is there when you query it!
And it is faster than the sleep(1); as suggested by author =)
You can find the official ElasticSearch documentation about it here.
Hope this will help someone.

Almost a year, later, I'm sure by now you've moved on.
You stated:
Elastic search doesnt index the created post. It shuold be indexed
Why would it be indexed? Unless, of course you have code to index in your setUp(), or your testing against an external ES server and assuming it's always available and contains the exact data you're testing against.
Another solution is to mock the request, since Elasticsearch returns JSON. All we need to do is mock a HTTP request that has a status of 200, and returns JSON. This JSON file we can place in our tests/ directory, and it will contain the sample results that Elasticsearch would return.
An example test would like this;
$handler = new MockHandler([
'status' => 200,
'transfer_stats' => [
'total_time' => 100
],
'body' => fopen(base_path('tests/Unit/mockelasticsearch.json'), 'r')
]);
$builder = ClientBuilder::create();
$builder->setHosts(['testing']);
$builder->setHandler($handler);
$client = $builder->build();
$response = $client->search([
'index' => 'my_index',
'type' => 'my_type',
'body' => [
[
'query' => [
'simple_query_string' => [
'query' => 'john',
'fields' => ['name']
]
]
]
]
]);
// Test against the "$response", i.e., $this->assertEquals(2 ...) etc.
Then in the JSON file, which you would need to customize based on your index;
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 121668,
"max_score": 1,
"hits": [
{
"_index": "test",
"_type": "test-type",
"_id": "1111",
"_score": 1,
"_source": {
"id": "1111",
"title": "Some Foo",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "2222",
"_score": 1,
"_source": {
"id": "2222",
"title": "Dolor Sit Amet",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "3333",
"_score": 1,
"_source": {
"id": "3333",
"title": "Consectetur Adipiscing Elit",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "4444",
"_score": 1,
"_source": {
"id": "4444",
"title": "Sed Do Eiusmod",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "5555",
"_score": 1,
"_source": {
"id": "5555",
"title": "Tempor Incididunt",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "6666",
"_score": 1,
"_source": {
"id": "6666",
"title": "Ut Labore Et Dolore",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "7777",
"_score": 1,
"_source": {
"id": "7777",
"title": "Magna Aliqua",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "8888",
"_score": 1,
"_source": {
"id": "8888",
"title": "Ut Enim Ad Minim",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "9999",
"_score": 1,
"_source": {
"id": "9999",
"title": "Veniam, Quis Nostrud",
"timestamp": "2017-08-02T15:45:22-05:00"
}
},
{
"_index": "test",
"_type": "test-type",
"_id": "0000",
"_score": 1,
"_source": {
"id": "0000",
"title": "Exercitation Ullamco Laboris",
"timestamp": "2017-08-02T15:45:22-05:00"
}
}
]
}
}

Related

Elasticsearch get data using search in PHP

I am trying to get data from elastic search in PHP.
Following is my elastic type (curated) structure :
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "qwerty",
"_type": "curated",
"_id": "2",
"_score": 1,
"_source": {
"shows_on_lfz": [
{
"sh_id": 14,
"sh_parent_id": 102,
"sh_name": "Veg Versions Of Global Dishes"
}
]
}
},
{
"_index": "qwerty",
"_type": "curated",
"_id": "1",
"_score": 1,
"_source": {
"top_stories": [
{
"ts_id": 515,
"ts_parent_id": 485,
}
]
}
}
]
}
}
How can I get all data of top_stories using search query in PHP?
I am getting data using match_all & get() query but I need data using search().
I tried -
$body['query']['bool']['should'] = ['match' => ['top_stories'=> '']];
But getting a null response.
This is the query that will bring all entries that have the field top_stories
GET qwerty/_search
{
"query": {
"exists": {
"field": "top_stories"
}
}
}
You can convert this to code and you can test it through kibana

how to sort within a functional boost query in elasticsearch

I have an index containing details regarding mobiles, case covers etc.
Below is the query used.
{
"query": {
"function_score": {
"query": { "match": {"title" :"Apple iPhone 6s"} },
"boost": "5",
"functions": [
{
"filter": { "match": { "main_category": "mobiles" } },
"weight": 8
},
{
"filter": { "match": {"main_category": "cases-and-covers" } },
"weight": 6
}
],
"max_boost": 8,
"score_mode": "max",
"boost_mode": "multiply",
"min_score" : 5
}
},
"_source":["title","main_category","selling_price"],
"size" : 1000
}
Is it possible to boost mobiles category like below and sort within the mobiles category by selling price ascending order.
Boost is working fine. How to sort within the specific boost function?
If user searched for apple iphone 6s,I want mobiles category to be boosted and lowest price should comes first and then case and cover category products also in selling price ascending order.
Below are the results needed
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 104,
"max_score": 40.645245,
"hits": [
{
"_index": "test",
"_type": "products",
"_id": "shop_24",
"_score": 40.645245,
"_source": {
"selling_price": 72000,
"main_category": "mobiles",
"title": "Apple iPhone 6s"
}
},
{
"_index": "test",
"_type": "products",
"_id": "shop_20",
"_score": 40.168346,
"_source": {
"selling_price": 82000,
"main_category": "mobiles",
"title": "Apple iPhone 6s Plus"
}
},
{
"_index": "test",
"_type": "products",
"_id": "shop_15",
"_score": 39.365562,
"_source": {
"selling_price": 92000,
"main_category": "mobiles",
"title": "Apple iPhone 6s Plus"
}
},
{
"_index": "test",
"_type": "products",
"_id": "shop_17",
"_score": 39.365562,
"_source": {
"selling_price": 2000,
"main_category": "cases-and-covers",
"title": "Case cover for Apple iPhone 6s"
}
},
{
"_index": "test",
"_type": "products",
"_id": "shop_18",
"_score": 39.365562,
"_source": {
"selling_price": 2300,
"main_category": "cases-and-covers",
"title": "Case cover for Apple iPhone 6s Plus"
}
}
]
}
}
Please help?.
Can you try following query:
{
"query": {
"function_score": {
"query": {
"match_all": {} // Change this to query as per your need
},
"boost": "5",
"functions": [
{
"filter": {
"match": {
"main_category": "mobiles"
}
},
"weight": 50
},
{
"filter": {
"match": {
"main_category": "cases-and-covers"
}
},
"weight": 25
}
]
}
},
"sort": [
{
"_score": {
"order": "desc"
}
},
{
"selling_price": {
"order": "asc"
}
}
]
}
We are providing high weight weight=50 to the documents which have mobile category and low weight weight=25 to the documents which have case-and-cover category.
Finally We are sorting first on the basis of score and then selling price.

Elasticsearch check if fieldA has value x then search fieldB for term xyz

Suppose I have stored bellow data and want to search for term xy in old_value and new_value fields of those documents that their field_name is curriculum_name_en or curriculum_name_pr:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 98,
"max_score": 1,
"hits": [
{
"_index": "my_index",
"_type": "audit_field",
"_id": "57526c197e83c",
"_score": 1,
"_source": {
"session_id": 119,
"trans_seq_no": 1,
"table_seq_no": 1,
"field_id": 2,
"field_name": "curriculum_id",
"new_value": 118,
"old_value": null
}
},
{
"_index": "my_index",
"_type": "audit_field",
"_id": "57526c197f2c3",
"_score": 1,
"_source": {
"session_id": 119,
"trans_seq_no": 1,
"table_seq_no": 1,
"field_id": 3,
"field_name": "curriculum_name_en",
"new_value": "Test Index creation",
"old_value": null
}
},
{
"_index": "my_index",
"_type": "audit_field",
"_id": "57526c198045c",
"_score": 1,
"_source": {
"session_id": 119,
"trans_seq_no": 1,
"table_seq_no": 1,
"field_id": 4,
"field_name": "curriculum_name_pr",
"new_value": null,
"old_value": null
}
},
{
"_index": "my_index",
"_type": "audit_field",
"_id": "57526c1981512",
"_score": 1,
"_source": {
"session_id": 119,
"trans_seq_no": 1,
"table_seq_no": 1,
"field_id": 5,
"field_name": "curriculum_name_pa",
"new_value": null,
"old_value": null
}
}
]
}
}
and many more fields may be there, now user may select one or more of those fields and define a search term across those fields that he/she selected, the challenge is here, how we can say elastic that consider field_name to match those fields that user selected, then search in old_value, and new_value.
for example if user select curriculum_name_en and curriculum_name_pr and then want to search for xy inside old_value and new_value fields of those documents that their field_name is above fields.
how we can do that?
The idea with this requirement is that you need to make something like: the query needs to match new_value and/or old_value only if field_name matches a certain value as well. There is no programmatic-like way of saying if this then that.
What I'm suggesting is something like this:
{
"query": {
"bool": {
"must": [
{
"terms": {
"field_name": [
"curriculum_name_en",
"curriculum_name_pr"
]
}
},
{
"multi_match": {
"query": "Test Index",
"fields": ["new_value","old_value"]
}
}
]
}
}
}
So, your if this then that condition is a must statement from a bool query where your if and then branches live inside the must.
This may solve your problem
{
"query": {
"filtered": {
"filter": {
"and": [
{
"query" : {
"terms" : {
"field_name" : [
"curriculum_name_en",
"curriculum_name_pr"
],
"minimum_match" : 1
}
}
},
{
"query" : {
"terms" : {
"new_value" : [
"test", "index"
],
"minimum_match" : 1
}
}
}
]
}
}
}
}
}

Getting unique results with elasticsearch acorrding to field

I'm using FOSElasticaBundle with Symfony2 on my project and there are entry and user tables on MySQL database and each entry belongs to one user.
I want to get just one entry per a user among the whole entries from the database.
Entries Representation
[
{
"id": 1,
"name": "Hello world",
"user": {
"id": 17,
"username": "foo"
}
},
{
"id": 2,
"name": "Lorem ipsum",
"user": {
"id": 15,
"username": "bar"
}
},
{
"id": 3,
"name": "Dolar sit amet",
"user": {
"id": 17,
"username": "foo"
}
},
]
Expected result is:
[
{
"id": 1,
"name": "Hello world",
"user": {
"id": 17,
"username": "foo"
}
},
{
"id": 2,
"name": "Lorem ipsum",
"user": {
"id": 15,
"username": "bar"
}
}
]
But it returns all entries on table. I've tried to add an aggregation to my elasticsearch query and nothing changed.
$distinctAgg = new \Elastica\Aggregation\Terms("distinctAgg");
$distinctAgg->setField("user.id");
$distinctAgg->setSize(1);
$query->addAggregation($distinctAgg);
Is there any way to do this via term filter or anything else? Any help would be great. Thank you.
Aggregations are not easy to understand when you are used to MySQL group by.
The first thing, is that aggregations results are not returned in hits, but in aggregations. So when you get the result of your search, you have to get aggregations like that :
$results = $search->search();
$aggregationsResults = $results->getAggregations();
The second thing is that aggregations wont return you the source. With the aggregation of your example, you will only know that you have 1 user with ID 15, and 2 users with ID 15.
E.g. with this query :
{
"query": {
"match_all": {}
},
"aggs": {
"byUser": {
"terms": {
"field": "user.id"
}
}
}
}
Result:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 3,
"max_score": 1,
"hits": [ ... ]
},
"aggregations": {
"byUser": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 17,
"doc_count": 2
},
{
"key": 15,
"doc_count": 1
}
]
}
}
}
If you want to get results, the same way you would do with a GROUP BY in MySQL, you have to use a top_hits sub-aggregation:
{
"query": {
"match_all": {}
},
"aggs": {
"byUser": {
"terms": {
"field": "user.id"
},
"aggs": {
"results": {
"top_hits": {
"size": 1
}
}
}
}
}
}
Result:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 3,
"max_score": 1,
"hits": [ ... ]
},
"aggregations": {
"byUser": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 17,
"doc_count": 2,
"results": {
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "test_stackoverflow",
"_type": "test1",
"_id": "1",
"_score": 1,
"_source": {
"id": 1,
"name": "Hello world",
"user": {
"id": 17,
"username": "foo"
}
}
}
]
}
}
},
{
"key": 15,
"doc_count": 1,
"results": {
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_stackoverflow",
"_type": "test1",
"_id": "2",
"_score": 1,
"_source": {
"id": 2,
"name": "Lorem ipsum",
"user": {
"id": 15,
"username": "bar"
}
}
}
]
}
}
}
]
}
}
}
More informations on this page : https://www.elastic.co/blog/top-hits-aggregation

Elasticsearch Updating / Deleting Nested

I've gone through a few examples and documentations and kind find a solution update a nested object in the this result set.
I can add one (if one does not exist)
I can append to it (if one does exist)
Can't figure out how to delete a selected entry.
Is there a method I can use (using the php client) to add an entry if it does not exist / update an entry if it does exist / delete the second entry.
I'm inheriting this problem and am new to Elastic search.
Thanks.
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "products",
"_type": "categories",
"_id": "AUpRjtKZfXI7LIe9OpNx",
"_score": 1,
"_source": {
"name": "Primary",
"description": "Primary Category",
"slug": "Primary",
"created": "2014-12-16 00:25:22",
"parent": [
{
"name": "First One",
"description": "Test",
"id": "ae74ea4e2e865ed3fd60c18a06e69c65",
"slug": "first-one"
},
{
"name": "Second One",
"description": "Testing Again",
"id": "c8dbe5143c8dfd6957fa33e6cea7a0a8",
"slug": "second-one"
}
]
}
}
]
}
}
Do you want to do all three in the same operation?
Deleting the second nested object is achieved through a script which removes the second element:
PUT /products
{
"mappings": {
"categories": {
"properties": {
"parent": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"description": { "type": "string" },
"id": { "type": "string", "index": "not_analyzed" },
"slug": { "type": "string" }
}
}
}
}
}
}
PUT /products/categories/1
{
"name": "Primary",
"description": "Primary Category",
"slug": "Primary",
"created": "2014-12-16 00:25:22",
"parent": [
{
"name": "First One",
"description": "Test",
"id": "ae74ea4e2e865ed3fd60c18a06e69c65",
"slug": "first-one"
},
{
"name": "Second One",
"description": "Testing Again",
"id": "c8dbe5143c8dfd6957fa33e6cea7a0a8",
"slug": "second-one"
}
]
}
POST /products/categories/1/_update
{
"script" : "ctx._source.parent.remove(1)",
"lang": "groovy"
}
GET /products/categories/1
So in PHP code (using the official PHP client), the update would look like:
$params = [
'index' => 'products',
'type' => 'categories',
'id' => 1,
'body' => [
'script' => 'ctx._source.parent.remove(1)',
'lang' => 'groovy'
]
];
$result = $client->update($params);

Categories