I want to index in my index buildings 2 types of documents: building & apartment.
A building can have several apartment, so I want to use parent/child relationships.
I use ES 7.10 and PHP with official PHP lib (elasticsearch/elasticsearch).
According to the doc (https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child-mapping.html), I have to use something like :
PUT index buildings
{
"mappings": {
"building": {},
"apartment": {
"_parent": {
"type": "building"
}
}
}
}
In my PHP I have :
$this->getClient()->indices()
->create([
'index' => 'buildings',
'body' => [
'mappings' => [
'building' => [],
'apartment' => [
'_parent' => [
'type' => 'building'
]
]
]
]
]);
Which throws the error:
Root mapping definition has unsupported parameters: [apartment: {_parent={type=building}}] [building: []]"}},"status":400}
Did I miss something?
Parent/type has been deprecated in favor of join type in Elasticsearch 7 as far as I remember, so this might be the reason you're getting the exception.
Link to docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html
Related
We started our web-based app back in 2012 when Perl was still popular. Now we want to reimplement it in PHP, preferably Laravel. However, I miss the power of DBIx::Class in their default ORM, Eloquent, and the alternative, Doctrine, from reading the docs seems more complicated, but doesn't solve the issue either. What I use DBIx::Class for:
Regenerate model classes from reading the DB structure, preserving any code that has been added to them (should have).
Generate complex queries (incl. nested joins, subqueries) from pure language constructs, no SQL needed (must have).
Code sample:
rows => $rows,
page => $page,
'select' => [
'entity_id',
{ group_concat => [ { distinct => 'pictures.id' } ], -as => 'photos' },
{ '' => [ 'entity.name' ], -as => 'entity_name' },
{ '' => [ 'municipality.name' ], -as => 'municipality_name' },
{ '' => [ 'me.birth_date' ], -as => 'bd' },
{ concat => [ { date_format => [ 'me.birth_date', \$c->config->{dateFormat} ] }, \"' ('", { timestampdiff => [ \'YEAR', 'me.birth_date', \'CURDATE()' ] }, \"')'" ], -as
{ date_format => [ 'me.birth_date', \$c->config->{dateFormat} ], -as => 'bd1' },
{ timestampdiff => [ \'YEAR', 'me.birth_date', \'CURDATE()' ], -as => 'bd2' },
{ '' => [ 'entity_state_type.name' ], -as => 'entity_state_name' },
{ '' => [ 'data_list_item.name' ], -as => 'entity_state_reason_name' },
{ '' => [ 'entity_state.note_text' ], -as => 'entity_state_note' },
{ '' => [ { if => [ 'entity.archived', \"'Yes'", \"''" ] } ], -as => 'entity_archived' },
],
join => {
entity => [ 'municipality', 'pictures', { 'entity_state' => [ 'entity_state_type', 'data_list_item' ] } ],
},
group_by => [ 'entity.id' ],
Subclass the result (row) and resultset classes to add user role-based limiting conditions transparently to the application code (must have, or equivalent).
The call and the resulting query:
$c->model('DB::Person')->all
SELECT * FROM person WHERE owner = <user_id>
...if the user role security settings indicate user should only access his own persons.
The security settings are read from DB on request start and specify which security conditions (separate classes) should apply to which table.
This moves all security away from application code, where mistakes are made most often.
Which PHP library would allow us to implement this?
I'm very new in Elasticsearch, I'm implementing it inside my Laravel project with Elasticsearch Scout Driver but I've got an error while insert model object inside index.
The model is a Post, made like this:
>>> $post = App\Models\Post::first();
=> App\Models\Post {#853
id: 1,
title: "First Post",
description: "My first post",
created_at: "2017-09-13 13:31:51",
updated_at: "2018-02-16 16:23:44",
deleted_at: null,
}
Inside model class, I declare the mapping options, I report my mapping options:
// Map elements to be saved in Elasticsearch
protected $mapping = [
'properties' => [
'id' => [
'type' => 'integer',
'index' => false
],
'title' => [
'type' => 'text'
],
'description' => [
'type' => 'text'
],
'created_at' => [
'type' => 'date',
'ignore_malformed' => true,
'format' => "yyyy-MM-dd HH:mm:ss"
],
'updated_at' => [
'type' => 'date',
'ignore_malformed' => true,
'format' => "yyyy-MM-dd HH:mm:ss",
],
'deleted_at' => [
'type' => 'date',
'ignore_malformed' => true,
'format' => "yyyy-MM-dd HH:mm:ss",
]
]
];
Every time I call $post->searchable(); to put my model inside my Elasticsearch index, I've got this error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Mapper for [deleted_at] conflicts with existing mapping in other types:\n[mapper [deleted_at] has different [format] values]"
}
],
"type": "illegal_argument_exception",
"reason": "Mapper for [deleted_at] conflicts with existing mapping in other types:\n[mapper [deleted_at] has different [format] values]"
},
"status": 400
}
I'm guessing that the problem is the null value of deleted_at property.
I need deleted_at == null because I manage soft deletion with Laravel: any another value will cause the soft deletion for Laravel framework (not retrieve element when query).
As you can see, I tried to put ignore_malformed => true but it doesn't work for me.
I tried to add another option null_value => NULL without success.
Where I am wrong?
How can I insert posts inside my Elasticsearch Index with deleted_at attribute set to null OR set to date with format yyyy-MM-dd HH:mm:ss?
Thanks
PS: I'm using Elasticsearch Version 6.1.2.
An index consists of multiple types (in version 6 this is no longer possible mainly due to this reason). The problem with different types is that they cannot store the same field name with a different mapping. This has to do with the way it is stored in Lucene.
Could it be you are inserting documents in two different types? Maybe by accident (Typo in the type while ingesting documents for instance). Then it might try to create a different field type by dynamic mapping say a string. This would cause the exception that you mention.
I am having trouble in returning matched embedded document using sokil PHPMongo ODM library. I am new to ODM concept and following is my document structure of collection Project:
{
"_id" : ObjectId("59f889e46803fa3713454b5d"),
"projectName" : "usecase-updated",
"classes" : [
{
"_id" : ObjectId("59f9d7776803faea30b895dd"),
"className" : "OLA"
},
{
"_id" : ObjectId("59f9d8ad6803fa4012b895df"),
"className" : "HELP"
},
{
"_id" : ObjectId("59f9d9086803fa4112b895de"),
"className" : "DOC"
},
{
"_id" : ObjectId("59f9d9186803fa4212b895de"),
"className" : "INVOC"
}
]
}
So in my query first criteria is to get the Project with a matched ID and 2nd criteria is to to retrieve only the class from the classes array of embedded documents with a specific id.This is how i am building the query:
$collection = $PHPMongoDBInstance->getCollection("Project");
$result = $collection->getDocument(
"59f889e46803fa3713454b5d",
function (\Sokil\Mongo\Cursor $cursor) {
// get embedded documents with matched id
$cursor->whereElemMatch("classes", $cursor->expression()->where("_id", new MongoId("59f9d7776803faea30b895dd")));
}
);
I was expecting it to return only the embedded document of OLA from the usecase-updated document like this:
{
"_id" : ObjectId("59f889e46803fa3713454b5d"),
"projectName" : "usecase-updated",
"classes" : [
{
"_id" : ObjectId("59f9d7776803faea30b895dd"),
"className" : "OLA"
}
]
}
But PHPMongo library is returning the whole Project Document (shown in the start of question) with all the classes. Someone suggested to look into aggregation framework. But problem is there is not good enough documentation on the PHPMongo for using array aggregation functions (like $filter)
I tried it by using native instance of MongoCollection and with that i can use the findOne method to project my final result using this way:
$result = $collection->getMongoCollection("Project")->
findOne(array("_id" => new MongoId("59f889e46803fa3713454b5d")),
array("classes" =>
array('$elemMatch' =>
array("_id" => new MongoId("59f9d7776803faea30b895dd")))));
If i want to achieve the similar projection using the getDocument method of sokil PHPMongo is there some possibility?
UPDATE:
I tried achieving with aggregation framework and following was the query:
$result = $collection->aggregate(array(
array(
'$match' => array(
"_id" => new MongoId("59f889e46803fa3713454b5d")
)
),
array(
'$project' => array(
'classes' => array(
'$filter' => array(
'input' => '$classes',
'as' => 'classItem',
'cond' => array(
'$eq' => array('$$classItem._id' => new MongoId("59f9d7776803faea30b895dd"))
)
)
)
)
)
));
But i get this exception:
Sokil\\Mongo\\Exception\nMessage: Aggregate error: Unrecognized expression '$$classItem._id'\nFile:
So after posting the problem on the PHPMongo GitHub issues i found so far there is no implementation of $elemMatch. Author posted some interesting insights on this issue and now this issue is marked as enhancement for the future.
Furthermore i have tried aggregation framework and i am succesfully able to achieve the desired results i found that there was a mistake in the syntax of following expression (for full query see above in the question):
'$eq' => array('$$classItem._id' => new MongoId("59f9d7776803faea30b895dd"))
I learned that expression of '$eq' shouldn't be a associative array rather each element in the expression should be a separate array item like this:
'$eq' => array('$$class._id', new MongoId("59f9d7776803faea30b895dd"))
So now my final aggregation query was like this:
$collection = $PHPMongoDBInstance->getCollection("Project");
$result = $collection->aggregate(array(
array(
'$match' => array(
"_id" => new MongoId('59f889e46803fa3713454b5d')
)
),
array(
'$project' => array(
'classes' => array(
'$filter' => array(
'input' => '$classes',
'as' => 'class',
'cond' => array(
'$eq' => array('$$class._id', new MongoId("59f9d7776803faea30b895dd"))
)
)
),
'projectName' => 1
)
)
));
Hope this will help people in future if they had similar problem like me as i tried finding solution and it wasn't there over the internet. I will try to update once there is a feature enhancement in PHPMongo of $elemMatch.
I want to use wildcard search using elastic search 2.3 using its official PHP client.
I am facing a issue which is like this:
Case 1. When i search for word wood, it returns the words which are having woodman, hollywood and hollywoodbolly.
Case 2. But when i search for hollywood, it does not return the words which are having hollywood in them.
However, everything is working fine when done in query string like this:
"query" => [
"query_string" => [
"query" => "*$keyword*",
"analyze_wildcard" => true,
"fields" => $fields
]
],
But when used like follwing, Case 2 is not working:
"query" => [
"bool" => [
"must" => [
[
"wildcard" => [
'name' => "*$keyword*",
]
],
[
"nested" => [
"path" => "address",
"score_mode" => "max",
"query" => [
"bool" => [
"must" => [..match[] parameters..]
]
]
]
]
]
]
I am not sure what I am doing wrong. Please help.
EDIT:
NOTE: I have made the field as not_analysed.
My query is returning cardboard when searching for card but not returning cardboard when searching for cardboard
Thanks.
Elasticsearch supports wildcard queries only on not_analyzed fields
So if you would like to use the wildcard capability you could either use it under the query_string object, or change the mapping for that field to index: not_analyzed and then you would be able to do a wildcard search.
I have created query in mongoDB. In MongoChef this query produces more than 10 thousand records in less than 2 seconds. Now I want to execute this query in PHP.
So i don't know how to write query in php as I read various documents on internet but confused how to implement it.
db.PMS.aggregate(
[
{$project:
{EventTS:1,MainsPower:1,PanelID:1}
},
{$unwind:
{path:"$MainsPower",includeArrayIndex:"arrayIndex",preserveNullAndEmptyArrays:true}
},
{ $match: { "MainsPower":{$ne:null}}},
{ $match: { "EventTS":{$gt:new Date("2016-01-01")}}},
{$project:
{MainsPower:1,
PanelID:1,
timestamp:{"$add":
[{'$subtract' : ["$EventTS",new Date("1970-01-01")]},
{"$multiply":[60000,"$arrayIndex"]}
]}
}
}
]
);
You can use some resources available on the php official documentation. A mapping of sql queries in php to mongoDB queries in php can be found here.
Also I have a demo login and registration script at my github. You can view those in this repo.
If you use MongoDB PHP Library you should be able to do something similar to this:
$mongo = new MongoClient();
$database = $mongo->examples;
$collection = $database->PMS;
$pipeline = [
[
'$project' => [
'EventTS' => 1,
'MainsPower' => 1,
'PanelID' => 1,
]
],
[
'$unwind' => [
'path' => '$MainsPower',
'includeArrayIndex' => 'arrayIndex',
'preserveNullAndEmptyArrays' => true
]
],
...
];
$cursor = $collection->aggregate($pipeline);