I am trying to insert records from mysql to elastic. I am inserting two records only. 'prod_id' is primary key in mysql DB.
below is my insert query.
$elastic->setIndex("tbl_product");
$elastic->setType("products");
//$elastic->setConnection(new MySQLiConnection());
//Where 1st param is the database connection and 2nd param is tableName
$elastic->setSqlQuery("SELECT *
FROM tbl_product
LEFT JOIN tbl_product_details ON tbl_product.prod_id = tbl_product_details.prod_id
LEFT JOIN tbl_product_seo ON tbl_product.prod_id = tbl_product_seo.prod_id
LEFT JOIN tbl_product_keywords ON tbl_product.prod_id = tbl_product_keywords.prod_id
LEFT JOIN tbl_category ON tbl_category.cat_id = tbl_product.cat_id
LEFT JOIN tbl_manufacturer ON tbl_manufacturer.manuf_id = tbl_product.manuf_id LIMIT 2000 OFFSET 18000");
$elastic->insertAllData($connection);
/***** Below is elastic array after inserting *****/
Search Function:
$params = [
'index' => $this->client->getIndex(),
'type' => $this->client->getType(),
"from" => 0, "size" => 1000,
//"scroll" => "1m",
'body' => [
"query" => [
"bool" => [
"must" => [
[
"multi_match" => [
"fields" => ["prod_name", "prod_seo_name"],
"type" => "phrase_prefix",
"query" => $query
]
],
[
"term"=> ["cat_type_id"=>1]
]
]
]
]
],
];
$query = $client->search($params);
//return array($this->extractResult($query),$this->extractResult($query1));
return $this->extractResult($query);
Above function returns below array. For example I have shown only two fields in array.
[0] => Array
(
[prod_id] => 16579
[lang_id] => 1
)
[1] => Array
(
[prod_id] =>
[lang_id] => 1
)
In second 'prod_id' is null. Any idea why this is happening?
Below is my mapping.
'mappings' => [
'article' => [
'properties' => [
'prod_id' => [
'type' => 'integer'
],
'lang_id' => [
'type' => 'string'
],
]
]
]
Related
I'm using delete by query in the PHP client for elasticsearch. I have elasticsearch index posts with some parent-child relations. So when I try to delete some records from child relation the query executes successfully and shows success but records are not deleted from the index, and most of the time query work properly. But this issue produced some time.
Field mapping type
"post_id" : {
"type": "integer"
}
Query
$query = [
'index' => 'posts',
'body' => [
'query' => [
'bool' => [
'should' => [
[
"bool" => [
"must" => [
['term' => ['type' => 'box_post']],
['term' => ['post_id' => (int) $post_id]]
]
]
],
[
"bool" => [
"must" => [
['term' => ['type' => 'post_box']],
['term' => ['post_id' => (int) $post_id]]
]
]
]
]
]
]
]
];
$this->client->deleteByQuery($query)
Response
Array
(
[took] => 0
[timed_out] =>
[total] => 0
[deleted] => 0
[batches] => 0
[version_conflicts] => 0
[noops] => 0
[retries] => Array
(
[bulk] => 0
[search] => 0
)
[throttled_millis] => 0
[requests_per_second] => -1
[throttled_until_millis] => 0
[failures] => Array
(
)
)
Is there any way to make sure that if the record exists on the index then the query must delete the record instead of showing the success message with no record deleted?
I want to know how can I query inner joins in mongodb? I have this users and payments table. I only want to display records that users has payments if not, do not include it in the result. So far, I only got left join like results.
Here is my code:
$getData=[
[
'$match' => ['profile_status'=> 'buyer']
],
[
'$project'=>[
'_id' => ['$toString' => '$_id'],
'profile_image' => '$profile_image',
'full_name' => '$full_name',
'profile_status' => '$profile_status'
]
],
[
'$lookup' => [
'from' => 'payment_details',
'let' => [
'user_id' => ['$toString' => '$_id']
],
'pipeline' => [
[
'$match' => [
'$expr' => ['$eq' => ['$buyer_id','$$user_id']]
]
],
[
'$project' => [
'_id' => ['$toString' => '$_id'],
'price' => '$price'
]
],
],
'as' => 'paymentData'
]
]
];
$getresult= $db->users->aggregate($getData);
$getfinalresult= iterator_to_array($getresult);
This code returns users data even paymentData is empty, I only want users that has paymentData. Im using PHP/Codeigniter 3.
Thank you!
MongoDB 3.4
I have a project and project_permission collections. The project_permission collection contains permissions to the projects for some users. A single user can have multiple different permissions to the project.
[
'$lookup' => [
'from' => ProjectPermission::collectionName(),
'localField' => '_id',
'foreignField' => 'project_id',
'as' => 'project_permissions'
]
],
[
'$project' => [
// ... irrelevant fields here
'permissions' => '$project_permissions'
]
],
this is how the project query results looks like without filtering:
// other project results
// ... other fields
'permissions' => [
0 => [
'_id' => '5d2873aafa873b2b7c000fad'
'project_id' => '56a9e5c5d18cacc72a485839'
'user_id' => '562f6bfc05dfe9570fb6e427'
'permission' => 'read'
'created_at' => 1562932138
'updated_at' => 1562932139
]
1 => [
'_id' => '5d2879fdfa873b2b7c000fbd'
'project_id' => '56a9e5c5d18cacc72a485839'
'user_id' => '562f6bfc05dfe9570fb6e427'
'permission' => 'write'
'created_at' => 1562932139
'updated_at' => 1562932140
]
2 => [
'_id' => '5db960b5fa873b1604005e8e'
'project_id' => '56a9e5c5d18cacc72a485839'
'user_id' => '582b30dd1e634e6362e1b504'
'permission' => 'write'
'created_at' => 1572430005
'updated_at' => 1572430005
]
]
What I would like to achieve is to return with only those projects where the client - who requested the query - has a specific permission to the project, for example write.
The way I tried it:
pipeline: [
0 => [
'$match' => [
// not related to the problem
]
]
1 => [
'$match' => [
'$and' => [
0 => [
'shared_permissions' => [
'$eq' => true
]
]
1 => [
'$or' => [
0 => [
'project_permissions' => [
'$exists' => true
'$ne' => []
]
]
1 => [
'owner_id' => [
'$ne' => MongoDB\BSON\ObjectId#1
(
[oid] => '582b30dd1e634e6362e1b504'
)
]
]
]
]
]
]
]
2 => [
'$lookup' => [
'from' => 'project_permission'
'localField' => '_id'
'foreignField' => 'project_id'
'as' => 'project_permissions'
]
]
3 => [
'$project' => [
// more not important fields here
'shared_permissions' => 1
'permissions' => [
'$map' => [
'input' => [
'$filter' => [
'input' => '$project_permissions'
'as' => 'project_permission'
'cond' => [
'$and' => [
0 => [
'$eq' => ['$$project_permission.user_id', MongoDB\BSON\ObjectId#1
(
[oid] => '582b30dd1e634e6362e1b504'
)
]
1 => [
'$eq' => ['$$project_permission.permission', 'write']
]
]
]
]
]
'as' => 'project_permission'
'in' => [
'user_id' => '$$project_permission.user_id'
'permission' => '$$project_permission.permission'
]
]
]
]
]
]
For this I almost get the correct response:
[
0 => [
'_id' => '56a9e5c5d18cacc72a485839'
'short_id' => 3
'title' => 'Modified title'
'owner_id' => '562f692a05dfe9560fb6e428'
'updated_at' => 1572435428
'owner_name' => 'Borat Sagdiyev'
'shared_permissions' => true
'permissions' => [
0 => [
'user_id' => '582b30dd1e634e6362e1b504'
'permission' => 'write'
]
]
]
1 => []
]
The problem with this is that empty array, where the result was filtered out - and it wouldn't be a problem if the empty array wouldn't be in the result, because if I use the pagination, then it says two results, instead of one. And we know that in the worst case we would get back an array of empty arrays only.
So what I would like to achieve is this last example results without empty arrays in a way where the pagination will be fine with it too.
ps.: unwind is not an option, because of some structural conventions.
Any ideas?
To perform an equality match between a field from the input documents with a field from the documents of the “joined” collection, the $lookup stage has the following syntax:
{
$lookup:
{
from: <collection to join>,
localField: <field from the input documents>,
foreignField: <field from the documents of the "from" collection>,
as: <output array field>
}
}
I have a data of stores, report_type is every type of data
- type 3 stored how much money that clients buy and how many transaction.
- type 5 stored how many people and average age of them.
I need an array like this:
"report_type3"
(
[_id] => 2441
[amountclient] => 3749078452,
[trans] => 1000
),
(
[_id] => 2442
[amountclient] => 15533754,
[trans] => 900
)
,
"report_type5"
(
[_id] => 2441
[people] => 120,
[avgage] => 23,
),
(
[_id] => 2441
[people] => 120,
[avgage] => 23,
)
My query looks like this, but it's not working
$cursor = $collection->aggregate([
['$match' => [ 'id_station' => ['$in' => $store ], 'report_type' => 3,
'date' => ['$gte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-07-01")*1000), '$lte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-08-05")*1000)]
]
],
['$group' => [
'_id' => [ 'id_station' => '$id_station', 'receiptid' => '$receiptid' ],
'amount' => [ '$sum' => '$amount' ]
]], .... *I group by receiptid because there are many items in one clients.*
[ '$group' => [
'_id' => '$_id.id_station',
'amount' => [ '$sum' => '$amount' ],
'trans' => [ '$sum' => '$trans' ]
]],
[
'$project' => [
'report_type3' => ['_id','$amountclient','$trans']
]
],
['$match' => [ 'id_station' => ['$in' => $store ], 'report_type' => 5,
'date' => ['$gte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-07-01")*1000), '$lte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-08-05")*1000)]
]
],
[ '$group' => [
'_id' => ['$_id.id_station', 'area_type' => '$area_type'],
'people' => [ '$sum' => '$people' ],
'avgage' => [ '$avg' => '$age' ]
]],
[
'$project' => [
'report_type5' => [ '_id', 'people', 'avgage' ]
]
]
]);
Could you please help me on this?
I'm using Elastic search in Laravel.
So I have query that returns correct result:
$results = Es::search(array(
'index' => 'testindex',
'type' => $type,
'body' => [
'query' => [
'filtered' => [
'query' => [
'match' => [
'_all' => '2015-02'
]
],
'filter' => [
'bool' => [
'must' => [
'term' => [
'type' => 11
]
],
]
]
]
],
'size' => 5,
'from' => 0
]
));
What it does is:
searches for $query in all fields AND
field "type" must be 11.
Here is output: http://pastebin.com/icWniix4
Total 9 results which is correct.
But when I add another must term then it returns invalid results
$results = Es::search(array(
'index' => 'testindex',
'type' => $type,
'body' => [
'query' => [
'filtered' => [
'query' => [
'match' => [
'_all' => '2015-02'
]
],
'filter' => [
'bool' => [
'must' => [
'term' => [
'type' => 11
],
'term' => [
'public' => 1
]
],
]
]
]
],
'size' => 5,
'from' => 0
]
));
So this only add term for "public".
What it does is:
searches for $query in all fields AND
field "type" must be 11 AND
field "public" must be 1
So now results are in total 429. It ignores "type" term and returns everything with "public" = 1. But as per docs if I use MUST then it should match all of them. Search result http://pastebin.com/cVcatcyi
So how can I write query that I need? $query + type + public
Official doc's fail to answer my questions.
Any advice?
This is your trouble spot:
'must' => [
'term' => [
'type' => 11
],
'term' => [
'public' => 1
]
]
Here, you're assigning the value of must as an associative array that has only one distinct key - term actually gets assigned twice, so presumably only one of the assignments will "survive" (presumably public survives because it appears last in the definition). The end result is that must ends up pointing at an associative array with only one key-value pair.
What I suspect you have to do is this:
'must' => [
[
'term' => [
'type' => 11
]
],
[
'term' => [
'public' => 1
]
]
]
Now must is actually pointing at an array with two items.