MongoDB very slow result with $lookup - php

I am using PHP to interact with MongoDB, recently I used $lookup on 5 collection and my query starts taking around 20-25 seconds to get the result. All my collections have testing data which is not more than 30-40 entries. I also used profiling but I am getting message: "Query Not Recording (too Large)"
I am sharing code from my PHP file. I have also tried creating index on field which are creating joins between collection but got same result. Please let me the the best way to create collection structure and to retrieve data from MongoDB.
Thanks in advance.
$collection = new MongoCollection($db, 'col_main');
$condition = array();
$condition[] = array(
'$lookup' => array(
"from" => "col_a",
"localField" => "a",
"foreignField" => "_id",
"as" => "col_a"
)
);
$condition[] = array(
'$lookup' => array(
"from" => "col_b",
"localField" => "b",
"foreignField" => "_id",
"as" => "col_b"
)
);
$condition[] = array(
'$lookup' => array(
"from" => "col_c",
"localField" => "c",
"foreignField" => "_id",
"as" => "col_c"
)
);
$condition[] = array(
'$lookup' => array(
"from" => "col_e",
"localField" => "e",
"foreignField" => "_id",
"as" => "col_e"
)
);
$condition[] = array(
'$lookup' => array(
"from" => "col_f",
"localField" => "f",
"foreignField" => "_id",
"as" => "col_f"
)
);
$condition[] = array(
'$project' => $projection
);
$condition[] = array (
'$unwind' => '$col_a'
);
$condition[] = array (
'$unwind' => '$col_a.translation'
);
$condition[] = array (
'$unwind' => '$col_b'
);
$condition[] = array (
'$unwind' => '$col_b.translation'
);
$condition[] = array (
'$unwind' => '$col_c'
);
$condition[] = array (
'$unwind' => '$col_c.translation'
);
$condition[] = array (
'$unwind' => '$col_e'
);
$condition[] = array (
'$unwind' => '$col_e.staff_name'
);
$condition[] = array (
'$unwind' => '$col_e.staff_surname'
);
$condition[] = array (
'$unwind' => '$col_f'
);
$condition[] = array (
'$unwind' => '$col_f.translation'
);
//Some More conditions
$condition[] = array(
'$match' => array( 'col_b.translation.language' => $val )
);
$condition[] = array(
'$match' => array( 'col_a.translation.language' => $val )
);
$condition[] = array(
'$match' => array( 'col_c.translation.language' => $val )
);
$condition[] = array(
'$match' => array( 'col_f.translation.language' => $val )
);
return $collection->aggregate($condition);

As you are multiplying object using many unwind operations this could produce a huge set of data, every $unwind muliplayes documnet set, so there could be a huge memory consumption and even swapping.
if we have 3 arrays 3 elements each, then we will perform unwind on a, b, c
we will get:
3 documents after first step,
9 documents after 2nd step
27 documents after last step..

Related

onwsignal web notification error

actually i am tring to get data from url which i already got but i can't able to insert data into the array please check values are dummy1 dummy2
Array ( [id] => 491 [headings] => ‘సౌందర్య లహరి’ ఎలా ఉంది అంటే..? [content] => శ్రీ వాసు దర్శకత్వం లో బెల... [userPic] => 682473.jpg ) JSON sent: {"app_id":"35fi6c27-5f39-4s9f-94db-79bf123g0f9","included_segments":["All"],"data":{"foo":"bar"},"contents":{"en":"dummy1"},"headings":{"en":"dummy2"},"url":"http:\/\/www.gggg.in\/view.php?id=[id]","chrome_web_image":"http:\/\/www.ggg.in\/admin\/user_images\/[userPic]"} JSON received: {"allresponses":"{\"id\":\"f66a03a4-1b17-8tf3-93ed-f3ad6rt7cdb9\",\"recipients\":6}"}
the above data got from url
http://www.ggg.in/gistfile1.php?id=491&headings=%E2%80%98%E0%B0%B8%E0%B1%8C%E0%B0%82%E0%B0%A6%E0%B0%B0%E0%B1%8D%E0%B0%AF%20%E0%B0%B2%E0%B0%B9%E0%B0%B0%E0%B0%BF%E2%80%99%20%E0%B0%8E%E0%B0%B2%E0%B0%BE%20%E0%B0%89%E0%B0%82%E0%B0%A6%E0%B0%BF%20%E0%B0%85%E0%B0%82%E0%B0%9F%E0%B1%87..?&content=%20%E0%B0%B6%E0%B1%8D%E0%B0%B0%E0%B1%80%20%E0%B0%B5%E0%B0%BE%E0%B0%B8%E0%B1%81%20%E0%B0%A6%E0%B0%B0%E0%B1%8D%E0%B0%B6%E0%B0%95%E0%B0%A4%E0%B1%8D%E0%B0%B5%E0%B0%82%20%E0%B0%B2%E0%B1%8B%20%E0%B0%AC%E0%B1%86%E0%B0%B2...&userPic=682473.jpg
and i need to insert the values in this function
function sendMessage() {
$content = array(
"en" => 'dummy1'
);
$headings = array(
"en" => 'dummy2'
);
$hashes_array = array();
$fields = array(
'app_id' => "35fi6c27-5f39-4s9f-94db-79bf123g0f9",
'included_segments' => array(
'All'
),
'data' => array(
"foo" => "bar"
),
'contents' => $content,
'headings' => $headings,
'url' => 'http://www.gggg.in/view.php?id=[id]',
'chrome_web_image' => 'http://www.ggg.in/admin/user_images/[userPic]',
);
OneSignal does not support substituting variable data directly into API calls. However, you can achieve this with Tag and Variable Substitution: https://documentation.onesignal.com/docs/personalization
this is the solutions for this Question i asked about
actually i have a varable passing throug url and i need them to insert into function
$array = [
'id' => 498,
'value2' => 'యూట్యూబ్‌లో “భరత్ అనే నేను” అన్ కట్ సీన్లు..!',
'value1' => 'కొరటాల శివ దర్శకత్వం లో సూ...',
'value3' => '525722.jpg'
];
sendMessage($array);
function sendMessage($array) {
$content = array(
"en" => $array['value1']
);
$headings = array(
"en" => $array['value2']
);
$hashes_array = array();
$fields = array(
'app_id' => "31pe2347-5i39-4y9f-2222-79b5875f00f9",
'included_segments' => array(
'All'
),
'data' => array(
"foo" => "bar"
),
'contents' => $content,
'headings' => $headings,
'url' => 'http://w...content-available-to-author-only...9.in/view.php?id=' . $array['id'],
'chrome_web_image' => 'http://w...content-available-to-author-only...9.in/admin/user_images/' . $array['value3'],
);
print_r($fields);
}

How to check one single array value into another multiple value?

Hello Developer I am trying to check one single array into multiple array value and My exact requirement is the value of one array should be check in each indexes of second array and replace if same otherwise append it in new array.
My Two array was like this:-
One array:-
$array1 = array(
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
);
Second Array:-
$array2 = array(
0 => array(
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
),
1 => array(
"extension_date" => "2017-05-31",
"extended_date" => "2017-05-31"
),
);
I am try it from yesterday but it's not be succeed so please help me to solve out this issue.
You can make use of array_search and array_push. You don't need to replace if you find the search array in the main array since it's the exact same thing.
$search = [
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
];
$data = [
[
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
],
[
"extension_date" => "2017-05-31",
"extended_date" => "2017-05-31"
]
];
if (array_search($search, $data) === false) {
array_push($data, $search);
}
// $data contains $search if it's missing
Here we are using array_search if needle doesn't exist then we add that in array.
Try this code snippet here
<?php
ini_set('display_errors', 1);
$array1 = array(
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
);
$array2 = array(
0 => array(
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
),
1 => array(
"extension_date" => "2017-05-31",
"extended_date" => "2017-05-31"
),
);
if(array_search($array1, $array2)===false)
{
$array2[]=$array1;
}
print_r($array2);
Inputs:
$array1 = array(
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
);
$array2 = array(
array(
"extension_date" => "2017-05-19",
"extended_date" => "2017-05-27"
),
array(
"extension_date" => "2017-05-31",
"extended_date" => "2017-05-31"
),
);
Method:
if(!in_array($array1,$array2)){
$array2[]=$array1;
}
Since you are only checking to see if the subarray exists and you don't care about its key, it doesn't make sense to use array_search(). in_array() was specifically designed to return true false -- so use it!
var_export($array2) output:
array (
0 =>
array (
'extension_date' => '2017-05-19',
'extended_date' => '2017-05-27',
),
1 =>
array (
'extension_date' => '2017-05-31',
'extended_date' => '2017-05-31',
),
)

How to convert mongoDB query into PHP?

My query is working in mongodb and produce output Properly. But same query i tried to run using php then it gives following error
Array
(
[ok] => 0
[errmsg] => A pipeline stage specification object must contain exactly one field.
[code] => 16435
)
Following is my MongoDB Query
db.sample_coll.aggregate(
{
$unwind: {
path:"$KeyValues",
includeArrayIndex:"arrayIndex",
preserveNullAndEmptyArrays:true
}
},
{
$project: {
timestamp:{
"$add":["$EventTS",{"$multiply":[60000,"$arrayIndex"]}]
} ,
"InputVolt":"$KeyValues.InputVolt",
arrayIndex:1
}
},
{
$match: {
$and: [
{InputVolt: {$ne: null}}
]
}
}
);
The Above query is converted in php
$pipeline = array(
array('$unwind' =>
array( 'path' => '$KeyValues'),
array( 'includeArrayIndex' => 'arrayIndex' ),
array( 'preserveNullAndEmptyArrays' => 'true' )
),
array(
'$project' => array(
'timestamp' => array(
'$add' => array(
'$EventTS',
array('$multiply' => array( 60000, '$arrayIndex' ))
)
),
array( 'InputVolt' => array( '$KeyValues', 'InputVolt' ) ) ,
array('arrayIndex' => 1)
)
),
array(
'$match' => array(
'$and' => array(
array('InputVolt' => array('$ne' => null )),
)
)
)
);
$result = $collection->aggregate($pipeline);
Kindly help to resolve this issue.
Thanks in advance
Your pipeline in PHP:
$pipeline = array(
array('$unwind' => array(
'path' => '$KeyValues',
'includeArrayIndex' => 'arrayIndex',
'preserveNullAndEmptyArrays' => true
)),
array('$project' => array(
'timestamp' => array(
'$add' => array('$EventTS', array('$multiply' => array(60000, '$arrayIndex')))
),
'InputVolt' => '$KeyValues.InputVolt',
'arrayIndex' => 1
)),
array('$match' => array(
'InputVolt' => array('$ne' => null)
))
);

mongodb aggregation php

I'm using mongodb 2.1 and how to translate this query into php
db.counter.aggregate([
{ $match:{ page_id:123456 }},
{ $group:{_id:"$page_id",total:{$sum:"$pageview"}} }
]);
Thanks
You can use the 'command()' method in PHP to run the aggregation framework as a database command. The precise syntax for your sample query would be:
$conn = new Mongo("localhost:$port");
$db = $conn->test;
$result = $db->command (
array(
"aggregate" => "counter",
"pipeline" =>
array(
array( '$match' => array( 'page_id' => 123456 )),
array( '$group' => array( "_id" => '$page_id',
'total' => array( '$sum' => '$pageview')
)
)
)
)
);
Try:
$m = new MongoClient();
$con = $m->selectDB("dbName")->selectCollection("counter");
$cond = array(
array('$match' => array('page_id' =>123456)),
array(
'$group' => array(
'_id' => '$page_id',
'total' => array('$sum' => '$pageview'),
),
)
);
$out = $con->aggregate($cond);
print_r($out);
For more detail click here

SQL select from three Tables and save in PHP array

I am working on a web application and would like to improve my code a little bit.
This is the SQL table structure I made for the application:
Now my question: Is there an easy way in PHP to select (with join or whatever?) the information from one specific event_id on all tables and save them in an array with followed structure for example:
$events = array(
'event_id' => 1,
'event_name' => '{"de":"Weltwirtschaftsforum","fr":"Forum \u00e9conomique mondial","it":"Forum Economico Mondiale"}',
'event_description' => '{"de":"Description DE","fr":"Description FR","it":"Description IT"}',
'event_lastedit' => 2011-12-01 10:23:35,
'locations' => array(
array(
'location_id' => 1,
'location_name' => 'Bern',
'location_date' => '01.01.2012',
'location_deadline' => 1323340607,
'timestamps' => array(
array(
'timestamp_id' => 1,
'timestamp_time' => '10:30',
'timestamp_seats' => 30
),
array(
'timestamp_id' => 2,
'timestamp_time' => '16:30',
'timestamp_seats' => 40
)
)
),
array(
'location_id' => 2,
'location_name' => 'Davos',
'location_date' => '02.02.2012',
'location_deadline' => 1323340607,
'timestamps' => array(
array(
'timestamp_id' => 3,
'timestamp_time' => '12:30',
'timestamp_seats' => 50
),
array(
'timestamp_id' => 4,
'timestamp_time' => '15:30',
'timestamp_seats' => 60
)
)
)
)
);
I hope the question is clear enough.
Greetings
Spinne
Edit:
What i did so far:
$event = $event_db->querySingle("SELECT * FROM rf_events WHERE event_id={$event_id}", true)
$rf_locations = $event_db->query("SELECT * FROM rf_locations WHERE event_id={$event_id}");
$locations = array();
$timestamps = array();
$counter = 0;
// Loop sql query result and save entries in $events array.
while ( $row = $rf_locations->fetchArray() ){
$locations[$counter] = array(
'location_id' => $row['location_id'],
'location_name' => json_decode($row['location_name'], true),
'location_date' => $row['location_date'],
'location_deadline' => $row['location_deadline']
);
$rf_timestamps = $event_db->query("SELECT * FROM rf_timestamps WHERE location_id={$row['location_id']}");
$counter2 = 0;
while ( $row2 = $rf_timestamps->fetchArray() ){
$locations[$counter]['timestamps'][$counter2] = array(
'timestamp_id' => $row2['timestamp_id'],
'timestamp_time' => $row2['timestamp_time'],
'timestamp_seats' => $row2['timestamp_seats']
);
$counter2++;
}
$counter++;
}
SELECT * FROM rf_events, rf_locations, rf_timestamps WHERE
rf_locations.event_id = rf_events.event_id AND
rf_timestamps.location_id = rf_locations.event_id
Then, use add the data into php accordingly.
You can refer to: MYSQL JOIN

Categories