I'm new with MapReduce, but I have a collection that I'd like to apply myself to as a chance to learn how mapreduce works.
Example Documents:
{ "filename" : "resume.doc",
"folder" : "work",
"completed": "0.5" },
{ "filename" : "spreadsheet.xls",
"folder" : "work",
"completed": "0.6" },
{ "filename" : "thesis.doc",
"folder" : "school",
"completed": "0.75" },
{ "filename" : "coverletter.doc",
"folder" : "work",
"completed": "0.6"}
So the whole idea is: I'd like to query:
{ "folder" : "work",
"completed": { $gt: 0.5 },
"filename" : new MongoRegex( "/\.[a-zA-Z]{2,}$/" ) }
And ultimately get the count of all documents by extension (.doc, .xls, etc.), as so:
{ ".doc" : 1,
".xls" : 1 }
I also realize i've got an issue because my %-completed are strings, not floats, so i think mongodb is going to need more instruction for comparing the strings.
I'm using (if it matters):
PHP extension: mongo/1.5.7
MongoDB: version 3.2.11
Seems I've stumbled upon my own answer.
Let me know if anybody comes up with a more concise/expert solution.
But this seems to work.
try {
$map = new MongoCode(
'function(){
var re = new RegExp(/(.+)\.([a-zA-Z]{2,})$/);
doctype = this.value.match(re);
if(parseFloat(this.completed)>0.5){
emit(doctype[2], parseFloat(this.completed));
}
}'
);
$reduce = new MongoCode(
'function(key, values){
var sum = 0, num = 0;
for(var i in values){
if(parseFloat(values[i])>0.5){
sum += values[i];
num += 1;
}
}
return { number_of_documents : num,
sum_of_document_completions : sum,
average_completion : (sum/num) };
}'
);
$query = array (
"folder" => "work"
);
$doctypes = $db->command(
array(
'mapReduce' => 'mydocuments',
"map" => $map,
"reduce" => $reduce,
"query" => $query,
"out" => array("inline"=>1)
)
);
print_r ( $doctypes);
}
catch(MongoCursorException $e) {
echo "error message: ".$e->getMessage()."\n";
echo "error code: ".$e->getCode()."\n";
}
Related
This question already has answers here:
How to loop through PHP object with dynamic keys [duplicate]
(16 answers)
Closed 5 years ago.
{
"took" : 363,
"timed_out" : false,
"num_reduce_phases" : 15,
"_shards" : {
"total" : 7195,
"successful" : 7195,
"failed" : 0
},
"hits" : {
"total" : 35672,
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"2" : {
"doc_count_error_upper_bound" : 54,
"sum_other_doc_count" : 1463,
"buckets" : [
{
"key" : "ふるさと納税",
"doc_count" : 30376
}
]
}
}
}
This is the json file . I need to access the doc_count field.
I am trying to do it like this
$trend_words = json_decode($file);
$aggregations = $trend_words->aggregations;
$buckets = $aggregations->{2}->buckets->{0};
But its not working.
Can someone help.
$trend_words = json_decode($file,true);
$aggregations = $trend_words['aggregations']['2']['buckets'][0]['doc_count'];
print_r($aggregations)
You should use ,true otherwise you will get an stdclass object and not an array you want.
try this
<?php
$sJson = '
{
"took" : 363,
"timed_out" : false,
"num_reduce_phases" : 15,
"_shards" : {
"total" : 7195,
"successful" : 7195,
"failed" : 0
},
"hits" : {
"total" : 35672,
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"2" : {
"doc_count_error_upper_bound" : 54,
"sum_other_doc_count" : 1463,
"buckets" : [
{
"key" : "ふるさと納税",
"doc_count" : 30376
}
]
}
}
}
';
$aJson = json_decode($sJson, true);
echo $aJson['aggregations'][2]['buckets'][0]['doc_count'];
https://3v4l.org/kHfdo
Here is the answer, As the buckets increases you need to have a loop through to read all the buckets
$trend_words = json_decode($file, true);
$aggregations = $trend_words['aggregations'][2];
$buckets = array();
foreach($aggregations as $element)
$buckets[] = $element['buckets'];
print_r($buckets);
Make Json to array and try
$trend_words = json_decode($file, true);
$aggregations = $trend_words['aggregations'];
$buckets =$aggregations[2]['buckets'][0];
I am working with highcharts to make a drilldown chart. This will display multiple projects, with an average value and will drilldown in columns with every value. The problem is, I can't get the average number per projects..
My JSON from the mongo looks like this:
{
"_id" : ObjectId("57f4dd6c45db428df9a5541b"),
"id" : 68,
"project_code" : 13110202,
"samples" : [
{
"sample_name" : "1678_2",
"perc" : "75.97"
},
{
"sample_name" : "2012_1B-5",
"perc" : "75.33",
},
{
"sample_name" : "2012_1B-8",
"perc" : "77.28",
}]
}, {
"_id" : ObjectId("57f4dd6c45db428df9a5541a"),
"id" : 70,
"project_code" : 1235125,
"samples" : [
{
"sample_name" : "Ben",
"perc" : "30.97"
},
{
"sample_name" : "Anna",
"perc" : "81.25",
},
{
"sample_name" : "Sarah",
"perc" : "80.23",
}]
},
I am trying to look through the document, through the embedded document samples, get all the perc_mapped values, calculate the average and put this in a JSON used for the HighChart. The code which I have now:
foreach ($id_array as $char) {
$cursor = $collection->find(array("_id" => new MongoId($char)));
foreach ($cursor as $document) {
if (is_array($document['samples'])) {
foreach ($document['samples'] as $samples) {
array_push($perc_total, $samples['perc']);
$average_of_foo = array_sum($perc_total) / count($perc_total);
}
}
}
}
var_dump($average_of_foo);
foreach ($id_array as $char) {
$cursor = $collection->find(array("_id" => new MongoId($char)));
foreach ($cursor as $document) {
if (is_array($document['samples'])) {
foreach ($document['samples'] as $samples) {
$array =
array("name" => $document['project_code'],
// Here is will add the average
"y" => (float) $samples['perc'],
"drilldown" => $char
);
array_push($array_data, $array);
}
}
}
}
so I have some PHP code which is displaying an email list (Name, Surname & Email) from a list I have hosted via API.
The problem I have is that in order to display the name of the subscriber, I have to separately pass the email through the API... and the only way to get the email is to use a separate function.
So, for a list of 100 people, I'm basically hitting calling their API 300 times... so it is taking forever to echo the information (and over 30ish people the server just times out).
Is there a way I can edit the blow code so that I'm not pulling their info EVERY time, and instead bring it all down at once?
Here's my code:
<?php
require('Mailin.php');
/////////Total Subscriber Count $subscribers
$mailin = new Mailin('https://api.sendinblue.com/v2.0','UNIQUEKEY');
$datacount = array( "id"=>13 );
$subscribercount = $mailin->get_list($datacount);
$subscribers = $subscribercount['data']['total_subscribers'] - 1;
/////////Get Emails of Subscribers
$dataemail = array( "listids" => array(2),
"page" => 1,
"page_limit" => 500
);
$getemails = $mailin->display_list_users($dataemail);
/////////Get Name of Subscribers
foreach (range(0, $subscribers) as $number) {
$subscriberemail = $getemails['data']['data'][$number]['email'];
echo $subscriberemail;
echo '<br/>';
$dataname = array( "email" => $subscriberemail );
$getname = $mailin->get_user($dataname);
$subscribername = $getname['data']['attributes']['NAME'];
$subscribersurname = $getname['data']['attributes']['SURNAME'];
echo $subscribername;
echo '<br/>';
echo $subscribersurname;
echo '<br/>';
}
?>
Here is the output you get calling the display_list_users:
{
"code":"success",
"message":"Retrieved details of all users for the given lists",
"data":{
"data":[
{
"blacklisted":0,
"email":"email1#domain.com",
"id":1,
"listid":[1],
"blacklisted_sms":1,
"last_modified" : "2015-05-22 15:30:00"
},
{
"blacklisted":1,
"email":"email2#domain.com",
"id":2,
"listid":[1,2],
"blacklisted_sms":0 ,
"last_modified" : "2015-05-25 19:10:30"
}
],
"page":1,
"page_limit":2,
"total_list_records":100
}
}
And the output you get calling the get_user:
{
"code":"success",
"message":"Data retrieved for email",
"data":{
"attributes":{
"EMAIL":"example#example.net",
"NAME" : "Name",
"SURNAME" : "surname"
},
"blacklisted":1,
"email":"example#example.net",
"entered":"2014-01-15",
"listid":[8],
"message_sent":[{
"camp_id" : 2,
"event_time" : "2013-12-18"
},
{ "camp_id" : 8,
"event_time" : "2014-01-03"
},
{ "camp_id" : 11,
"event_time" : "2014-01-07"
}],
"hard_bounces":[{
"camp_id" : 11,
"event_time" : "2014-01-07"
}],
"soft_bounces":[],
"spam":[{
"camp_id" : 2,
"event_time" : "2014-01-09"
}],
"unsubscription":{
"user_unsubscribe":[
{
"event_time":"2014-02-06",
"camp_id":2,
"ip":"1.2.3.4"
},
{
"event_time":"2014-03-06",
"camp_id":8,
"ip":"1.2.3.4"
}
],
"admin_unsubscribe":[
{
"event_time":"2014-04-06",
"ip":"5.6.7.8"
},
{
"event_time":"2014-04-16",
"ip":"5.6.7.8"
}
]
},
"opened":[{
"camp_id" : 8,
"event_time" : "2014-01-03",
"ip" : "1.2.3.4"
}],
"clicks":[],
"transactional_attributes":[
{
"ORDER_DATE":"2015-07-01",
"ORDER_PRICE":100000,
"ORDER_ID":"1"
},
{
"ORDER_DATE":"2015-07-05",
"ORDER_PRICE":500000,
"ORDER_ID":"2"
}
],
"blacklisted_sms":1
}
}
So I have this JSON for example
{
"top" : [
{
"info" : {
"ID" : 0,
"TID" : 1
},
"geo" : {
"poins" : [
[
[
-5.9,
57.1
],
[
-5.99,
57.0
]
]
]
}
},
{
"info" : {
"ID" : 1,
"TID" : 2
},
"geo" : {
"points" : [
[
[
-5.4,
57.0
],
[
-5.9,
57.0
]
]
]
}
}
]
}
I need to put this information in the DataBase with php
So I have a colum in the DB called points and it need the data inside to be looking like:
[-5.4, 57.0],[-5.9, 57.0]
I have a columb with ID so all I need is to put the points from the JSON for every ID
My php should be loking like:
connection to the database
$str = file_get_contents(the JSON);
$json = json_decode($str, true);
foreach ($json['top'] as $field) {
query='UPDATE poins_table
SET points='$field['geo']['points']'
WHERE ID='$field['info']['ID']' '
}
The code seems to be not working. What I am missing ...Any siggestions will be helpfull. Thank you
You could try following:
<?php
$str = file_get_contents('the JSON URL');
$json = json_decode($str, true);
try{
$db = new PDO("dbtype:host=yourhost;dbname=yourdbname;charset=utf8","username","password");
$query = $db->prepare('UPDATE poins_table SET points=? WHERE ID=?');
foreach ($json['top'] as $field) {
$query->execute(array(json_encode($field['geo']['points']), $field['info']['ID']));
}
} catch(PDOException $e) {
echo "Error: ". $e;
}
?>
I'm trying to search my collection for all occurrences where the body property contains all of the search keywords.
Example string - "The black cat is definitely purple."
Keywords "black", "purple" would return the string.
Keywords "black", "dog" would not return that string.
I've been cruising some topics and Googling, but cannot seem to find the proper syntax to do this.
Currently, I am taking an string of keywords separated by commas, exploding it into an array, and then putting that into a MongoRegex Object. I know my syntax is off, because when I send just one keyword it works, but when there is more than one, I do not get any results that I would expect to get.
Current Approach:
<?php
function search_topics($array)
{
include_once('config.php');
$collection = get_connection($array['flag']);
$x = 0;
$string = null;
$search_results = null;
$keywords = explode(',', $array['search']);
$end_of_list = count($keywords);
while ($x < $end_of_list)
{
$string = $string."/".$keywords[$x];
$x++;
if($x >= $end_of_list)
{
$string = $string."/i";
}
}
if ($string != null)
{
try
{
$regex_obj = new MongoRegex($string);
$cursor = $collection->find(array('body' => $regex_obj));
}
catch (MongoCursorException $e)
{
return array('error' => true, 'msg' => $e->getCode());
}
foreach($cursor as $post)
{
$search_results[] = $post;
}
if ($search_results != null && count($search_results) > 1)
{
usort($search_results, 'sort_trending');
}
return array('error' => false, 'results' => $search_results);
}
else
{
return array('error' => false, 'results' => null);
}
}
?>
So, if I send the string black in $array['search'], my object is formed with /black/i and would return that string.
If I send the string black,cat in $array['search'], my object is formed with /black/cat/i and returns null.
Can anyone point me in the right direction with this regex syntax stuff?
Thanks in advance for any help!
Nathan
Instead of regular expressions, I would suggest you look at MongoDB's text search functionality instead, which is specifically made for situations like this: http://docs.mongodb.org/manual/core/text-search/
You would use that like this (on the MongoDB shell):
use admin
db.runCommand( { setParameter: 1, 'textSearchEnabled' : 1 } );
use test
db.so.ensureIndex( { string: 'text' } );
db.so.insert( { string: "The black cat is definitely purple." } );
db.so.runCommand( 'text', { search: '"cat" AND "dog"' } )
db.so.runCommand( 'text', { search: '"cat" AND "purple"' } )
A command doesn't return a cursor, but instead it will return one document containing all the query results in the results field. For the last search command, the result is:
{
"queryDebugString" : "cat|purpl||||cat|purple||",
"language" : "english",
"results" : [
{
"score" : 2.25,
"obj" : {
"_id" : ObjectId("51f8db63c0913ecf728ff4d2"),
"string" : "The black cat is definitely purple."
}
}
],
"stats" : {
"nscanned" : 2,
"nscannedObjects" : 0,
"n" : 1,
"nfound" : 1,
"timeMicros" : 135
},
"ok" : 1
}
In PHP, for the runCommand to turn on text search, you'd use:
$client->database->command( array(
'setParameter' => 1,
'textSearchEnabled' => 1
) );
And the text search itself as:
$client->database->command( array(
'text' => 'collectionName',
'search' => '"cat" AND "purple"'
) );