I have documents with a array property timestamps as seen in the document below. I wish to find all documents with timestamps between two dates. My find query looks as follow:
$subset = $db->col->find(
array(
'timestamps'=> array(
'$elemMatch'=> array(
'$gte'=> new MongoDate(strtotime('2016-05-25 13:00:00')),
'$lte'=> new MongoDate(strtotime('2016-05-25 14:00:00'))
)
)
)
);
But it does not return the expected documents. I tried to Google and search StackOverflow for answer but I cannot find anything explaining how to search for timestamps in a array, between two dates.
EDIT:
It seems the problem lies with PHP because the same Python query works:
import datetime
f = '%Y-%m-%d %H:%M:%S'
c = db.col.find({
'timestamps': {
'$elemMatch': {
'$gte': datetime.datetime.strptime('2016-05-25 13:00:00', f),
'$lte': datetime.datetime.strptime('2016-05-25 14:00:00', f)
}
}
})
Any help would be appreciated.
{
"month": {
"$date": 1464134400000
},
"timestamps": [
{
"$date": 1464181803000
},
{
"$date": 1464182103000
},
{
"$date": 1464182403000
},
{
"$date": 1464183003000
},
{
"$date": 1464183603000
},
{
"$date": 1464184203000
},
{
"$date": 1464184803000
},
{
"$date": 1464185403000
},
{
"$date": 1464186003000
}
],
"status_history": [
1,
1,
1,
0,
1,
1,
1,
0,
1
],
"_id": 12345
}
I had a look at find in array between dates in mongo and Get data between two dates mongo.
It was a timezone issue. I created a DateTime and set the timezone to Etc/GMT and the expected data returned correctly.
$gte = new MongoDate (new DateTime( '2016-05-25 13:00:00', new DateTimeZone( 'Etc/GMT' )));
$lte = new MongoDate (new DateTime( '2016-05-25 14:00:00', new DateTimeZone( 'Etc/GMT' )));
Related
I have an array of $dates like below
[
"2022-30",
"2022-31",
"2022-32",
"2022-33",
"2022-34",
"2022-35",
]
and I have a $collection with an output like below
[
{
"new": 60,
"settled": "1",
"date": "2022-31"
},
{
"new": 50,
"settled": "1",
"date": "2022-32"
},
]
how can I achieve a result like below which merge the value of date from the $collection item if it matches from the date on $dates array
[
{
"new": 0,
"settled": "0",
"date": "2022-30"
},
{
"new": 60,
"settled": "1",
"date": "2022-31"
},
{
"new": 50,
"settled": "1",
"date": "2022-32"
},
{
"new": 0,
"settled": "0",
"date": "2022-33"
},
{
"new": 0,
"settled": "0",
"date": "2022-34"
},
{
"new": 0,
"settled": "0",
"date": "2022-35"
}
]
I tried making the dates into a collection and formatting the output like the $collection format and use merge but this just combine the two collection together with duplicates.
$out = collect($dates)->map( function($d, $k) {
return [
'new' => 0,
'settled' => 0,
'date' => $d
];
});
return $out->merge($collection);
appreciate any help
$dates = [
"2022-30",
"2022-31",
"2022-32",
"2022-33",
"2022-34",
"2022-35",
];
$collection = collect([
[
"new" => 60,
"settled" => "1",
"date" => "2022-31"
],
[
"new" => 50,
"settled" => "1",
"date" => "2022-32"
]
]);
foreach ($dates as $date) {
if (!$collection->where('date', $date)->count()) {
$collection->push([
'new' => 0,
'settled' => 0,
'date' => $date
]);
}
}
return $collection->sortBy('date')->values()->all();
You can key the collection by dates and then add in the dates that are not in there:
$out = $collection->keyBy('date')
$out = $out->merge(
collect($dates)
->whereNotIn(null, $out->keys())
->mapWithKeys(fn ($date) => [
$date => [
'new' => 0,
'settled' => 0,
'date' => $date
]
])
)->sortKeys()->values();
What this does is convert the collection to one that has the dates as keys, then filters from dates all the dates that are already in the collection, maps the rest into your desired format, sorts them all to how they should be then discards the keys.
For the $dates maybe you can filter only the dates that does not exist in the collections? Then after you can map them out and merge.
Okay so not 100% sure how to even ask this correctly but I have an array that I'm trying to consolidate and group together based on each hour, and then data for each year underneath that time.
My current array looks like this:
[
{
"name": "7:00 AM",
"data": [
{
"x": "2019",
"y": 1
}
]
},
{
"name": "7:00 AM",
"data": [
{
"x": "2020",
"y": 221
}
]
}
]
So I'm looking to combine these into a single array based on the name value and then all of the data will be combined:
[
{
"name": "7:00 AM",
"data": [
{
"x": "2019",
"y": 1
},
{
"x": "2020",
"y": 221
}
]
}
]
EDIT: And just to add how I'm originally getting and formatting these values, I'm using Larvel and running the following query:
$shipments = Orders::whereNotNull('shipped_at')->groupBy(DB::raw('YEAR(created_at)'))->selectRaw('count(*) as count, HOUR(shipped_at) as hour')->selectRaw("DATE_FORMAT(created_at, '%Y') label")->orderBy('label')->orderBy('hour')->groupBy('hour')->get();
And that gives me the following data:
And then I'm building the original array like this:
$shipping_chart_year = [];
foreach ($shipments as $item) {
$hour = Carbon::parse($item['hour'] . ':00:00')->timezone('America/Chicago')->format('g:i A');
$shipping_chart_year[] = ['name' => $hour, 'data' => [['x' => $item['label'], 'y' => $item['count']]]];
}
This functions formats array as you expected to have.
function formatArray($array){
$response = [];
$keysIndexes = [];
foreach($array as $value) {
if(!array_key_exists($value['name'], $keysIndexes)) {
array_push($response, $value);
$keysIndexes[$value['name']] = sizeof($keysIndexes);
} else {
array_push($response[$keysIndexes[$value['name']]]['data'], $value['data'][0]);
}
}
return $response;
}
if you want to achieve this with the Laravel Collections (you can do so much with it!), here you go (based on your current array structure).
It is much more readable and way easier to achieve.
$output = collect(json_decode($data, true))
->groupBy('name')
->transform(function ($items, $name) {
// Merge nested arrays into one
$data = $items->map(function ($item) {
return $item['data'][0];
})->toArray();
return [
'name' => $name,
'data' => $data,
];
})
->values() // removes the key
->toJson(); // returns the collection to json
Hi I am MySQL user and new to mongoDB.
I get data from my IOT devices as following:
$body={
"key": "121239",
"secrete": "Your_Device_Secrete",
"data": [
{
"Temperature":50,
"Humidity":30,
"Vibration":100,
"Time":"2020-1-26 00:00:01"
},
{
"Temperature":55,
"Humidity":34,
"Vibration":50,
"Time":"2020-1-26 00:00:02"
}
]
}
I am inserting it into mongoDB using PHP CodeIgnitor as following:
$this->mongo->batch_insert($body["key"],$body["data"]);
data is getting inserted in collection named as given key as following:
{
"Temperature": 50,
"Humidity": 30,
"Vibration": 100,
"Time": "2020-1-26 00:00:01",
"_id": {
"$id": "5e330be3f7577f640d2a0922"
}
},
{
"Temperature": 55,
"Humidity": 34,
"Vibration": 50,
"Time": "2020-1-26 00:00:02",
"_id": {
"$id": "5e330be3f7577f640d2a0923"
}
}
Now I want to add timestamp to every row getting inserted. I want data getting inserted as following :
{
"Temperature": 50,
"Humidity": 30,
"Vibration": 100,
"Time": "2020-1-26 00:00:01",
"_id": {
"$id": "5e330be3f7577f640d2a0922"
},
timestamp:<CURRUNT TIME>
},
{
"Temperature": 55,
"Humidity": 34,
"Vibration": 50,
"Time": "2020-1-26 00:00:02",
"_id": {
"$id": "5e330be3f7577f640d2a0923"
},
timestamp:<CURRUNT TIME>
}
is there any way to make mongoDB add current timestamp auto like MySQL?
First of all you should not store date/time values as string! It will just generate trouble at later times. Better use
"Time": ISODate("2020-01-26T00:00:02Z")
"Time": new Date("2020-01-26 00:00:02") may also work
Then in general you don't need to insert a timestamp. Every document gets an internal _id identifier which contains also the time when it was inserted. You can try like this:
db.collection.aggregate([
{
$project: {
timestamp: {
$toDate: "$_id"
},
key: 1,
secrete: 1,
data: 1,
_id: 0
}
}
])
Result:
{
"data": [
{
"Humidity": 30,
"Temperature": 50,
"Time": "2018-02-26 01:00:00",
"Vibration": 100
},
{
"Humidity": 34,
"Temperature": 55,
"Time": "2018-02-26 01:00:00",
"Vibration": 50
}
],
"key": "121239",
"secrete": "Your_Device_Secrete",
"timestamp": ISODate("2018-02-26T00:00:00Z")
}
You can also query for it, e.g.
db.collection.find({
$expr: {
$gte: [ {"$toDate": "$_id"}, ISODate("2020-01-03T11:00:00Z") ]
}
})
However, if you like to insert timestamp explicitly then simply do it like this.
{
"Temperature": 50,
"Humidity": 30,
"Vibration": 100,
"_id": {
"$id": "5e330be3f7577f640d2a0923"
},
"timestamp": new Date()
}
Or "timestamp": ISODate() should also work
Another note, this embedded document "_id": {"$id": "5e330be3f7577f640d2a0923"} looks really strange. I assume this is a bug. The string look like a ObjectId object used for _id. Somehow you scratched it.
Update
I really recommend to use proper data types. In PHP the insert could look like this:
$mongo->db->collection->insertOne([
"timestamp" => new MongoDB\BSON\UTCDateTime(NULL),
"key" => "121239",
"secrete" => "Your_Device_Secrete",
"data" => [
[
"Temperature" => 50,
"Humidity" => 30,
"Vibration" => 100,
"Time" => new MongoDB\BSON\UTCDateTime(new DateTime("2020-1-26 00:00:01"))
],
[
"Temperature" => 55,
"Humidity" => 34,
"Vibration" => 50,
"Time" => new MongoDB\BSON\UTCDateTime(new DateTime("2020-1-26 00:00:02"))
]
]
]);
I also had similar requirements as yours while reading sensor values from DHT11 sensor in my IoT class. I was able to insert all the temperature and humidity readings into MongoDB but later while plotting the graph I realized that I would need the timestamps as well. After researching for a while, I came up with this query. However, it will create a new collection in the database.
Here, the new collection name is dht11-sensor-readings.
Please also refer to https://docs.mongodb.com/manual/reference/operator/aggregation/out/ for more.
db.collection.aggregate([
{
$addFields:{timestamp:{$toDate:"$_id"}
}
},
{ $out:"dht11-sensor-readings"
}
])
I use service API which provide a schedule in json in format describes date ranges:
.... // Other items
{
"registration": "SP-TEST",
"records": [
{
"from": "2014-12-06T13:40Z",
"available": true
},
{
"from": "2014-12-07T14:30Z",
"available": false
},
{
"from": "2014-12-13T14:30Z",
"available": true
},
{
"from": "2014-12-13T16:30Z",
"available": false
},
{
"from": "2014-12-15T14:30Z",
"available": true
}
]
},
....
But it uncomfortable for use and search. I need import it to MySQL DB and perform search in date range where is available, so I need combine arrays something like:
[{
"registration": "SP-TEST",
"from": "2014-12-06T13:40Z",
"to": "2014-12-07T14:30Z"
},
{
"registration": "SP-TEST",
"from": "2014-12-13T14:30Z",
"to": "2014-12-13T16:30Z"
},
{
"registration": "SP-TEST",
"from": "2014-06-06T13:40Z",
"to": "2014-06-07T14:30Z"
},
{
"registration": "SP-TEST",
"from": "2014-12-15T14:30Z",
"to": "2014-02-07T14:30Z"
}]
I use usort function to sort by time source array (json_decode($schedule)):
usort($schedule->records, function($a, $b) {
return strtotime($a->from) - strtotime($b->from);
});
So, if this code is correct I can use foreach to populate new array, but it does not work, because a little problem: "records" can contain just one record. It can have "available": true or "available": false, which means that it available or not from current date up 2 month.
Maybe somebody prompts me a right way?
Resolved. Sort by date, then foreach.
function schedule($records) {
date_default_timezone_set('UTC'); // 0 timezone
$result = array();
foreach ($records as $k => $v) {
$record = array_merge($record, array(
'available' => $v->available,
'location' => $v->location,
'from' => $v->from,
'depart' => '',
'arrive' => '',
'to'=> date("Y-m-d\TH:i\Z", strtotime("+2 month", strtotime($v->from)))
));
if (count($records) > 1) {
$record['to'] = date("Y-m-d\TH:i\Z", strtotime("+2 month", strtotime($record['from'])));
}
if (isset($records[$k+1])) {
$record['to'] = $records[$k+1]->from;
$record['depart'] = $v->location;
$record['arrive'] = $records[$k+1]->location;
}
$result[] = $record;
}
return $result;
}
I have my data in mongodb database as
{
"_id": ObjectId("5319acf0b06f6e98371ca505"),
"field1": "",
"date": "07-03-2014",
"os": "android",
"time": "11:26:40",
"userid": "xxxx"
}
I wanted to get count of all records having time from 00 to 23.
I have written my query as
$time_start = date('00');
$time_end = date('23');
$keys = array("time" => 1);
$initial = array("counttime" => array());
$reduce = "function(obj, prev) {
if (obj.time != null)
if (obj.time instanceof Array) prev.counttime += obj.time.length;
else prev.counttime++; }";
$condition = array(
'condition' => array(
"time" => array(
'$gt' => $time_start,
'$lte' => $time_end
)
)
);
$g = $collection->group($keys, $initial, $reduce , $condition);`
I have tried to get hour but thats not working for me.
I have used as
$condition = array(
'condition' => array(
"HOUR(time)" => array(
'$gt' => $time_start,'$lte' => $time_end)
)
);
Anyone who can help me out?
You can do this using .aggregate(), which you really should favor over .group() in any case, as it uses native code rather than JavaScript for it's results:
db.collection.aggregate([
// Match one date (tell you later)
{ "$match": { "date": "07-03-2014" } },
// Project the hour
{ "$project": {
"hour": { "$substr": [ "$time", 0, 2 ] }
}},
// Group on the hour
{ "$group": {
"_id": "$hour",
"count": { "$sum": 1 }
}}
])
Now this relies on that you have your "time" stored as a string. So you $project the "hour" part of that "time" and then $group on it, keeping a count of records found.
I restricted this to one "date" because as with "time" this is also a string. Depending on what you want to do you can possibly get unexpected results from this, so generally speaking, using "strings" for date representation is not a good idea.
If you just had a proper BSON date in a field called "timestamp" then you can do things like this, where you can even set ranges:
var start_date = new Date("2014-02-01");
var end_date = new Date("2014-03-01");
db.collection.aggregate([
// Match on the date range
{ "$match": { "timestamp": { "$gte": start_date, "$lt": end_date } },
// Project day and hour
{ "$project": {
"day": {
"year": { "$year": "$timestamp" },
"month": { "$month": "$timestamp" },
"day": { "$dayOfMonth": "$timestamp" },
"hour": { "$hour": "$timestamp" }
}
}},
// Group on the day
{ "$group": {
"_id": "$day",
"count": { "$sum": 1 }
}}
])
But with your present date format, ranges are going to be a problem. So best to convert where you can.