I have following mongo document structure.
{
"geo": {
"0": {
"deliveryArea": {
"0": {
"lat": 50.449234773334,
"lng": 30.52300029031
},
"1": {
"lat": 50.449234773334,
"lng": 30.52542980606
},
"2": {
"lat": 50.45154573136,
"lng": 30.52542980606
},
"3": {
"lat": 50.45154573136,
"lng": 30.52300029031
}
},
"title": "Kiev, ....",
"coords": {
"lat": "50.4501",
"lngt": "30.523400000000038"
},
"wholeCityDelivery": "false"
}
Questions:
How to ensure_index for every element in deliveryArea [lat,lng] for geolocationg?
How to build query to find documents with the point N[lat,lng] in(belongs to) polygon deliveryArea.$
If you can do PHP - please)
Thanks!
First of all, you need to change your data structure. MongoDB wants "longitude, lattitude" and it doesn't care which names you give to the fields—in general, I would use them without the field names. You also need to store your document without the "0" keys
So like:
{
"geo": {
"deliveryArea": [
[ 30.52300029031, 50.449234773334 ],
]
}
}
Then you need to set a 2D index on "geo.deliveryArea":
$collection->ensureIndex( array( "geo.deliveryArea" => "2d" ) );
Information on how to build queries can be at http://www.mongodb.org/display/DOCS/Geospatial+Indexing#GeospatialIndexing-BoundsQueries However, in your case you want to find a delivery area for a point, instead of checking whether stored points fit in a given area, and MongoDB can not do that directly as its 2d index can only store points.
Please file a feature request at http://jira.mongodb.org
Related
by doing a GET-Request, I am receiving routing informations from the HERE Routing API:
https://router.hereapi.com/v8/routes?apikey=MY_API_KEY&destination=52.530394,13.400683&origin=52.530728,13.383833&return=polyline,travelSummary&transportMode=truck&&vehicle[speedCap]=30&spans=names,length,duration,speedLimit
Now I want to find the coordinates for example in the middle of the route with respect to the total time. So I the example below, the total duration is 274 seconds. How can I find out, on which position I will be after 137 seconds? (In real application these times are much longer. Here, for simplicity and for a small JSON file size, I have chosen only a short distance)
First, I thought of adding starting and ending coordinates of the spans, however it seems not to be possible with the API.
Second, I thought of using the polyline. From that I receive a lot of coordinates, however I don't see a possiblity to connect one of these coordinates to a certain duration of travel.
Is there any way how I can get the information I am looking for with the HERE Routing API or with any PHP calculation?
{
"routes": [
{
"id": "90be4eb8-d0ba-47f8-9954-9be444576a17",
"sections": [
{
"id": "bfd32e45-662b-4b7e-a297-21eeee09dd68",
"type": "vehicle",
"departure": {
"time": "2021-12-11T23:42:04+01:00",
"place": {
"type": "place",
"location": {
"lat": 52.5307744,
"lng": 13.3838015
},
"originalLocation": {
"lat": 52.5307279,
"lng": 13.383833
}
}
},
"arrival": {
"time": "2021-12-11T23:46:38+01:00",
"place": {
"type": "place",
"location": {
"lat": 52.5303982,
"lng": 13.4006967
},
"originalLocation": {
"lat": 52.5303939,
"lng": 13.4006829
}
}
},
"travelSummary": {
"duration": 274,
"length": 1338,
"baseDuration": 264
},
"polyline": "BGslnmkDyn8wZ8CmL4Iof0F0U8BoGsEoQwCsJsEkSoBoG8BsJsE0U8BgK8BoLoB4IoB0KoBoLoBkNwC8a8B0UoB0UoBkNsEgtBkDsd8BsTkDgZsEgtB4D0jBgFwvBoG46B8B8QwCoV8BwMgFgtBUwHkD8akDgeU4NoB4XAkIoB0ZoB8pBU0K8Boa8B0PkDkS7GkD3I0F3DwC7foa7G0Fzeoaze0ZvTiQ",
"spans": [
{
"offset": 0,
"names": [
{
"value": "Invalidenstraße",
"language": "de"
}
],
"length": 189,
"duration": 31,
"speedLimit": 13.8888893
},
{
"offset": 11,
"names": [
{
"value": "Invalidenstraße",
"language": "de"
}
],
"length": 872,
"duration": 184,
"speedLimit": 8.333334
},
{
"offset": 44,
"names": [
{
"value": "Brunnenstraße",
"language": "de"
}
],
"length": 277,
"duration": 59,
"speedLimit": 8.333334
}
],
"transport": {
"mode": "truck"
}
}
]
}
]
}
Using the information in the spans object is definitely the way to go. What you need is to break up the spans into as many pieces as possible. You can do that by adding these values to the parameter in your request:
&spans=duration,length,segmentId,names,speedLimit,dynamicSpeedInfo,baseDuration,typicalDuration,segmentRef
You'll see that the response includes a list of spans identified by the offset attribute, which tells you what coordinate in your polyline that span refers to. This means that you want to know what is the offset (coordinate index) where the sum of span durations is 137.
This procedure will get you the best approximation to the middle of the route relative to travel time:
Loop through the list of spans and sum the value in the duration attribute; the loop should stop when the sum is equal or greater than the desired duration (137 in your example).
Get the value of the offset attribute, and add 1.
Decode your polyline, and get the coordinates at the index that is equal to the number you got in step 2 (offset + 1).
For the route in your example, the span that meets the condition in step 1 is offset=31, so you're interested in the coordinates at index 32 from your polyline.
I'm trying to have an array that is gonna be passed as JSON to an angular View.
The structure of the sql result is roughly :
Order Number
Supplier
Qty
Date
OrderState
ProductA(including size)
GenericA(excluding size)
LibA
TypeA(Package,Batch,Product)
SizeA
SizeNameA
ProductB
GenericB
LibB
TypeB
SizeB
SizeNameB
QtyPerSizeB
ProductC
GenericC
LibC
TypeC
SizeC
SizeNameC
QtyPerSizeC
It is done with left joins, so the cost concerns the last non null product.
If it's a package :
ProductA -> Package
ProductB -> Batch (Child of A)
ProductC -> Product (Child of B)
If it's a batch or a package with no batches :
ProductA -> Package/Batch
ProductB -> Product
ProductC -> null
If it's only products :
ProductA -> Product
ProductB/C -> null
So I need to have a hierarchy of some sort so I can loop through these objects.
Something similar to this :
{
"order": {
"infos": {},
"products": {
"name": "x",
"type": "Package",
"sizes": {
"1": {
"qty": "",
"price": ""
},
"2": {
"qty": "",
"price": ""
}
},
"children": {
"name": "x",
"type": "Batch",
"sizes": {
"1": {
"qty": "",
"price": ""
},
"2": {
"qty": "",
"price": ""
}
},
"children": {
"name": "x",
"type": "Batch",
"sizes": {
"1": {
"qty": "",
"price": ""
},
"2": {
"qty": "",
"price": ""
}
},
}
}
}
}
}
I tried setting each of them by looping through all the results but due to the complexity, it takes too long to develop/modify/maintain.
So I was wondering if there was any way to specify another array containing keys and instructions (something like "parent of","sum of","child of"), and then merge the sql result into a multidimensional array that follows the structure above.
I look forward to reading your comments, you geniuses, and thanks for reading !
Raekh
Okay no responses.
I ended up creating classes that describe the data that I want.
If you're in need of a similar structure, contact me and I'll share how I did it.
In DynamoDB i have a table with the following structure.
The actions "field" contains all the info (and this is the field i would like to search into) and orderId it's the primary key
{
"actions": [
{
"actionDescription": "8f23029def1d6baa4",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533730680,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "21857e61037bc29ec",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731788,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "cf10abd44e24cef56",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731788,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "7787fe7a5bf4d22de",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "OOOOOO",
"userName": "ooooo#oooo.ooo",
}
},
{
"actionDescription": "9528c439021f504bf",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "bfba100e0e54934b2",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "f789dc12f1dbe3be2",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "OOOOOO",
"userName": "ooooo#oooo.ooo",
}
},
{
"actionDescription": "4cd6b68dfea7cf8ee",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "1e3a0e95f8e5106d7",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731790,
"user": {
"fullName": "OOOOOO",
"userName": "ooooo#oooo.ooo",
}
}
],
"orderId": "13aae31"
}
What i would like to do it's to make the scan terms in PHP to be able to search by userName. or by any field inside the actions array (timestamp, actionTitle, etc, etc).
Bellow it's one of the many terms i tried to use but i was unable to achieve any results
$params = [
'TableName' => $this->tableName,
'FilterExpression' => "userName = :searchTerm",
'ExpressionAttributeValues' => [
':searchTerm' => 'ooooo#oooo.ooo',
],
'ReturnConsumedCapacity' => 'TOTAL',
];
$results = $this->dynamoDbClient->scan($params);
Can you please guide my by telling me what i'm missing?
Also, please note: I don't want to get a specific orderId, i would like to get ALL orderIds containing the searchTerm (in this case userName)
Your best bet with this item schema is to filter the table items yourself. That is to say, scan the table with no filter expression and write your own code to filter the results. Scanning without the filter expression will consume the same amount of read capacity units.
You can set the filter expression to something like this, however this isn't scalable and only works if you have a fixed number of items in the actions list.
actions[0].user.userName == :searchTerm OR actions[1].user.userName == :searchTerm OR actions[2].user.userName == :searchTerm OR ....
If you need complex search abilities you are probably better off using a dedicated search database. AWS provides two services around this, AWS CloudSearch and AWS ElasticSearch. You can use DynamoDB streams to keep your search indexes up to date.
If you are set on scanning the DynamoDB table with a filter you can refactor your structure to include additional attributes that have all the searchable information in a set (or concatenated string)
{
"actions": [....],
"actionsDescriptions": Set["8f23029def1d6baa4", "21857e61037bc29ec", "cf10abd44e24cef56", "7787fe7a5bf4d22de", "9528c439021f504bf", "bfba100e0e54934b2", "f789dc12f1dbe3be2", "4cd6b68dfea7cf8ee", "1e3a0e95f8e5106d7"],
"actionTitles": Set["UNDEFINED_ACTION"],
"timestamps": Set[1533730680, 1533731788, 1533731789, 1533731790],
"user_fullNames": Set["XXXXX"],
"user_userNames": Set["ooooo#oooo.ooo", "xxxxx#xxxx.xxx"],
"orderId": "13aae31"
}
Notice you have to use a Set (or concatenate all the values into a string) since the contains functions only works on strings and sets.
Then you can use a filter expression like this
contains(user_userNames, :searchTerm)
The DynamoDB QueryFilter and ScanFilter options do not currently support the CONTAINS operator for maps. You'll need to build another lookup table indexed by userName to avoid scanning the entire table.
E.g. new table schema:
{
"userName": "xxxxx#xxxx.xxx"
"orderId": "13aae31"
}
Where the hash key is userName and orderId is the ID of an order in the other table.
The closest you can get with the current schema is to use #cementblocks's suggestions to scan the whole table and filter application-side or query each element in the list individually.
If you are adding a "Search" like feature to your application, then scanning may not be the best approach.
DynamoDB scan can be expensive and slow, especially when you have many rows.
So, if you intend on adding a "Search" feature you may consider using AWS CloudSearch. It is a scalable "Search" feature. You can quickly enable "Search" from a DynamoDB table.
I have read about json_encode but still lack the logic in using it for my needs on this particular JSON structure.
Assuming the JSON structure is as follows :
{
"_id": "23441324",
"api_rev": "1.0",
"type": "router",
"hostname": "something",
"lat": -31.805412,
"lon": -64.424677,
"aliases": [
{
"type": "olsr",
"alias": "104.201.0.29"
}
],
"site": "roof town hall",
"community": "Freifunk/Berlin",
"attributes": {
"firmware": {
"name": "meshkit",
"url": "http:k.net/"
}
}
}
Some of the values of the attributes will be taken from the database while some are going to be hardcoded(static) like "type","api_rev". I was thinking of just using concatenation to build the structure but learnt its a bad idea. So if i am to use json_encode how may i be able to handle this structure ? array dimensions etc.
document structure example is:
{
"dob": "12-13-2001",
"name": "Kam",
"visits": {
"0": {
"service_date": "12-5-2011",
"payment": "40",
"chk_number": "1234455",
},
"1": {
"service_date": "12-15-2011",
"payment": "45",
"chk_number": "3461234",
},
"2": {
"service_date": "12-25-2011",
"payment": "25",
"chk_number": "9821234",
}
}
}
{
"dob": "10-01-1998",
"name": "Sam",
"visits": {
"0": {
"service_date": "12-5-2011",
"payment": "30",
"chk_number": "86786464",
},
"1": {
"service_date": "12-15-2011",
"payment": "35",
"chk_number": "45643461234",
},
"2": {
"service_date": "12-25-2011",
"payment": "20",
"chk_number": "4569821234",
}
}
}
In PHP i want to list all those "visits" information (and corresponding "name" ) for which payment is less than "30".
I want to print only the visits with "payment" < "30" not others. Is such query possible, or do i have to get entire document first using search and then use PHP to select such visits??
In the example document, the "payment" values are given as strings which may not work as intended with the $lt command. For this response, I have converted them to integers.
Wildcard queries are not possible with MongoDB, so with the given document structure, the key (0,1,2, etcetera) of the sub-document must be known. For instance, the following query will work:
> db.test.find({"visits.2.payment":{$lt:35}})
However,
> db.test.find({"visits.payment":{$lt:35}})
Will not work in this case, and
> db.test.find({"visits.*.payment":{$lt:35}})
will also not return any results.
In order to be able to query the embedded "visits" documents, you must change your document structure and make "visits" into an array or embedded documents, like so:
> db.test2.find().pretty()
{
"_id" : ObjectId("4f16199d3563af4cb141c547"),
"dob" : "10-01-1998",
"name" : "Sam",
"visits" : [
{
"service_date" : "12-5-2011",
"payment" : 30,
"chk_number" : "86786464"
},
{
"service_date" : "12-15-2011",
"payment" : 35,
"chk_number" : "45643461234"
},
{
"service_date" : "12-25-2011",
"payment" : 20,
"chk_number" : "4569821234"
}
]
}
Now you can query all of the embedded documents in "visits":
> db.test2.find({"visits.payment":{$lt:35}})
For more information, please refer to the Mongo documentation on dot notation:
http://www.mongodb.org/display/DOCS/Dot+Notation+%28Reaching+into+Objects%29
Now on to the second part of your question: it is not possible to return only a conditional sub-set of embedded documents.
With either document format, it is not possible to return a document containing ONLY the sub-documents that match the query. If one of the sub-documents matches the query , then the entire document matches the query, and it will be returned.
As per the Mongo Document "Retrieving a subset of fields"
http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields
We can return parts of embedded documents like so:
> db.test2.find({"visits.payment":{$lt:35}},{"visits.service_date":1}).pretty()
{
"_id" : ObjectId("4f16199d3563af4cb141c547"),
"visits" : [
{
"service_date" : "12-5-2011"
},
{
"service_date" : "12-15-2011"
},
{
"service_date" : "12-25-2011"
}
]
}
But we cannot have conditional retrieval of some sub documents. The closest that we can get is the $slice operator, but this is not conditional, and you will have to first know the location of each sub-document in the array:
http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields#RetrievingaSubsetofFields-RetrievingaSubrangeofArrayElements
In order for the application to display only the embedded documents that match the query, it will have to be done programmatically.
You may try:
$results = $mongodb->find(array("visits.payment" => array('$lt' => 30)));
But i don't know if it will work since visits is an object. BTW judging from what you posted it could be transfered to array (or should since numerical property names tends to cause confusion)
try - db.test2.find({"visits.payment":"35"})