I'm trying to loop through some JSON data and pull out specific values. Here is the JSON data and the partially working code.
$jsondata = '
[
{
"id" : "421356",
"trip_update" : {
"trip" : {
"trip_id" : "421356",
"start_time" : "12:05:00",
"start_date" : "20130926",
"route_id" : "15"
},
"stop_time_update" : {
"stop_sequence" :70,
"departure" : {
"delay" : 240,
"time" : 1380215057
},
"stop_id" : "6090"
},
"stop_time_update" : {
"stop_sequence" :71,
"departure" : {
"delay" : 240,
"time" : 1380215075
},
"stop_id" : "6095"
}
}
}]';
$result = json_decode($jsondata);
foreach($result as $value) {
echo "trip_id: ".$value->trip_update->trip->trip_id;
if (gettype($value->trip_update ) == "object") {
foreach($value->trip_update as $item) {
echo " - stop_sequence: ".$item->stop_sequence;
}
}
}
I can get the first level of data under 'trip_update->trip'. But there can be any number of 'stop_time_update' data within 'trip_update' as well. Since this data relates to the trip_update data, I need to loop through it and correlate it.
The end goal is to save this data to a database (not shown in the code), so for clarity, this would be the simplified 2 rows of DB data I would like to save in this example:
trip_id,stop_sequence
421356,70
421356,71
There can be any number of stop_sequences in the source data.
Here is an interactive link to the code for you to edit or mess with:
http://sandbox.onlinephpfunctions.com/code/f21ca8928da7de3e9fb351edb075d0a446906937
You might get better results if you write your own parser or use a stream-parser with callbacks. Here's a PHP implementation of such a parser that works with callbacks. So instead of reading the whole JSON data into memory, the parser will read the data in chunks and notify your "listener-class" whenever a new object starts or a property was read in etc. By doing this, you should get separate callback events for each stop_time_update property instead of just one value in the parsed array.
Very similar to what SAX is for XML.
Hi maybe you can change the name.
function next_update($coincidencias){
$replace=$coincidencias[0].$GLOBALS["json_stop_time_update"]++;
return $replace;
}
$result= preg_replace_callback("/stop_time_update/","next_update",$jsondata);
$result = json_decode($result);
You should rework your JSON - you have multiple keys with the same name, try to do print_r($result) to see what I am talking about - PHP will override the "stop_time_update" key time after time and you will be able to access only the last entry. Instead, you should organize your JSON like that:
[
{
"id" : "421356",
"trip_update" : {
"trip" : {
"trip_id" : "421356",
"start_time" : "12:05:00",
"start_date" : "20130926",
"route_id" : "15"
},
"stop_time_update" : [{
"stop_sequence" :70,
"departure" : {
"delay" : 240,
"time" : 1380215057
},
"stop_id" : "6090"
}, {
"stop_sequence" :71,
"departure" : {
"delay" : 240,
"time" : 1380215075
},
"stop_id" : "6095"
}]
}
}]
then you will be able to iterate through your data like this:
foreach($result[0]->trip_update->stop_time_update as $update)
{
$time = $update->departure->time;
...
}
If you cannot change the data structure, then what probably could help you is a PULL parser - one that does not return parsed data structure, but allows you to use a data stream instead - this way you could iterate over each node. The only one I managed to find is an extension to PHP:
https://github.com/shevron/ext-jsonreader
Check the usage section.
This JSON response is invalid because it contains duplicate keys but JSON doesn't allow duplicate keys.
You should contact the service you're trying to request this response from.
If you have a valid JSON response then you can decode it using the json_decode function which returns an object or an array (depends on the second parameter);
You cannot use a JSON parser for this as it will always overwrite the first element due to the same keys. The only proper solution would be asking whoever creates that "JSON" to fix his code to either use an array or an object with unique keys.
Another option is to write your own decoder function for parse it
Related
I have a document in mongodb with 2 level deep nested array of objects that I need to update, something like this:
{
id: 1,
items: [
{
id: 2,
blocks: [
{
id: 3
txt: 'hello'
}
]
}
]
}
If there was only one level deep array I could use positional operator to update objects in it but for second level the only option I've came up is to use positional operator with nested object's index, like this:
db.objects.update({'items.id': 2}, {'$set': {'items.$.blocks.0.txt': 'hi'}})
This approach works but it seems dangerous to me since I'm building a web service and index number should come from client which can send say 100000 as index and this will force mongodb to create an array with 100000 indexes with null value.
Are there any other ways to update such nested objects where I can refer to object's ID instead of it's position or maybe ways to check if supplied index is out of bounds before using it in query?
Here's the big question, do you need to leverage Mongo's "addToSet" and "push" operations? If you really plan to modify just individual items in the array, then you should probably build these arrays as objects.
Here's how I would structure this:
{
id: 1,
items:
{
"2" : { "blocks" : { "3" : { txt : 'hello' } } },
"5" : { "blocks" : { "1" : { txt : 'foo'}, "2" : { txt : 'bar'} } }
}
}
This basically transforms everything in to JSON objects instead of arrays. You lose the ability to use $push and $addToSet but I think this makes everything easier. For example, your query would look like this:
db.objects.update({'items.2': {$exists:true} }, {'$set': {'items.2.blocks.0.txt': 'hi'}})
You'll also notice that I've dumped the "IDs". When you're nesting things like this you can generally replace "ID" with simply using that number as an index. The "ID" concept is now implied.
This feature has been added in 3.6 with expressive updates.
db.objects.update( {id: 1 }, { $set: { 'items.$[itm].blocks.$[blk].txt': "hi", } }, { multi: false, arrayFilters: [ { 'itm.id': 2 }, { 'blk.id': 3} ] } )
The ids which you are using are linear number and it has to come from somewhere like an additional field such 'max_idx' or something similar.
This means one lookup for the id and then update. UUID/ObjectId can be used for ids which will ensure that you can use Distributed CRUD as well.
Building on Gates' answer, I came up with this solution which works with nested object arrays:
db.objects.updateOne({
["items.id"]: 2
}, {
$set: {
"items.$.blocks.$[block].txt": "hi",
},
}, {
arrayFilters: [{
"block.id": 3,
}],
});
MongoDB 3.6 added all positional operator $[] so if you know the id of block that need update, you can do something like:
db.objects.update({'items.blocks.id': id_here}, {'$set': {'items.$[].blocks.$.txt': 'hi'}})
db.col.update({"items.blocks.id": 3},
{ $set: {"items.$[].blocks.$[b].txt": "bonjour"}},
{ arrayFilters: [{"b.id": 3}] }
)
https://docs.mongodb.com/manual/reference/operator/update/positional-filtered/#update-nested-arrays-in-conjunction-with
This is pymongo function for find_one_and_update. I searched a lot to find the pymongo function. Hope this will be useful
find_one_and_update(filter, update, projection=None, sort=None, return_document=ReturnDocument.BEFORE, array_filters=None, hint=None, session=None, **kwargs)
Added reference and pymongo documentation in comments
Background Information
I have the following data in my mongo database:
{ "_id" :
ObjectId("581c97b573df465d63af53ae"),
"ph" : "+17771111234",
"fax" : false,
"city" : "abd",
"department" : "",
"description" : "a test"
}
I am now writing a script that will loop through a CSV file that contains data that I need to append to the document. For example, the data might look like this:
+17771111234, 10:15, 12:15, test#yahoo.com
+17771111234, 1:00, 9:00, anothertest#yahoo.com
Ultimately I want to end up with a mongo document that looks like this:
{ "_id" :
ObjectId("581c97b573df465d63af53ae"),
"ph" : "+17771111234",
"fax" : false,
"city" : "abd",
"department" : "",
"description" : "a test",
"contact_locations": [
{
"stime": "10:15",
"etime": "12:15",
"email": "test#yahoo.com"
},
{
"stime": "1:00",
"etime": "9:00",
"email": "anothertest#yahoo.com"
},
]
}
Problem
The code I've written is actually creating new documents instead of appending to the existing ones. And actually, it's not even creating a new document per row in the CSV file... which I haven't debugged enough yet to really understand why.
Code
For each row in the csv file, I'm running the following logic
while(!$csv->eof() && ($row = $csv->fgetcsv()) && $row[0] !== null) {
//code that massages the $row into the way I need it to look.
$data_to_submit = array('contact_locations' => $row);
echo "proving that the record already exists...: <BR>";
$cursor = $contact_collection->find(array('phnum'=>$row[0]));
var_dump(iterator_to_array($cursor));
echo "now attempting to update it....<BR>";
// $cursor = $contact_collection->update(array('phnum'=>$row[0]), $data_to_submit, array('upsert'=>true));
$cursor = $contact_collection->insert(array('phnum'=>$row[0]), $data_to_submit);
echo "AFTER UPDATE <BR><BR>";
$cursor = $contact_collection->find(array('phnum'=>$row[0]));
var_dump(iterator_to_array($cursor));
}
}
Questions
Is there a way to "append" to documents? Or do I need to grab the existing document, save as an array, merge my contact locations array with the main document and then resave?
how can I query to see if the "contact_locations" object already exists inside a document?
Hi yes you can do it !
1st you need to find your document and push the new value you need :
use findAndModify and $addToSet :
$cursor = $contact_collection->findAndModify(
array("ph" => "+17771111234"),
array('$addToSet' =>
array(
"contact_locations" => array(
"stime"=> "10:15",
"etime"=> "12:15",
"email"=> "test#yahoo.com"
)
)
)
);
The best part is $addToSet wont add 2 time the same stuff so you will not have twice the same value :)
Here the docs https://docs.mongodb.com/manual/reference/operator/update/addToSet/
I'm not sure the exact syntax in PHP as I've never done it before but I'm currently doing the same thing in JS with MongoDB and $push is the method you're looking for. Also if I may be a bit nitpicky I recommend changing $contact_collection to $contact_locations as a variable name. Array variable names are usually plural and being more descriptive is always better. Also make sure you find the array in the MongoDB first that you want to append to and that you use the MongoDb "update" command
I am using MongoDB with the PHP Library. I inserted a valid JSON document inside MongoDB using PHP. I am now retrieving the document using findOne and am getting a MongoDB\Model\BSONDocument object as a result. How do I get back my JSON document easily? Is there any inbuilt function or will I have to write logic to convert the BSONDocument to JSON?
I didn't see any answers here and I was having the same issue. I did some research and it appears that when you create a document of MongoDB\Model\BSONDocument there is a bsonSerialize() method. This method will return a stdClass Object which is really the PHP Array Class. According to documentation one can then convert from PHP to BSON and then to JSON.
This is crazy looking, but it works. Here is my example $accountResultDoc is of MongoDB\Model\BSONDocument type.
$json = MongoDB\BSON\toJSON(MongoDB\BSON\fromPHP($accountResultDoc))
Results
{
"_id": {
"$oid": "56e1d8c31d41c849fb292184"
},
"accountName": "Michael's Test Company",
"accountType": "Partner",
"subsidiary_id": {
"$oid": "563c3ffbaca6f518d80303ce"
},
"salesforceId": "WERWERWEr2",
"netsuiteExternalId": "56e1d8c31d41c849fb292184",
"suspendBilling": false,
"testAccount": false,
"serviceOrder_ids": null,
"invoice_ids": null
}
The BSONDocument object has a jsonSerialize method. Use that:
Example
{"_id" : 12345,
"filename" : "myfile",
"header" : {
"version" : 2,
"registry" : "test",
"serial" : 20080215,
"records" : 17806,
"startDate" : 19850701,
"endDate" : 20080214
},
}
$connect = new MongoDB\Client('mongodb://yourconnection');
$db = $connect->YourDB;
$collection = $db->YourCollection;
$test = $collection->findOne(array("_id"=>12345));
$data = $test->jsonSerialize();
echo $data->_id;
echo $data->filename;
Will output this:
12345
myfile
Another way would be:
json_encode( $bsonDoc->getArrayCopy() );
I had the same problem and this is how I accessed the values inside. This works with find.
foreach ($result as $entry) {
echo $entry['_id'], $entry['val1'], ['val2'];
}
Hope this helps someone.
I’m working on my master thesis where one of my goals is to run tests and experiments against the CouchDB database and tune the performance.
To do that I need some test data. I’ve created a piece of php code to generate some simple relational data for a MySQL database. The tables are:
Customer
Product
Brand
Color
Checkout
I’ve made some relations between for example Product and Colorid and Brandid and in the Checkout table I’ve a relation to Customerid and Productid.
I want to export this entirely data structure and the data with its relations to a JSON format for CouchDB.
So I’ve a JSON string which should contain each customer with its attributes and its purchase with all the parameters for this product, and so on.
I’m thinking that it would look something like:
{
"customer": {
"customerid" : "1",
"firstname" : "somefirstname",
"lastname" : "somelastname",
"email" : "my#mail.com",
"country" : "USA",
"datecreated" : "11111111111111"
}
"purchase" : {
"purchaseid" : "1",
"product": {
"productname" : "mightymouse",
"productcolor": "blue",
"productbrand" : "Apple",
"productprice" : "200",
"checkoutdate" : "1111111111112"
}
"purchaseid" : "2",
"product": {
"productname" : "something nice",
"productcolor": "yellow",
"productbrand" : "Google",
"productprice" : "5000",
"checkoutdate" : "11111111113333"
}
}
}
It’s probably not the right data structure I’ve shown but it something like that.
Can this be done in PHP and if so, how do I create this kind of “CouchDB” ready JSON strings??
If I haven’t explained myself clearly, please let me know.
Thank you
Sincere
- Mestika
PHP has function json_encode, you can use it to convert any PHP object to its JSON representation (and json_decode to convert JSON string to object).
So:
Use PHP database functions to read data from database.
Create object of stdClass:
$data = new stdClass;
Fill this object with properties read from database like this:
$customer = new stdClass;
$customer->customerid = "1"
...
$data->customer = $customer;
Encode generated object with json_encode.
Is that possible to sort data in sub array in mongo database?
{ "_id" : ObjectId("4e3f8c7de7c7914b87d2e0eb"),
"list" : [
{
"id" : ObjectId("4e3f8d0be62883f70c00031c"),
"datetime" : 1312787723,
"comments" :
{
"id" : ObjectId("4e3f8d0be62883f70c00031d")
"datetime": 1312787723,
},
{
"id" : ObjectId("4e3f8d0be62883f70c00031d")
"datetime": 1312787724,
},
{
"id" : ObjectId("4e3f8d0be62883f70c00031d")
"datetime": 1312787725,
},
}
],
"user_id" : "3" }
For example I want to sort comments by field "datetime". Thanks. Or only variant is to select all data and sort it in PHP code, but my query works with limit from mongo...
With MongoDB, you can sort the documents or select only some parts of the documents, but you can't modify the documents returned by a search query.
If the current order of your comments can be changed, then the best solution would be to sort them in the MongoDB documents (find(), then for each doc, sort its comments and update()). If you want to keep the current internal order of comments, then you'll have to sort each document after each query.
In both case, the sort will be done with PHP. Something like:
foreach ($doc['list'] as $list) {
// uses a lambda function, PHP 5.3 required
usort($list['comments'], function($a,$b){ return $a["datetime"] < $b["datetime"] ? -1 : 1; });
}
If you can't use PHP 5.3, replace the lambda function by a normal one. See usort() examples.