I have put Index on mongoDB.
when I ensureIndex using php it put 1 as NumberLong(1) .
I want to know:
will this Indexing work or not.
If not
how to remove this NumberLong(1)
Code:
"0": {
"v" : 1,
"key" : {
"telNum" : NumberLong(1),
"requestID" : NumberLong(1)
},
"ns" : "database.collections",
"background" : true,
"name" : "tel_req"
}
Yes, the index will still work.
This is due to a change in the way that the PHP driver works ( https://jira.mongodb.org/browse/PHP-955 ).
I will come out immediately and say that it is not advised to remove it. That NumberLong object supports throwing in 64bit integers by default so it is extremely useful to have the defaults in the new PHP driver on by default and to treat NumberLong as the new generic number object.
However, to go into this a bit more and explain to you what's happening, there is this runtime configuration option called native_long: http://www.php.net/manual/en/mongo.configuration.php#ini.mongo.native-long which essentially allows MongoDB to store 64bit numbers by storing them into NumberLong objects. In PHP it would automatically convert to int data type and back again, making this transparent to the end user.
If you have this turned off MongoDB can only store 32bit integers.
This option used to be off by default but now it is on, that is why you are seeing this behaviour.
You can turn it off in the PHP configuration to remove the NumberLongs
Related
I have a vary large dataset in MongoDB, in which there are documents with numeric fields. Due to some issue in the data import, some of these fields ended up in int32 datatype with some are in int64 datatype.
I need to convert all of them to int32. Since many of the fields are nested documents/array I cannot use MongoChef or RoboMongo to edit the field and do a collection wide replace.
What is my next best option? Would I need to write a script that loop through each document/field and explicitly typecast them to NumberInt(). I could do this in PHP or Python, but I was wondering if there is a way to do this without writing extra code.
Is there any mongoshell magic that can be done? I would appreciate if any Mongo Masters can give me any insights.
To anyone looking to do this and coming here. You can run
db.foo.find().forEach(doc => {
const newBar = bar.valueOf()
db.foo.update({
"_id" : doc._id
}, {
"$set" : {
"bar" : newBar
}
})
})
in the mongo shell. This might not be doable in large collections. The key is to use .valueOf() on the Int64. You might want to check that this doesn't overflow
i am using
$user = User::find($user_id);
return response()->json(array('user'=>$user),200);
On local server with php5.9 it returns all the keys except id as string.
{
"success": "1",
"message": "success",
"user": {
"id": 75,
"name": "",
"postal_code": "73733",
}
}
But the same code on production server with php7.0 returns other keys as of type integer ex. check this postal_code value.
{
"success": "1",
"message": "success",
"user": {
"id": 75,
"name": "",
"postal_code": 73733,
}
}
So solve this i am using $cast=[] in User.php.
But i have used raw queries also. What is the best way to convert all the values in response json to be of string type.
The difference in the type is most likely due to the fact that your local server is using the php5-mysqld (non-native) driver, whereas your production server is probably using the php-mysqlnd (native) driver. One of the main differences is that the non-native driver reads all fields as strings, whereas the native driver will automatically convert integer fields to PHP integers.
The id shows up as an integer in both cases because Laravel automatically adds the primary key field ('id') to the casts array with the type defined by the $keyType property (default to int).
I would say you have four options:
Attempt to install the non-native mysql driver for PHP 7 on your production server (not recommended).
Don't do anything on the PHP side, and just make sure that whatever is consuming the json can handle strings or integers (probably a good idea either way, but still doesn't solve the real issue).
Add postal_code to your casts array, to ensure it is always casted to a string (not bad; this will work for most cases, but is not exactly the correct solution).
Change the field type of the postal_code field from an integer to a varchar(5), varchar(9), or varchar(10), depending on if you're going to store the +4, and with or without a hyphen (best solution).
Changing the postal_code data type in the database to a varchar really is the best option. The main reason is that, while they are comprised of numbers, postal codes are not actually integers, or numeric in nature. You will never be doing any arithmetic with them. Additionally, there are postal codes that start with a leading 0, and if the value is stored as an integer, this is an extra edge case condition you have to contend with whenever displaying postal codes.
Enabling PDO::ATTR_STRINGIFY_FETCHES setting will solve this problem. You can add this PDO setting under database configurations.
'options' => array(
PDO::ATTR_STRINGIFY_FETCHES => true,
),
You could cast to string like:
$user['postal_code'] = (string) $user['postal_code'];
Further information on variable type casting is in the PHP manual.
Hope this helps :)
I have the following structure within a mongoDB collection:
{
"_id" : ObjectId("5301d337fa46346a048b4567"),
"delivery_attempts" : {
"0" : {
"live_feed_id" : 107,
"remaining_attempts" : 2,
"delivered" : false,
"determined_status" : null,
"date" : 1392628536
}
}
}
// > db.lead.find({}, {delivery_attempts:1}).pretty();
I'm trying to select any data from that collection where remaining_attempts are greater than 0 and a live_feed_id is equal to 107. Note that the "delivery_attempts" field is of a type hash.
I've tried using an addAnd within an elemMatch (not sure if this is the correct way to achieve this).
$qb = $this->dm->createQueryBuilder($this->getDocumentName());
$qb->expr()->field('delivery_attempts')
->elemMatch(
$qb->expr()
->field('remaining_attempts')->gt(0)
->addAnd($qb->expr()->field('live_feed_id')->equals(107))
);
I do appear to be getting the record detailed above. However, changing the greater than
test to 3
->field('remaining_attempts')->gt(3)
still returns the record (which is incorrect). Is there a way to achieve this?
EDIT: I've updated the delivery_attempts field type from a "Hash" to a "Collection". This shows the data being stored as an array rather than an object:
"delivery_attempts" : [
{
"live_feed_id" : 107,
"remaining_attempts" : 2,
"delivered" : false,
"determined_status" : null,
"date" : 1392648433
}
]
However, the original issue still applies.
You can use a dot notation to reference elements within a collection.
$qb->field('delivery_attempts.remaining_attempts')->gt(0)
->field('delivery_attempts.live_feed_id')->equals(107);
It works fine for me if I run the query on mongo.
db.testQ.find({"delivery_attempts.remaining_attempts" : {"$gt" : 0}, "delivery_attempts.live_feed_id" : 107}).pretty()
so it seems something wrong with your PHP query, I suggest running profiler to see which query is actually run against mongo
db.setProfilingLevel(2)
This will log all operation since you enable profiling. Then you can query the log to see which the actual queries
db.system.profile.find().pretty()
This might help you to find the culprit.
It sounds like your solved your first problem, which was using the Hash type mapping (instead for storing BSON objects, or associative arrays in PHP) instead of the Collection mapping (intended for real arrays); however, the query criteria in the answer you submitted still seems incorrect.
$qb->field('delivery_attempts.remaining_attempts')->gt(0)
->field('delivery_attempts.live_feed_id')->equals(107);
You said in your original question:
I'm trying to select any data from that collection where remaining_attempts are greater than 0 and a live_feed_id is equal to 107.
I assume you'd like that criteria to be satisfied by a single element within the delivery_attempts array. If that's correct, the criteria you specified above may match more than you expect, since delivery_attempts.remaining_attempts can refer to any element in the array, as can the live_feed_id criteria. You'll want to use $elemMatch to restrict the field criteria to a single array element.
I see you were using elemMatch() in your original question, but the syntax looked a bit odd. There should be no need to use addAnd() (i.e. an $and operator) unless you were attempting to apply two query operators to the same field name. Simply add extra field() calls to the same query expression you're using for the elemMatch() method. One example of this from ODM's test suite is QueryTest::testElemMatch(). You can also use the debug() method on the query to see the raw MongoDB query object created by ODM's query builder.
I have a database with a "Text" col which contains a javascript object like this:
{
"description" : "",
"title" : " diagramm1",
"xlabel" : "Zeit",
"ylabel" : "",
"ylabel1" : "Anzahl Stabis aufgelegt",
"ylabel2" : "Anzahli.O.",
"tablename" : "edmat1",
"xvaluecol" : "timestamp",
"y1valuecol" : "EDMAT1Q001",
"y2valuecol" : "EDMAT1Q002",
"showRangeSelector" : true,
"divid" : "diagramm1",
"refreshtime" : 30000
}
If I get this out of the database with php it is interpreted as a php array. What can I do to force php to treat this like a string? Afterwards I want to give this javascript object to javascript.
Tried json_encode on php side and JSON.parse at Javascript side, there must be a other solution.
EDIT:
$db=mysql_connect("localhost","root","");
mysql_select_db("visualization");
$anfrage="SELECT options FROM mat1 WHERE id=1";
$a=mysql_query($anfrage);
$b=mysql_fetch_row($a);
echo $b;
only have on row!
If I get this out of the database with php it is interpreted as a php array. What can I do to force php to treat this like a string?
If you store it as a string, then it will stay a string. Even in JavaScript you have to parse it to have an object or an array.
You use mysql_fetch_row, that means you will have your row with your result in an indexed array.
As yoy selected only one column, your array will contain only one column too.
So your solution is (as said in comments) $your_result = $db[0];
Moreover, acording to the php doc I advise you to switch your mysql_ function to mysqli_ functions that are newer, more performant and which support newer version of Mysql SGBD.
I am considering using MongoDB to store documents that include a list of key/value pairs. The safe but ugly and bloated way to store this is as
[ ['k1' : 'v1'] , ['k2' : 'v2'], ...]
But document elements are inherently ordered within the underlying BSON data structure, so in principle:
{k1 : 'v1',
k2 : 'v2', ...}
should be enough. However I expect most language bindings will interpret these as associative arrays, and thus potentially scramble the ordering. So what I need to know is:
Does MongoDB itself promise to preserve item ordering of the second form.
Do language bindings have some API which can extract it ordered form -- even if the usual "convenient" API returns an associative array.
I am mostly interested in Javascript and PHP here, but I would also like to know about other languages. Any help is appreciated, or just a link to some documentation where I can go RTM.
From Version 2.6 on, MongoDB preserves the order of fields where possible. However, the _id field always comes first an renaming fields can lead to re-ordering. However, I'd generally try not to rely on details like this. As the original question mentions, there are also additional layers to consider which each must provide some sort of guarantee for the stability of the order...
Original Answer:
No, MongoDB does not make guarantees about the ordering of fields:
"There is no guarantee that the field order will be consistent, or the same, after an update."
In particular, in-place updates that change the document size will usually change the ordering of fields. For example, if you $set a field whose old value was of type number and the new value is NumberLong, fields usually get re-ordered.
However, arrays preserve ordering correctly:
[ {'key1' : 'value1'}, {'key2' : 'value2'}, ... ]
I don't see why this is "ugly" and "bloated" at all. Storing a list of complex objects couldn't be easier. However, abusing objects as lists is definitely ugly: Objects have associative array semantics (i.e. there can only be one field of a given name), while lists/arrays don't:
// not ok:
db.foo2.insert({"foo" : "bar", "foo" : "lala" });
db.foo2.find();
{ "_id" : ObjectId("4ef09cd9b37bc3cdb0e7fb26"), "foo" : "lala" }
// a list can do that
db.foo2.insert({ 'array' : [ {'foo' : 'bar'}, { 'foo' : 'lala' } ]});
db.foo2.find();
{ "_id" : ObjectId("4ef09e01b37bc3cdb0e7fb27"), "array" :
[ { "foo" : "bar" }, { "foo" : "lala" } ] }
Keep in mind that MongoDB is an object database, not a key/value store.
As of Mongo 2.6.1, it DOES keep the order of your fields:
MongoDB preserves the order of the document fields following write operations except for the following cases:
The _id field is always the first field in the document.
Updates that
include renaming of field names may result in the reordering of
fields in the document.
http://docs.mongodb.org/manual/release-notes/2.6/#insert-and-update-improvements
One of the pain points of this is comparing documents to one another in the shell.
I've created a project that creates a custom mongorc.js which sorts the document keys by default for you when they are printed out so at least you can see what is going on clearly in the shell. It's called Mongo Hacker if you want to give it a whirl.
Though it's true that, as of Mongo 2.6.1, it does preserve order, one should still be careful with update operations.
mattwad makes the point that updates can reorder things, but there's at least one other concern I can think of.
For example $addToSet:
https://docs.mongodb.com/manual/reference/operator/update/addToSet/
$addToSet when used on embedded documents in an array is discussed / exemplified here:
https://stackoverflow.com/a/21578556/3643190
In the post, mnemosyn explains how $addToSet disregards the order when matching elements in its deep value by value comparison.
($addToSet only adds records when they're unique)
This is relevant if one decided to structure data like this:
[{key1: v1, key2: v2}, {key1: v3, key2: v4}]
With an update like this (notice the different order on the embedded doc):
db.collection.update({_id: "id"},{$addToSet: {field:
{key2: v2, key1: v1}
}});
Mongo will see this as a duplicate and NOT this object to the array.