I'm using the Mongo PHP extension.
My data looks like:
users
{
"_id": "4ca30369fd0e910ecc000006",
"login": "user11",
"pass": "example_pass",
"date": "2010-09-29"
},
{
"_id": "4ca30373fd0e910ecc000007",
"login": "user22",
"pass": "example_pass",
"date": "2010-09-29"
}
news
{
"_id": "4ca305c2fd0e910ecc000003",
"name": "news 333",
"content": "news content 3333",
"user_id": "4ca30373fd0e910ecc000007",
"date": "2010-09-29"
},
{
"_id": "4ca305c2fd0e910ecc00000b",
"name": "news 222",
"content": "news content 2222",
"user_id": "4ca30373fd0e910ecc000007",
"date": "2010-09-29"
},
{
"_id": "4ca305b5fd0e910ecc00000a",
"name": "news 111",
"content": "news content",
"user_id": "4ca30369fd0e910ecc000006",
"date": "2010-09-29"
}
How to run a query similar like this, from PHP?
SELECT n.*, u.*
FROM news AS n
INNER JOIN users AS u ON n.user_id = u.id
MongoDB does not support joins. If you want to map users to the news, you can do the following
1) Do this at the application-layer. Get the list of users, and get the list of news and map them in your application. This method is very expensive if you need this often.
2) If you need to do the previous-step often, you should redesign your schema so that the news articles are stored as embedded documents along with the user documents.
{
"_id": "4ca30373fd0e910ecc000007",
"login": "user22",
"pass": "example_pass",
"date": "2010-09-29"
"news" : [{
"name": "news 222",
"content": "news content 2222",
"date": "2010-09-29"
},
{
"name": "news 222",
"content": "news content 2222",
"date": "2010-09-29"
}]
}
Once you have your data in this format, the query that you are trying to run is implicit. One thing to note, though, is that analytics queries become difficult on such a schema. You will need to use MapReduce to get the most recently added news articles and such queries.
In the end the schema-design and how much denormalization your application can handle depends upon what kind of queries you expect your application to run.
You may find these links useful.
http://www.mongodb.org/display/DOCS/Schema+Design
http://www.blip.tv/file/3704083
I hope that was helpful.
Forget about joins.
do a find on your news. Apply the skip number and limit for paging the results.
$newscollection->find().skip(20).limit(10);
then loop through the collection and grab the user_id in this example you would be limited to 10 items. Now do a query on users for the found user_id items.
// replace 1,2,3,4 with array of userids you found in the news collection.
$usercollection.find( { _id : { $in : [1,2,3,4] } } );
Then when you print out the news it can display user information from the user collection based on the user_id.
You did 2 queries to the database. No messing around with joins and figuring out field names etc. SIMPLE!!!
If you are using the new version of MongoDB (3.2), then you would get something similar with the $lookup operator.
The drawbacks with using this operator are that it is highly inefficient when run over large result sets and it only supports equality for the match where the equality has to be between a single key from each collection. The other limitation is that the right-collection should be an unsharded collection in the same database as the left-collection.
The following aggregation operation on the news collection joins the documents from news with the documents from the users collection using the fields user_id from the news collection and the _id field from the users collection:
db.news.aggregate([
{
"$lookup": {
"from": "users",
"localField": "user_id",
"foreignField": "_id",
"as": "user_docs"
}
}
])
The equivalent PHP example implementation:
<?php
$m = new MongoClient("localhost");
$c = $m->selectDB("test")->selectCollection("news");
$ops = array(
array(
"$lookup" => array(
"from" => "users",
"localField" => "user_id",
"foreignField" => "_id",
"as" => "user_docs"
)
)
);
$results = $c->aggregate($ops);
var_dump($results);
?>
You might be better off embedding the "news" within the users' documents.
You can't do that in mongoDB. And from version 3 Eval() is deprecated, so you shouldn't use stored procedures either.
The only way I know to achieve a server side query involving multiple collections right now it's to use Node.js or similar. But if you are going to try this method, I strongly recommend you to limit the ip addresses allowed to access your machine, for security reasons.
Also, if your collections aren't too big, you can avoid inner joins denormalizing them.
Related
In DynamoDB i have a table with the following structure.
The actions "field" contains all the info (and this is the field i would like to search into) and orderId it's the primary key
{
"actions": [
{
"actionDescription": "8f23029def1d6baa4",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533730680,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "21857e61037bc29ec",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731788,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "cf10abd44e24cef56",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731788,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "7787fe7a5bf4d22de",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "OOOOOO",
"userName": "ooooo#oooo.ooo",
}
},
{
"actionDescription": "9528c439021f504bf",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "bfba100e0e54934b2",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "f789dc12f1dbe3be2",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "OOOOOO",
"userName": "ooooo#oooo.ooo",
}
},
{
"actionDescription": "4cd6b68dfea7cf8ee",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731789,
"user": {
"fullName": "XXXXX",
"userName": "xxxxx#xxxx.xxx",
}
},
{
"actionDescription": "1e3a0e95f8e5106d7",
"actionTitle": "UNDEFINED_ACTION",
"timestamp": 1533731790,
"user": {
"fullName": "OOOOOO",
"userName": "ooooo#oooo.ooo",
}
}
],
"orderId": "13aae31"
}
What i would like to do it's to make the scan terms in PHP to be able to search by userName. or by any field inside the actions array (timestamp, actionTitle, etc, etc).
Bellow it's one of the many terms i tried to use but i was unable to achieve any results
$params = [
'TableName' => $this->tableName,
'FilterExpression' => "userName = :searchTerm",
'ExpressionAttributeValues' => [
':searchTerm' => 'ooooo#oooo.ooo',
],
'ReturnConsumedCapacity' => 'TOTAL',
];
$results = $this->dynamoDbClient->scan($params);
Can you please guide my by telling me what i'm missing?
Also, please note: I don't want to get a specific orderId, i would like to get ALL orderIds containing the searchTerm (in this case userName)
Your best bet with this item schema is to filter the table items yourself. That is to say, scan the table with no filter expression and write your own code to filter the results. Scanning without the filter expression will consume the same amount of read capacity units.
You can set the filter expression to something like this, however this isn't scalable and only works if you have a fixed number of items in the actions list.
actions[0].user.userName == :searchTerm OR actions[1].user.userName == :searchTerm OR actions[2].user.userName == :searchTerm OR ....
If you need complex search abilities you are probably better off using a dedicated search database. AWS provides two services around this, AWS CloudSearch and AWS ElasticSearch. You can use DynamoDB streams to keep your search indexes up to date.
If you are set on scanning the DynamoDB table with a filter you can refactor your structure to include additional attributes that have all the searchable information in a set (or concatenated string)
{
"actions": [....],
"actionsDescriptions": Set["8f23029def1d6baa4", "21857e61037bc29ec", "cf10abd44e24cef56", "7787fe7a5bf4d22de", "9528c439021f504bf", "bfba100e0e54934b2", "f789dc12f1dbe3be2", "4cd6b68dfea7cf8ee", "1e3a0e95f8e5106d7"],
"actionTitles": Set["UNDEFINED_ACTION"],
"timestamps": Set[1533730680, 1533731788, 1533731789, 1533731790],
"user_fullNames": Set["XXXXX"],
"user_userNames": Set["ooooo#oooo.ooo", "xxxxx#xxxx.xxx"],
"orderId": "13aae31"
}
Notice you have to use a Set (or concatenate all the values into a string) since the contains functions only works on strings and sets.
Then you can use a filter expression like this
contains(user_userNames, :searchTerm)
The DynamoDB QueryFilter and ScanFilter options do not currently support the CONTAINS operator for maps. You'll need to build another lookup table indexed by userName to avoid scanning the entire table.
E.g. new table schema:
{
"userName": "xxxxx#xxxx.xxx"
"orderId": "13aae31"
}
Where the hash key is userName and orderId is the ID of an order in the other table.
The closest you can get with the current schema is to use #cementblocks's suggestions to scan the whole table and filter application-side or query each element in the list individually.
If you are adding a "Search" like feature to your application, then scanning may not be the best approach.
DynamoDB scan can be expensive and slow, especially when you have many rows.
So, if you intend on adding a "Search" feature you may consider using AWS CloudSearch. It is a scalable "Search" feature. You can quickly enable "Search" from a DynamoDB table.
I have two collection in MongoDB database, i want join two Collection in PHP
I have searched but unfortunately I have not found a compelling answer.
Data look like this:
users
{
"_id": "4ca30369fd0e910ecc000006",
"login": "user11",
"pass": "example_pass",
"date": "2017-12-15"
}
news
"_id": "4ca305c2fd0e910ecc000003",
"name": "news 333",
"content": "news content",
"user_id": "4ca30373fd0e910ecc000007",
"date": "2017-12-15"
}
Already answer in this thread
Note : I'm a MEAN developer
In mean we use .populate() method (mongoose) to achieve joins upto a level.
as for php
You can use different approach from RDBMS
Data Replication
"news": {
"_id": "4ca305c2fd0e910ecc000003",
"name": "news one",
"content": "news one",
"user": {
"_id": "4ca30369fd0e910ecc000006",
"login": "user11"
},
"date": "2017-12-15"
}
I've been struggling to figure out a table relationship for the past two days, I'm sure that the solution is simple but it is alluding me.
Four tables/models are involved (including a pivot table):
Skill - skills table. A Skill belongs to a SkillGroup and belongs to many Candidates (candidate_skill pivot table)
Candidate - candidates table. Contains personal information on a candidate, not terribly related to the issue.
SkillGroup - skill_groups table. Each Skill Group has many Skills.
I want to be able to retrieve Skill objects possessed by a Candidate grouped by the SkillGroup. For example:
[
{
"id": 1,
"title": "Information Technology (Skill Group)",
"slug": "information-technology",
"created_at": "2016-05-07 23:58:23",
"updated_at": "2016-05-07 23:58:23",
"skills": [
{
"id": 1,
"title": "Web Development (Skill)",
"slug": "web-development",
"description": "Web developers primarily focus on the back-end of websites",
"created_at": "2016-05-07 23:58:55",
"updated_at": "2016-05-07 23:58:55",
"skill_group_id": 1,
"candidates": [
{
"first_name": "John (Candidate)",
"last_name": "Smith",
"pivot": {
"skill_id": 1,
"candidate_id": 6
}
}
]
}
]
}
]
This is easy to accomplish with the following code, but I want to retrieve only results (SkillGroup -> Skills -> Candidate) for a specific candidate.
\App\SkillGroup::with('skills.candidates')->get();
I have tried the following (and everything else I can think of), the candidate_id does not seem to affect the query - I see skills that are not possessed by the given candidate.
\App\SkillGroup::with(['skills.candidates' => function($query) {
$query->whereCandidateId(6);
}])->get();
Any help would be greatly appreciated, thanks!
EDIT
Thanks to #Giedrius Kiršys, I was able to come up with the following:
\App\SkillGroup::with(['skills.candidates' => function($query) {
$query->wherePivot('candidate_id', 8)->addSelect('candidates.id', 'first_name', 'last_name');
}])->whereHas('skills.candidates', function($q) {
$q->whereCandidateId(8);
})->get();
This only retrieves SkillGroup results with Skills with a Candidate with the given ID.
You want to query by pivot table attribute, but You are querying by candidates.candidate_id attribute.
You can do it like this:
\App\SkillGroup::with(['skills.candidates' => function($query) {
$query->wherePivot('candidate_id', 6);
}])->get();
I've create a small application with PHP and I use ES.
My request is good, but I've got the good result.
My request look-like that:
link:9200/index/_search?from=0&size=130&q=try:'yes'
%2Bbrand:'BMW' %2Bmodel:'SERIE 5' %2Bprice:[500 TO 700000]
When I send this query, ES reply me with model 'SERIE 3' and 'SERIE 5', it's great, but when I send this query, I would like to recover only 'BMW' and 'SERIE 5'.
How can I fix this?
First, you should take a look at the documentation to be more familiar with these notions (analyze / difference between query and filters) which are very important for a good use of ElasticSearch. You can find a good getting started documentation here.
Your problem is that your "model" field is a string, which by default is analyzed using the standard analyzer.
It outputs 2 tokens because of the whitespace in the model name as you can see if you use the _analyze endpoint :
GET _analyze?analyzer=standard&text='Serie 5'
{
"tokens": [
{
"token": "serie",
"start_offset": 1,
"end_offset": 6,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "5",
"start_offset": 7,
"end_offset": 8,
"type": "<NUM>",
"position": 2
}
]
}
On top of that, you're using a query and though will return all results matching even partially. So, you're certainly having the two cars in your results, but the "SERIE 5" car must be the first (as it matches better) than the car "SERIE 3", which is represented by a higher _score attribute.
You need to use a term filter which will return only the documents containing the term value you provided.
However, as it works on terms, you have to change the mapping of your field to "not_analyzed" like this to keep it as it is :
PUT /test/car/_mapping
{
"properties":{
"model":{
"type": "string",
"index":"not_analyzed"
}
}
}
Finally, the search request will be something like this (with price criteria as range filter and the use of a and filter to combine both) :
GET /test/car/_search
{
"query": {
"filtered": {
"filter": {
"and": {
"filters": [
{
"term": {
"model": "Serie 3"
}
},
{
"range": {
"price": {
"from": 500,
"to": 70000
}
}
}
]
}
}
}
}
}
Your query (url_decoded) looks like
link:9200/index/_search?from=0&size=130&q=try:'yes' +brand:'BMW' +model:'SERIE 5' +price:[500 TO 700000]
I think you are using '+' incorrectely, so that it is doing or operation for your query,
If you want to get with try:yes, brand:BMW and model:SERIE 5 then you have to join these query by AND keyword.
like.
link:9200/index/_search?from=0&size=130&q=try:'yes'
AND brand:'BMW' AND model:'SERIE 5' AND price:[500 TO 700000]
And you should be aware of choosing analyzer (in mapping of fields), so that things are indexed as you want.
It will work, Thanks
Reference
Hi i am using mongoDb for my new project. I am trying to find values from mongodb database.
My mongoDB database collection name is: test and json format is below
{
"College": [
{
"name": "tamy",
"roll_no": "1"
},
{
"name": "abhi",
"roll_no": "2"
},
{
"name": "jack",
"roll_no": "3"
}
],
"School": [
{
"name": "zack",
"roll_no": "1"
},
{
"name": "mac",
"roll_no": "2"
},
{
"name": "john",
"roll_no": "3"
}
]
}
**And i want to find the name:abhi from test collection **
and my mongodb find query in php
$criteria = array(
'College'=> array(
'name'=> 'abhi'
));
$cursor = $collection->find($criteria);
but its not returning any value.
if i am displaying in php by using print_r.
print_r(iterator_to_array($cursor));
displaying empty array()
I need output like this:
name : jack
roll_no : 3
plz help me..
The key "College" is an array, your trying to treat it as a "subdocument" (but that wouldnt allow for multiple colleges to be stored)
You want the elemMatch keyword here I believe: http://docs.mongodb.org/manual/reference/operator/query/elemMatch/
EDIT: After looking at this again, I think you have your structure a little out of whack. How many "students" will there be per college or school? Keep in mind each parent document can not exceed 10mb, and when you select a document, the entire document must be sent back across the connection (unless you are using aggregate or map/reduce. I would think you may want to have a collection of Persons and then a key associating a person to... something else.