The situation is as follows: I've got 2 models: 'Action' and 'User'. These models refer to the tables 'actions' and 'users', respectively.
My action table contains a column user_id. At this moment, I need an overview of all actions, and the users to which they are assigned to. When i use $action->fetchAll(), I only have the user ID, so I want to be able to join the data from the user model, preferably without making a call to findDependentRowset().
I thought about creating custom fetchAll(), fetchRow() and find() methods in my model, but this would break default behaviour.
What is the best way to solve this issue? Any help would be greatly appreciated.
I designed and implemented the table-relationships feature in Zend Framework.
My first comment is that you wouldn't use findDependentRowset() anyway -- you'd use findParentRow() if the Action has a foreign key reference to User.
$actionTable = new Action();
$actionRowset = $actionTable->fetchAll();
foreach ($actionRowset as $actionRow) {
$userRow = $actionRow->findParentRow('User');
}
Edit: In the loop, you now have an $actionRow and a $userRow object. You can write changes back to the database through either object by changing object fields and calling save() on the object.
You can also use the Zend_Db_Table_Select class (which was implemented after I left the project) to retrieve a Rowset based on a join between Action and User.
$actionTable = new Action();
$actionQuery = $actionTable->select()
->setIntegrityCheck(false) // allows joins
->from($actionTable)
->join('user', 'user.id = action.user_id');
$joinedRowset = $actionTable->fetchAll($actionQuery);
foreach ($joinedRowset as $joinedRow) {
print_r($joinedRow->toArray());
}
Note that such a Rowset based on a join query is read-only. You cannot set field values in the Row objects and call save() to post changes back to the database.
Edit: There is no way to make an arbitrary joined result set writable. Consider a simple example based on the joined result set above:
action_id action_type user_id user_name
1 Buy 1 Bill
2 Sell 1 Bill
3 Buy 2 Aron
4 Sell 2 Aron
Next for the row with action_id=1, I change one of the fields that came from the User object:
$joinedRow->user_name = 'William';
$joinedRow->save();
Questions: when I view the next row with action_id=2, should I see 'Bill' or 'William'? If 'William', does this mean that saving row 1 has to automatically update 'Bill' to 'William' in all other rows in this result set? Or does it mean that save() automatically re-runs the SQL query to get a refreshed result set from the database? What if the query is time-consuming?
Also consider the object-oriented design. Each Row is a separate object. Is it appropriate that calling save() on one object has the side effect of changing values in a separate object (even if they are part of the same collection of objects)? That seems like a form of Content Coupling to me.
The example above is a relatively simple query, but much more complex queries are also permitted. Zend_Db cannot analyze queries with the intention to tell writable results from read-only results. That's also why MySQL views are not updateable.
You could always make a view in your database that does the join for you.
CREATE OR REPLACE VIEW VwAction AS
SELECT [columns]
FROM action
LEFT JOIN user
ON user.id = action.user_id
Then just use
$vwAction->fetchAll();
Just remember that views in MySQL are read-only (assuming this is MySQL)
isn't creating a view sql table a good solution to make joint ?
and after a simple table class to access it
I would think it's better if your logic is in sql than in php
Related
In a nutshell, the title best discribes my question, but here I am showing the core of the problem.
I have two databases in my web application, One is MariaDB, the other is MongoDB, To give some context, the "user" table in MariaDB stores user information with column "id" it's primary key, there is another "badge" table which stores badge information with also column "id" it's primary key, at last there is "user_badge" collection in MongoDB having documents of fields
{_id, user_id, badge_id, date}
which just links the User with his/her Badges. This is what I meant by pseudo-relation, Unfortunately I don't know what is it called in this situation.
An example:
I want to query and get all users that have a badge with ID 1. So my pseudo-query should do something like "Select all fields from user table where badge_id in user_badge collection is 1". I highlighted like here because this is impossible to be done in a query (based on my knowledge) somehow a query ought to be made on the MongoDB database first then a second have to be made in the MariaDB database against the results of the former query.
Edit: My original question was about how to implement this in Yii2 PHP framework, but when I googled for sometime and I found out no information to do such a thing even in pure PHP, So I decide to end my edited question here, asking for a way to query between a table in an sql database and a collection in a no-sql database, Yet below I leave my old question which just asks for how to do this more specifically in the PHP framework. really if I knew how to do this in pure PHP I can just make a function somehow that does that in the framework if there wasn't any.
Obviously there cannot be a direct primarykey-foriegnkey relation between two database types but I overrided this issue by having a ::hasMany ActiveRecord method in my User Model, and that worked perfectly fine; When I have a User model between hands I just call $model->userBadges to get from MongoDB all documents having that User ID, also vice versa. The problem is when I do a Query involving this relation, I get error
Calling unknown method: yii\mongodb\ActiveQuery::getTableNameAndAlias()
Parts of my Application
User getUserBadges method in User model
public function getUserBadges(){
return $this->hasMany(UserBadge::className(), ['user_id' => 'id']);
}
UserBadge model extending yii\mongodb\ActiveRecord
class UserBadge extends ActiveRecord
{
public static function collectionName()
{
return 'user_badge';
}
public function attributes()
{
return ['_id', 'user_id', 'badge_id', 'date'];
}
public function getUser(){
return $this->hasOne(User::className(), ['id' => 'user_id']);
}
public function getBadge(){
return $this->hasOne(Badge::className(), ['id' => 'badge_id']);
}
}
My query
$query = User::find()->joinWith(['userBadges']);
Edit: I figured out that the previous query is not really what I want, I simplified it to be clear but the real query that I want to do and you will get the point of why I am doing all of this is
$query = User::find()->joinWith(['userBadges'])->where(['badge_id' => 1]);
And with that I can get users from the user table who have a certain badge with id for example 1.
And here the code fails and throws the error stated above. After inspecting for sometime I found the API for the joinWith method
This method allows you to reuse existing relation definitions to perform JOIN queries. Based on the definition of the specified relation(s), the method will append one or multiple JOIN statements to the current query.
And here I knew that it's normal for this error to occur, In my query I am joining a document in a collection of the MongoDB database not a record in a table in a SQL database which definitely wouldn't work. I got stuck here and don't know what to exactly do, I am sticking to have user table in a SQL database and having the user_badge collection in a no-SQL database, what shall I do in such scenario? query on the no-SQL first and then query a SQL query against the result of the former query? or there is already a solution to such a problem in the methods of AcitveQuery? Or my Database structure is invalid?
Thanks in advance.
So after some good time I knew how to do it with the help of this question, where a SQL query is made against a PHP array.
So, first MongoDB will be queried and the results will be stored in an array, then A MariaDB SQL query will be made against the array generated from former query, I am pretty sure that this is not the best option; what if the result of the MongoDB query 100,000? well an array will be made with 100,000 entries, the SQL query will be made using also that 100,000 item array. Yet this is the best answer I could get (until now).
How to implement it in Yii2
// This line query from the MongoDB database and format the data returned well
$userBadges = UserBadge::find()->select(['user_id'])->where(['badge_id' => 1])->column();
// This line make the SQL query using the array generated from the former line
$userQuery = User::find()->where(['id' => $userBadges]);
I hope there can be a better answer for this question that someone can know, But I thought of sharing what I have reached so far.
I'm using Yii2's ActiveRecord implementation in (hopefully) exactly the way it should be used, according to the docs.
Problem
In a quite simple setup with simple relations betweens the tables, fetching 10 results is fast, 100 is slow. 1000 is impossible. The database is extremely small and indexed perfectly. The problem is definitly Yii2's way to request data, not the db itself.
I'm using a standard ActiveDataProvider like:
$provider = new ActiveDataProvider([
'query' => Post::find(),
'pagination' => false // to get all records
]);
What I suspect
Debugging with the Yii2 toolbar showed thousands of single SELECTs for a simple request that should just get 50 rows from table A with some simple "JOINs" to table B to table C. In plain SQL everybody would solve this with one SQL statement and two joins. Yii2 however fires a SELECT for every relation in every row (which makes sense to keep the ORM clean). Resulting in (more or less) 1 * 50 * 30 = 1500 queries for just getting two relations of each row.
Question
Why is Yii2 using so many single SELECTs, or is this a mistake on my side ?
Addionally, does anybody know how to "fix" this ?
As this is a very important issue for me I'll provide 500 bounty on May 14th.
By default, Yii2 uses lazy loading for better performance. The effect of this is that any relation is only fetched when you access it, hence the thousands of sql queries. You need to use eager loading. You can do this with \yii\db\ActiveQuery::with() which:
Specifies the relations with which this query should be performed
Say your relation is comments, the solution is as follows:
'query' => Post::find()->with('comments'),
From the guide for Relations, with will perform an extra query to get the relations i.e:
SELECT * FROM `post`;
SELECT * FROM `comment` WHERE `postid` IN (....);
To use proper joining, use joinWith with the eagerLoading parameter set to true instead:
This method allows you to reuse existing relation definitions to perform JOIN queries. Based on the definition of the specified relation(s), the method will append one or multiple JOIN statements to the current query.
So
'query' => Post::find()->joinWith('comments', true);
will result in the following queries:
SELECT `post`.* FROM `post` LEFT JOIN `comment` comments ON post.`id` = comments.`post_id`;
SELECT * FROM `comment` WHERE `postid` IN (....);
From #laslov's comment and https://github.com/yiisoft/yii2/issues/2379
it's important to realise that using joinWith() will not use the JOIN query to eagerly load the related data. For various reasons, even with the JOIN, the WHERE postid IN (...) query will still be executed to handle the eager loading. Thus, you should only use joinWith() when you specifically need a JOIN, e.g. to filter or order on one of the related table's columns
TLDR:
joinWith = with plus an actual JOIN (and therefore the ability to filter/order/group etc by one of the related columns)
In order to use relational AR, it is recommended that primary-foreign key constraints are declared for tables that need to be joined. The constraints will help to keep the consistency and integrity of the relational data.
Support for foreign key constraints varies in different DBMS. SQLite 3.6.19 or prior does not support foreign key constraints, but you can still declare the constraints when creating tables. MySQL’s MyISAM engine does not support foreign keys at all.
In AR, there are four types of relationships:
BELONGS_TO: if the relationship between table A and B is one-to-many, then B belongs to A (e.g. Post belongs to User);
HAS_MANY: if the relationship between table A and B is one-to-many, then A has many B (e.g. User has many Post);
HAS_ONE: this is special case of HAS_MANY where A has at most one B (e.g. User has at most one Profile);
MANY_MANY: this corresponds to the many-to-many relationship in database. An associative table is needed to break a many-to-many relationship into one-to-many relationships, as most DBMS do not support many-to-many relationship directly. In our example database schema, the tbl_post_category serves for this purpose. In AR terminology, we can explain MANY_MANY as the combination of BELONGS_TO and HAS_MANY. For example, Post belongs to many Category and Category has many Post.
The following code shows how we declare the relationships for the User and Post classes.
class Post extends CActiveRecord
{
......
public function relations()
{
return array(
'author'=>array(self::BELONGS_TO, 'User', 'author_id'),
'categories'=>array(self::MANY_MANY, 'Category',
'tbl_post_category(post_id, category_id)'),
);
}
}
class User extends CActiveRecord
{
......
public function relations()
{
return array(
'posts'=>array(self::HAS_MANY, 'Post', 'author_id'),
'profile'=>array(self::HAS_ONE, 'Profile', 'owner_id'),
);
}
}
The query result will be saved to the property as instance(s) of the related AR class. This is known as the lazy loading approach, i.e., the relational query is performed only when the related objects are initially accessed. The example below shows how to use this approach:
// retrieve the post whose ID is 10
$post=Post::model()->findByPk(10);
// retrieve the post's author: a relational query will be performed here
$author=$post->author;
You are somehow doing it the wrong please go through from the documentaion here http://www.yiiframework.com/doc/guide/1.1/en/database.arr
I'm really struggling with a recurring OOP / database concept.
Please allow me to explain the issue with pseudo-PHP-code.
Say you have a "user" class, which loads its data from the users table in its constructor:
class User {
public $name;
public $height;
public function __construct($user_id) {
$result = Query the database where the `users` table has `user_id` of $user_id
$this->name= $result['name'];
$this->height = $result['height'];
}
}
Simple, awesome.
Now, we have a "group" class, which loads its data from the groups table joined with the groups_users table and creates user objects from the returned user_ids:
class Group {
public $type;
public $schedule;
public $users;
public function __construct($group_id) {
$result = Query the `groups` table, joining the `groups_users` table,
where `group_id` = $group_id
$this->type = $result['type'];
$this->schedule = $result['schedule'];
foreach ($result['user_ids'] as $user_id) {
// Make the user objects
$users[] = new User($user_id);
}
}
}
A group can have any number of users.
Beautiful, elegant, amazing... on paper. In reality, however, making a new group object...
$group = new Group(21); // Get the 21st group, which happens to have 4 users
...performs 5 queries instead of 1. (1 for the group and 1 for each user.) And worse, if I make a community class, which has many groups in it that each have many users within them, an ungodly number of queries are ran!
The Solution, Which Doesn't Sit Right To Me
For years, the way I've got around this, is to not code in the above fashion, but instead, when making a group for instance, I would join the groups table to the groups_users table to the users table as well and create an array of user-object-like arrays within the group object (never using/touching the user class):
class Group {
public $type;
public $schedule;
public $users;
public function __construct($group_id) {
$result = Query the `groups` table, joining the `groups_users` table,
**and also joining the `users` table,**
where `group_id` = $group_id
$this->type = $result['type'];
$this->schedule = $result['schedule'];
foreach ($result['users'] as $user) {
// Make user arrays
$users[] = array_of_user_data_crafted_from_the_query_result;
}
}
}
...but then, of course, if I make a "community" class, in its constructor I'll need to join the communities table with the communities_groups table with the groups table with the groups_users table with the users table.
...and if I make a "city" class, in its constructor I'll need to join the cities table with the cities_communities table with the communities table with the communities_groups table with the groups table with the groups_users table with the users table.
What an unmitigated disaster!
Do I have to choose between beautiful OOP code with a million queries VS. 1 query and writing these joins by hand for every single superset? Is there no system that automates this?
I'm using CodeIgniter, and looking into countless other MVC's, and projects that were built in them, and cannot find a single good example of anyone using models without resorting to one of the two flawed methods I've outlined.
It appears this has never been done before.
One of my coworkers is writing a framework that does exactly this - you create a class that includes a model of your data. Other, higher models can include that single model, and it crafts and automates the table joins to create the higher model that includes object instantiations of the lower model, all in a single query. He claims he's never seen a framework or system for doing this before, either.
Please Note:
I do indeed always use separate classes for logic and persistence. (VOs and DAOs - this is the entire point of MVCs). I have merely combined the two in this thought-experiment, outside of an MVC-like architecture, for simplicity's sake. Rest assured that this issue persists regardless of the separation of logic and persistence. I believe this article, introduced to me by James in the comments below this question, seems to indicate that my proposed solution (which I've been following for years) is, in fact, what developers currently do to solve this issue. This question is, however, attempting to find ways of automating that exact solution, so it doesn't always need to be coded by hand for every superset. From what I can see, this has never been done in PHP before, and my coworker's framework will be the first to do so, unless someone can point me towards one that does.
And, also, of course I never load data in constructors, and I only call the load() methods that I create when I actually need the data. However, that is unrelated to this issue, as in this thought experiment (and in the real-life situations where I need to automate this), I always need to eager-load the data of all subsets of children as far down the line as it goes, and not lazy-load them at some future point in time as needed. The thought experiment is concise -- that it doesn't follow best practices is a moot point, and answers that attempt to address its layout are likewise missing the point.
EDIT : Here is a database schema, for clarity.
CREATE TABLE `groups` (
`group_id` int(11) NOT NULL, <-- Auto increment
`make` varchar(20) NOT NULL,
`model` varchar(20) NOT NULL
)
CREATE TABLE `groups_users` ( <-- Relational table (many users to one group)
`group_id` int(11) NOT NULL,
`user_id` int(11) NOT NULL
)
CREATE TABLE `users` (
`user_id` int(11) NOT NULL, <-- Auto increment
`name` varchar(20) NOT NULL,
`height` int(11) NOT NULL,
)
(Also note that I originally used the concepts of wheels and cars, but that was foolish, and this example is much clearer.)
SOLUTION:
I ended up finding a PHP ORM that does exactly this. It is Laravel's Eloquent. You can specify the relationships between your models, and it intelligently builds optimized queries for eager loading using syntax like this:
Group::with('users')->get();
It is an absolute life saver. I haven't had to write a single query. It also doesn't work using joins, it intelligently compiles and selects based on foreign keys.
Say you have a "wheel" class, which loads its data from the wheels table in its constructor
Constructors should not be doing any work. Instead they should contain only assignments. Otherwise you make it very hard to test the behavior of the instance.
Now, we have a "car" class, which loads its data from the cars table joined with the cars_wheels table and creates wheel objects from the returned wheel_ids:
No. There are two problems with this.
Your Car class should not contain both code for implementing "car logic" and "persistence logic". Otherwise you are breaking SRP. And wheels are a dependency for the class, which means that the wheels should be injected as parameter for the constructor (most likely - as a collection of wheels, or maybe an array).
Instead you should have a mapper class, which can retrieve data from database and store it in the WheelCollection instance. And a mapper for car, which will store data in Car instance.
$car = new Car;
$car->setId( 42 );
$mapper = new CarMapper( $pdo );
if ( $mapper->fetch($car) ) //if there was a car in DB
{
$wheels = new WheelCollection;
$otherMapper = new WheelMapper( $pdo );
$car->addWheels( $wheels );
$wheels->setType($car->getWheelType());
// I am not a mechanic. There is probably some name for describing
// wheels that a car can use
$otherMapper->fetch( $wheels );
}
Something like this. The mapper in this case are responsible for performing the queries. And you can have several source for them, for example: have one mapper that checks the cache and only, if that fails, pull data from SQL.
Do I really have to choose between beautiful OOP code with a million queries VS. 1 query and disgusting, un-OOP code?
No, the ugliness comes from fact that active record pattern is only meant for the simplest of usecases (where there is almost no logic associated, glorified value-objects with persistence). For any non-trivial situation it is preferable to apply data mapper pattern.
..and if I make a "city" class, in its constructor I'll need to join the cities table with the cities_dealerships table with the dealerships table with the dealerships_cars table with the cars table with the cars_wheels table with the wheels table.
Jut because you need data about "available cares per dealership in Moscow" does not mean that you need to create Car instances, and you definitely will not care about wheels there. Different parts of site will have different scale at which they operate.
The other thing is that you should stop thinking of classes as table abstractions. There is no rule that says "you must have 1:1 relation between classes and tables".
Take the Car example again. If you look at it, having separate Wheel (or even WheelSet) class is just stupid. Instead you should just have a Car class which already contains all it's parts.
$car = new Car;
$car->setId( 616 );
$mapper = new CarMapper( $cache );
$mapper->fetch( $car );
The mapper can easily fetch data not only from "Cars" table but also from "Wheel" and "Engines" and other tables and populate the $car object.
Bottom line: stop using active record.
P.S.: also, if you care about code quality, you should start reading PoEAA book. Or at least start watching lectures listed here.
my 2 cents
ActiveRecord in Rails implements the concept of lazy loading, that is deferring database queries until you actually need the data. So if you instantiate a my_car = Car.find(12) object, it only queries the cars table for that one row. If later you want my_car.wheels then it queries the wheels table.
My suggestion for your pseudo code above is to not load every associated object in the constructor. The car constructor should query for the car only, and should have a method to query for all of it's wheels, and another to query it's dealership, which only queries for the dealership and defers collecting all of the other dealership's cars until you specifically say something like my_car.dealership.cars
Postscript
ORMs are database abstraction layers, and thus they must be tuned for ease of querying and not fine tuning. They allow you to rapidly build queries. If later you decide that you need to fine tune your queries, then you can switch to issuing raw sql commands or trying to otherwise optimize how many objects you're fetching. This is standard practice in Rails when you start doing performance tuning - look for queries that would be more efficient when issued with raw sql, and also look for ways to avoid eager loading (the opposite of lazy loading) of objects before you need them.
In general, I'd recommend having a constructor that takes effectively a query row, or a part of a larger query. How do do this will depend on your ORM. That way, you can get efficient queries but you can construct the other model objects after the fact.
Some ORMs (django's models, and I believe some of the ruby ORMs) try to be clever about how they construct queries and may be able to automate this for you. The trick is to figure out when the automation is going to be required. I do not have personal familiarity with PHP ORMs.
specs: PHP 5 with mySQL built on top of Codeigniter Framework.
I have a database table called game and then sport specific tables like soccerGame and footballGame. these sport specific tables have a gameId field linking back to the game table. I have corresponding classes game and soccerGame/footballGame, which both extend game.
When I look up game information to display to the user, I'm having trouble figuring out how to dynamically link the two tables. i'm curious if it's possible to get all the information with with one query. The problem is, I need to query the game table first to figure out the sport name.
if that's not possible, my next thought is to do it with two queries. have my game_model query the game table, then based off the sport name, call the appropriate sport specific model (i.e. soccer_game_model) and get the sport specific info.
I would also pass the game object into the soccer_model, and the soccer_model would use that object to build me a soccerGame object. this seems a little silly to me because i'm building the parent object and then giving it to the extending class to make a whole new object?
thoughts?
thanks for the help.
EDIT:
game table
gameId
sport (soccer, basketball, football, etc)
date
other data
soccerGame table
soccerGameId
gameId
soccer specific information
footballGame table
footballGameId
gameId
football specific information
and so on for other sports
So I need to know what the sport is before I can decide which sport specific table I need to pull info from.
UPDATE:
Thanks all for the input. It seems like dynamic SQL is only possible through stored procedures, something I'm not well versed on right now. And even with them it's still a little messy. Right now I will go the two query route, one to get the sport name, and then a switch to get the right model.
From the PHP side of things now, it seems a little silly to get a game object, pass it to, say, my soccer_game_model, and then have that return me a soccer_game object, which is a child of the original game. Is that how it has to be done? or am I missing something from an OO perspective here?
To extend on Devin Young's answer, you would achieve this using Codeigniter's active record class like so:
public function get_game_by_id($game_id, $table)
{
return $this->db->join('game', 'game.id = ' . $table . '.gameId', 'left')
->where($table . '.gameId', $game_id)
->get('game')
->result();
}
So you're joining the table by the gameId which is shared, then using a where clause to find the correct one. Finally you use result() to return an array of objects.
EDIT: I've added a second table paramater to allow you to pass in the name of the table you can join either soccerGame, footballGame table etc.
If you don't know which sport to choose at this point in the program then you may want to take a step back and look at how you can add that so you do know. I would be reluctant to add multiple joins to all sport tables as you''ll run into issues down the line.
UPDATE
Consider passing the "sport" parameter when you look up game data. As a hidden field, most likely. You can then use a switch statement in your model:
switch($gameValue) {
case 'football': $gameTable = "footballGame"; break;
case 'soccer': $gameTable = "soccerGame"; break;
}
Then base your query off this:
"SELECT *
FROM ". $gameTable . "
...etc
You can combine the tables with joins. http://www.w3schools.com/sql/sql_join.asp
For example, if you need to get all the data from game and footballGame based on a footballGameId of 15:
SELECT *
FROM footballGame a
LEFT OUTER JOIN game b ON a.id = b.gameId
WHERE footballGameId = 15
Check this Stack Overflow answer for options on how to do it via a standard query. Then you can turn it into active record if you want (though that may be complicated and not worth your time if you don't need DB-agnostic calls in your app).
Fow what it's worth, there's nothing wrong with doing multiple queries, it just might be slower than an alternative. Try a few options, see what works best for you and your app.
For example I have the following tables in my database:
People
Groups
And in the People table I have the following columns:
Name
Birthdate
Group
In the Groups table I have the following columns:
Name
Colour
Now I want to fetch results from my database to make a list of all the People.
So I would use this:
$this->set('people', $this->Person->find("all"));
And obviously in my view I would loop through the returned array and display it. Now I also want to find out for each person in the list what Colour and Group name they are (retrieved from the groups table).
How would I go about this in CakePHP.
First make sure your model relationships are defined. Second, you might want to consider using "id" fields for your related tables (I assume you did and maybe just did not include them in your original questions table definitions).
And yes, run debug($this->Person->find("all")) as Elwhis noted to see what your array is dumping out.
And if this is a mission critical application, be sure to use containable instead of recursive to prevent taxing queries to your db.
You have to set the recursive attribute. But I guess it should work with the default value. Put this line in your controller debug($this->Person->find("all")) and check whether the data doesn't already contain you desired group information.
If not, try setting $this->Person->recursive = 1; before calling the find() function
For more information about recursive: http://book.cakephp.org/view/1063/recursive