I have some sharded tables in a MySQL environment. I use Yii so I wanted to add some support for the sharding. I have managed to make a custom CActiveRecord class that is mother to all the sharded models:
class ShardedActiveRecord extends CActiveRecord{
private $_shard;
public function tableName(){
return get_class($this) . $this->getShard();
}
public function setShard($shard) {
$this->_shard = $shard;
call_user_func(array(get_class($this), 'model'))->_shard = $shard;
$this->refreshMetaData();
return $this;
}
public function getShard() {
if($this->_shard === null){
$this->setShard("");
}
return $this->_shard;
}
public function createShardedTable($shard){
$connection=Yii::app()->db;
$command = $connection->createCommand("CREATE TABLE IF NOT EXISTS ".get_class($this).$shard." LIKE ".get_class($this));
$command->execute();
}
}
Everything works fine on insert, but when I need to retrieve some data, I don't know how to proceed. I would like in my model to have a parameter that would be the sharded tables to unite. Here is an example of what I would like to be possible:
MyModel::model()->shard("01,"02",03")->findAll();
And eventually that to return the last 3 tables:
$data = new CActiveDataProvider("MyModel")
The sql for retrieving the data shoold look like this if I want to select the first 3 tables:
SELECT * FROM stats01
UNION SELECT * FROM stats02
UNION SELECT * FROM stats03
I have tried to redefine the fetchData() function but it didn't work, the redefined function was not altering the dataset... I need to have all the datas as one big chunk as if it was a single table.
Thank you!
Update:
I found a solution to the problem. I created a function in my ShardedActiveRecord class that generates a custom CActiveRecord and returns it. Here is how it looks:
public function getMergedDataProvider($back){
$modelArray = array();
$model = array();
$now = date("Ym");
for($i=0; $i<$back; $i++){
array_push($modelArray, $this->model()->setShard(date("Ym", strtotime("-".$i." month", strtotime($now))))->findAll());
}
foreach($modelArray as $ma){
$model = array_merge($model, $ma);
}
$dataProvider = new CActiveDataProvider(get_class($this), array(
'data'=>$model,
));
return $dataProvider;
}
I added a parameter that will allow me to go get the last 3 table shards (they are relative to a date).
In an action that requires the use of a sharded model, I can call the function as so, and use it as a normal CActiveDataProvider:
$dataProvider = MyModel::model()->getMergedDataProvider(3);
I hope this will help someone!
Related
Is it possible to override the ->get() methods from Eloquent in order to always customize the output of the selected fields from the DB query?
For example, if you do a usual Eloquent query like:
User::where('email','like','%wherever.com')->get();
Instead of it making the query:
Select name, email, address, created_at from users where email like '%wherever.com'
Is it possible to override the method to always return something like:
Select CONCAT('CL::',name), lower(email), address, created_at from users where email like '%wherever.com'
I ask for ->get because I have seen that I can pass an array with the columns to be selected but I don't want to define them on all queries.
So, after digging around the functions regarding the query I found this solution:
I created a new class CustomQueryBuilder that extends the Illuminate\Database\Query\Builder
There you can override the get() / select() / where() methods from Eloquent.
Then, in the models that you want to change the way the query is made, define the fields to change like:
protected $encrypted = [
'name',
'email',
];
After this, I created a new class CustomModel that extends the Illuminate\Database\Eloquent\Model and there override the newBaseQueryBuilder like this:
protected function newBaseQueryBuilder()
{
$connection = $this->getConnection();
return new CustomQueryBuilder(
$connection, $connection->getQueryGrammar(), $connection->getPostProcessor(), $this
);
}
Inside the CustomQueryBuilder you can customise all the methods from builder for your needs.
With this setup, you can in any Illuminate\Database\Eloquent\Model change it to extend your CustomModel and inherit this special behaviour for the designated columns.
Be aware that all queries made from Models extending your CustomModel will get this new methods, so do all the needed checks to don't mess up with Eloquent normal behaviour, something like this:
public function where($column, $operator = null, $value = null, $boolean = 'and')
{
if ($this->model !== null && isset($this->model::$encrypted))
{
if (in_array($column, $this->model::$encrypted))
{
$column = DB::raw("CONCAT('CL::',$column)");
}
}
parent::where($column, $operator, $value, $boolean);
}
PS: I know this sound silly with the CONCAT example but with the property $encrypted you can figure out that it's not for concatenating string.
You can use model to update the result for the specific field like below
Use this code in your model file to contact CL::' with thename` value
public function getNameAttribute($value) {
return 'CL::'.$value;
}
You can just call User::where('email','like','%wherever.com')->get(); like this
This getNameAttribute function will always return name value with "CL::"
So you don't need to add CONCAT('CL::'.name) with your query.
Same way you can add for other fields also
Updated
Solution when querying the result
Add this geofields in your model
protected $geofields = array("concat('CL::',name) as name");
Add this newQuery function to override the columns
public function newQuery($excludeDeleted = true)
{
$raw='';
foreach($this->geofields as $column){
$raw .= $column;
}
return parent::newQuery($excludeDeleted)->addSelect('*',\DB::raw($raw));
}
Hope this is what you expect.
Hello and thanks in advance for your time and help. I have 3 very simple tables. A user table with a user_id, a games table with a game_id as well as some other fields (scheduled date/time) and a GamesAttendee table that just has user_id and game_id field. I am trying to select all games that user is connected to and only return ones that are scheduled for the future/past.
What I ended up going with is:
$cur = GamesAttendee::where('user_id',$user_id)->pluck('game_id')->all();
$cur = Game::whereIn('id', $cur)->where('scheduled','>=',$now)->get();
But I feel like there has to be a more efficient way of doing this. I have looked around and tried various things like eager loading and just messing with my models and nothing seems to work. I feel like this a very simple and essential case that is extremely common and I am wondering how this is actually supposed to be done in laravel.
I have tried:
$cur = Game::with(['attendees'=>function($q) use ($user_id){
return $q->where('user_id',$user_id);
}])->where('scheduled','>=',$now)->get();
But that was not what I wanted. I am basically trying to do:
SELECT * FROM GameAttendees
JOIN `games` on games.id = GameAttendees.game_id
WHERE GameAttendees.user_id = 'x' AND games.scheduled >= '2016/05/01' ;
I quickly jotted that mysql code so just ignore any mistakes. Any ideas?
Thank you.
UPDATE:
Resolved by adding the following into my user model:
public function future_games()
{
$now = gmdate('Y-m-d H:i:s',strtotime('+4 hours'));
return $this->belongsToMany('App\Game','games_attendees')->where('scheduled','>=',$now);
}
then in my controller I was able to do:
$future_games = User::with('future_games')->get();
First define many-to-many relation in your Game and User models:
class Game extends Model {
public function users() {
return $this->belongsToMany(User::class, 'GameAttendees');
}
}
class User extends Model {
public function games() {
return $this->belongsToMany(Game::class, 'GameAttendees');
}
}
With that in place you should be able to get all games given user is attending with:
$games = $user->games;
If you want to add some additional conditions, do the following:
$futureGames = $user->games()->where('scheduled','>=',$now)->get();
Or just create another relation in your User model:
class User extends Model {
public function futureGames() {
return $this->belongsToMany(Game::class, 'GameAttendees')->where('scheduled','>=',$now);
}
}
and access them by:
$futureGames = $user->futureGames;
I have a table called bonus. A user can get a bonus (it's like an reward) for certain actions. Well, the bonus can be assigned to many users and many users can get the same bonus. So it's a many to many relation between user and bonus.
This is no problem so far. But users can get the same bonus for different actions. So let's say there is a bonus for voting on a picture. Well, one user could vote on one picture and another one could vote on another picture which I'd like to save in the many-to-many table.
Furthermore there could be a bonus for writing a comment which is clearly another table than picture votes.
The problem here is that I would need to save the polymorphic type in the bonus table and the ID in the many-to-many table.
I think this should be the best way but how would I realize it with laravel? I think this is not a normal use case. But still I'd like to use it as other relations in laravel so that I could fetch a user and get his bonuses with the correct polymorphic relation.
Do you have any ideas?
You are probably going to have to develop your own relationship classes.
Ex:
MODEL
public function answers()
{
$instance = new Response();
$instance->setSid($this->sid);
return new QuestionAnswerRelation($instance->newQuery(),$this);
}
RELATIONSHIP
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Database\Eloquent\Collection;
use Illuminate\Database\Eloquent\Relations\Relation;
use Pivotal\Survey\Models\Answer;
use Pivotal\Survey\Models\Collections\AnswerCollection;
use Pivotal\Survey\Models\QuestionInterface;
use Pivotal\Survey\Models\SurveyInterface;
class QuestionAnswerRelation extends Relation
{
/**
* Create a new relation instance.
*
* #param \Illuminate\Database\Eloquent\Builder $query
* #param \Illuminate\Database\Eloquent\Model $parent
* #return void
*/
public function __construct(Builder $query, QuestionInterface $parent)
{
$table = $query->getModel()->getTable();
$this->query = $query
->select(array(
\DB::raw($parent->sid.'X'.$parent->gid.'X'.$parent->qid . ' AS value'),
'id'
));
$this->query = $query;
$this->parent = $parent;
$this->related = $query->getModel();
$this->addConstraints();
}
public function addEagerConstraints(array $models)
{
parent::addEagerConstraints($models);
}
public function initRelation(array $models, $relation)
{
}
public function addConstraints()
{
}
public function match(array $models, Collection $results, $relation)
{
}
public function getResults()
{
$results = $this->query->get();
$answerCollection = new AnswerCollection();
foreach($results as $result)
{
$answer = new Answer($result->toArray());
$answer->question = $this->parent;
$answerCollection->add($answer);
}
return $answerCollection;
}
In this case we are using Lime Survey which creates a unique table (note the $instance->setSid() changes the table name) for each of its surveys and a unique column for each of its answer -> question values. ( note $parent->sid.'X'.$parent->gid.'X'.$parent->qid. 'AS value')
Where sid = survey_id, gid = group_id(I think) and qid = question_id
Its was quite irritating.
Note how I reference values from the parent to further develop the query.
You should be able to follow a similar route to achieve whatever your heart desires and still maintain the feasibility to use Eloquent.
In Laravel 4.2, I have this models
// models ticket.php
class Ticket extends Eloquent {
public function feedback()
{
return $this->hasMany('Feedback');
}
}
// models/feedback.php
class Feedback extends Eloquent {
public function ticket()
{
return $this->belongsTo('Ticket');
}
}
When I do:
$tickets = Ticket::with('feedback')->get();
It returns an array of all tickets with feedback in one array as expected.
Next I want to get one ticket with all related feedback:
$tickets = Ticket::find($id)->with('feedback')->get();
This returns also all tickets with their feedback.
I tried:
$tickets = Ticket::find($id)->with('feedback')->first();
This seems to work but ignores $id and always shows the first row/ticket in the table. $id is not empty, I checked that.
find() already runs a query. Then with()->get() runs another one without the where clause on the id. Do this instead:
$ticket = Ticket::with('feedback')->find($id);
I have a model Listing that inherits through its belongsTo('Model') relationship should inherently belong to the Manufacturer that its corresponding Model belongs to.
Here's from my Listing model:
public function model()
{
return $this->belongsTo('Model', 'model_id');
}
public function manufacturer()
{
return $this->belongsTo('Manufacturer', 'models.manufacturer_id');
/*
$manufacturer_id = $this->model->manufacturer_id;
return Manufacturer::find($manufacturer_id)->name;*/
}
and my Manufacturer model:
public function listings()
{
return $this->hasManyThrough('Listing', 'Model', 'manufacturer_id', 'model_id');
}
public function models()
{
return $this->hasMany('Model', 'manufacturer_id');
}
I am able to echo $listing->model->name in a view, but not $listing->manufacturer->name. That throws an error. I tried the commented out 2 lines in the Listing model just to get the effect so then I could echo $listing->manufacturer() and that would work, but that doesn't properly establish their relationship. How do I do this? Thanks.
Revised Listing model (thanks to answerer):
public function model()
{
return $this->belongsTo('Model', 'model_id');
}
public function manufacturer()
{
return $this->belongsTo('Model', 'model_id')
->join('manufacturers', 'manufacturers.id', '=', 'models.manufacturer_id');
}
I found a solution, but it's not extremely straight forward. I've posted it below, but I posted what I think is the better solution first.
You shouldn't be able to access manufacturer directly from the listing, since manufacturer applies to the Model only. Though you can eager-load the manufacturer relationships from the listing object, see below.
class Listing extends Eloquent
{
public function model()
{
return $this->belongsTo('Model', 'model_id');
}
}
class Model extends Eloquent
{
public function manufacturer()
{
return $this->belongsTo('manufacturer');
}
}
class Manufacturer extends Eloquent
{
}
$listings = Listing::with('model.manufacturer')->all();
foreach($listings as $listing) {
echo $listing->model->name . ' by ' . $listing->model->manufacturer->name;
}
It took a bit of finagling, to get your requested solution working. The solution looks like this:
public function manufacturer()
{
$instance = new Manufacturer();
$instance->setTable('models');
$query = $instance->newQuery();
return (new BelongsTo($query, $this, 'model_id', $instance->getKeyName(), 'manufacturer'))
->join('manufacturers', 'manufacturers.id', '=', 'models.manufacturer_id')
->select(DB::raw('manufacturers.*'));
}
I started off by working with the query and building the response from that. The query I was looking to create was something along the lines of:
SELECT * FROM manufacturers ma
JOIN models m on m.manufacturer_id = ma.id
WHERE m.id in (?)
The query that would be normally created by doing return $this->belongsTo('Manufacturer');
select * from `manufacturers` where `manufacturers`.`id` in (?)
The ? would be replaced by the value of manufacturer_id columns from the listings table. This column doesn't exist, so a single 0 would be inserted and you'd never return a manufacturer.
In the query I wanted to recreate I was constraining by models.id. I could easily access that value in my relationship by defining the foreign key. So the relationship became
return $this->belongsTo('Manufacturer', 'model_id');
This produces the same query as it did before, but populates the ? with the model_ids. So this returns results, but generally incorrect results. Then I aimed to change the base table that I was selecting from. This value is derived from the model, so I changed the passed in model to Model.
return $this->belongsTo('Model', 'model_id');
We've now mimic the model relationship, so that's great I hadn't really got anywhere. But at least now, I could make the join to the manufacturers table. So again I updated the relationship:
return $this->belongsTo('Model', 'model_id')
->join('manufacturers', 'manufacturers.id', '=', 'models.manufacturer_id');
This got us one step closer, generating the following query:
select * from `models`
inner join `manufacturers` on `manufacturers`.`id` = `models`.`manufacturer_id`
where `models`.`id` in (?)
From here, I wanted to limit the columns I was querying for to just the manufacturer columns, to do this I added the select specification. This brought the relationship to:
return $this->belongsTo('Model', 'model_id')
->join('manufacturers', 'manufacturers.id', '=', 'models.manufacturer_id')
->select(DB::raw('manufacturers.*'));
And got the query to
select manufacturers.* from `models`
inner join `manufacturers` on `manufacturers`.`id` = `models`.`manufacturer_id`
where `models`.`id` in (?)
Now we have a 100% valid query, but the objects being returned from the relationship are of type Model not Manufacturer. And that's where the last bit of trickery came in. I needed to return a Manufacturer, but wanted it to constrain by themodelstable in the where clause. I created a new instance of Manufacturer and set the table tomodels` and manually create the relationship.
It is important to note, that saving will not work.
$listing = Listing::find(1);
$listing->manufacturer()->associate(Manufacturer::create([]));
$listing->save();
This will create a new Manufacturer and then update listings.model_id to the new manufacturer's id.
I guess that this could help, it helped me:
class Car extends Model
{
public function mechanical()
{
return $this->belongsTo(Mechanical::class);
}
}
class CarPiece extends Model
{
public function car()
{
return $this->belongsTo(Car::class);
}
public function mechanical()
{
return $this->car->mechanical();
}
}
At least, it was this need that made me think of the existence of a belongsToThrough
You can do something like this (Student Group -> Users -> Poll results):
// poll result
public function studentGroup(): HasOneDeep
{
return $this->hasOneDeepFromRelations($this->user(), (new User())->studentGroup());
}