I'm working on a Laravel 5 app using the jenssegers\laravel-mongodb package and I'm trying to get up and running with Factory Muffin to help with rapidly testing and seeding etc.
I have defined relationships in my factory definitions (code below) - and they work when I run seeds to create new records in the database. By "work", I mean they attach related data correctly to the parent model and persist all those records to the database. But they do not work when I run the $muffin->instance('My\Object') method, which creates a new instance without persisting it. All the relations come back as null.
In a way this makes sense. When the models are stored in the database, they are related by the object _id key. That key doesn't exist until the model is stored. So when I call the instance method, I actually can see via debugging that it does generate the models that would be related, but it does not yet have any key to establish the relation, so that data just kinda goes poof.
This is a bummer because I'd like to be able to generate a fully fleshed-out model with its relations and see for example whether it is saved or passes validation and whatnot when I submit its data to my routes and things. I.e., right now I would not be able to check that the contact's email was valid, etc.
Simplified code samples are below - am I going about this in an entirely wrong way? This is my first time working with Mongo and Factory Muffin. Should this even be able to work the way I want it to?
// factories/all.php
$fm->define('My\Models\Event', [
'title' => Faker::text(75),
'contact' => 'factory|My\Models\Contact'
]);
$fm->define('My\Models\Contact', [
'first_name' => Faker::firstName(),
'email' => Faker::email(),
... etc
]);
// EventControllerTest.php
...
/**
* #var League\FactoryMuffin\FactoryMuffin
*/
protected $muffin;
public function setUp()
{
parent::setUp();
$this->muffin = new FactoryMuffin();
$this->muffin->loadFactories(base_path('tests/factories'));
}
...
public function testCanStoreEvent()
{
$event = $this->muffin->instance('Quirks\Models\Event');
echo $event->contact; // returns null
$event_data = $event->toArray();
$this->call('POST', 'events', $event_data);
$retrieved_event = Event::where('title', '=', $event->title)->get()->first();
$this->assertNotNull($retrieved_event); // passes
$this->assertRedirectedToRoute('events.edit', $retrieved_event->id); // passes
// So, the event is persisted and the controller redirects etc
// BUT, related data is not persisted
}
Ok, in order to get factory muffin to properly flesh out mongo embedded relationships, you have to do like this:
// factories/all.php
$embedded_contact = function() {
$definitions = [
'first_name' => Faker::firstName(),
'email' => Faker::email(),
// ... etc
];
return array_map(
function($item) { return $item(); },
$definitions
);
}
$fm->define('My\Models\Event', [
'title' => Faker::text(75),
'contact' => $embedded_contact
]);
If that seems onerous and hacky ... I agree! I'm definitely interested to hear of easier methods for this. For now I'm proceeding as above and it's working for both testing and seeding.
Related
I am looking to extend Silverstripe's CSVBulkLoader class to do some business logic before/upon import.
In the WineAdmin Class (extending ModelAdmin), I have a custom loader defined with $model_importers property:
//WineAdmin.php
private static $model_importers = [
'Wine' => 'WineCsvBulkLoader'
];
In the WineCsvBulkLoader Class, the $columnMap property maps CSV columns to SS DataObject columns:
//WineCsvBulkLoader.php
use SilverStripe\Dev\CsvBulkLoader;
class WineCsvBulkLoader extends CsvBulkLoader
{
public $columnMap = [
// csv columns // SS DO columns
'Item Number' => 'ItemNumber',
'COUNTRY' => 'Country',
'Producer' => 'Producer',
'BrandName' => 'BrandName',
// etc
];
When the import is run, the WineCsvBulkLoader class is being invoked, but, the column mappings do not seem to work properly, returning values only where the [key] and [value] in the array are identical. Otherwise, the imported columns are empty. What could be causing this?
Additionally, the $duplicateChecks property is set to look for duplicates.
public $duplicateChecks = [
'ItemNumber' => 'ItemNumber'
];
}
What does the $duplicateChecks property actually do when there is a duplicate? Does it skip the record?
Can I use callbacks here?
In the docs, I found some code for an example method that splits data in a column into 2 parts and maps those parts to separate columns on the class:
public static function importFirstAndLastName(&$obj, $val, $record)
{
$parts = explode(' ', $val);
if(count($parts) != 2) return false;
$obj->FirstName = $parts[0];
$obj->LastName = $parts[1];
}
Is $obj the final import object? How does it get processed?
$val seems to be the value of the column in the csv being imported. Is that correct?
What is contained in $record?
Here are some additional enhancements I hope to make:
Read the Byte Order Marker, if present, upon import, and do something useful with it
Upon import, check for duplicate records, and if there are duplicates, I’d like to only update the columns in the record that have changed.
Delete records that are already in the database, that are not in the CSV being imported
Add whatever security measures are necessary to use this custom class securely.
Export CSV with BOM (byte order mark as UTF8)
I'm not looking for a complete answer, but appreciative of any insights.
I'll attempt to answer some of your questions based on SilverStripe 4.2.0:
Judging by the logic in CsvBulkLoader::findExistingObject the duplicateChecks property is used to help finding an existing record in order to update it (rather than create it). It will use defined values in the array in order to find the first record that matches a given value and return it.
What does the $duplicateChecks property actually do when there is a duplicate? Does it skip the record?
Nothing, it will just return the first record it finds.
Can I use callbacks here?
Kind of. You can use a method on the instance of CsvBulkLoader, but you can't pass it a callback directly (e.g. from _config.php etc). Example:
public $duplicateChecks = [
'YourFieldName' => [
'callback' => 'loadRecordByMyFieldName'
]
];
/**
* Take responsibility for loading a record based on "MyFieldName" property
* given the CSV value for "MyFieldName" and the original array record for the row
*
* #return DataObject|false
*/
public function loadRecordByMyFieldName($inputFieldName, array $record)
{
// ....
Note: duplicateChecks callbacks are not currently covered by unit tests. There's a todo in CsvBulkLoaderTest to add them.
Is $obj the final import object? How does it get processed?
You can see where these magic-ish methods get called in CsvBulkLoader::processRecord:
if ($mapped && strpos($this->columnMap[$fieldName], '->') === 0) {
$funcName = substr($this->columnMap[$fieldName], 2);
$this->$funcName($obj, $val, $record); // <-------- here: option 1
} elseif ($obj->hasMethod("import{$fieldName}")) {
$obj->{"import{$fieldName}"}($val, $record); // <----- here: option 2
} else {
$obj->update(array($fieldName => $val));
}
This is actually a little misleading, especially because the method's PHPDoc says "Note that columnMap isn't used." Nevertheless, the priority will be given to a value in the columnMap property being ->myMethodName. In both the documentation you linked to and the CustomLoader test implementation in the framework's unit tests, they both use this syntax to specifically target the handler for that column:
$loader->columnMap = array(
'FirstName' => '->importFirstName',
In this case, $obj is the DataObject that you're going to update (e.g. a Member).
If you don't do that, you can define importFirstName on the DataObject that's being imported, and the elseif in the code above will then call that function. In that case the $obj is not provided because you can use $this instead.
"Is it the final import object" - yes. It gets written after the loop that code is in:
// write record
if (!$preview) {
$obj->write();
}
Your custom functions would be required to set the data to the $obj (or $this if using importFieldName style) but not to write it.
$val seems to be the value of the column in the csv being imported. Is that correct?
Yes, after any formatting has been applied.
What is contained in $record?
It's the source row for the record in the CSV after formatting callbacks have been run on it, provided for context.
I hope this helps and that you can achieve what you want to achieve! This part of the framework probably hasn't had a lot of love in recent times, so please feel free to make a pull request to improve it in any way, even if it's only documentation updates! Good luck.
Is there a way to invoke eloquent relationship methods without changing the original eloquent collection that the method runs on? Currently I have to employ a temporary collection to run the method immutable and to prevent adding entire related record to the response return:
$result = Item::find($id);
$array = array_values($result->toArray());
$temp = Item::find($id);
$title = $temp->article->title;
dd($temp); //This prints entire article record added to the temp collection data.
array_push($array, $title);
return response()->json($array);
You are not dealing with collections here but with models. Item::find($id) will get you an object of class Item (or null if not found).
As far as I know, there is no way to load a relation without storing it in the relation accessor. But you can always unset the accessor again to delete the loaded relation (from memory).
For your example, this process yields:
$result = Item::find($id);
$title = $result->article->title;
unset($result->article);
return response()->json(array_merge($result->toArray(), [$title]));
The above works but is no very nice code. Instead, you could do one of the following three things:
Use attributesToArray() instead of toArray() (which merges attributes and relations):
$result = Item::find($id);
return response()->json(array_merge($result->attributesToArray(), [$result->article->title]));
Add your own getter method on the Item class that will return all the data you want. Then use it in the controller:
class Item
{
public function getMyData(): array
{
return array_merge($this->attributesToArray(), [$this->article->title]);
}
}
Controller:
$result = Item::find($id);
return response()->json($result->getMyData());
Create your own response resource:
<?php
namespace App\Http\Resources;
use Illuminate\Http\Resources\Json\JsonResource;
class ItemResource extends JsonResource
{
public function toArray($request)
{
return [
'id' => $this->id,
'name' => $this->name,
'title' => $this->article->title,
'author' => $this->article->author,
'created_at' => $this->created_at,
'updated_at' => $this->updated_at,
];
}
}
Which can then be used like this:
return new ItemResource(Item::find($id));
The cleanest approach is option 3. Of course you could also use $this->attributesToArray() instead of enumerating the fields, but enumerating them will yield you security in future considering you might extend the model and do not want to expose the new fields.
I see two ways you can achieve that.
First, you can use an eloquent Resource. Basically it'll allow you to return exactly what you want from the model, so in your case, you'll be able to exclude the article. You can find the documentation here.
The second way is pretty new and is still undocumented (as fas i know), but it actually works well. You can use the unsetRelation method. So in your case, you just have to do:
$article = $result->article; // The article is loaded
$result->unsetRelation('article'); // It is unloaded and will not appear in the response
You can find the unsetRelation documentation here
There is not as far as I know. When dealing with Model outputs, I usually construct them manually like this:
$item = Item::find($id);
$result = $item->only('id', 'name', 'description', ...);
$result['title'] = $item->article->title;
return $result;
Should you need more power or a reusable solution, Resources are your best bet.
https://laravel.com/docs/5.6/eloquent-resources#concept-overview
In a function in my controller I call this:
$item = Item::where('i_id', $Id)->where('type', 1)->first();
$firebaseData = app('firebase')->getDatabase()->getReference('items/'.$Id)->getSnapshot()->getValue();
Then I do a lot of "validation" between the data from the two sources above like:
if ($item->time_expires < strtotime(Carbon::now()) && $firebaseData['active'] == 1) {
return response()->json(['errors' => [trans('api.pleaserenew')]], 422);
}
And since this is not data coming from a user/request I cant use Laravels validate method
I dont want to keep this kind of logic inside my controller but where should I put it? Since part of my data is coming from Firebase I cant setup a Eloquent model to handle it either.
I recommend to receive the firebase data via a method within the model:
public function getFirebaseData()
{
app('firebase')->getDatabase()->getReference('items'/ . $this->i_id)->getSnapshot()->getValue();
}
That way you have the logic to receive the data decoupled from controller logic and moved it to where it makes more sense. Adding a validation method could work similarily within the model then:
public function validateData()
{
$combined = array_merge($this->toArray(), $this->getFirebaseData());
Validator::make($combined, [
'active' => 'in:1',
'time_expires' => 'before:' . Carbon::now(),
]);
}
The caveat with this is that the validation error will be thrown within the model instead of the controller, but that shouldn't really be an issue I don't think.
For any data you have in your application you can use Laravel validation.
You can merge your data and process it using Validator facade like this:
$combinedData = array_merge($item->toArray(), $firebaseData);
Validator::make($combinedData, [
'active' => 'required|in:1',
'time_expires' => 'required|before:' . Carbon::now()->toDateTimeString()
], $customMessageArray);
I think the best place for this code is some kind of service class you will inject to controller or other service class using Laravel dependency injection.
I am using Yii 2.1 and Yii2-elasticsearch 2.0.3 with a elasticsearch 1.5.0 server to try and index a Member model for a more powerful search. I have a common\indexes\Member model that extends yii\elasticsearch\ActiveRecord and set up the attributes I want to index.
namespace common\indexes;
use yii\elasticsearch\ActiveRecord;
class Member extends ActiveRecord {
/**
* #return array the list of attributes for this record
*/
public function attributes() {
// path mapping for '_id' is setup to field 'id'
return ['id', 'given_name', 'family_name', 'email'];
}
}
I am having trouble setting the attributes I want in the common\indexes\Member model.
I am creating a new instance of the object and trying to set the attribute values via the ActiveRecord setAttributes() method but it doesn't seem to set any values.
$index = new common\indexes\Member();
$index->setAttributes(['given_name' => 'Test', 'family_name' => 'User', 'email' => 'test.member#test.com']);
$index->save();
This seems to create an empty record. If I manually set the attributes one by one everything seems to work and a record with the correct attributes is created in the elasticsearch database.
$index = new common\indexes\Member();
$index->given_name = 'Test';
$index->family_name = 'User';
$index->email = 'test.member#test.com';
$index->save();
Am I using the setAttributes() method incorrectly for elasticsearch ActiveRecord's? Do I need to set up my elasticsearch Model differently?
By default setAttributes only sets attributes that have at least a single validation rule defined or those that are - as a minimum - defined as being "safe" (either via safeAttributes() or via the safe-validator).
You can force it to assign everything by just changing the call to
$index->setAttributes([
'given_name' => 'Test',
'family_name' => 'User',
'email' => 'test.member#test.com'
], false);
This tells it it is okay to also assign non-safe attributes.
But I usually prefer to make sure the validation is configured correctly
let me explain my problem, hope you get an idea of what it is.
I have a web service which hide from public access, and have designed a secure way of mysql sql querying using to the service across the websites. so i dont think i can really use the current model layer of Yii2, and that also means i hardly can use activeDataProvider as no database present.
Currently what i do is to write raw sqls and get all the results and then feed into dataprovider using ArrayDataProvider.
e.g.
$sql="select * from a_table";
$result=$remote->select($sql);
$dataProvider = new ArrayDataProvider([
'allModels' => $result,
'sort' => [
'attributes' => ['date', 'name'],
],
'pagination' => [
'pageSize' => 10,
],
]);
return $this->render('index', [
'dataProvider' => $dataProvider,
]);
that pose a problem, everytime i need to query the full table. This is not idea if the table is very large. It is better to query in the size of 10 something, however if i do
$sql="select * from a_table LIMIT 10";
no pagination will appear in my case...How do i solve this problem? And if this is not an idea way to talk to external data services, what is ur suggestion?
The pagination won't appear because you only feed 10 rows to the ArrayDataProvider and it has no way of knowing there is more.
If you want to use DataProvider with remote fetching I would advise you to create your own MyRemoteDataProvider class by extending BaseDataProvider and overriding at least these four methods:
use yii\data\BaseDataProvider;
class MyRemoteDataProvider extends BaseDataProvider
{
protected function prepareModels()
{
}
protected function prepareKeys($models)
{
}
protected function prepareTotalCount()
{
}
protected function sortModels($models, $sort)
{
}
}
.. and then of course use your MyRemoteDataProvider instead of ArrayDataProvider. If you don't know what the methods should return, please read documentation for BaseDataProvider or get inspired by ArrayDataProvider implementation.