How to structure/modularize an application with a cross-module search? - php

I'm developing a Zend Framework 2 application, that should provide:
a search in projects with a result list of projects
a single project view
a search in images with a result list of images (every image belongs to a project)
Also there are two main entities: image and project andthe first idea was to create two modules (Image and Project) and to give to each of them a separate search fuctionaly. But it would mean much of duplicated code. OK, maybe a separate Search module? But then this module would essentially depend on the other modules.
What would be a good approach for such case? Are there a best practice for this?

I have written a ZF2 elastic search module but it is closed source. It enables modules to put searchable data into ES. I can't post the code, but I can explain how it works.
The central module is Search. The search contains two main interfaces: Search\Repository\ItemRepositoryInterface and Search\Service\ItemServiceInterface. The repository is for searching items, the service for storing/removing items.
interface ItemRepositoryInterface
{
public function search($query, $limit = 10);
}
interface ItemServiceInterface
{
public function insert(SearchableInterface $object);
public function remove(SearchableInterface $object);
}
SearchableInterface is an interface my models/entities can use to "make" it searchable. It let's ES set the ES id and grabs the type. Usually, every entity gets its own type (so I can search all images and projects, or query only for image types).
I have implemented this for a blog/event system. The service class which persists the blog article into the database triggers events and ElasticSearch is one of the listeners:
public function onBootstrap(MvcEvent $e)
{
$app = $e->getApplication();
$sm = $app->getServiceManager();
$em = $app->getSharedManager();
$em->attach('Blog\Service\ArticleService', 'create', function($e) use ($sm) {
$searchService = $sm->get('Search\Service\ItemService');
$article = $e->getArticle();
$searchService->insert($article);
});
}
Because Article implements SearchableInterface, this works great. Now my blog doesn't have to deal with search and search doesn't have to deal with the blog. But, how is the Article transformed into a search document, you wonder?
I have a hydrator mechanism which works like the ZF2 hydrator. It does not convert any object into an array (and vice versa). It transforms a SearchableInterface object into an Elastic Search Document (for storing the object) and it transforms an Elastic Search Result (which is returned after a search query) into a SearchableInterface object again.
interface HydratorInterface
{
public function extract(SearchableInterface $object);
public function hydrate(Result $object);
}
Every type has its own hydrator registered. All these different hydrators are collected into a HydratorManager which is basically a Zend\ServiceManager\AbstractPluginManager. This plugin manager is injected into the repository and service.
So, inside the service, the following happens:
The $object->getType() is checked
For its type, the corresponding hydrator is fetched
The hydrator's extract() is called to turn the $object into a $document
The underlying ES client is used to persist the document (either it is added or updated, depending on the result of $object->getElasticSearchId()
For the repository, given a query type:image name:Mountain, the following happens:
The repository's search() is called with given query
The string is used for a ES query object and is executed
The results are iterated
For every result, the type is checked
For its type, the corresponding hydrator is fetched
The hydrator's hydrate() is called to turn the $result into an $object
The collection of objects is returned

I would create some kind of "search interface" that each module would implement to search its own data. Then your search module could check all available modules if they contain this interface, and if they do, use it to search their data...
The drawback I guess is that search code is implemented in each module...

Related

FOSElasticaBundle: Is it possible to change "query_builder_method" in controller?

According to FOSElasticaBundle documentation it is possible to configure application to use custom query builder method like this:
user:
persistence:
elastica_to_model_transformer:
query_builder_method: createSearchQueryBuilder
But is it possible to choose QB method live, e.g. in controller action?
I'd like to be able to control what's being fetched from DB while transforming Elastica results to Doctrine entities. E.g. sometimes I'll want to do eager fetch on some relations, but can't do that by default.
Since FOSElasticaBundle documentation is not very precise, I went through its code and found it impossible to control what query builder is used on controller level.
It is possible to change whole elastica_to_model_transformer to a custom service, but still it's statically defined in configuration. Maybe with some dirty solution it would be possible going this way, but I don't think it's worth it.
I decided to just not using this feature of FOSElasticaBundle. The main problem I had was that when you use fos_elastica.index instead of fos_elastica.finder or elastica repository (in order to get plain not transformed results Elastica\Resultset), there's no findPaginated method with returns Pagerfanta paginator object, which is very helpful in my case.
Fortunately although it's not mentioned in documentation it's possible to create the Pagerfanta this way too, but a little bit more manually.
Here's a code snippet:
//generate ElaticaQuery somehow.
$browseQuery = $browseData->getBrowseQuery();
$search = $this->container->get('fos_elastica.index.indexName.typName');
//create pagerfanta's adapter manually
$adapter = new \Pagerfanta\Adapter\ElasticaAdapterElasticaAdapter($search, $browseQuery);
// now you can create the paginator too.
$pager = new Pagerfanta($adapter);
//do some paging work on it...
$pager->setMaxPerPage($browseData->getPerPage());
try {
$pager->setCurrentPage($browseData->getPage());
} catch(OutOfRangeCurrentPageException $e) {
$pager->setCurrentPage(1);
}
//and get current page results.
/** #var Result[] $elasticaResults */
$elasticaResults = $pager->getCurrentPageResults();
// we have to grab ids manyally, but it's done the same way inside FOSElasticaBundle with previous approach
$ids = array();
foreach($elasticaResults as $elasticaResult) {
$ids[] = $elasticaResult->getId();
}
//use regular Doctrine's repository to fetch Entities any way you want.
$entities = $this->getDoctrine()->getRepository(MyEntity::class)->findByIdentifiers($ids);
This actually has a few advantages. In general it gives you back control over your data and doesn't tie ElasticSearch with Doctrine. Therefore you can resign on fetching data from Doctrine if you have all needed data in ElasticSearch (if they are read only data of course). This lets you optimize your application performance but reducing amount of SQL queries.
The code above may be wrapped with some kind of service in order to prevent making mess in controllers.

How can I include multiple models in one view for in a Joomla 3.x component built with Component Creator

Joomla components use the MVC model. Component Creator is a widely used tool whose paid level supports creation of multi tabled views with SQL import. Also, developers build components from scratch according to Joomla documentation.
I want to build an advanced component that functions as a "dashboard" displaying data from multiple database tables with all the administrator back-end and visitor front-end CRUD (CREATE, READ, UPDATE, DELETE) capabilities of Joomla. This means that I need multiple models (from the MVC philosophy) drawing from multiple database tables shown on the screen simultaneously.
Joomla Documentation suggests the following code be inserted into the "controller task-method" to make the information available:
$view = $this->getView( 'model-a', 'html' );
$view->setModel( $this->getModel( 'model-a' ), true );
$view->setModel( $this->getModel( 'model-b' ) );
$view->display();
and then later call upon those models with the in the views display method:
$item1 = $this->get( 'data1' );
$item2 = $this->get( 'data2', 'model-b' );
However, these instructions provided in Joomla documentation are insufficient or incompatible with the component built when following the provided Joomla Hello World Tutorial tutorial or components built from the widely used and popular Component Creator tool. Either the component will fail to load the page when called upon or will not pass the data to the view with a simple copy and paste into any one of the multiple controller created by component creator or Joomla hello world tutorial.
How can I call upon multiple models in the same view for a Joomla 3.X component?
I was able to successfully use multiple models from the same view by making calls directly in the two view files to properly formed models. I did not follow Joomla documentation because I didn't modify either possible controller (one being the controller for the entire component and the other controller being view-specific). I also did not use the functions provided in Joomla documentation, as those produced errors.
According to proper Joomla MVC convention, a view is created by two files in the relevant view directory and subfolder:
/site/views/multiviewname/view.html.php (which passes the model to the view)
/site/views/multiviewname/tmpl/default.php (which has the HTML template)
Both these need to be changed to view data from more than one model at the same time. This works assuming that all of your other views, controllers, and models are built properly, as is done automatically when using the 'Component Creator' tool. My component had hundreds of files, including css, backend administration, installation, language, etc. All of these were build in moments with the component creator tool.
The abridged but still completely functional code is as follows:
/site/views/multiviewname/view.html.php
<?php
jimport('joomla.application.component.view');
class ComponentnameViewMultiviewname extends JViewLegacy
{
// $items is for the default model
protected $items;
// $ItemsOtherModel is for second model. Notice the '$' used here but not elsewhere
protected $ItemsOtherModel;
public function display($tpl = null)
{
$app = JFactory::getApplication();
$this->state = $this->get('State');
$this->items = $this->get('Items');
$this->pagination = $this->get('Pagination');
$this->params = $app->getParams('com_componentname');
// sets default model
$this->setModel( $this->getModel( 'model-a' ), true );
// sets second model & uses 'JModelLegacy,' contrary to documentation
$this->setModel(JModelLegacy::getInstance('model-b', 'componentnameModel'));
// assigns array from the second model to 'ItemsOtherModel.' there is no '$' sign used.
$this->ItemsOtherModel = $this->get('Items','model-b');
parent::display($tpl);
}
}
/site/views/multiviewname/tmpl/default.php
<?php
echo "<h3>Items from default model</h3> ";
echo var_dump($this->items);
echo "<h3>items from secondary model</h3> ";
// notice that the '$' is absent from 'ItemsOtherModel'
echo var_dump($this->ItemsOtherModel);
This breakthrough was only possible after days of research. The paid Component Creator tool was invaluable to start me off with well formed code that adheres to Joomla MVC component standards. After working with and examining all the files for days, the I found the prompt I needed in this google groups thread, calling my attention to the JModelLegacy class, found when searching google for terms from the PHP error message PHP Notice: Undefined index: left on my server when attempting to use the officially documented methods.
This page rendered in the browser simply dumps out all information from the database table to the page, but further development can create the formatted and functional dashboard that I will ultimately need.
This code is for displaying lists of information, as opposed to multiple single items. The Joomla documentation for adding multiple models to one view is set up for multiple single items, rather than arrays of items shown here.

Where does pagination live when using datamappers, services and domain object trio

What is considered the correct place for my pagination to live when using a service/datamapper/domain object trio?
Example:
Fetch ports with a few given criteria
Paginate the results
Know what page we are on, number of pages in total, number of results etc.. from the view class
The below is just something I wrote here and now, but it is similar to my application.
class PostController extends Controller
{
function viewPosts()
{
return $this->serviceFactory
->build('post')
->getPosts($aCriteria, $this->request->_get('page'), 10);
}
}
I am currently both calculating and storing pagination parameters in each service. Note that I am not pushing any data to the view from my controller.
class PostService extends AbstractService
{
public $posts;
public $iTotalPages; // This does not belong here does it?
function getPosts($aCriteria, $iPage, $iItemsPerPage)
{
$mapper = $this->dataMapperFactory->build('post');
// I do not know where to put the below
// (The parameters and the code itself)
$iCount = $mapper->count($aCriteria);
$iOffset = $iItemsPerPage * $iPage;
$this->iTotalPages = $iCount / $iItemsPerPage;
$this->posts = $mapper->find($aCriteria, $iOffset, $iOffset + $iItemsPerPage);
return $this->posts;
}
}
My views have access to the same instance of the Model layer as my controller, so I could call $service->iTotalPages from the view, but that feels wrong.
class PostsView extends AbstractView
{
function viewPosts()
{
$service = $this->serviceFactory->build('post');
if(count($service->posts)>0) {
$this->template->assign_var('TOTAL_PAGES', $service->iTotalPages);
$this->template->assign_vars('POSTS', $service->posts);
}
}
}
Solutions?
1) Create a service for pagination and have the controller exchange data with the post service as required?
2) Create a helper class for pagination that each service can include? (How would such a class look like?)
3) Add generic pagination to the AbstractService?
4) Add some sort of pagination support to my Repos?
You answer is pretty good, though I have a few suggestions.
Layer location:
The pagination infrastructure should live in the data access layer, this is so that you have low level control on how the data are retrieved.
Highest invocation
The pagination's interface should be abstracted and exposed in the UI through the service, since its a front-end concern. (I think you already have this covered in your answer)
The Abstraction:
What the UI should know are the page index, number of pages, number of items per page, and total items. Not the Offset nor the limit, these are infrastructure terms that should be encapsulated.
Input: (part of your fitlers)
Search filters
Sorting filters (if necessary)
Page index.
Number of items per page. (if the UI has control, if not then this should be encapsulated)
Output:
Filtered collection
Number of pages (for navigation)
Number of total items (if necessary in the UI)
Due to lack of answers to this question I am posting the solution I came up with. Appreciate comments/additions to it if it can be improved.
// Controller method
public function viewPosts()
{
// Create service objects
$postService = $this->serviceFactory->build('post', true);
$searchService = $this->serviceFactory->build('search');
// Set the search/filter parameters
$searchService->addFilter('author_id', $this->visitor->user_id);
$searchService->setOffset($this->request->_get('offset'));
$searchService->setLimit($this->request->_get('limit'));
// Search service will call the 'find' method on $articleService
$searchService->setServiceObject($articleService, 'find');
// Search service will return a filtered collection
return $searchService->search();
}
Doing it this way I am not leaking business logic into my controller (I think), and I have a single class to do all my filtering and analysis of the returned sql data, while still keeping each specific service in control of their specific find() method and datamappers.

CakePHP 2.0 Object not Array

I am currently a beginner in CakePHP, and have played around with CakePHP 1.3, but recently CakePHP 2.0 has been released.
So far I like it but the only thing is being a pain is the fact that it doesn't return Objects, rather it just returns arrays. I mean, it hardly makes sense to have to do $post['Post']['id']. It is (in my opinion) much more practical to just do $post->id.
Now after Google I stumbled upon this link, however, this kept generating errors about indexes not being defined when using the Form class (guessing this is because it was getting the objectified version rather than the array version).
I am following the Blog tutorial (already have followed it under 1.3 but going over it again for 2.0)
So, anyone know how to achieve this without it interfering with the Form class?
Hosh
Little known fact: Cake DOES return them as objects, or well properties of an object, anyway. The arrays are the syntactical sugar:
// In your View:
debug($this->viewVars);
Shwoing $this is a View object and the viewVars property corresponds with the $this->set('key', $variable) or $this->set(compact('data', 'for', 'view')) from the controller action.
The problem with squashing them into $Post->id for the sake of keystrokes is Cake is why. Cake is designed to be a heavy lifter, so its built-in ORM is ridiculously powerful, unavoidable, and intended for addressing infinity rows of infinity associated tables - auto callbacks, automatic data passing, query generation, etc. Base depth of multidimensional arrays depends on your find method, as soon as you're working with more than one $Post with multiple associated models (for example), you've introduced arrays into the mix and there's just no avoiding that.
Different find methods return arrays of different depths. From the default generated controller code, you can see that index uses $this->set('posts', $this->paginate()); - view uses $this->set('post', $this->Post->read(null, $id)); and edit doesn't use $this->set with a Post find at all - it assigns $this->data = $this->Post->read(null, $id);.
FWIW, Set::map probably throws those undefined index errors because (guessing) you happen to be trying to map an edit action, amirite? By default, edit actions only use $this->set to set associated model finds to the View. The result of $this->read is sent to $this->data instead. That's probably why Set::map is failing. Either way, you're still going to end up aiming at $Post[0]->id or $Post->id (depending on what you find method you used), which isn't much of an improvement.
Here's some generic examples of Set::map() property depth for these actions:
// In posts/index.ctp
$Post = Set::map($posts);
debug($Post);
debug($Post[0]->id);
// In posts/edit/1
debug($this-viewVars);
debug($this->data);
// In posts/view/1
debug($this-viewVars);
$Post = Set::map($post);
debug($Post->id);
http://api13.cakephp.org/class/controller#method-Controllerset
http://api13.cakephp.org/class/model#method-Modelread
http://api13.cakephp.org/class/model#method-ModelsaveAll
HTH.
You could create additional object vars. This way you wouldn't interfere with Cake's automagic but could access data using a format like $modelNameObj->id; format.
Firstly, create an AppController.php in /app/Controller if you don't already have one. Then create a beforeRender() function. This will look for data in Cake's standard naming conventions, and from it create additional object vars.
<?php
App::uses('Controller', 'Controller');
class AppController extends Controller {
public function beforeRender() {
parent::beforeRender();
// camelcase plural of current model
$plural = lcfirst(Inflector::pluralize($this->modelClass));
// create a new object
if (!empty($this->viewVars[$plural])) {
$objects = Set::map($this->viewVars[$plural]);
$this->set($plural . 'Obj', $objects);
}
// camelcase singular of current model
$singular = lcfirst(Inflector::singularize($this->modelClass));
// create new object
if (!empty($this->viewVars[$singular])) {
$object = Set::map($this->viewVars[$singular]);
$this->set($singular . 'Obj', $object);
}
}
}
Then in your views you can access the objects like so:
index.ctp
$productsObj;
view.ctp
$productObj->id;
All we're doing is adding 'Obj' to the variable names that Cake would already provide. Some example mappings:
Products -> $productsObj
ProductType -> $productTypesObj
I know this is not perfect but it would essentially achieve what you wanted and would be available across all of your models.
While I like the idea Moz proposes there are a number of existing solutions to this problem.
The quickest one I found is https://github.com/kanshin/CakeEntity - but it looks like you might need to refactor it for 2.x - there might even already be a 2.x branch or fork but I didn't look.
I also ran this question couple of time in my head. Now a few Cake based apps later, I see the benefit to be able to branch and merge (am, in_array etc.) result sets more conveniently with arrays than using objects.
The $Post->id form would be a sweet syntactic sugar, but not a real benefit over arrays.
You could write a function that iterates over your public propertys (see ReflectionClass::getProperties) and save it in an array (and return the array).
If you have access to the class, you can implement the ArrayAccess Interface and easily access your object as an array.
P.S.: Sorry, i've never used CakePHP but i think object-to-array conversion doesn't have to be a framework specific problem

Filtering with symfony2

Is there any open source (or example) code for Symfony2 which can filter certain model using multiple parameters? A good example of what I'm looking for can be seen on this Trulia web page.
http://www.trulia.com/for_sale/30000-1000000_price/10001_zip/
http://www.trulia.com/for_rent/Chicago,IL/#for_rent/Chicago,IL/0-500_price/wd,dw_amenities/sm_dogs_pets"
http://www.trulia.com/for_rent/Chicago,IL/#for_rent/Chicago,IL/400-500_price/wd,dw_amenities
http://www.trulia.com/for_rent/Chicago,IL/#for_rent/Chicago,IL/wd,dw_amenities"
http://www.trulia.com/for_rent/Chicago,IL/#for_rent/Chicago,IL/400p_price/dw,cs_amenities
http://www.trulia.com/for_rent/Chicago,IL/#for_rent/Chicago,IL/1p_beds/1p_baths/400p_price/dw,cs_amenities
Note how URL are build when clicking in the form, I guess is using one controller for all this routes, How is it done?.
I Don't think it will be redirecting all the possible routes to a specific controller, (shown below), maybe some sort of dynamic routing?
/**
* #Route("/for_rent/{state}/{beds}_beds/{bath}_bath/{mix_price}-{max_price}_price /{amenities_list}
* #Route("/for_rent/{state}/{mix_price}-{max_price}_price/{amenities_list}
* #Route("/for_rent/{state}/{bath}_bath/{mix_price}-{max_price}_price/{amenities_list}
* #Route("/for_rent/{state}/{mix_price}_price/{amenities_list}
* #Route("/for_rent/{state}/{beds}_beds/{bath}_bath/{amenities_list}
* ........
*/
public function filterAction($state, $beds, $bath, $min_price, $max_price ....)
{
....
}
Thanks.
For simple queries (i.e. where you don't need to have a data range, such as min-max value), you can use the entity repository to find entities by the request parameters given. Assuming that your entity is Acme\FooBundle\Entity\Bar:
$em = $this->getDoctrine()->getEntityManager();
$repo = $em->getRepository('AcmeFooBundle:Bar');
$criteria = array(
'state' => $state,
'beds' => $beds,
// and so on...
);
$data = $repo->findBy($criteria);
When building the $criteria array, you'll probably want some logic so that you only sort by criteria that have been provided, instead of all possible values. $data will then contain all entities that match the criteria.
For more complex queries, you'll want to look into DQL (and perhaps a custom repository) for finer-grained control of the entities that you're pulling out.
To construct your routes, i'm sure you had a look at the Routing page of the documentation, but did you notice that you can put requirements on routes? This page explains how to do it with annotations.
As for the filtering, I suppose DQL would be ok, but you can also write straight up SQL with Doctrine, and map the results of your query to one or more entities. This is described here. It may be more flexible than DQL.
csg, your solution is good (with #Route("/search/{q}) if you only need to use routing in "one-way". But what if you will need to print some price filter links on page accessible by url:
http://www.trulia.com/for_sale/30000-1000000_price/10001_zip/
In case of #Route("/search/{q} you will not be able to use route method url generate with params.
There is a great Bundle called LexikFormFilterBundle "lexik/form-filter-bundle": "~2.0" that helps you generate the complex DQL after the Filter form completed by the user.
I created a Bundle, that depends on it, that changes the types of a given FormType (like the one generated by SencioGeneratorBundle) So you can display the right FilterForm and then create the DQL after it (with Lexik).
You can install it with Composer, following this README.md
All it does is override the Doctrine Type Guesser, that suggests the required FormType for each Entity field, and replace the given Type by the proper LexikFormFilterType. For instance, replaces a simple NumberType by a filter_number which renders as two numbers, Max and Min interval boundaries.
private function createFilterForm($formType)
{
$adapter = $this->get('dd_form.form_adapter');
$form = $adapter->adaptForm(
$formType,
$this->generateUrl('document_search'),
array('fieldToRemove1', 'fieldToRemove2')
);
return $form;
}
Upon form Submit, you just give it to Lexik and run the generated query, as shown in my example.
public function searchAction(Request $request)
{
// $docType = new FormType/FQCN() could do too.
$docType = 'FormType/FQCN';
$filterForm = $this->createFilterForm($docType);
$filterForm->handleRequest($request);
$filterBuilder = $this->getDocRepo($docType)
->createQueryBuilder('e');
$this->get('lexik_form_filter.query_builder_updater')
->addFilterConditions($filterForm, $filterBuilder);
$entities = $filterBuilder->getQuery()->execute();
return array(
'entities' => $entities,
'filterForm' => $filterForm->createView(),
);
}

Categories