For context:
I am setting up a PubSub Emitter for snowplow. (For other readers PubSub is a simple queue on Google Cloud Platforms that takes in messages which are an array as input).
['data' => 'Name', 'attributes' => 'key pair values of whatever data you are sending']
The above is irrelevant except that I must create a custom Emitter class in order to achieve this goal since Google Cloud PubSub has some different connectors than the stereotypical http request/sockets/others that snowplow provides.
Actual problem:
I want to set a specific schema for each event I am sending. How do you associate the schema to each payload?
The PHP Tracker SyncEmitter (the most standard snowplow provided Emitter) doesn't allow any custom setting for the schema (as shown below)
private function getPostRequest($buffer) {
$data = array("schema" => self::POST_REQ_SCEHMA, "data" => $buffer);
return $data;
}
It is hardcoded in to every event tracked.
So I investigated. And read up on snowplow trackers a bit more. I am still baffled, and I know I can extend the Payload class and force my own schemas as a variable, but why is it not this way already? I am asking because I am assuming the opensource programmer did it right, and I am not understanding it correctly.
I figured it out.
The Tracker class contains trackUnstructuredEvent:
/**
* Tracks an unstructured event with the aforementioned metrics
*
* #param array $event_json - The properties of the event. Has two fields:
* - A "data" field containing the event properties and
* - A "schema" field identifying the schema against which the data is validated
* #param array|null $context - Event Context
* #param int|null $tstamp - Event Timestamp
*/
public function trackUnstructEvent($event_json, $context = NULL, $tstamp = NULL) {
$envelope = array("schema" => self::UNSTRUCT_EVENT_SCHEMA, "data" => $event_json);
$ep = new Payload($tstamp);
$ep->add("e", "ue");
$ep->addJson($envelope, $this->encode_base64, "ue_px", "ue_pr");
$this->track($ep, $context);
}
Which accepts the schema as input. Snowplow wants you to use the Tracker's default function and provided the above as a solution to my issue.
But it still has a schema wrapped around the data(that contains the input schema).... More questions from my own answer...
Related
I'm trying to figure out how to add a task using the Google tasks API. The docs leave a lot to investigate and I seen to be stuck. This is my code:
// Get the API client and construct the service object.
$client = getClient();
$service = new Google_Service_Tasks($client);
...
... get token etc...
...
// Create a task
$task = new Google_Service_Tasks_Task();
$task->setTitle("Call mum");
$task->setNotes("Inserted by app");
$task->setStatus("needsAction");
$task->setDue(new DateTime('2020-01-01T00:00:01.000Z'));
// constructor needs 4 params: service, serviceName, resourceName, resource
$res = new Google_Service_Tasks_Resource_Tasks('', '', '', '');
$res->insert($lastListID, $task);
When creating the new Google_Service_Tasks_Resource_Tasks (second-last line) I need to provide 4 parameters to the constructor: service, serviceName, resourceName, resource.
I cant find any docs explaining the last three params. I use this class because it has an insert method and I think that's the one I need. The examples all stop at listing the tasks (works).
I tried making sense of these docs:
https://developers.google.com/tasks/v1/reference/tasks/insert
https://developers.google.com/resources/api-libraries/documentation/tasks/v1/php/latest/index.html
I couldn't make sense of the actual classes in my vendor dir.
Does anyone know how to tame this API?
I understand that you want to create a task using the PHP Tasks API. Your approach is correct, you only need to use the constructor with three parameters instead of four. As we can see on the Google_Service_Tasks_Resource_Tasks documentation, the insert operation is defined as:
/**
* Creates a new task on the specified task list. (tasks.insert)
*
* #param string $tasklist Task list identifier.
* #param Google_Service_Tasks_Task $postBody
* #param array $optParams Optional parameters.
*
* #opt_param string parent Parent task identifier. If the task is created at
* the top level, this parameter is omitted. Optional.
* #opt_param string previous Previous sibling task identifier. If the task is
* created at the first position among its siblings, this parameter is omitted.
* Optional.
* #return Google_Service_Tasks_Task
*/
public function insert($tasklist, Google_Service_Tasks_Task $postBody, $optParams = array())
{
$params = array('tasklist' => $tasklist, 'postBody' => $postBody);
$params = array_merge($params, $optParams);
return $this->call('insert', array($params), "Google_Service_Tasks_Task");
}
That comment define the three parameters as:
Task list identifier. This is the id of the task list as defined on its resource.
Content of the task object to be created, like the one on your code.
Optional object containing two elements:
Parent task identifier. This element is called id on the resource documentation. If the task has no parent task, this parameter can be omitted.
Previous sibling task. This element is the same as the previous point, but makes reference to the previous element of the list. If the new element will be the first one between its sibling (or it will be the only one) this parameter can be omitted.
A working example, based on the variables of your code, will be:
$optParams = array("{PARENT TASK ID}", "{PREVIOUS TASK ID}");
$res = new Google_Service_Tasks_Resource_Tasks();
$res->insert($taskListID, $task, $optParams);
With this method you can create a task using your approach. Please, do not hesitate to ask for further clarification if you have any questions.
I basically want to do this: Add custom field to Laravel queued job records?
However the top-voted answer there doesn't really address how to do this, rather it advocates for a work-around.
I know in order to actually accomplish this I need to tap into the dispatch function. The issue is it isn't being called directly in my (in this case) event listener.
According to this and this dispatch is a global helper function. Where in the source code is this registered and how should I overwrite it in my event listener in order to run handle() (that which dispatch itself calls) in my Job class after a specified delay and yet be able to add additional entries/fields to the database?
My prime reason for this is because:
1). The job is sending a notificaiton via email.
2). Emails should only go out after a delay IF the user hasn't logged in/been active during that delay.
3). AND emails should only go out if that user doesn't have another email queued up since the first was dispatched, in which case the first should be consolidated and deleted into the second email.
Therefore I need additional database fields in order to find, delete and modify entries in the jobs table.
These ideas come from an article on Medium which teaches that you don't want to spam users with emails/notifications and you need to consolidate/group/prioritize them and I'm trying to do this in Laravel.
I believe it is possible in Laravel but I am unsure how to overwrite the functionality when dispatch is a global and I don't know from where it is invoked. The Laravel docs on the Command Bus don't go beyond Laravel 5.0.
Edit: I have to use Redis now, according to this (because I am getting that beginTransaction() error in my queue):
Redis Driver > Database Driver
I think a model watcher would be best but I'm not sure if Job is an Eloquent model. I need something that will work consistently across database drivers.
When you create a job instance everything assigned to a public property is serialized and thereafter available to you during the job processing. You can pass in parameters to the job constructor and then assign them to properties like
public $foo;
public function __constructor($bar) {
$this->foo = $bar;
}
Then in the handle function you can get the value $bar from $this->foo
Edit: If found this method within the DatabaseQueue class. Perhaps you could override to suit your needs:
/**
* Create an array to insert for the given job.
*
* #param string|null $queue
* #param string $payload
* #param int $availableAt
* #param int $attempts
* #return array
*/
protected function buildDatabaseRecord($queue, $payload, $availableAt, $attempts = 0)
{
return [
'queue' => $queue,
'attempts' => $attempts,
'reserved_at' => null,
'available_at' => $availableAt,
'created_at' => $this->currentTime(),
'payload' => $payload,
];
}
I am using API Platform 2.0 with Symfony 3.1.
I followed the WIP documentation to add groups to serialisation context conditionally. For that I decorated the ContextBuilder.
This works well to set some groups based on currently logged in user.
But now I additionally want to add some groups depending on the requested resource item - so I need that to be available as Object, already fetched from persistence layer.
Something like this:
public function createFromRequest(Request $request, bool $normalization, array $extractedAttributes = null) : array {
/* #var $currentUser User */
$currentUser = $this->tokenStorage->getToken()->getUser();
/* #var $requestedProduct Product */
$requestedProduct = $this->getRequestedItem();
if ($product->getAuthoringOrganization() === $currentUser->getOrganization() {
$context['groups'][] = 'api_products_get_item_current_user_is_owner';
}
return $context;
}
I am afraid i cant get the requested item / collection in ContextBuilder. If so, i would be really happy to get some advice where to build my serialisation groups.
In EventListener i can do that to get what i called "$requestedProduct" here:
$subject = $event->getControllerResult())
Thanks a lot for your help.
Cheers Ben
It is available from the request attributes:
$request->attributes->get('data');
See https://github.com/api-platform/core/blob/2d252cfe1a188cc0d05c70a78d111b7e54c6d650/src/EventListener/ReadListener.php#L61
This is related to my other question: Persisting entities using a REST API.
For a project in Symfony2 I need to be able to persist entities using an remote (third-party) RESTful API. I also want to be able to retrieve entities with data from that API.
In other words, my objects are saved in the third-party database. They are not saved in my own database. Whenever I need to save data, or find data, I use their REST API.
I have been pointed to several libraries, including one made by Doctrine itself. However, none of them offers me what I'm looking for. The one made by Doctrine is the best option, but uses the Active Record pattern and doesn't offer all the sweet Doctrine 2 stuff. Don't get me wrong, I've been using Active Record implementations for a long time, but I've fallen in love with Doctrine's Data Mapper pattern now.
Ideally, I'd like to be able to use Doctrine's ORM and simply replace the database-specific part with logic that saves entities using an API call. (and of course retrieves them using that same API). This way I can save my entities using roughly the same syntax:
// current way to save $entity in database:
$em = $this->getDoctrine()->getManager();
$em->persist($entity);
$em->flush();
// desired way to save $entity using REST API:
// (just an example, it doesn't have to be exactly like this)
$em = $this->getDoctrine()->getManager('rest');
$em->persist($entity);
$em->flush();
Note that I'm not trying to build my own API, I'm simply trying to communicate with a third party API in order to save my entities. I'm relatively new to Doctrine, but I'm liking it so far. I really like the idea of seperating the persistence logic from the entities, but so far I can't find out how I can use that to save them using an API.
There is an article in Symfony's documentation, describing how to work with multiple Entity Managers. I'm looking for a solution similar to this, but with an entity manager that enables me to use REST instead of the DB.
I've been trying to tweak Doctrine's ORM myself, but I only end up rewriting half their code because it (seems to be) too tightly coupled to the Database-specific logic. I might be doing something stupid of course.
So my question is, is there a way to replace / override the database-specific parts of Doctrine's ORM with custom ones? Without rewriting a lot of things that should be common for all persistence methods? Has it been done before? Or is it simply not possible because Doctrine is intended for use with a database and isn't flexible enough for other uses?
My own progress
CakePHP seems to be able to do this, by letting you define a custom DataSource. This way you can save your models using an SQL database, but also using an API, sessions, etc. I want to do roughly the same, but using Doctrine instead of CakePHP.
Update 1
The actual database queries seem to be executed by the
Doctrine\ORM\Persisters\BasicEntityPersister class. There are several other xxxPersister classes, to deal with different types of inheritance. It might be possible to replace the xxxPersister classes with our own, so we can replace the DB code with REST API code.
The persister objects are created within the getEntityPersister() method of the Doctrine\ORM\UnitOfWork class. The classnames are hardcoded so we need to override Doctrine\ORM\UnitOfWork if we want to use our own persisters.
Update 2
Doctrine\ORM\UnitOfWork seems to be hardcoded into Doctrine\ORM\EntityManager, so we need to override that one as well. However, this class seems to contain some database-specific parts. For instance, it's constructor requires a Doctrine\DBAL\Connection object as parameter. Perhaps it's better to create our own EntityManger (implementing the Doctrine\Common\Persistence\ObjectManager interface), as long as that doesn't take too much time / effort.
Update 3
The database-specific code for retrieving/loading/finding objects lives in the same class as the code for persisting / deleting etc: the Doctrine\ORM\Persisters\xxxPersister classes. So if we are able to replace them with our own, in order to persist objects, we can retrieve objects as well. When you call $entityRepository->findAll(), for instance, it will return $entityRepository->findBy(array()) (because findAll() is simply an alias for findBy(array())) which will run the following code:
$persister = $this->_em->getUnitOfWork()->getEntityPersister($this->_entityName);
return $persister->loadAll($criteria, $orderBy, $limit, $offset);
In other words, once we get EntityManager to create the right UnitOfWork and xxxPersister objects, we will be able to use the find methods in the EntityRepository.
Update 4
I discovered that a new feature is developed for Doctrine: custom persisters (also see this). This should make it easier to use a custom persister class. I don't know yet if it will enable us to create a non-DB persister, but it looks promising. However, the last updates were in August, so I'm not sure if it's still in active development.
You might use https://github.com/doctrine/rest to build a REST client, which talks to the target server. The essential part here is the mapping from entity (local) to REST API (target).
In short: Doctrine2 (local DB) -> Rest client (entity to rest mapping) -> Request (target server)
Doctrine/Rest provides also the other way around: a Doctrine Rest Server, to expose your local entities via REST (requests to your server). But thats not what you are looking for.
DoctrineRestDriver is exactly doing what you are looking for.
https://github.com/CircleOfNice/DoctrineRestDriver
Configure Doctrine:
doctrine:
dbal:
driver_class: "Circle\\DoctrineRestDriver\\Driver"
host: "http://www.your-url.com/api"
port: 80
user: "Circle"
password: "CantRenember"
Build entity:
/**
* This annotation marks the class as managed entity:
*
* #ORM\Entity
*
* You can either only use a resource name or the whole url of
* the resource to define your target. In the first case the target
* url will consist of the host, configured in your options and the
* given name. In the second one your argument is used as it is.
* Important: The resource name must begin with its protocol.
*
* #ORM\Table("products|http://www.yourSite.com/api/products")
*/
class Product {
/**
* #ORM\Column(type="integer")
* #ORM\Id
* #ORM\GeneratedValue(strategy="AUTO")
*/
private $id;
/**
* #ORM\Column(type="string", length=100)
*/
private $name;
public function getId() {
return $this->id;
}
public function setName($name) {
$this->name = $name;
return $this;
}
public function getName() {
return $this->name;
}
}
Let's assume we have used the value http://www.yourSite.com/api/products for the product entity's #Table annotation.
Controller:
<?php
namespace CircleBundle\Controller;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\HttpFoundation\Response;
class UserController extends Controller {
/**
* Sends the following request to the API:
* POST http://www.yourSite.com/api/products HTTP/1.1
* {"name": "Circle"}
*
* Let's assume the API responded with:
* HTTP/1.1 200 OK
* {"id": 1, "name": "Circle"}
*
* Response body is "1"
*/
public function createAction() {
$em = $this->getDoctrine()->getManager();
$entity = new CircleBundle\Entity\Product();
$entity->setName('Circle');
$em->persist($entity);
$em->flush();
return new Response($entity->getId());
}
/**
* Sends the following request to the API by default:
* GET http://www.yourSite.com/api/products/1 HTTP/1.1
*
* which might respond with:
* HTTP/1.1 200 OK
* {"id": 1, "name": "Circle"}
*
* Response body is "Circle"
*/
public function readAction($id = 1) {
$em = $this->getDoctrine()->getManager();
$entity = $em->find('CircleBundle\Entity\Product', $id);
return new Response($entity->getName());
}
/**
* Sends the following request to the API:
* GET http://www.yourSite.com/api/products HTTP/1.1
*
* Example response:
* HTTP/1.1 200 OK
* [{"id": 1, "name": "Circle"}]
*
* Response body is "Circle"
*/
public function readAllAction() {
$em = $this->getDoctrine()->getManager();
$entities = $em->getRepository('CircleBundle\Entity\Product')->findAll();
return new Response($entities->first()->getName());
}
/**
* After sending a GET request (readAction) it sends the following
* request to the API by default:
* PUT http://www.yourSite.com/api/products/1 HTTP/1.1
* {"name": "myName"}
*
* Let's assume the API responded the GET request with:
* HTTP/1.1 200 OK
* {"id": 1, "name": "Circle"}
*
* and the PUT request with:
* HTTP/1.1 200 OK
* {"id": 1, "name": "myName"}
*
* Then the response body is "myName"
*/
public function updateAction($id = 1) {
$em = $this->getDoctrine()->getManager();
$entity = $em->find('CircleBundle\Entity\Product', $id);
$entity->setName('myName');
$em->flush();
return new Response($entity->getName());
}
/**
* After sending a GET request (readAction) it sends the following
* request to the API by default:
* DELETE http://www.yourSite.com/api/products/1 HTTP/1.1
*
* If the response is:
* HTTP/1.1 204 No Content
*
* the response body is ""
*/
public function deleteAction($id = 1) {
$em = $this->getDoctrine()->getManager();
$entity = $em->find('CircleBundle\Entity\Product', $id);
$em->remove($entity);
$em->flush();
return new Response();
}
}
You can even use DQL or native queries.
As a ready-to-use solution wasn't available, I decided to write my own. I called it RAPL. It's heavily inspired by Doctrine's ORM (in fact, it uses many of the interfaces provided by Doctrine Common).
Using RAPL I can simply write a small YAML file to configure the mapping between my entities and the web service, allowing me to persist/retrieve entities using the custom EntityManager.
I think you are in not right way.
I'm not ready to dig into the documentation now, but I understand doctrine stack as:
ORM -> DQL (doctrine query language) ->dbal ->Some database sql
And point for implementation you feature in DBAL as custom database driver.
I think create common REST-Driver realy interesting feature and it will do easy integration with third-party services.
I'm not sure, but you can try to use lifecycle callback events for entities to perform persisting logic via REST.
I wanted to do a similar thing, so I built this library to help expose doctrine entities as RESTful resources. It has a fair amount of features, and allows you to define exactly what you want to have exposed via both pull (GET) and push (POST/PUT/PATCH) methods.
http://leedavis81.github.io/drest/
https://github.com/leedavis81/drest
Hope it helps
Passing uid as an argument works fine with this code:
$bouts = views_get_view_result('Results', 'page_1', array($user->uid));
The key line in views_get_view_result that sets arguments is:
$view->set_arguments($args);
But what about passing date ranges?
Also, if something is specified as a filter on a view, is there a way to prorammatically alter it?
views_get_view_result:
/**
* Investigate the result of a view.
* from Drupal.org.
*
* #param string $viewname
* The name of the view to retrieve the data from.
* #param string $display_id
* The display id. On the edit page for the view in question, you'll find
* a list of displays at the left side of the control area. "Defaults"
* will be at the top of that list. Hover your cursor over the name of the
* display you want to use. A URL will appear in the status bar of your
* browser. This is usually at the bottom of the window, in the chrome.
* Everything after #views-tab- is the display ID, e.g. page_1.
* #param array $args
* Array of arguments. (no keys, just args)
* #return
* array
* An array containing an object for each view item.
* string
* If the view is not found a message is returned.
*/
function views_get_view_result($viewname, $display_id = NULL, $args = NULL) {
$view = views_get_view($viewname);
if (is_object($view)) {
if (is_array($args)) {
$view->set_arguments($args);
}
if (is_string($display_id)) {
$view->set_display($display_id);
}
else {
$view->init_display();
}
$view->pre_execute();
$view->execute();
/* print "<pre> $viewname: $display_id";
print_r(get_class_methods($view)); */
return $view->result;
}
else {
return t('View %viewname not found.', array('%viewname' => $viewname));
}
}
As for passing data ranges and given the posted function definition, you could pass date ranges to that only if the view would accept them as arguments. I'm not 100% sure, but afaik date ranges can only be defined as filters, not as arguments, which leads to your second Question:
Programmatically altering the views filter settings is possible, but a bit messy, given the rather complicated view object/array mashup structure. In your posted function above, the first line is
$view = views_get_view($viewname);
After that, $view contains the whole view object. The filter settings are defined per display, so assuming you have a view with only a default display, you will find the filter settings under
$view->display['default']->display_options['filters']
(Note the object/array notation mix - the display is a contained object of type views_display)
The 'filters' array contains one entry per filter, with varying elements depending on the filter type. For your purpose, I would suggest to create a dummy view with just the filter you are interested in, with preconfigured/hardcoded values. Using a debugger (or var_dump/print_r) you can then take a look at the filter array after view creation. From what you find there, you should be able to deduce how to inject your custom date range.
Disclaimer: Poking around in the view like this is a bit annoying and not to effective, but it works. As of yet, I have not found a concise documentation of Views2 that would explain the innards in a straight forward way, as I find the official API documentation a bit lacking concerning the usage from code. (Of course this could well be just me being to stupid ;)
If you're using views 2, you can use the GUI to add a date argument. Then in the url, you can put :
www.yousite.com/yourview/startDate--finishDate
For the startDate/finishDate, the format is YYYY-MM-DD-HH.
GL!