Override/Update Variable/Array in php file - php

In my recent project, working on console command where I need to perform/run various action mentioned in json based on the linux standard convention as
what:mv(Move),type:file,dir
what:mkdir(Make Directory)
what:touch(Make File)
what:cp(Copy), type:file,dir
what:echo (Write into File), type:override,append
what:sed (Find and Replace in file)
and param schema would be same almost exact to linux convention.
Current SetUp (Mkdir, touch)
Json Schema (Array)
"actions" => [
[
'what' => "mkdir",
'param' => [
'name' => "cache",
'in' => "bootstrap",
],
],
[
'what' => "touch",
'param' => [
'name' => ".gitignore",
'in' => "bootstrap/cache",
],
]
],
and its iterate through all action and resolve action class per what type (mkdir,touch) like MkdirOperation for mkdir and call handle functions respectively.
<?php
use Symfony\Component\Filesystem\Filesystem;
use Symfony\Component\Filesystem\Exception\IOExceptionInterface;
class MkdirOperation extends Operation
{
const ATTR_IN = "in";
const ATTR_NAME = "name";
public function handle()
{
$path = $this->_path();
$this->oIO->comment($path);
if ($this->oFileSystem->isAbsolutePath($path)) {
try {
$this->oFileSystem->mkdir($path);
} catch (IOExceptionInterface $e) {
echo "An error occurred while creating your directory at "
.$e->getPath();
}
$this->oIO->info("Directory created at:".$path);
}
}
private function _path()
{
return $this->oConfig->getBaseDir()
.$this->aParam[self::ATTR_IN].DIRECTORY_SEPARATOR
.$this->aParam[self::ATTR_NAME]
.DIRECTORY_SEPARATOR;
}
}
Requirement:
//somefile.php
$path = "/var/www/ins/"
//someotherfile.php
return [
'files' = [
'Path\\To\\NameSpace',
'Path\\To\\NameSpace'
]
];
So, basically I want to update/override my mentioned variable/array according to specific rules, for that purpose, I tried to prepare rules in json schema:
"actions": [
{
"what": "sed",
"in": "path/to/somefile.php",
"find": {
"type": "variable",
"value": "path"
},
"replace": {
"type": "variable",
"value": "__DIR__.'/../vendor/compiled.php';"
}
},{
"what": "put",
"value": "Path\\To\\NameSpace",
"in": "path/to/someotherfile.php",
"find": {
"type": "array",
"at": "files"
}
}
]
The Component I'm using
symfony/console
symfony/finder
Symfony/filesystem
Looking for:
Suggestion to organize rules set schema in such manner to iterate through all actions for update/override variable or push/pull element from array and perform action.
Mechanism to update the value of specific variable and also push/pull element from array/subarray using php.
If still something unclear from my side let me know.
Thanks in advance.

Related

ElasticSearch 7 & PHP - Create mapping for parent / child relation

I want to index in my index buildings 2 types of documents: building & apartment.
A building can have several apartment, so I want to use parent/child relationships.
I use ES 7.10 and PHP with official PHP lib (elasticsearch/elasticsearch).
According to the doc (https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child-mapping.html), I have to use something like :
PUT index buildings
{
"mappings": {
"building": {},
"apartment": {
"_parent": {
"type": "building"
}
}
}
}
In my PHP I have :
$this->getClient()->indices()
->create([
'index' => 'buildings',
'body' => [
'mappings' => [
'building' => [],
'apartment' => [
'_parent' => [
'type' => 'building'
]
]
]
]
]);
Which throws the error:
Root mapping definition has unsupported parameters: [apartment: {_parent={type=building}}] [building: []]"}},"status":400}
Did I miss something?
Parent/type has been deprecated in favor of join type in Elasticsearch 7 as far as I remember, so this might be the reason you're getting the exception.
Link to docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html

Elasticsearch in php doesn't recognize dash

I'm working on a project and try to make a search with elasticsearch but my field can contain dash and when I search with it I can't find the result I'm looking for, so I tried to change the mapping but the index doesn't work at all. I don't have any error message but I can't find what I indexed even using a different field. So what I did was :
$params = [
'index' => 'arc',
'type' => 'purchase',
'id' => $purchase['id'],
'body' => $purchase
];
It worked great with that except for the field with the dash. My $purchase looks like that :
array:34 [
"id" => 163160
"distant" => "MOR-938BBM28147090"
[...]
]
so when I search for "MOR" I find the result but when I do "MOR-" nothing. I tried to change the mapping by doing that :
$params = [
'index' => 'arc',
'type' => 'purchase',
'id' => $purchase['id'],
'body' => [
'mappings' => [
'_default_' => [
'properties' => [
'distant' => [
'type' => 'string',
'index' => 'not_analyzed'
]
]
]
],
$purchase
]
];
But with that even if I try to search "163160" I can't find any result.
Whitespace analyzer could be the right solution in this case. It takes into account only whitespaces while breaking text into tokens, and characters like "-" or "_" are still treated as a part of a term.
But if you need to do a partial matching, for example with "MOR-" token, then it requires a bit more complicated mapping.
As I don't know php, I'll be using Elasticsearch syntax. First, create a proper mapping:
PUT http://127.0.0.1:9200/arc
{
"settings": {
"analysis": {
"analyzer": {
"edge_ngram_analyzer": {
"tokenizer": "my_tokenizer"
}
},
"tokenizer": {
"my_tokenizer": {
"type": "edge_ngram",
"min_gram": 3,
"max_gram": 18,
"token_chars": [
"letter",
"digit",
"punctuation"
]
}
}
}
},
"mappings": {
"purchase": {
"properties": {
"distant": {
"type": "string",
"analyzer": "edge_ngram_analyzer"
}
}
}
}
}
As you can see, I use EdgeNGram tokenizer here. When you index a document with MOR-938BBM28147090 in distant field, it will create following tokens:
[MOR, MOR-, MOR-9, MOR-93, MOR-938, MOR-938B, MOR-938BB, ...]
The core point here is punctuation character class in token_chars list, that tells elasticsearch, that dash character (and some others like ! or ") should be included in a token and not treated as a "split char".
Now when I index the document:
PUT http://127.0.0.1:9200/arc/purchase/163160
{
"distant": "MOR-938BBM28147090"
}
and run a term search query:
POST http://127.0.0.1:9200/arc/purchase/_search
{
"query": {
"bool" : {
"must" : {
"term" : {
"distant": "MOR-93"
}
}
}
}
}
I get in response:
"hits": {
"total": 1,
"max_score": 0.6337049,
"hits": [
{
"_index": "arc",
"_type": "purchase",
"_id": "163160",
"_score": 0.6337049,
"_source": {
"distant": "MOR-938BBM28147090"
}
}
]
}

Yii2 Elasticsearch extension - how do I handle type mapping?

I want to be able to store a json object in my ES index. Here's an example of what I'm trying to store (this a serialized model, a request body that is sent to ES):
"{"id":218,"name":"Test2","category_id":1,"address":"Pushkin street","phone":null,"site":null,"location":{"lat":64,"lon":70},"city":"Heaven","description":"Super company","tags":["#test1","#test2"]}"
When I try to store it (via the extension, of course), here's the error that ES returns:
"{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [location]"}],"type":"mapper_parsing_exception","reason":"failed to parse [location]","caused_by":{"type":"illegal_argument_exception","reason":"unknown property [lat]"}},"status":400}"
It seems that I am unable to do so without having a specific type mapping, like in the docs:
https://www.elastic.co/guide/en/elasticsearch/reference/1.4/mapping-object-type.html
However, I don't seem to find a way to provide that mapping inside the model. The extension's documentation doesn't really say anything about it.
So, my question is: do I need it at all, and if I do, how?
Appreciate all feedback.
I'll assume your model is \yii\elasticsearch\ActiveRecord. You'll need to describe its attributes:
public function attributes()
{
return [
'name',
'category_id',
'address',
'phone',
'site',
'location',
'city',
'description',
'tags'
];
}
Don't forget to configure index() and type(). In the following example type is my_address.
Then you'll need to create an index with proper field mapping. Here's what your mapping should look like:
"mappings" : {
"my_address" : {
"properties" : {
"name" : { "type" : "string"},
"category_id" : { "type" : "integer"},
"address" : { "type" : "string"},
"phone" : { "type" : "string"},
"site" : { "type" : "string"},
"location" : { "type" : "geo_point"},
"city" : { "type" : "string"},
"description" : { "type" : "string"},
"tags" : { "type" : "string"}
}
}
}
Note three things:
Location is of type geo_point.
Tags are declared as string. This will also allow them to be arrays of strings.
I didn't include the id field. If it's unique, I suggest you just set your yii model's id to the necessary value ($model->primaryKey = '123'). Otherwise your ES model will have its internal id set to something like AVDXmfJ3Ou7LzqD1DDMj and also have an id field which is not very convenient.
I encourage you to take a closer look at the mappings - they are very important when it comes to configuring how exactly the strings are being analyzed.
UPDATE: You don't really describe the mapping anywhere in your model. Do it in a migration - similar to creating tables in SQL.
In case you using ElasticSearch ActiveRecord , you could define a method for setupMapping
Class BookIndex extends yii\elasticsearch\ActiveRecord
{
/**
* sets up the index for this record
*
*/
public static function setUpMapping()
{
$db = static::getDb();
//in case you are not using elasticsearch ActiveRecord so current class extends database ActiveRecord yii/db/activeRecord
// $db = yii\elasticsearch\ActiveRecord::getDb();
$command = $db->createCommand();
/*
* you can delete the current mapping for fresh mapping but this not recommended and can be dangrous.
*/
// $command->deleteMapping(static::index(), static::type());
$command->setMapping(static::index(), static::type(), [
static::type() => [
// "_id" => ["path" => "id", "store" => "yes"],
"properties" => [
'name' => ["type" => "string"],
'author_name' => ["type" => "string"],
'publisher_name' => ["type" => "string"],
'created_at' => ["type" => "long"],
'updated_at' => ["type" => "long"],
'status' => ["type" => "long"],
],
],
]);
}
}
Later on you just need to call this method any time you want to apply the new mapping.

elasticsearch php search exists

How might one do the following request
GET /giata_index/giata_type/_search/exists
{
"query": {
"bool": {
"must": [
{
"term": {
"status": 2
}
},
{
"term": {
"ids": "26744"
}
}
]
}
}
}
with ElasticSearch's PHP library?
I have played around with the exists endpoint, but as it turns out, that can only check whether a specific uid is existant or not. So I guess I need to do a search. But I can't find a parameter in the Search endpoints's whitelist that would allow a simple check for exists or not.
The reason why I would like to avoid getting the entire document and just ask whether it exists or not is because I have multiple hundreds of thousands of imports and just as many documents in ES, so I would like it to put as little work into it as possible.
Note: I have also looked into head requests that are possible via HTTP requests (only retrieve the header of a document - either 200 or 404). But that would probably only exist for requests via HTTP.
If worse comes to worse I could shoot a curl via php and simply do it via HTTP. But I would prefer it otherwise.
It seems indeed that there's no endpoint voor search exists, but I think you use a simple alternative:
Use an empty "fields" array. And count the results of your query. If == 0: false. If > 0: true
GET /giata_index/giata_type/_search
{
"fields": [],
"query": {
"bool": {
"must": [
{
"term": {
"status": 2
}
},
{
"term": {
"ids": "26744"
}
}
]
}
}
}
An other alternative is to use _count : https://www.elastic.co/guide/en/elasticsearch/reference/1.6/search-count.html
It should be possible with the latest 2.x version.
Code sample could be something like this:
$clientBuilder = Elasticsearch\ClientBuilder::create();
// Additional client options, hosts, etc.
$client = $clientBuilder->build();
$index = 'your_index';
$type = 'your_type';
$params = [
'index' => $index,
'type' => $type,
'body' => [
'query' => [
'bool' => [
'must' => [
[
'term' => [
"status" => 2
]
],
[
'term' => [
'ids' => "26744"
]
]
]
]
]
];
try {
$client->searchExists($params);
} catch (Exception $e) {
// Not found. You might want to return FALSE if wrapped in a function.
// return FALSE;
}
// Found.
It is worth noting that if search is not wrapped in try/catch block it can break execution and throw an exception (status code 4xx if not found).
Also, it can not be used effectively in future mode.

Laravel & PHP: Return special formatted JSON

After doing a query, how can I create and echo a formatted JSON like this:
{
"results": [
{
"user_id": "1",
"name": "Apple",
"address": "7538 N LA CHOLLA BLVD",
"city": "Palo Alto",
"state": "CA",
"latlon": [
-111.012654,
32.339807
],
},
{
"user_id": "2",
"name": "Microsoft",
"address": "75 S BWY STE 400",
"city": "Palo Alto",
"state": "CA",
"latlon": [
-73.764497,
41.031858
],
},
],
"meta": {
"page": 1,
"per_page": 10,
"count": 493,
"total_pages": 50
}
}
This is my current query:
public function getAgenciesJson() {
$agencies = DB::table('users')->where('type','a')->orWhere('type','l');
}
Haven't figured out how to output JSON like that, considering I have a "latlon" field like [-111.012654,32.339807], also a "results" tag and a "meta" tag.
Thanks in advance
What you need is something called a transformer (or presenter) to convert your raw model into a format that can be sent to your users.
A very popular package is called Fractal (http://fractal.thephpleague.com/) by Phil Sturgeon. There's a Laravel package, that might make it a bit easier to use, called Larasponse (https://github.com/salebab/larasponse).
Phil actually a blog post about this just the other day - https://philsturgeon.uk/api/2015/05/30/serializing-api-output/ - that goes into why you should always have some kind of transformer between your models and what you send to your users.
There's also a guide about using Larasponse and Fractal that might be of use here - http://laravelista.com/laravel-fractal/.
The gist of it boils down to passing the model through another class that will take the models values and build an array/object in a known/fixed format, e.g. (from Phil's blog post)
return [
'id' => (int) $book->id,
'title' => $book->title,
'year' => (int) $book->yr,
'author' => [
'name' => $book->author_name,
'email' => $book->author_email,
],
'links' => [
[
'rel' => 'self',
'uri' => '/books/'.$book->id,
]
]
];
This way you're not exposing your original field names and if at any point your column names should change you only need to update that in 1 place rather than having to get any user of your JSON to update their site/app. It will also allow you to do string manipulation of your latlon column so that you can split it into 2 different values.
Using a slightly modified example from the Fractal documentation. Say you have a transformer for a User
class UserTransformer extends Fractal\TransformerAbstract
{
public function transform(User $user)
{
return [
'id' => (int) $user->id,
'name' => $user->first_name . ' ' . $user->last_name,
];
}
}
You can then use this class to either transform a single item of a collection
$user = User::find(1);
$resource = new Fractal\Resource\Item($user, new UserTransformer);
// Or transform a collection
// $users = User::all();
// $resource = new Fractal\Resource\Collection($users, new UserTransformer);
// Use a manager to convert the data into an array or json
$json = (new League\Fractal\Manager)->createData($resource)->toJson();
Fractal includes a paginator adapter for Laravel that can be used straight away
$paginator = User::paginate();
$users = $paginator->getCollection();
$resource = new Collection($users, new UserTransformer);
$resource->setPaginator(new IlluminatePaginatorAdapter($paginator));
The method exists:
->toJson()
REF: http://laravel.com/docs/4.2/eloquent#converting-to-arrays-or-json
Update your getAgenciesJson to :
public function getAgenciesJson() {
return DB::table('users')->where('type','a')->orWhere('type','l')->toJson();
}
Then you could echo by:
<?= SomeModel::getAgenciesJson(); ?>
To modify the column names you can update your select. Here is an example taken from the Laravel Docs:
$users = DB::table('users')->select('name as user_name')->get();
Here would be a more fully realized version of what you are looking for minus the column aliases since you didn't really mention what they were.
public function getAgenciesJson($page, $per_page = 10) {
$output = [
'results' => [],
'meta' => [],
];
// Get Results
$output['results'] = DB::table('users')->where('type','a')->orWhere('type','l')->take($per_page)->skip($per_page * ($page - 1))->get();
// Set Meta
$output['meta'] = [
'page' => $page,
'per_page' => $per_page,
'count' => DB::select('SELECT FOUND_ROWS()'),
'total_pages' => DB::table('users')->count() / $per_page
];
// Return
return json_encode($output);
}
Your original code didn't attempt to get or handle the pagination information but this example covers that in order to provide the meta data you indicated you wanted returned.
Not sure if you wanted Count to be the number of the current result set or a count of all the records on that table. If you don't want the current set you but rather the entire count you can use DB::table('users')->count() though I would assign it to a variable and use that rather than calling it twice in the meta info.
public function getAgenciesJson() {
$agencies = DB::table('users')->where('type','a')->orWhere('type','l')->get();
return response()->tojson($agencies,200);//for response with status code 200
}
You should call get function to get data in array format and response function will format your output to json. Second paramater signifies which status code you have to assign for your response.

Categories