Predis Alias Sharding - php

I'm trying to use Predis sharding by alias, as described here. My code is basically identical, but I'm only returning empty arrays. Do my hash keys need {} around them? (EDIT: Nope, just tried it)
$api->get("/test", function () {
$servers = [
["alias" => "metadata", "port" => 6380],
["alias" => "relations", "port" => 6381],
["alias" => "dim_provider", "port" => 6382],
["alias" => "dim_revctrcode", "port" => 6383],
["alias" => "dim_enccode", "port" => 6384],
["alias" => "dim_pos", "port" => 6385]
];
$options = [
"nodehash" => function ($connection) { return $connection->getParameters()->alias; },
"cluster" => function ($options) {
$replicas = Predis\Cluster\Distribution\HashRing::DEFAULT_REPLICAS;
$hashring = new Predis\Cluster\Distribution\HashRing($replicas, $options->nodehash);
$cluster = new Predis\Connection\PredisCluster($hashring);
return $cluster;
}
];
$redis = new Predis\Client($servers, $options);
try {
$test = $redis->scard("dim_provider");
print_r($test); // Prints 0 for scard or empty Array for hgetall
} catch (Exception $e) {
print $e->getMessage();
}
$redis = new Predis\Client(["port" => 6382]);
$test = $redis->scard("dim_provider");
print_r($test); // Works.
});
EDIT: It also works if I only put one server in the $servers array. So it seems the hashing is not working right. When I throw some echos in front of the return value in nodehash I can see that it's returning the alias.

Assigning a dim_provider alias to a Redis connection and trying to get a key named dim_provider from a server are two different things.
In your script you are trying to set up a cluster of Redis instances using connection aliases (instead of the usual ip:port pairs) to calculate the distribution of your keyspace among multiple Redis servers acting as your data shards. Using this setup, the key dim_provider is sharded accordingly to the underlying distribution algorithm and could be stored on any of the 6 servers composing your cluster and defined in the $servers array.

I wanted to add how trivially easy it was to implement my clustering strategy once nrk got me on the right track. This is a really well-written library.
$api->get("/test", function () {
Class KeyCluster extends Predis\Connection\PredisCluster {
public function __construct() {
$this->pool = Array();
}
public function add (Predis\Connection\SingleConnectionInterface $connection) {
$parameters = $connection->getParameters();
if (isset($parameters->table)) {
$this->pool[$parameters->table] = $connection;
} else {
$this->pool[] = $connection;
}
}
public function getConnection (Command\CommandInterface $command) {
$key = $command->getArgument(0);
$table = explode(":", $key)[0];
return isset($this->pool[$table]) ? $this->pool[$table] : null;
}
}
$redis = new Predis\Client([
"tcp://127.0.0.1:6382?table=dim_provider",
"tcp://127.0.0.1:6383?table=dim_pos"
],[
"cluster" => new KeyCluster
]);
$result = $redis->scard("dim_provider");
print_r($result);
});

Related

Mocking Symfony Ldap::create for unit tests

Recently I have been working on an LDAP authentication provider for MediaWiki. In my mind, I have been trying to tackle this issue for a number of days now and cannot come up with a solution.
Context
The way I have developed this plugin is to allow configuration of a number of servers to which we will connect. If we cannot connect to one server, we will try the next... And so on until all are exhausted.
To facilitate this, I have a function in my class that loops over the servers attempting a connection until one succeeds:
private function connect( LdapAuthenticationRequest $req ) {
$dn = $this->config->get( 'BindDN' )[$req->domain];
$pass = $this->config->get( 'BindPass' )[$req->domain];
$servers = $this->config->get( 'Servers' )[$req->domain];
$encryption = $this->config->get( 'EncryptionType' )[$req->domain];
if ( false === $dn ) {
$msgkey = 'ldapauth-attempt-bind-search';
$bind_with = [ null, null ];
} else {
$msgkey = 'ldapauth-attempt-bind-dn-search';
$bind_with = [ $dn, $pass ];
}
$message = new Message( $msgkey, [
'dn' => "{$dn}#{$req->domain}",
] );
$this->logger->info( $message->text() );
foreach ( $servers as $server ) {
if ( false === $server ) {
continue;
}
$ldap = Ldap::create( 'ext_ldap', [
'host' => $server,
'encryption' => $encryption
] );
// Attempt bind - on failure, throw an exception
try {
call_user_func_array( [ $ldap, 'bind' ], $bind_with );
$this->server = $server;
$this->encryption = $encryption;
// log successful bind
$msgkey = 'ldapauth-bind-success';
$message = wfMessage( $msgkey )->text();
$this->logger->info( $message );
return $ldap;
} catch ( SymException $e ) {
if ( false === $dn ) {
$msgkey = 'ldapauth-no-bind-search';
} else {
$msgkey = 'ldapauth-no-bind-dn-search';
}
$message = new Message( $msgkey, [
'dn' => "{$dn}#{$req->domain}",
] );
$message = $message->text();
$this->logger->info( $message );
$this->logger->debug( $e->getMessage() );
}
}
I have been trying to come up with a better way to do this, one that would permit me to better unit-test this class, but thus far I am drawing blanks.
A big part of the reason that I am stuck on this issue is that Symfony's LDAP adapter is essentially hard-coupled into my code, as the call to connect is a static call into Symfony's codebase. i.e. I cannot pass in a connector instance of some description that would then attempt the connection. Do I simply wrap Ldap::create with my own connection wrapper, perhaps?
Since you are using Symfony, I guess your best bet would be to inject LDap object using the framework's dependency injection. However I am not an expert in Symfony. So as a simple hack, I would do it like this :
private function connect($req)
{
$dn = $this->config->get('BindDN')[$req->domain];
$pass = $this->config->get('BindPass')[$req->domain];
$servers = $this->config->get('Servers')[$req->domain];
$encryption = $this->config->get('EncryptionType')[$req->domain];
if (false === $dn) {
$msgkey = 'ldapauth-attempt-bind-search';
$bind_with = [null, null];
} else {
$msgkey = 'ldapauth-attempt-bind-dn-search';
$bind_with = [$dn, $pass];
}
$message = new Message($msgkey, [
'dn' => "{$dn}#{$req->domain}",
]);
$this->logger->info($message->text());
foreach ($servers as $server) {
if (false === $server) {
continue;
}
$ldap = $this->createLDAPObject($server, $encryption);
// Attempt bind - on failure, throw an exception
try {
call_user_func_array([$ldap, 'bind'], $bind_with);
$this->server = $server;
$this->encryption = $encryption;
// log successful bind
$msgkey = 'ldapauth-bind-success';
$message = wfMessage($msgkey)->text();
$this->logger->info($message);
return $ldap;
} catch (SymException $e) {
if (false === $dn) {
$msgkey = 'ldapauth-no-bind-search';
} else {
$msgkey = 'ldapauth-no-bind-dn-search';
}
$message = new Message($msgkey, [
'dn' => "{$dn}#{$req->domain}",
]);
$message = $message->text();
$this->logger->info($message);
$this->logger->debug($e->getMessage());
}
}
}
/**
* #param $server
* #param $encryption
* #return mixed
*/
public function createLDAPObject($server, $encryption)
{
return Ldap::create('ext_ldap', [
'host' => $server,
'encryption' => $encryption
]);
}
Then, you can mock the member method createLDAPObject instead of mocking the static method Ldap::create, which should be easier.
However, I would recommend that you refactor your code, so that it is more readable and testable.
1- First of all, call_user_func_array() is not really test-friendly, and I think your requirements here are not too dynamic so you can replace that line with $ldap->bind($bind_with[0],$bind_with[1]);
2- Your connect method is too large to be tested. Please read about Code Smells - Long Methods
3- The method can be refactored into smaller version by decoupling the presentation from the logic. Like for example you are getting the Message object to get the text from the $msgkey just to log it, which is not helping in code readability and test-ability.
These are my first thoughts on the thing :)
Happy coding & testing :)

pass value for last default parameter of function

From very long time i am working on php.
But one question may I have no idea about
like I have one function as bellow:
function hello($param1, $param2="2", $param3="3", $param4="4")
Now whenever I will use this function and if I need 4th params thats the $param4 then still I need to call all as blank like this one:
hello(1, '', '', "param4");
So is there any another way to just pass 1st and 4th param in call rather then long list of blanks ?
Or is there any other standard way for this ?
There was an RFC for this named skipparams but it was declined.
PHP has no syntactic sugar such as hello(1, , , "param4"); nor hello(1, default, default, "param4"); (per the RFC) for skipping optional parameters when calling a function.
If this is your own function then you can choose the common jQuery style of passing options into plug-ins like this:
function hello( $param1, $more_params = [] )
{
static $default_params = [
'param2' => '2',
'param3' => '3',
'param4' => '4'
];
$more_params = array_merge( $default_params, $more_params );
}
Now you can:
hello( 1, [ 'param4'=>'not 4, muahaha!' ] );
If your function requires some advanced stuff such as type hinting then instead of array_merge() you will need to manually loop $more_params and enforce the types.
One potential way you can do this, while a little bit hacky, may work well in some situations.
Instead of passing multiple variables, pass a single array variable, and inside the function check if the specific keys exist.
function hello($param1, $variables = ["param2" => "2", "param3" => "3", "param4" => "4"]) {
if(!array_key_exists("param2", $variables)) $variables['param2'] = "2";
if(!array_key_exists("param3", $variables)) $variables['param3'] = "3";
if(!array_key_exists("param4", $variables)) $variables['param4'] = "4";
echo "<pre>".print_r($variables, true)."</pre>";
}
This will allow you to set "param4" in the above variable, while still remaining default on all of the others.
Calling the function this way:
hello("test", ["param4" => "filling in variable 4"]);
Will result in the output being:
Array
(
[param4] => filling in variable 4
[param2] => 2
[param3] => 3
)
I don't generally recommend this if it can be avoided, but if you absolutely need this functionality, this may work for you.
The key here is that you have a specifically named index inside the array being passed, that you can check against inside the function itself.
The answer, as I see it, is yes and no.
No, because there's no way to do this in a standard fashion.
Yes, because you can hack around it. This is hacky, but it works ;)
Example:
function some_call($parm1, $parm2='', $parm3='', $parm4='') { ... }
and the sauce:
function some_call_4($parm1, $parm4) {
return some_call($parm1, '', '', $parm4);
}
So if you make that call ALOT and are tired of typing it out, you can just hack around it.
Sorry, that's all I've got for you.
It is an overhead, but you can use ReflectionFunction to create a class, instance of which that can be invoked with named parameters:
final class FunctionWithNamedParams
{
private $func;
public function __construct($func)
{
$this->func = $func;
}
public function __invoke($params = [])
{
return ($this->func)(...$this->resolveParams($params));
}
private function resolveParams($params)
{
$rf = new ReflectionFunction($this->func);
return array_reduce(
$rf->getParameters(),
function ($carry, $param) use ($params) {
if (isset($params[$param->getName()])) {
$carry[] = $params[$param->getName()];
} else if ($param->isDefaultValueAvailable()) {
$carry[] = $param->getDefaultValue();
} else {
throw new BadFunctionCallException;
}
return $carry;
},
[]
);
}
}
Then you can use it like this:
function hello($param1, $param2 = "2", $param3 = "3", $param4 = "4")
{
var_dump($param1, $param2, $param3, $param4);
}
$func = new FunctionWithNamedParams('hello');
$func(['param1' => '1', 'param4' => 'foo']);
Here is the demo.

View JSON differences when HTTP testing in Laravel

I am running some HTTP tests in Laravel 5.4, mainly using the assertJson helper method with phpunit. When I run other tests on my models using assertEquals I usually get very good feedback about specifically which properties, fields, etc. are different than expected. However, the assertJson method only tells me that there are differences, but not what those differences are. For instance, let's say I have a route my/route that returns this JSON:
{
"name": "test",
"foo": "bar"
}
I might run this Laravel test:
$response = $this->get("my/route");
$response->assertJson([
'name' => 'test',
'foo' => 'baz',
]);
My test fails as expected. However, the resulting message is pretty unhelpful:
Failed asserting that an array has the subset Array &0 (
'name' => 'test'
'foo' => 'baz'
).
For a non-trivial example with larger response, it can get pretty annoying to try to figure out what is different between the JSON responses. Is there any way to view the specific differences between the expected and actual outputs, instead of just knowing that something is different between the two?
You could wrap your assertion in a try-catch and then if the assertion fails you'll be able to create a new message and throw a new exception.
/**
* #test
* #group new
*/
public function testExample()
{
$response = $this->get('test');
try {
$expected = [
'name' => 'test',
'foo' => 'barz',
];
$response->assertJson($expected);
} catch (\Exception $e) {
$exporter = new \SebastianBergmann\Exporter\Exporter();
$message = 'The following items we\'re expected to be different ' .
$exporter->export($this->arrayRecursiveDiff($expected, $response->decodeResponseJson()));
throw new \PHPUnit_Framework_ExpectationFailedException($message);
}
}
public function arrayRecursiveDiff($array1, $array2)
{
$return = [];
foreach ($array1 as $key => $value) {
if (is_array($array2) && array_key_exists($key, $array2)) {
if (is_array($value)) {
$recursiveDiff = $this->arrayRecursiveDiff($value, $array2[$key]);
if (count($recursiveDiff)) {
$return[$key] = $recursiveDiff;
}
} else {
if ($value != $array2[$key]) {
$return[$key] = $value;
}
}
} else {
$return[$key] = $value;
}
}
return $return;
}
Please note the arrayRecursiveDiff method isn't fully tested, however, there are quite a few different examples floating around for how to compare multidimensional arrays.
Hope this helps!

Auto-increment in MongoDB with CodeIgniter Alex Bilbie library

First of all, sorry for my english and I'm beginning to study MongoDB. :)
I'm trying to insert a record using the CI library to MongoDB (https://github.com/alexbilbie/codeigniter-mongodb-library/tree/v2).
The insert works perfectly, but I can't insert using the recommended function to auto-increment getNextSequence (http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/).
I tried the following ways without success.
Controller:
$data = array('_id' => 'getNextSequence("myid")',
'name' => 'Test '.time(),
'email' => 'test#test.com');
$this->default_model->add($this->collection, $data);
Model:
function add($collection, $data){
return $this->mongo_db->insert($collection, $data);
}
This returns the string 'getNextSequence("relatoriosid")' as the "_id".
I also tried used the command function, like this:
Controller:
$query = 'db.collection.insert({_id: getNextSequence("myid"), name: "Test '.time().'"});';
$ret = $this->default_model->execute($query);
var_dump($ret);
Model:
function execute($query){
return $this->mongo_db->command($query);
}
This way, returns the error:
["errmsg"]=> string(136) "exception: JavaScript execution failed: ReferenceError: getNextSequence is not defined near 'quence("myid"), name: "Teste 1374' "
["code"]=> int(16722)
["ok"]=> float(0)
Has anyone implemented something similar?
Thanks in advance!
I have implemented once mongoDB in one of my project.
I had faced similar problem. But one important thing i remember that we can not modify _id. Its MongoDB inbuilt properly. Neither its good to have the auto-increament in MongoDB because you need to implement that manually.
I will recommend that have another column in structure, you can call it anything, which will be virtual unique/primary key for you, lets say that column name is data_id.
$data = array('data_id' => md5(timestamp() . "<your string>"),
'name' => 'Test '.time(),
'email' => 'test#test.com');
if we check the documentation, is specified that Generally in MongoDB, you would not use an auto-increment pattern for the _id field, or any field, because it does not scale for databases with large numbers of documents. Typically the default value ObjectId is more ideal for the _id. Here i am referring to this link.
Even on the same link, its specified that getNextSequence() we need to write it. Its not inbuilt function.
Hope this helps you.
I got that problem too here is my solution.
Add this function to appflow\application\libraries
public function getNextSequence($name = ""){
if (empty($name))
{
show_error("In order to retrieve documents from MongoDB, a collection name must be passed", 500);
}
try{
$collection_1 = 'counters';
$collection = $this->db->{$collection_1};
$result = $collection->findAndModify(
['_id' => $name],
['$inc' => ['seq' => 1]],
['seq' => true],
['new' => true, 'upsert' => true]
);
if (isset($result['seq']))
{
return $result['seq'];
}
else
{
return false;
}
}
catch (MongoCursorException $e)
{
if(isset($this->debug) == TRUE && $this->debug == TRUE)
{
show_error("MongoDB query failed: {$e->getMessage()}", 500);
}
else
{
show_error("MongoDB query failed.", 500);
}
}
}
This is the way to use.
$nextID = $this->mongo_db->getNextSequence("maindata_child");
$this->mongo_db->where(array('_id'=>$mongo_id))->push('maindata_child', array('id'=>$nextID,'data'=>$maindata_child))->update('main_data');
and don't forget to add collection "counters".
db.counters.insert(
{
_id: "userid",
seq: 0
}
)
PS. I'm use this library https://github.com/bcit-ci/CodeIgniter/wiki/Using-MongoDB-in-Codeigniter
Reference for solution : http://tbsmc.com/a/mwbwswmm-mongodb-how-to-insert-record-using-autoincrement-functionality.html

How to get values that doesn't pass from FilterIterator

I'm using FilterIterator to filter out the values and implemented the accept() method successfully. However I was wondering how would it be possible to get the values that returned false from my accept method in single iteration. Let's take the code below as an example (taken from php.net);
class UserFilter extends FilterIterator
{
private $userFilter;
public function __construct(Iterator $iterator , $filter )
{
parent::__construct($iterator);
$this->userFilter = $filter;
}
public function accept()
{
$user = $this->getInnerIterator()->current();
if( strcasecmp($user['name'],$this->userFilter) == 0) {
return false;
}
return true;
}
}
On the code above, it directly filters out the values and returns the values that pass from the filteriterator. Implemented as;
$array = array(
array('name' => 'Jonathan','id' => '5'),
array('name' => 'Abdul' ,'id' => '22')
);
$object = new ArrayObject($array);
$iterator = new UserFilter($object->getIterator(),'abdul');
It will contain only the array with name Jonathan. However I was wondering would it be possible to store the object with name Abdul in another variable using the same filter with a slight addition instead of reimplementing the entire filter to do the opposite?. One way I was thinking would exactly copy paste the FilterIterator and basically change values of true and false. However are there any neat ways of doing it, since it will require another traversal on the list.
I think you must rewrite the accept() mechanic. Instead of returning true or false, you may want to break down the array to
$result = array(
'passed' => array(...),
'not_passed' => array(...)
);
Your code may look like this
if (strcasecmp($user['name'], $this->userFilter) == 0) {
$result['not_passed'][] = $user;
} else {
$result['passed'][] = $user;
}
return $result;

Categories