How to have both clustered and non-clustered redis connections in laravel - php

Question

Background
In the past I was able to use a non-clustered redis just fine in my config like so:
'redis' => [
'default' => [
'host' => env('REDIS_HOST', '127.0.0.1'),
'password' => env('REDIS_PASSWORD', null),
'port' => 6379,
'database' => 0,
'cluster' => true,
]
],
However due to the load on our redis servers, I have to cluster my redis, This config works fine when the only redis connection I have is clustered (figured it out after a lot of work):
'redis' => [
'client' => 'predis',
'cluster' => true,
'options' => [
'cluster' => 'redis',
'parameters' => [
'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_DEFAULT_PORT', 6379),
'database' => 0,
],
],
'clusters' => [
'default' => [
'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_1_PORT', 6379),
'database' => 0,
],
'shard2' => [
'host' => env('REDIS_SHARD_2_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_2_PORT', 6379),
'database' => 0,
],
'shard3' => [
'host' => env('REDIS_SHARD_3_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_3_PORT', 6379),
'database' => 0,
],
'options' => [
'cluster' => 'redis'
],
]
]
any my env file looks like this (For my localhost, anyway):
QUEUE_DRIVER=redis // cluster compatible
BROADCAST_DRIVER=redis // cluster compatible
CACHE_CONNECTION=redis // cluster incompatible
REDIS_CLUSTER=true
REDIS_HOST=localhost
REDIS_DEFAULT_PORT=7000
REDIS_SHARD_1_HOST=localhost
REDIS_SHARD_2_HOST=localhost
REDIS_SHARD_3_HOST=localhost
REDIS_SHARD_1_PORT=7000
REDIS_SHARD_2_PORT=7001
REDIS_SHARD_3_PORT=7002
Problem
The fact is that currently, we use the non-clustered redis for the following:
Cache: supports redis clustering
Queue/Jobs: supports redis clustering
Broadcast (ie websockets): does not support redis clustering
That's why we need to have both redis connections simultaneously, so that we can use the clustered connection for caching/queues, and non-clustered connection for websockets.
But this isn't working:
'redis' => [
'default' => [
'host' => env('REDIS_HOST', '127.0.0.1'),
'password' => env('REDIS_PASSWORD', null),
'port' => 6379,
'database' => 0,
'cluster' => true,
],
'clustered' => [
'client' => 'predis',
'cluster' => true,
'options' => [
'cluster' => 'redis',
'parameters' => [
'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_DEFAULT_PORT', 6379),
'database' => 0,
],
],
'clusters' => [
'default' => [
'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_1_PORT', 6379),
'database' => 0,
],
'shard2' => [
'host' => env('REDIS_SHARD_2_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_2_PORT', 6379),
'database' => 0,
],
'shard3' => [
'host' => env('REDIS_SHARD_3_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_3_PORT', 6379),
'database' => 0,
],
'options' => [
'cluster' => 'redis'
],
]
further, some users state that such a task is simply impossible for redis. Is that true?
update
I tried this
in Cache.php
'redis' => [
'driver' => 'redis',
'connection' => 'clustered',
],
note: in the above connection i simply couldn't just copy/paste the cluster options, since it would crash if i didn't put a driver option
In database.php I was inspired by this answer and simply put different connections under the redis key: (ie `database.redis.connection-1, database.redis.connection-2 etc)
'redis' => [
'clustered' => [
// clustered settings copied from above
],
],
'default' => [
// non clustered settings
],
]
To test, I ran the following tinker
>>> use Illuminate\Support\Facades\Cache;
>>> Cache::put('foo','bar',1);
Predis/Response/ServerException with message 'MOVED 7837 127.0.0.1:7001'
The move error is a known one, it's simply saying that i'm dealing with a non-clustered redis connection.
thoughts?

Answer 1

I was able to fix this problem with this PR.
This is what my config looks like now:
'redis' => [
'clustered' => [
'client' => 'predis',
'cluster' => true,
'options' => [ 'cluster' => 'redis' ],
'clusters' => [
[
'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_1_PORT', 6379),
'database' => 0,
],
[
'host' => env('REDIS_SHARD_2_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_2_PORT', 6379),
'database' => 0,
],
[
'host' => env('REDIS_SHARD_3_HOST', '127.0.01'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_SHARD_3_PORT', 6379),
'database' => 0,
],
],
],
'default' => [
'host' => env('REDIS_HOST', '127.0.0.1'),
'password' => env('REDIS_PASSWORD', null),
'port' => 6379,
'database' => 0,
'cluster' => false,
],
]
Will be sending PR to Laravel itself shortly.