Redis Clustering
Overview
Redis’s exceptional performance, simplicity, and atomic
manipulation of data structures lend itself to solving problems that are
difficult or perform poorly when implemented with traditional relational
databases.
I
would like to discuss Redis in
1. Setup Redis clusters
This is guidelines
for installing Redis Cluster on ubuntu 18.04
Step 1: open the terminal
Step 2: Then Enter
· $
sudo apt update
· $
sudo apt install redis-server
Step 3: open the file and
search for "supervised". It is set to No by default. Change it to
"systemd"
Step 4: now you want to
restart Redis server
· $
sudo systemctl restart redis.service
(if it not works, try this)
· $
sudo systemctl enable redis-server
Step 5: check whether the
Redis service is running.
· $ sudo systemctl status redis
Now to deploy a
Redis cluster, there are 2 ways to do this.
1.
By creating empty Redis instances running in cluster
mode
2.
By using the create-cluster script
2. Redis manages it storage in distributed concept
Redis cluster topology
The
minimal cluster that works as expected requires;
Redis Cluster TCP ports
Every
Redis Cluster node requires two TCP connections open. The normal Redis TCP port
used to serve clients, for instance let’s take 7000, plus the port obtained by
adding 10000 to the data port, so 17000.
This
second high port (In here, 17000) is used for the Cluster bus, that is a
node-to-node communication channel using a binary protocol. The Cluster bus is
used by nodes for failure detection, configuration update, failover
authorization and so forth. If you don’t open both TCP ports, your cluster will
not work as expected. So make sure that you open both ports in your firewall.
Note
that for a Redis Cluster to work properly you need, for each node:
· The normal client communication port (in here 7000) used to communicate
with clients to be opened to all the clients that need to reach the cluster,
plus all the other cluster nodes (that use the client port for keys migrations)
·
The cluster bus port (the client port + 10000) must be reachable from all the other
cluster nodes.
Redis Cluster data sharding
Redis
Cluster does not use consistent hashing, but a different form of sharding where
every key is conceptually part of what we call an hash slot.
There
are 16384 hash slots in Redis Cluster (Redis clustering follows a distributed
slot map approach which distribute 16384 slots among Cluster nodes), and to
compute what is the hash slot of a given key, we simply take the CRC16 of the
key modulo 16384.
Every
node in a Redis Cluster is responsible for a subset of the hash slots, so for
example you may have a cluster with 3 nodes, where:
·
Node A contains hash slots from 0 to 5500.
·
Node B contains hash slots from 5501 to 11000.
·
Node C contains hash slots from 11001 to 16383.
This
allows to add and remove nodes (scale) in the cluster easily and does not
require any downtime.
3. How to handle failover
Redis follows a Master-Slave replication process.
Server………… Master……………… Slave
...A…………..…6379………………....6381
...B……………. 6380…………………6379
...C……………. 6381…………………6380
...A…………..…6379………………....6381
...B……………. 6380…………………6379
...C……………. 6381…………………6380
Let’s take above details as a example. The cluster of A, B, and C.
The client is directly talking with the cluster, and one of the masters will be
accepting the request. In here if the B fails, the whole cluster will fail
since we don’t have the Hash Slots from 6380 to 6379.
But, what if we have a replica
for each of A, B, and C nodes? Let’s say these replicas are A1, B1, and C1.
They are called “Slaves” in the Redis world, and A, B,
and C is known as “Masters”. Each Slave is an exact copy of
its master (theoretically) and one Master can have multiple Slaves as well.
So does how this ensures the
high availability or what happens in a failover scenario?
Let's say B had some internal
issues and stopped working. Others in the cluster will get to know it by
the “Gossip Port” since they run node-to-node
communication (including health checks) through this port. Then A and C does
voting and, will promote one of the B’s replicas to be a Master. In this
example, since we have only B1, it will be promoted to be the new master. If the
B rejoins the cluster, it has to be a slave, not a master.
In a nutshell, even if the B
fails as a master, since its slave takes his position and does his work, the
cluster does not fail. Hence, it ensures maximum availability as well.
The replication mechanism
provides basic two uses in the Redis clusters.
1. Multiple Reads or Sorts at a time — performance
improvement
2. Recovery in the case of failure — high availability
and fault-tolerance
This is only a small scenrio.
This concept can apply for large scale situations as well.
4. How Utilized Performance
In-memory data store
All Redis data resides in the server’s
main memory, in contrast to databases such as PostgreSQL, Cassandra, MongoDB
and others that store most data on disk or on SSDs. In comparison to
traditional disk-based databases where most operations require a roundtrip to
disk, in-memory data stores such as Redis don’t suffer the same penalty. They
can therefore support an order of magnitude more operations and faster response
times. The result is – blazing fast performance with average read or write
operations taking less than a millisecond and support for millions of
operations per second.
Flexible data structures
Unlike simplistic key-value data stores
that offer limited data structures, Redis has a vast variety of data structures
to meet your application needs. Redis data types include:
Strings – text or binary data up to 512MB
in size
Lists – a collection of Strings in the
order they were added
Sets – an unordered collection of strings
with the ability to intersect, union, and diff other Set types
Sorted Sets – Sets ordered by a value
Hashes – a data structure for storing a
list of fields and values
Bitmaps – a data type that offers bit
level operations
HyperLogLogs – a probabilistic data
structure to estimate the unique items in a data set
Simplicity and ease-of-use
Redis simplifies your code by enabling you
to write fewer lines of code to store, access, and use data in your
applications. For example, if your application has data stored in a hashmap,
and you want to store that data in a data store – you can simply use the Redis
hash data structure to store the data. A similar task on a data store with no
hash data structures would require many lines of code to convert from one
format to another. Redis comes with native data structures and many options to
manipulate and interact with your data. Over a hundred open source clients are
available for Redis developers. Supported languages include Java, Python, PHP,
C, C++, C#, JavaScript, Node.js, Ruby, R, Go and many others.
Replication and persistence
Redis employs a primary-replica
architecture and supports asynchronous replication where data can be replicated
to multiple replica servers. This provides improved read performance (as
requests can be split among the servers) and faster recovery when the primary
server experiences an outage. For persistence, Redis supports point-in-time
backups (copying the Redis data set to disk).
High availability and scalability
Redis offers a primary-replica
architecture in a single node primary or a clustered topology. This allows you
to build highly available solutions providing consistent performance and
reliability. When you need to adjust your cluster size, various options to
scale up and scale in or out are also available. This allows your cluster to
grow with your demands.
Extensibility
Redis is an open source project supported
by a vibrant community. There’s no vendor or technology lock in as Redis is
open standards based, supports open data formats, and features a rich set of
clients.
Comments
Post a Comment