Redis Deep Dive

Redis Deep Dive — A Senior Engineer's Reference

Engineering Deep Dive · In-Memory Data Stores

Redis:
Beyond the Cache

A practical reference for engineers who need more than a hello-world tutorial — covering architecture, all five data types, pub/sub, transactions, and real-world patterns.

Data Structures Performance Patterns Production Tips

1. What is Redis — and Why Should You Care?

Redis (Remote Dictionary Server) is a powerful, extremely fast in-memory data store that persists data as key-value pairs. The killer feature is that the value isn't limited to a simple string — it can be one of five rich data structures (Strings, Lists, Sets, Sorted Sets, Hashes), making Redis a genuine multi-paradigm engine rather than just a dumb cache.

⚡ Performance insight

Storing data in memory eliminates disk and network I/O overhead during reads and writes. Benchmark it against a typical MySQL read — the latency difference is often 10–100×. The trade-off: memory is your ceiling for dataset size.

Replication — Master/Slave Architecture

Redis supports master–replica (formerly master–slave) replication. The master propagates all writes to one or more replicas synchronously or asynchronously. If the master dies, a replica can be promoted to take over — a critical pattern for production HA setups. Combine with Redis Sentinel or Redis Cluster for automatic failover.

Key Characteristics

Open-source NoSQL store. No built-in indexing out of the box (though you can build secondary indexes using Sorted Sets). No SQL query language, but Lua scripting gives you server-side logic equivalent to PL/SQL stored procedures without round-trips. Data can be optionally persisted to disk via RDB snapshots or AOF (Append-Only File) logging.

2. SQL vs NoSQL — Choosing Deliberately

Redis sits firmly in the NoSQL world. Here's a crisp decision framework:

Dimension SQL (RDBMS) NoSQL (incl. Redis)
Schema Pre-defined, rigid tables. Each table is an entity. RDBMS examples: MySQL, PostgreSQL, Oracle, MS SQL Server, SQLite. Dynamic / schema-less. Stores data in many formats: key-value, document, wide-column, graph.
Scaling Vertically scalable (bigger CPU/RAM) Horizontally scalable (more nodes)
ACID Full ACID compliance.
Atomicity — transaction happens or doesn't.
Consistency — always valid state.
Isolation — transactions don't see each other mid-flight.
Durability — DB functional even after catastrophes.
No ACID compliance by default — traded for performance and scale.
Best for Structured data, ACID requirements, no speed bottleneck. Large unstructured data volumes, cloud/distributed storage, speed, rapid prototyping.

⚠️ Senior take

Redis isn't a MySQL replacement. It's a complement. A common production pattern: MySQL as the source of truth + Redis as a read-through cache / session store / leaderboard engine. The two work together, not against each other.

3. Where Redis Actually Shines

Redis's in-memory nature makes it the right tool for latency-sensitive, high-throughput operations. Common production patterns:

User session management Caching / read-through cache Pub/Sub queues & notifications Gaming leaderboards (Sorted Sets) Geospatial — nearby objects, distance, radius Rate limiting / token buckets Distributed locks HyperLogLog — unique visitor counting

For geospatial applications: Redis's GEOADD / GEODIST / GEORADIUS commands let you store lat/long coordinates and find objects within a radius — a feature set that would require PostGIS or external services otherwise. It's a genuinely underrated use case.

4. The Five Data Types, In Depth

Strings

Simplest type. Holds text, integers, or binary data up to 512 MB. Supports atomic increment/decrement.

SET name "Jesse"
INCR age
APPEND name " Doe"

Hashes

Map of field→value pairs. Ideal for storing objects (user profiles, config objects). Avoids key explosion.

HSET user:1 name "Lee"
HMSET scores u1 0 u2 3
HGETALL user:1

Lists

Ordered, allows duplicates. Doubly linked list. O(1) for head/tail operations. Use for queues, timelines.

LPUSH colors red green
LRANGE colors 0 -1
LINSERT colors BEFORE blue purple

Sets

Unordered, unique members. O(1) add/remove/check. Powerful set operations: UNION, DIFF, INTER.

SADD art pencil brush
SUNION art food
SDIFF art food

Sorted Sets

Unique members, each with a floating-point score. Auto-sorted by score. Perfect for leaderboards & priority queues.

ZADD players 10 "ben"
ZRANGE players 0 -1
ZRANGEBYSCORE players 0 50

Strings — Full Command Reference

# Basic get/set
SET    name    Jesse
SET    hobby   "Computer reading"
GET    name
APPEND name    " is my name!"

# Numeric operations
INCRBY  age 5     # increment by 5
INCR    age       # increment by 1
DECR    age
DECRBY  age 10

# Bulk operations
MSET   mammal whale insect "June bug"
MGET   mammal insect

# Key management
SETNX  name Jesse   # set only if key doesn't exist
MSETNX mammal pig fish salmon  # atomic multi-set if none exist
RENAME name newname
EXISTS name
DEL    name
TYPE   age           # returns the type of value
TTL    animal        # time to live (-1 = no expiry)

# Expiry
EXPIRE  animal 5000  # set TTL in milliseconds
PERSIST animal       # remove expiry

# Utility
KEYS   *            # list all keys (avoid in prod)
KEYS   na*          # wildcard search
FLUSHDB             # remove all keys from current db
FLUSHALL            # remove all keys from ALL dbs

Hashes

HSET    people   john 12
HMSET   people   kim 9   ted 12   ben 13
HGETALL people
HGET    people   john
HMGET   people   sam kim
HDEL    people   ben
HLEN    people
HKEYS   people
HVALS   people
HINCRBY scores   u2 5
HEXISTS people   u4

Lists

Unlike Sets, Lists are ordered and allow duplicates. They're implemented as doubly-linked lists — constant time for push/pop at either end, but O(n) for index access.

LPUSH   colors   red green blue   # push left (head)
RPUSH   colors   yellow            # push right (tail)
LLEN    colors
LRANGE  colors   0 -1             # -1 = get all
LINSERT colors   BEFORE blue purple
LREM    colors   1 blue            # remove 1 occurrence
LPOP    colors
RPOP    colors
LPUSHX  colors   red              # push only if key exists

Sets

Unordered, unique members. The set operations (UNION, DIFF, INTER) are among the most useful patterns in Redis — e.g., computing mutual friends, tagging systems, permission sets.

SADD      art      pencil brush canvas
SMEMBERS  art
SREM      art      pencil
SCARD     art
SISMEMBER art      pencil

SUNION    art food
SDIFF     art food
SINTER    art food

# Store set operation results into a new key
SUNIONSTORE  result   art food
SDIFFSTORE   result   art food
SINTERSTORE  result   art food

Sorted Sets

Every member gets a floating-point score. Members are unique; scores can repeat. Perfect for leaderboards, rate limiting, and time-series data (use Unix timestamp as score).

ZADD    players   1 john
ZADD    players   10 ben   20 kim   30 sam   50 tammy
ZRANGE  players   0 -1                   # asc by score
ZRANGE  players   0 -1   'withscores'
ZREVRANGE players 0 -1                   # desc
ZRANK   players   kim                    # rank (0-indexed)
ZCARD   players                          # count
ZCOUNT  players   0 -1                   # count by score
ZRANGEBYSCORE players 0 50
ZRANGEBYSCORE players 0 80 'withscores'

5. Pub/Sub — Database-Agnostic Messaging

Redis Pub/Sub is a fire-and-forget messaging system. It's database-agnostic — a subscriber on redis db 0 can receive messages published from a client connected to a different db. This makes it suitable as a lightweight event bus.

📌 Key constraint

Pub/Sub does not persist messages. If a subscriber is offline when a message is published, it misses that message. For durable queues, use Redis Streams or an external broker like Kafka / RabbitMQ.

# Subscriber
SUBSCRIBE    general
PSUBSCRIBE   p*          # pattern — anything starting with "p"
PSUBSCRIBE   h?llo       # matches "hello", "hallo", etc.
UNSUBSCRIBE  general
UNSUBSCRIBE              # unsubscribe from all
PUNSUBSCRIBE             # unsubscribe all pattern subs

# Publisher
PUBLISH  general  "How are you?"

# Introspection
PUBSUB   channels *

6. Transactions with MULTI/EXEC

Redis transactions let you queue multiple commands and execute them atomically. Useful when inserting values that depend on each other — e.g., updating a user's score and simultaneously recording the event.

# Queue commands atomically
MULTI
  SADD   people   joe fred suzanne
  SET    life     fun
  SET    day      bright
EXEC

# Discard a queued transaction
MULTI
  SADD   animals   horse dog
  SET    age       5
DISCARD

⚠️ Gotcha

Redis transactions are not ACID-equivalent. Commands inside MULTI are queued; if one fails at runtime, the rest still execute. There's no rollback. For compare-and-swap patterns, use WATCH to implement optimistic locking.

7. HyperLogLog — Probabilistic Counting at Scale

Available since Redis 2.8.9. HyperLogLog is designed for one thing: counting unique elements in a massive dataset with minimal memory. Google and Facebook use this pattern. It gives ~98% accuracy (±2% error) and uses at most 12 KB of memory, regardless of how many unique items you've tracked.

The classic use case: counting unique daily active users or signups. Storing 20k raw user IDs per hour × 24 hours would be ridiculous; HyperLogLog keeps that footprint trivially small.

# Timestamp = Redis key; values = unique user IDs
PFADD   signups:102514   jesse@mail.com
PFADD   signups:102514   tom@mail.com
PFADD   signups:102515   jane@mail.com
PFADD   signups:102516   jenny@mail.com

PFCOUNT  signups:102514
PFCOUNT  signups:102515

# Merge multiple HLLs into one key
PFMERGE  total:weekly   signups:102514   signups:102515   signups:102516
PFCOUNT  total:weekly

✅ When to use

Use HyperLogLog when you need approximate cardinality at scale and exact counts aren't critical — analytics dashboards, event counters, unique reach metrics. For exact counts on small-medium datasets, a Set with SCARD is simpler.

8. Client Quickstart (PHP & Python)

PHP — Predis

Install via Composer: "predis/predis": "^1.0.0". Then composer update.

<?php
require '../vendor/autoload.php';

$redis = new Predis\Client([
    'scheme' => 'tcp',
    'host'   => 'localhost',
    'port'   => 6379,
]);

$exists = $redis->exists('age');
if (!$exists) {
    $redis->set('age', 1);
} else {
    $redis->incr('age');
}

Python — redis-py

# pip install redis
import redis

r = redis.Redis(host='localhost', port=6379)

r.set('numerical', 50)
r.hset('animals', 'mammals', 'dog')
r.hset('animals', 'mammals', 'pig')

r.hmset('humans', {
    'ted': 20,
    'jane': 40
})

r.hgetall('animals')
r.hgetall('humans')

9. Production Checklist

Security — Enable AUTH

By default Redis has no password. In production, edit /etc/redis/redis.conf and uncomment requirepass. Restart the server, then authenticate:

# In redis-cli
AUTH   your_secret_password
SET    name 5    # → returns OK after auth

Database Selection

Redis ships with 16 logical databases (0–15) by default. Use SELECT to switch:

SELECT  0   # default
SELECT  3   # switch to db 3

Monitoring

redis-cli monitor   # live command stream — use sparingly in prod
redis-server --version
sudo service redis-server start
sudo service redis-server stop

🏗️ Architecture note

For production HA: run Redis Sentinel (monitors masters/replicas, handles failover) or Redis Cluster (shards data across nodes for horizontal scale). Sentinel is simpler to operate; Cluster is needed when a single node's memory is the bottleneck.

Connection Commands Reference

AUTH     password
ECHO     message
PING                # → PONG
SELECT   index
QUIT
Notes compiled from Redis learning sessions · redis.io for authoritative docs