Redis Deep Dive
Engineering Deep Dive · In-Memory Data Stores
Redis:
Beyond the Cache
A practical reference for engineers who need more than a hello-world tutorial — covering architecture, all five data types, pub/sub, transactions, and real-world patterns.
1. What is Redis — and Why Should You Care?
Redis (Remote Dictionary Server) is a powerful, extremely fast in-memory data store that persists data as key-value pairs. The killer feature is that the value isn't limited to a simple string — it can be one of five rich data structures (Strings, Lists, Sets, Sorted Sets, Hashes), making Redis a genuine multi-paradigm engine rather than just a dumb cache.
⚡ Performance insight
Storing data in memory eliminates disk and network I/O overhead during reads and writes. Benchmark it against a typical MySQL read — the latency difference is often 10–100×. The trade-off: memory is your ceiling for dataset size.
Replication — Master/Slave Architecture
Redis supports master–replica (formerly master–slave) replication. The master propagates all writes to one or more replicas synchronously or asynchronously. If the master dies, a replica can be promoted to take over — a critical pattern for production HA setups. Combine with Redis Sentinel or Redis Cluster for automatic failover.
Key Characteristics
Open-source NoSQL store. No built-in indexing out of the box (though you can build secondary indexes using Sorted Sets). No SQL query language, but Lua scripting gives you server-side logic equivalent to PL/SQL stored procedures without round-trips. Data can be optionally persisted to disk via RDB snapshots or AOF (Append-Only File) logging.
2. SQL vs NoSQL — Choosing Deliberately
Redis sits firmly in the NoSQL world. Here's a crisp decision framework:
| Dimension | SQL (RDBMS) | NoSQL (incl. Redis) |
|---|---|---|
| Schema | Pre-defined, rigid tables. Each table is an entity. RDBMS examples: MySQL, PostgreSQL, Oracle, MS SQL Server, SQLite. | Dynamic / schema-less. Stores data in many formats: key-value, document, wide-column, graph. |
| Scaling | Vertically scalable (bigger CPU/RAM) | Horizontally scalable (more nodes) |
| ACID | Full ACID compliance. Atomicity — transaction happens or doesn't. Consistency — always valid state. Isolation — transactions don't see each other mid-flight. Durability — DB functional even after catastrophes. |
No ACID compliance by default — traded for performance and scale. |
| Best for | Structured data, ACID requirements, no speed bottleneck. | Large unstructured data volumes, cloud/distributed storage, speed, rapid prototyping. |
⚠️ Senior take
Redis isn't a MySQL replacement. It's a complement. A common production pattern: MySQL as the source of truth + Redis as a read-through cache / session store / leaderboard engine. The two work together, not against each other.
3. Where Redis Actually Shines
Redis's in-memory nature makes it the right tool for latency-sensitive, high-throughput operations. Common production patterns:
For geospatial applications: Redis's GEOADD / GEODIST / GEORADIUS commands let you store lat/long coordinates and find objects within a radius — a feature set that would require PostGIS or external services otherwise. It's a genuinely underrated use case.
4. The Five Data Types, In Depth
Strings
Simplest type. Holds text, integers, or binary data up to 512 MB. Supports atomic increment/decrement.
INCR age
APPEND name " Doe"
Hashes
Map of field→value pairs. Ideal for storing objects (user profiles, config objects). Avoids key explosion.
HMSET scores u1 0 u2 3
HGETALL user:1
Lists
Ordered, allows duplicates. Doubly linked list. O(1) for head/tail operations. Use for queues, timelines.
LRANGE colors 0 -1
LINSERT colors BEFORE blue purple
Sets
Unordered, unique members. O(1) add/remove/check. Powerful set operations: UNION, DIFF, INTER.
SUNION art food
SDIFF art food
Sorted Sets
Unique members, each with a floating-point score. Auto-sorted by score. Perfect for leaderboards & priority queues.
ZRANGE players 0 -1
ZRANGEBYSCORE players 0 50
Strings — Full Command Reference
# Basic get/set
SET name Jesse
SET hobby "Computer reading"
GET name
APPEND name " is my name!"
# Numeric operations
INCRBY age 5 # increment by 5
INCR age # increment by 1
DECR age
DECRBY age 10
# Bulk operations
MSET mammal whale insect "June bug"
MGET mammal insect
# Key management
SETNX name Jesse # set only if key doesn't exist
MSETNX mammal pig fish salmon # atomic multi-set if none exist
RENAME name newname
EXISTS name
DEL name
TYPE age # returns the type of value
TTL animal # time to live (-1 = no expiry)
# Expiry
EXPIRE animal 5000 # set TTL in milliseconds
PERSIST animal # remove expiry
# Utility
KEYS * # list all keys (avoid in prod)
KEYS na* # wildcard search
FLUSHDB # remove all keys from current db
FLUSHALL # remove all keys from ALL dbs
Hashes
HSET people john 12
HMSET people kim 9 ted 12 ben 13
HGETALL people
HGET people john
HMGET people sam kim
HDEL people ben
HLEN people
HKEYS people
HVALS people
HINCRBY scores u2 5
HEXISTS people u4
Lists
Unlike Sets, Lists are ordered and allow duplicates. They're implemented as doubly-linked lists — constant time for push/pop at either end, but O(n) for index access.
LPUSH colors red green blue # push left (head)
RPUSH colors yellow # push right (tail)
LLEN colors
LRANGE colors 0 -1 # -1 = get all
LINSERT colors BEFORE blue purple
LREM colors 1 blue # remove 1 occurrence
LPOP colors
RPOP colors
LPUSHX colors red # push only if key exists
Sets
Unordered, unique members. The set operations (UNION, DIFF, INTER) are among the most useful patterns in Redis — e.g., computing mutual friends, tagging systems, permission sets.
SADD art pencil brush canvas
SMEMBERS art
SREM art pencil
SCARD art
SISMEMBER art pencil
SUNION art food
SDIFF art food
SINTER art food
# Store set operation results into a new key
SUNIONSTORE result art food
SDIFFSTORE result art food
SINTERSTORE result art food
Sorted Sets
Every member gets a floating-point score. Members are unique; scores can repeat. Perfect for leaderboards, rate limiting, and time-series data (use Unix timestamp as score).
ZADD players 1 john
ZADD players 10 ben 20 kim 30 sam 50 tammy
ZRANGE players 0 -1 # asc by score
ZRANGE players 0 -1 'withscores'
ZREVRANGE players 0 -1 # desc
ZRANK players kim # rank (0-indexed)
ZCARD players # count
ZCOUNT players 0 -1 # count by score
ZRANGEBYSCORE players 0 50
ZRANGEBYSCORE players 0 80 'withscores'
5. Pub/Sub — Database-Agnostic Messaging
Redis Pub/Sub is a fire-and-forget messaging system. It's database-agnostic — a subscriber on redis db 0 can receive messages published from a client connected to a different db. This makes it suitable as a lightweight event bus.
📌 Key constraint
Pub/Sub does not persist messages. If a subscriber is offline when a message is published, it misses that message. For durable queues, use Redis Streams or an external broker like Kafka / RabbitMQ.
# Subscriber
SUBSCRIBE general
PSUBSCRIBE p* # pattern — anything starting with "p"
PSUBSCRIBE h?llo # matches "hello", "hallo", etc.
UNSUBSCRIBE general
UNSUBSCRIBE # unsubscribe from all
PUNSUBSCRIBE # unsubscribe all pattern subs
# Publisher
PUBLISH general "How are you?"
# Introspection
PUBSUB channels *
6. Transactions with MULTI/EXEC
Redis transactions let you queue multiple commands and execute them atomically. Useful when inserting values that depend on each other — e.g., updating a user's score and simultaneously recording the event.
# Queue commands atomically
MULTI
SADD people joe fred suzanne
SET life fun
SET day bright
EXEC
# Discard a queued transaction
MULTI
SADD animals horse dog
SET age 5
DISCARD
⚠️ Gotcha
Redis transactions are not ACID-equivalent. Commands inside MULTI are queued; if one fails at runtime, the rest still execute. There's no rollback. For compare-and-swap patterns, use WATCH to implement optimistic locking.
7. HyperLogLog — Probabilistic Counting at Scale
Available since Redis 2.8.9. HyperLogLog is designed for one thing: counting unique elements in a massive dataset with minimal memory. Google and Facebook use this pattern. It gives ~98% accuracy (±2% error) and uses at most 12 KB of memory, regardless of how many unique items you've tracked.
The classic use case: counting unique daily active users or signups. Storing 20k raw user IDs per hour × 24 hours would be ridiculous; HyperLogLog keeps that footprint trivially small.
# Timestamp = Redis key; values = unique user IDs
PFADD signups:102514 jesse@mail.com
PFADD signups:102514 tom@mail.com
PFADD signups:102515 jane@mail.com
PFADD signups:102516 jenny@mail.com
PFCOUNT signups:102514
PFCOUNT signups:102515
# Merge multiple HLLs into one key
PFMERGE total:weekly signups:102514 signups:102515 signups:102516
PFCOUNT total:weekly
✅ When to use
Use HyperLogLog when you need approximate cardinality at scale and exact counts aren't critical — analytics dashboards, event counters, unique reach metrics. For exact counts on small-medium datasets, a Set with SCARD is simpler.
8. Client Quickstart (PHP & Python)
PHP — Predis
Install via Composer: "predis/predis": "^1.0.0". Then composer update.
<?php
require '../vendor/autoload.php';
$redis = new Predis\Client([
'scheme' => 'tcp',
'host' => 'localhost',
'port' => 6379,
]);
$exists = $redis->exists('age');
if (!$exists) {
$redis->set('age', 1);
} else {
$redis->incr('age');
}
Python — redis-py
# pip install redis
import redis
r = redis.Redis(host='localhost', port=6379)
r.set('numerical', 50)
r.hset('animals', 'mammals', 'dog')
r.hset('animals', 'mammals', 'pig')
r.hmset('humans', {
'ted': 20,
'jane': 40
})
r.hgetall('animals')
r.hgetall('humans')
9. Production Checklist
Security — Enable AUTH
By default Redis has no password. In production, edit /etc/redis/redis.conf and uncomment requirepass. Restart the server, then authenticate:
# In redis-cli
AUTH your_secret_password
SET name 5 # → returns OK after auth
Database Selection
Redis ships with 16 logical databases (0–15) by default. Use SELECT to switch:
SELECT 0 # default
SELECT 3 # switch to db 3
Monitoring
redis-cli monitor # live command stream — use sparingly in prod
redis-server --version
sudo service redis-server start
sudo service redis-server stop
🏗️ Architecture note
For production HA: run Redis Sentinel (monitors masters/replicas, handles failover) or Redis Cluster (shards data across nodes for horizontal scale). Sentinel is simpler to operate; Cluster is needed when a single node's memory is the bottleneck.
Connection Commands Reference
AUTH password
ECHO message
PING # → PONG
SELECT index
QUIT