This series will cover Redis caching from fundamental concepts to advanced real-world applications, including performance tuning, clustering, and security.

Mastering Caching with Redis

A Complete Guide to Efficient Caching with Redis

Series Overview

Redis is a powerful in-memory data structure store widely used for caching. This tutorial series will take you from the basics of Redis caching to advanced strategies for optimizing performance and ensuring reliability.

📖 Table of Contents

1. Introduction to Caching and Redis

What is caching? Why is it important?
Redis vs. other caching solutions (Memcached, etc.)
Installing and setting up Redis
Connecting to Redis with different programming languages (Node.js, Python, Golang)

2. Redis Data Structures for Caching

Strings as cache values
Hashes for object storage
Lists & Sets for caching collections
Sorted Sets for ranking systems
Expiration and TTL management

3. Basic Caching Patterns

Cache-aside (Lazy Loading)
Write-through caching
Read-through caching
Write-back caching
When to use each pattern

4. Handling Cache Expiration & Eviction

Redis eviction policies (volatile-lru, allkeys-lfu, etc.)
Setting TTLs and expirations (EX, PX, TTL)
Managing cache invalidation effectively
Using Redis for session management

5. Performance Optimization

Benchmarking Redis performance
Connection pooling
Using pipelining for batch requests
Redis vs. database query performance

6. Scaling Redis for High Availability

Running Redis in production
Master-slave replication
Redis Sentinel for failover
Redis Cluster for sharding and horizontal scaling

7. Securing Redis

Authentication and ACLs
Protecting against cache poisoning
Encrypting Redis traffic
Avoiding common security pitfalls

8. Real-World Use Cases

Redis for API rate limiting
Full-page caching with Redis
Caching database queries (PostgreSQL, MySQL)
Distributed caching for microservices
Event-driven caching with Redis Streams

9. Monitoring and Debugging Redis

Redis CLI commands for debugging (INFO, MONITOR, SLOWLOG)
Setting up monitoring with RedisInsight
Logging and analyzing Redis metrics
Troubleshooting performance bottlenecks

10. Advanced Topics

Using Lua scripting for advanced caching logic
Implementing Redis as a message broker
Redis vs. CDN caching
Combining Redis with other tools like Kafka & RabbitMQ

📌 Target Audience

Backend developers working with caching
DevOps engineers optimizing performance
Architects designing scalable distributed systems
Anyone interested in mastering Redis caching

🚀 Hands-on Code Examples

Each section will include real-world examples in: ✅ Node.js
✅ Golang
✅ Python
✅ TypeScript

🚀 Part 2: Handling Cache Expiration & Eviction Strategies in Redis

In this tutorial, we will cover cache expiration, eviction policies, and how to prevent stale cache issues to optimize Redis performance.

1️⃣ Understanding Cache Expiration (TTL)

Redis allows setting a Time-To-Live (TTL) for cached keys to prevent storing stale or unnecessary data.

📌 Setting TTL in Redis

SET user:1 "John Doe" EX 60  # Expires in 60 seconds
TTL user:1                   # Check time left

📌 Setting TTL in Code

Node.js

await redis.set("session:123", "user-data", "EX", 300); // Expires in 300 sec

Python

client.set("session:123", "user-data", ex=300)  # Expires in 300 sec

Golang

client.Set(ctx, "session:123", "user-data", 300*time.Second)

2️⃣ Redis Eviction Policies

When Redis memory is full, it needs to evict (remove) old data. Redis supports multiple eviction policies:

Policy	Description
noeviction	Returns an error when memory is full (default for persistence use cases)
allkeys-lru	Removes least recently used (LRU) keys (good for caching)
volatile-lru	Removes LRU keys that have an expiration set
allkeys-lfu	Removes least frequently used (LFU) keys
volatile-lfu	Removes LFU keys with expiration
volatile-ttl	Removes keys with the shortest TTL first

📌 Setting an Eviction Policy

Edit redis.conf:

maxmemory 256mb
maxmemory-policy allkeys-lru

Or set dynamically:

CONFIG SET maxmemory 256mb
CONFIG SET maxmemory-policy allkeys-lfu

3️⃣ Implementing Cache Eviction Strategies

A. Least Recently Used (LRU) Cache Implementation

LRU ensures that old, unused cache is removed when new data is added.

Node.js Example

await redis.set("user:1", JSON.stringify({ name: "Alice" }), "EX", 300);
await redis.set("user:2", JSON.stringify({ name: "Bob" }), "EX", 300);
await redis.set("user:3", JSON.stringify({ name: "Charlie" }), "EX", 300);

// Fetching makes user:1 the most recently used
await redis.get("user:1"); 

// If memory is full, LRU will remove the least accessed key first

B. Least Frequently Used (LFU) Cache Implementation

LFU removes the least accessed keys first instead of the oldest ones.

Setting LFU Policy

CONFIG SET maxmemory-policy allkeys-lfu

Golang Example

client.Set(ctx, "user:1", "Alice", 300*time.Second)
client.Set(ctx, "user:2", "Bob", 300*time.Second)

// Simulating access frequency
client.Get(ctx, "user:1") 
client.Get(ctx, "user:1") // Access user:1 more times

// If memory is full, LFU will remove user:2 first

4️⃣ Preventing Cache Stale Issues

A. Cache Invalidation

1. Time-based Expiration

Using EX (seconds) or PX (milliseconds) ensures cache expires automatically.

2. Manual Cache Invalidation

When updating the database, delete or update the cache to prevent stale data.

Example:

await redis.del("user:1"); // Remove cache when user updates info

B. Cache Stampede Prevention

A cache stampede happens when multiple requests hit a missing key at the same time, overloading the database.

Solutions

Locking (Mutex)
- If a cache miss occurs, set a temporary lock to prevent multiple requests.
Randomized TTL
- Prevents all keys from expiring at the same time.

Example:

await redis.set("product:123", JSON.stringify(product), "EX", Math.floor(Math.random() * 300) + 300);

✅ Next Steps

Now that we’ve covered cache expiration and eviction, we can explore:

Distributed caching in microservices
Rate limiting with Redis
Using Redis for full-page caching in web applications

🚀 Part 3: Distributed Caching with Redis

In this tutorial, we’ll explore how to implement distributed caching with Redis to scale applications, reduce latency, and ensure high availability.

1️⃣ Why Distributed Caching?

📌 Benefits of Distributed Caching

Scalability: Allows multiple instances of an application to share cached data.
Reduced Load on Databases: Prevents database overload by caching frequently requested data.
High Availability: Ensures cache remains available even if one Redis node fails.

📌 When to Use Distributed Caching?

Multi-instance microservices architectures.
Handling high-traffic web applications (e.g., e-commerce, social media).
Scaling APIs using rate-limiting.

2️⃣ Approaches to Distributed Caching with Redis

There are three main approaches to setting up distributed caching with Redis:

Approach	Description
Single Redis Instance	All services access the same Redis node (Simple but has a single point of failure).
Redis Sentinel	Ensures high availability with failover mechanisms.
Redis Cluster	Distributes data across multiple nodes for better scalability.

3️⃣ Setting Up a Distributed Redis Cache

A. Using a Single Redis Instance (Basic Setup)

This is the simplest setup where all microservices share one Redis instance.

📌 Docker Setup

docker run --name redis -p 6379:6379 -d redis:latest

📌 Node.js Example (Connecting from Multiple Services)

const Redis = require("ioredis");
const redis = new Redis("redis://localhost:6379");

async function cacheData() {
    await redis.set("product:123", JSON.stringify({ name: "Laptop", price: 1200 }), "EX", 300);
    console.log("Product cached!");
}

cacheData();

✅ Limitation: If Redis crashes, all cache data is lost. Use Sentinel or Cluster for high availability.

B. Using Redis Sentinel for High Availability

Redis Sentinel monitors Redis instances and automatically promotes a replica if the master fails.

📌 Redis Sentinel Architecture

Master node: Handles writes and reads.
Replica nodes: Read-only copies of the master.
Sentinel nodes: Monitors Redis nodes and performs failover.

📌 Setting Up Redis Sentinel (Docker)

docker network create redis-net

# Run Redis master
docker run --name redis-master --network redis-net -p 6379:6379 -d redis --appendonly yes

# Run Redis replica
docker run --name redis-replica --network redis-net -p 6380:6379 -d redis --appendonly yes --replicaof redis-master 6379

# Run Sentinel
docker run --name redis-sentinel --network redis-net -d redis redis-sentinel /etc/redis/sentinel.conf

📌 Connecting with Sentinel in Node.js

const Redis = require("ioredis");
const redis = new Redis({
  sentinels: [{ host: "localhost", port: 26379 }],
  name: "mymaster",
});

(async () => {
    await redis.set("user:1", JSON.stringify({ name: "Alice" }), "EX", 300);
    console.log(await redis.get("user:1"));
})();

✅ Advantage: High availability. If the master fails, Sentinel promotes a replica.

C. Using Redis Cluster for Scalability

Redis Cluster allows data to be sharded across multiple nodes for better load balancing.

📌 Setting Up Redis Cluster with Docker

docker network create redis-cluster

# Run multiple Redis nodes
docker run --name redis-node1 --network redis-cluster -d redis --port 7001
docker run --name redis-node2 --network redis-cluster -d redis --port 7002
docker run --name redis-node3 --network redis-cluster -d redis --port 7003

# Create the cluster
docker exec -it redis-node1 redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 --cluster-replicas 1

📌 Connecting with Redis Cluster (Node.js)

const Redis = require("ioredis");
const cluster = new Redis.Cluster([
  { host: "localhost", port: 7001 },
  { host: "localhost", port: 7002 },
  { host: "localhost", port: 7003 },
]);

(async () => {
    await cluster.set("session:123", "User Data", "EX", 300);
    console.log(await cluster.get("session:123"));
})();

✅ Advantage: Scalability and better load distribution.

4️⃣ Handling Cache Consistency in Distributed Caching

📌 Strategies to Keep Cache Updated

Write-through caching: Every update to the database also updates Redis.
Cache invalidation: Use TTLs and delete outdated cache when data changes.
Event-driven updates: Use message queues (Kafka, RabbitMQ) to sync changes.

Example: Using Event-driven Caching with Redis Pub/Sub

const redis = new Redis();
const subscriber = new Redis();

// Subscribe to an update event
subscriber.subscribe("user_updates", (err, count) => {
    console.log(`Subscribed to ${count} channels.`);
});

// Listen for updates
subscriber.on("message", (channel, message) => {
    console.log(`Received update: ${message}`);
    redis.del(`user:${message}`);  // Invalidate cache
});

✅ Advantage: Ensures all instances have fresh cache.

5️⃣ Monitoring Redis in a Distributed Setup

Use Redis CLI, RedisInsight, and Prometheus/Grafana for monitoring.

📌 Basic Monitoring Commands

INFO          # Get Redis statistics
MONITOR       # Debug live commands
SLOWLOG GET   # Find slow queries

📌 Using RedisInsight for Monitoring

docker run --name redisinsight -p 8001:8001 -d redislabs/redisinsight

Access UI at http://localhost:8001
Monitor keys, memory usage, and commands.

✅ Next Steps

Now that we’ve covered Distributed Caching with Redis, we can explore:

API Rate Limiting with Redis
Caching API responses for performance
Redis as a Session Store

🚀 Part 4: API Rate Limiting with Redis

API rate limiting is essential for preventing abuse, controlling request loads, and ensuring fair resource usage in web applications. In this tutorial, we’ll implement rate limiting with Redis using different strategies.

1️⃣ What is API Rate Limiting?

Rate limiting restricts the number of API requests a user or client can make within a specified time window. This helps:

Prevent DDoS attacks & API abuse.
Ensure fair usage across users.
Reduce backend load.

2️⃣ Rate Limiting Strategies

Strategy	Description
Fixed Window Counter	Counts requests in fixed time windows (e.g., 100 requests per minute).
Sliding Window Log	Tracks precise timestamps to allow smoother request distribution.
Token Bucket	Uses a bucket of tokens that refills over time; requests consume tokens.
Leaky Bucket	Ensures requests are processed at a steady rate to avoid bursts.

3️⃣ Implementing Rate Limiting in Node.js with Redis

We’ll implement Fixed Window Counter and Sliding Window Log strategies.

A. Setting Up Redis & Express

First, install the required dependencies:

npm install express ioredis

Create a server.js file:

const express = require("express");
const Redis = require("ioredis");

const app = express();
const redis = new Redis();

const WINDOW_SIZE = 60; // 1 minute
const MAX_REQUESTS = 5; // Allow 5 requests per window

B. Fixed Window Rate Limiting

This method resets the count at the start of each time window.

const fixedWindowRateLimiter = async (req, res, next) => {
    const ip = req.ip; // Track requests by IP
    const key = `rate:${ip}`;

    let requests = await redis.get(key);
    if (!requests) {
        await redis.set(key, 1, "EX", WINDOW_SIZE);
        return next();
    }

    if (requests >= MAX_REQUESTS) {
        return res.status(429).json({ error: "Too many requests. Try again later." });
    }

    await redis.incr(key);
    next();
};

app.use(fixedWindowRateLimiter);

app.get("/", (req, res) => {
    res.send("Welcome! You are within rate limits.");
});

app.listen(3000, () => console.log("Server running on port 3000"));

✅ Pros: Simple to implement.
❌ Cons: Can cause bursts of requests at window boundaries.

C. Sliding Window Rate Limiting

A more precise method using timestamps.

const slidingWindowRateLimiter = async (req, res, next) => {
    const ip = req.ip;
    const key = `rate:${ip}`;
    const now = Date.now();

    await redis.zremrangebyscore(key, 0, now - WINDOW_SIZE * 1000);
    const requests = await redis.zcard(key);

    if (requests >= MAX_REQUESTS) {
        return res.status(429).json({ error: "Too many requests. Try again later." });
    }

    await redis.zadd(key, now, now);
    await redis.expire(key, WINDOW_SIZE);
    next();
};

app.use(slidingWindowRateLimiter);

✅ Pros: Prevents bursts, smoother distribution.
❌ Cons: Uses more memory due to timestamps.

4️⃣ Implementing Rate Limiting in Golang

For Golang, install Redis and Fiber:

go get github.com/gofiber/fiber/v2
go get github.com/redis/go-redis/v9

Create main.go:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/gofiber/fiber/v2"
	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{
	Addr: "localhost:6379",
})

const WINDOW_SIZE = 60
const MAX_REQUESTS = 5

func rateLimiter(c *fiber.Ctx) error {
	ip := c.IP()
	key := fmt.Sprintf("rate:%s", ip)

	reqs, err := rdb.Get(ctx, key).Int()
	if err != nil && err != redis.Nil {
		return c.Status(500).SendString("Redis error")
	}

	if reqs >= MAX_REQUESTS {
		return c.Status(429).SendString("Too many requests, try later")
	}

	rdb.Incr(ctx, key)
	rdb.Expire(ctx, key, time.Duration(WINDOW_SIZE)*time.Second)

	return c.Next()
}

func main() {
	app := fiber.New()
	app.Use(rateLimiter)

	app.Get("/", func(c *fiber.Ctx) error {
		return c.SendString("Welcome! You are within rate limits.")
	})

	app.Listen(":3000")
}

✅ Pros: Fast and lightweight.
❌ Cons: May need tuning for large-scale use.

5️⃣ Monitoring & Debugging Rate Limits

Check Redis keys:

redis-cli keys 'rate:*'

Check request count:

redis-cli get rate:127.0.0.1

✅ Next Steps

Now that we’ve covered API rate limiting with Redis, we can explore:

Full-page caching with Redis
Using Redis for session storage
Event-driven caching with Redis Streams

🚀 Part 5: Using Redis as a Session Store

Managing user sessions efficiently is crucial for scalable web applications. In this tutorial, we’ll explore how to use Redis as a session store to handle user authentication and stateful data efficiently.

1️⃣ Why Use Redis for Session Storage?

Feature	Benefit
Fast Performance	Redis operates in-memory, making it much faster than database-backed sessions.
Scalability	Supports distributed session storage for multiple app instances.
Expiration Handling	Sessions automatically expire after a defined TTL.
Persistence Options	Can persist sessions to disk if needed (RDB, AOF).

2️⃣ Setting Up Redis for Session Storage

📌 Installing Dependencies

For Node.js (Express + Redis session store):

npm install express express-session connect-redis ioredis

For Golang (Fiber + Redis session middleware):

go get github.com/gofiber/fiber/v2
go get github.com/gofiber/session/v2
go get github.com/redis/go-redis/v9

3️⃣ Implementing Redis Sessions

A. Using Redis for Sessions in Node.js

📌 Step 1: Setup Express & Redis Store

const express = require("express");
const session = require("express-session");
const RedisStore = require("connect-redis").default;
const Redis = require("ioredis");

const app = express();
const redisClient = new Redis();

app.use(
  session({
    store: new RedisStore({ client: redisClient }),
    secret: "supersecret",
    resave: false,
    saveUninitialized: false,
    cookie: { secure: false, maxAge: 60000 }, // 60s expiration
  })
);

app.get("/", (req, res) => {
  req.session.username = "Alice";
  res.send("Session stored!");
});

app.get("/session", (req, res) => {
  res.json({ session: req.session });
});

app.listen(3000, () => console.log("Server running on port 3000"));

📌 Step 2: Check Stored Sessions in Redis

Run:

redis-cli keys '*'

You’ll see a session key like:

sess:xyz123abc

Get session data:

redis-cli get sess:xyz123abc

B. Using Redis for Sessions in Golang (Fiber)

📌 Step 1: Setup Fiber & Redis Sessions

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/gofiber/fiber/v2"
	"github.com/gofiber/session/v2"
	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{
	Addr: "localhost:6379",
})

func main() {
	app := fiber.New()
	store := session.New(session.Config{
		Storage: rdb,
	})

	app.Get("/", func(c *fiber.Ctx) error {
		sess, err := store.Get(c)
		if err != nil {
			return err
		}
		sess.Set("username", "Alice")
		sess.Save()
		return c.SendString("Session saved")
	})

	app.Get("/session", func(c *fiber.Ctx) error {
		sess, err := store.Get(c)
		if err != nil {
			return err
		}
		username := sess.Get("username")
		return c.JSON(fiber.Map{"username": username})
	})

	app.Listen(":3000")
}

✅ Check Stored Session in Redis

redis-cli keys '*'
redis-cli get session:xyz123abc

4️⃣ Managing Session Expiration & Cleanup

📌 Setting Session TTL

To ensure sessions expire automatically, set a TTL:

redis-cli config set timeout 1800  # 30 mins expiration

Or dynamically in code:

Node.js

session({
  store: new RedisStore({ client: redisClient, ttl: 1800 }), // 30 min TTL
});

Golang

sess.SetExpiry(30 * time.Minute)

5️⃣ Scaling Redis Session Storage

If using multiple instances of your app, ensure Redis Sentinel or Cluster is used.

📌 Load Balancing Session Storage

Use Redis Sentinel to handle failover.
Use Redis Cluster for high availability.
Use sticky sessions with a load balancer.

Example Redis Cluster connection:

const cluster = new Redis.Cluster([
  { host: "redis-node1", port: 7001 },
  { host: "redis-node2", port: 7002 },
  { host: "redis-node3", port: 7003 },
]);

✅ Next Steps

Now that we’ve covered Redis as a session store, we can explore:

Caching API responses for performance
Redis for real-time notifications (Pub/Sub)
Event-driven caching with Redis Streams

🚀 Part 6: Redis Pub/Sub for Real-Time Notifications

Redis Publish/Subscribe (Pub/Sub) is a messaging system that allows real-time communication between different parts of your application. It is commonly used for: ✅ Real-time notifications (chat apps, live updates)
✅ Event-driven systems (microservices communication)
✅ Live streaming dashboards (stocks, sports scores)

1️⃣ How Redis Pub/Sub Works

Publishers send messages to a channel.
Subscribers listen for messages on that channel.
Messages are broadcasted instantly to all subscribers.

📌 Key Commands:

PUBLISH channel "Hello, Subscribers!"
SUBSCRIBE channel

2️⃣ Setting Up Redis Pub/Sub

📌 Install Dependencies

For Node.js:

npm install ioredis

For Golang:

go get github.com/redis/go-redis/v9

3️⃣ Implementing Pub/Sub in Node.js

A. Publisher (send messages)

Create publisher.js:

const Redis = require("ioredis");
const publisher = new Redis();

setInterval(() => {
    const message = `Message at ${new Date().toLocaleTimeString()}`;
    publisher.publish("notifications", message);
    console.log("Published:", message);
}, 5000);

B. Subscriber (listen for messages)

Create subscriber.js:

const Redis = require("ioredis");
const subscriber = new Redis();

subscriber.subscribe("notifications", (err, count) => {
    console.log(`Subscribed to ${count} channels.`);
});

subscriber.on("message", (channel, message) => {
    console.log(`Received on ${channel}: ${message}`);
});

📌 Running Pub/Sub

Open two terminals and run:

node subscriber.js
node publisher.js

✅ Result: Messages will appear in the subscriber every 5 seconds.

4️⃣ Implementing Pub/Sub in Golang

A. Publisher (send messages)

Create publisher.go:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	for {
		message := fmt.Sprintf("Message at %v", time.Now().Format("15:04:05"))
		rdb.Publish(ctx, "notifications", message)
		fmt.Println("Published:", message)
		time.Sleep(5 * time.Second)
	}
}

B. Subscriber (listen for messages)

Create subscriber.go:

package main

import (
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	pubsub := rdb.Subscribe(ctx, "notifications")
	defer pubsub.Close()

	ch := pubsub.Channel()
	for msg := range ch {
		fmt.Println("Received:", msg.Payload)
	}
}

📌 Running Pub/Sub

Open two terminals and run:

go run subscriber.go
go run publisher.go

✅ Result: Subscriber receives real-time messages.

5️⃣ Scaling Redis Pub/Sub

A. Handling Multiple Subscribers

Each subscriber receives all messages, but to balance workload:

Use Redis Streams for message persistence.
Use message queues (RabbitMQ, Kafka) for async processing.

B. Using Pub/Sub with Microservices

One service publishes (e.g., order events).
Multiple services subscribe (e.g., inventory, notifications).

Example Event-Driven Microservice:

publisher.publish("order_created", JSON.stringify({ orderId: 123 }));

✅ Next Steps

Now that you’ve implemented Redis Pub/Sub, we can explore:

Redis Streams for event-driven architecture
Redis for real-time analytics & leaderboards
Full-page caching with Redis

🚀 Part 7: Redis Streams for Event-Driven Systems

Redis Streams is an advanced message queue system that enables real-time event processing, data pipelines, and microservice communication.

1️⃣ Why Use Redis Streams?

Feature	Benefit
Persistent Messaging	Unlike Pub/Sub, messages persist even if consumers disconnect.
Multiple Consumers	Supports consumer groups for load balancing.
Replayability	Can replay old messages using IDs.
Scalable & Fast	Optimized for high-throughput event processing.

2️⃣ Redis Streams vs. Pub/Sub

Feature	Redis Streams	Redis Pub/Sub
Message Persistence	✅ Yes, stores messages	❌ No, messages lost if no subscribers
Multiple Consumers	✅ Supports consumer groups	❌ Messages sent to all subscribers
Replay Messages	✅ Yes	❌ No

✅ Use Streams for durable event processing
❌ Use Pub/Sub for real-time notifications

3️⃣ Basic Redis Stream Commands

# Add an event to the stream
XADD orders * order_id 123 status "pending"

# Read events from the stream
XRANGE orders - +

# Read new events as they arrive
XREAD BLOCK 5000 STREAMS orders $

4️⃣ Implementing Redis Streams in Node.js

A. Producer (Writing Events)

Create producer.js:

const Redis = require("ioredis");
const redis = new Redis();

async function produce() {
    const orderId = Math.floor(Math.random() * 1000);
    await redis.xadd("orders", "*", "order_id", orderId, "status", "pending");
    console.log(`Produced Order: ${orderId}`);
}

setInterval(produce, 5000);

B. Consumer (Reading Events)

Create consumer.js:

const Redis = require("ioredis");
const redis = new Redis();

async function consume() {
    while (true) {
        const response = await redis.xread("BLOCK", 5000, "STREAMS", "orders", "$");
        if (response) {
            const [stream, messages] = response[0];
            for (const [id, fields] of messages) {
                console.log(`Consumed: ${id}`, fields);
            }
        }
    }
}

consume();

5️⃣ Implementing Redis Streams in Golang

A. Producer (Writing Events)

Create producer.go:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	for {
		orderID := fmt.Sprintf("%d", time.Now().UnixNano())
		rdb.XAdd(ctx, &redis.XAddArgs{
			Stream: "orders",
			Values: map[string]interface{}{"order_id": orderID, "status": "pending"},
		})
		fmt.Println("Produced Order:", orderID)
		time.Sleep(5 * time.Second)
	}
}

B. Consumer (Reading Events)

Create consumer.go:

package main

import (
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	for {
		streams, err := rdb.XRead(ctx, &redis.XReadArgs{
			Streams: []string{"orders", "$"},
			Block:   5000,
			Count:   1,
		}).Result()

		if err == nil && len(streams) > 0 {
			for _, msg := range streams[0].Messages {
				fmt.Println("Consumed:", msg.Values)
			}
		}
	}
}

6️⃣ Using Consumer Groups for Load Balancing

Consumer groups distribute messages across multiple consumers.

# Create a consumer group
XGROUP CREATE orders order_group $ MKSTREAM

# Read with a consumer
XREADGROUP GROUP order_group consumer-1 COUNT 1 STREAMS orders >

✅ Advantage: Supports parallel processing across services.

7️⃣ Scaling Redis Streams

Partition events across multiple Redis nodes
Use Consumer Groups for parallel processing
Limit stream size using MAXLEN to prevent memory issues

XADD orders MAXLEN ~1000 * order_id 124 status "pending"

✅ Next Steps

Now that you’ve mastered Redis Streams, we can explore:

Redis for full-page caching
Using Redis for real-time leaderboards
Building a Redis-based job queue

🚀 Part 8: Full-Page Caching with Redis

Caching full pages in Redis improves performance and reduces load on your web server by serving precomputed responses instantly.

1️⃣ What is Full-Page Caching?

Instead of hitting the database or rendering a page on every request, full-page caching stores entire responses in Redis.

✅ Speeds up API & web responses
✅ Reduces database queries
✅ Handles high traffic efficiently

2️⃣ How Full-Page Caching Works

Check if the requested page is in Redis (GET key)
If found (cache hit), return it instantly.
If not (cache miss), generate the page, store it in Redis (SET key), and return it.

3️⃣ Implementing Full-Page Caching in Node.js

A. Install Dependencies

npm install express ioredis

B. Create `server.js`

const express = require("express");
const Redis = require("ioredis");

const app = express();
const redis = new Redis();
const CACHE_TTL = 60; // Cache pages for 60 seconds

// Middleware to check cache
async function cacheMiddleware(req, res, next) {
    const cachedPage = await redis.get(req.originalUrl);
    if (cachedPage) {
        console.log("Cache Hit");
        return res.send(cachedPage);
    }
    next();
}

// Route with caching
app.get("/page/:id", cacheMiddleware, async (req, res) => {
    console.log("Cache Miss. Generating page...");
    const pageContent = `<h1>Page ${req.params.id}</h1><p>Generated at ${new Date()}</p>`;
    
    // Store in Redis
    await redis.set(req.originalUrl, pageContent, "EX", CACHE_TTL);
    
    res.send(pageContent);
});

app.listen(3000, () => console.log("Server running on port 3000"));

4️⃣ Implementing Full-Page Caching in Golang

A. Install Dependencies

go get github.com/gofiber/fiber/v2
go get github.com/redis/go-redis/v9

B. Create `main.go`

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/gofiber/fiber/v2"
	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	app := fiber.New()

	app.Get("/page/:id", func(c *fiber.Ctx) error {
		pageID := c.Params("id")
		key := fmt.Sprintf("page:%s", pageID)

		// Check cache
		cached, err := rdb.Get(ctx, key).Result()
		if err == nil {
			fmt.Println("Cache Hit")
			return c.SendString(cached)
		}

		fmt.Println("Cache Miss. Generating page...")
		pageContent := fmt.Sprintf("<h1>Page %s</h1><p>Generated at %s</p>", pageID, time.Now().String())

		// Store in Redis
		rdb.Set(ctx, key, pageContent, 60*time.Second)

		return c.SendString(pageContent)
	})

	app.Listen(":3000")
}

5️⃣ Clearing the Cache

If the page content changes, delete the cached version:

📌 In Node.js

await redis.del("/page/1");

📌 In Golang

rdb.Del(ctx, "page:1")

6️⃣ Optimizing Cache Performance

✅ Use Compression: Store compressed HTML

const zlib = require("zlib");
await redis.set(req.originalUrl, zlib.gzipSync(pageContent), "EX", CACHE_TTL);

✅ Use Redis Expiry Policies: Set volatile-lru to evict older pages

CONFIG SET maxmemory-policy volatile-lru

✅ Cache Only Important Pages: Don’t cache user-specific content (like dashboards).

✅ Next Steps

Now that you’ve implemented full-page caching with Redis, we can explore:

Building a real-time leaderboard with Redis
Using Redis as a job queue for background tasks
Optimizing Redis performance for high-traffic applications

🚀 Part 9: Building a Real-Time Leaderboard with Redis

A leaderboard ranks users based on scores, often used in gaming, ranking systems, and competitive apps. Redis Sorted Sets (ZSETs) make it easy to manage a real-time leaderboard efficiently.

1️⃣ Why Use Redis for Leaderboards?

✅ Fast operations: Insert, update, and rank retrieval in O(log N) time.
✅ Efficient sorting: Automatically keeps items sorted by score.
✅ Range queries: Fetch top rankings or a specific user’s rank instantly.
✅ Low latency: Ideal for real-time applications like gaming and competition platforms.

2️⃣ How Redis Sorted Sets Work

Redis Sorted Sets (ZSETs) store values with a score and keep them sorted automatically.

Command	Description
`ZADD leaderboard score user`	Add or update a user’s score
`ZRANGE leaderboard 0 -1 WITHSCORES`	Get all rankings (lowest to highest)
`ZREVRANGE leaderboard 0 9 WITHSCORES`	Get top 10 rankings
`ZRANK leaderboard user`	Get user’s current rank
`ZREM leaderboard user`	Remove a user from the leaderboard

3️⃣ Implementing a Redis Leaderboard in Node.js

A. Install Dependencies

npm install express ioredis

B. Create `leaderboard.js`

const express = require("express");
const Redis = require("ioredis");

const app = express();
const redis = new Redis();
const LEADERBOARD = "game_leaderboard";

app.use(express.json());

// Add or update player score
app.post("/score", async (req, res) => {
    const { player, score } = req.body;
    await redis.zadd(LEADERBOARD, score, player);
    res.json({ message: "Score updated!" });
});

// Get top 10 players
app.get("/leaderboard", async (req, res) => {
    const scores = await redis.zrevrange(LEADERBOARD, 0, 9, "WITHSCORES");
    res.json(scores);
});

// Get a player’s rank
app.get("/rank/:player", async (req, res) => {
    const rank = await redis.zrevrank(LEADERBOARD, req.params.player);
    res.json({ player: req.params.player, rank: rank !== null ? rank + 1 : "Not Ranked" });
});

// Remove a player from the leaderboard
app.delete("/remove/:player", async (req, res) => {
    await redis.zrem(LEADERBOARD, req.params.player);
    res.json({ message: "Player removed from leaderboard" });
});

app.listen(3000, () => console.log("Leaderboard API running on port 3000"));

✅ Test the API using CURL

curl -X POST http://localhost:3000/score -H "Content-Type: application/json" -d '{"player": "Alice", "score": 100}'
curl -X POST http://localhost:3000/score -H "Content-Type: application/json" -d '{"player": "Bob", "score": 150}'
curl http://localhost:3000/leaderboard
curl http://localhost:3000/rank/Alice
curl -X DELETE http://localhost:3000/remove/Alice

4️⃣ Implementing a Redis Leaderboard in Golang

A. Install Dependencies

go get github.com/gofiber/fiber/v2
go get github.com/redis/go-redis/v9

B. Create `main.go`

package main

import (
	"context"
	"fmt"

	"github.com/gofiber/fiber/v2"
	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})
const leaderboard = "game_leaderboard"

func main() {
	app := fiber.New()

	// Add or update player score
	app.Post("/score", func(c *fiber.Ctx) error {
		data := struct {
			Player string  `json:"player"`
			Score  float64 `json:"score"`
		}{}
		if err := c.BodyParser(&data); err != nil {
			return err
		}
		rdb.ZAdd(ctx, leaderboard, redis.Z{Score: data.Score, Member: data.Player})
		return c.JSON(fiber.Map{"message": "Score updated!"})
	})

	// Get top 10 players
	app.Get("/leaderboard", func(c *fiber.Ctx) error {
		scores, _ := rdb.ZRevRangeWithScores(ctx, leaderboard, 0, 9).Result()
		return c.JSON(scores)
	})

	// Get player rank
	app.Get("/rank/:player", func(c *fiber.Ctx) error {
		rank, err := rdb.ZRevRank(ctx, leaderboard, c.Params("player")).Result()
		if err != nil {
			return c.JSON(fiber.Map{"player": c.Params("player"), "rank": "Not Ranked"})
		}
		return c.JSON(fiber.Map{"player": c.Params("player"), "rank": rank + 1})
	})

	// Remove player
	app.Delete("/remove/:player", func(c *fiber.Ctx) error {
		rdb.ZRem(ctx, leaderboard, c.Params("player"))
		return c.JSON(fiber.Map{"message": "Player removed from leaderboard"})
	})

	app.Listen(":3000")
}

✅ Test the API using CURL

curl -X POST http://localhost:3000/score -H "Content-Type: application/json" -d '{"player": "Alice", "score": 100}'
curl -X POST http://localhost:3000/score -H "Content-Type: application/json" -d '{"player": "Bob", "score": 150}'
curl http://localhost:3000/leaderboard
curl http://localhost:3000/rank/Alice
curl -X DELETE http://localhost:3000/remove/Alice

5️⃣ Handling Real-Time Updates

To broadcast leaderboard updates in real-time, use Redis Pub/Sub:

📌 Publisher (Send Updates)

await redis.publish("leaderboard_updates", JSON.stringify({ player: "Alice", score: 120 }));

📌 Subscriber (Listen for Updates)

const subscriber = new Redis();
subscriber.subscribe("leaderboard_updates");
subscriber.on("message", (channel, message) => {
    console.log(`Leaderboard Update: ${message}`);
});

✅ Use WebSockets to update the frontend when a new score is added.

6️⃣ Optimizing Leaderboard Performance

✅ Set a max length to prevent unlimited storage:

ZREMRANGEBYRANK game_leaderboard 0 -101

(Keep only top 100 players)

✅ Expire inactive players:

ZREMRANGEBYSCORE game_leaderboard -inf 50

(Remove players below a score of 50)

✅ Use a Redis Cluster for scaling:

docker run -d --name redis-cluster -p 7000:7000 redis:6 --cluster-enabled yes

✅ Next Steps

Now that you’ve built a real-time leaderboard with Redis, we can explore:

Using Redis as a job queue for background tasks
Optimizing Redis performance for high-traffic applications
Building an AI-powered recommendation system using Redis

🚀 Part 10: Using Redis as a Job Queue for Background Tasks

Redis job queues help process tasks asynchronously, improving application responsiveness and handling large workloads efficiently.

1️⃣ Why Use Redis as a Job Queue?

✅ Efficient Task Processing: Offloads work from the main application thread.
✅ Scalable: Multiple workers can process jobs concurrently.
✅ Retry Mechanism: Failed jobs can be retried automatically.
✅ Supports Delayed Jobs: Schedule tasks to run later.

2️⃣ How a Redis Job Queue Works

Producer pushes jobs to a Redis list (LPUSH queue_name job_data).
Worker(s) fetch jobs using BRPOP (blocking pop).
The job is processed, and results are stored.

3️⃣ Implementing a Redis Job Queue in Node.js

A. Install Dependencies

npm install bull ioredis express

B. Create a Job Queue with `bull`

📌 Producer (producer.js)

const Queue = require("bull");

const jobQueue = new Queue("email_jobs", { redis: { host: "localhost", port: 6379 } });

async function addJob(email) {
    await jobQueue.add({ email }, { attempts: 3, delay: 5000 });
    console.log(`Job added for: ${email}`);
}

// Add a job every 5 seconds
setInterval(() => addJob(`user${Math.floor(Math.random() * 100)}@example.com`), 5000);

📌 Worker (worker.js)

const Queue = require("bull");

const jobQueue = new Queue("email_jobs", { redis: { host: "localhost", port: 6379 } });

jobQueue.process(async (job) => {
    console.log(`Processing job: Sending email to ${job.data.email}`);
    await new Promise((resolve) => setTimeout(resolve, 2000)); // Simulate email sending
    console.log(`Email sent to ${job.data.email}`);
});

✅ Start Producer & Worker

node producer.js  # Starts adding jobs
node worker.js    # Starts processing jobs

4️⃣ Implementing a Redis Job Queue in Golang

A. Install Dependencies

go get github.com/go-redis/redis/v9

B. Create a Job Queue

📌 Producer (producer.go)

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	for {
		jobID := fmt.Sprintf("job-%d", time.Now().UnixNano())
		rdb.LPush(ctx, "email_jobs", jobID)
		fmt.Println("Added Job:", jobID)
		time.Sleep(5 * time.Second)
	}
}

📌 Worker (worker.go)

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()

func main() {
	rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})

	for {
		job, err := rdb.BRPop(ctx, 0, "email_jobs").Result()
		if err != nil {
			fmt.Println("Error fetching job:", err)
			continue
		}

		fmt.Println("Processing job:", job[1])
		time.Sleep(2 * time.Second) // Simulate processing time
		fmt.Println("Job completed:", job[1])
	}
}

✅ Start Producer & Worker

go run producer.go  # Starts adding jobs
go run worker.go    # Starts processing jobs

5️⃣ Handling Delayed & Failed Jobs

✅ Delay a Job (Bull in Node.js)

await jobQueue.add({ email }, { delay: 10000 }); // Run after 10 sec

✅ Retry Failed Jobs (Bull in Node.js)

await jobQueue.add({ email }, { attempts: 5 }); // Retry 5 times on failure

✅ Move Failed Jobs to a Retry Queue (Golang)

rdb.RPush(ctx, "failed_jobs", jobID)

6️⃣ Monitoring Redis Job Queues

📌 Bull Dashboard for Node.js

npm install bull-board

const { createBullBoard } = require("bull-board");
const { BullAdapter } = require("bull-board/bullAdapter");
const express = require("express");

const app = express();
const { router } = createBullBoard([new BullAdapter(jobQueue)]);

app.use("/admin/queues", router);
app.listen(3001, () => console.log("Bull Dashboard running on port 3001"));

Visit: http://localhost:3001/admin/queues

📌 Redis CLI for Golang

redis-cli LRANGE email_jobs 0 -1  # View pending jobs
redis-cli LLEN email_jobs         # Count pending jobs

✅ Next Steps

Now that you’ve built a Redis Job Queue, we can explore:

Optimizing Redis performance for high-traffic applications
Building an AI-powered recommendation system using Redis
Using Redis with GraphQL for real-time subscriptions

🚀 Part 11: Optimizing Redis Performance for High-Traffic Applications

Redis is designed for speed, but when handling millions of requests per second, optimization is key to ensuring low latency, high availability, and efficient memory usage.

1️⃣ Key Performance Bottlenecks

Issue	Cause	Solution
High Latency	Too many concurrent connections	Use connection pooling
Memory Overload	No eviction policy	Set eviction rules (`maxmemory-policy`)
Slow Queries	Large data scans	Use indexing & key expiration
CPU Spikes	Expensive Lua scripts or blocking commands	Optimize with pipelining & async tasks

2️⃣ Optimize Redis Connection Handling

A. Use Connection Pooling

Instead of opening a new connection for every request, use a connection pool.

✅ Node.js

const Redis = require("ioredis");
const redis = new Redis({ maxRetriesPerRequest: null, enableAutoPipelining: true });

✅ Golang

rdb := redis.NewClient(&redis.Options{
    Addr: "localhost:6379",
    PoolSize: 10, // 10 concurrent connections
})

3️⃣ Use Redis Pipelining for Bulk Operations

Instead of sending multiple requests, use pipelining to execute them in one batch.

✅ Node.js

const pipeline = redis.pipeline();
pipeline.set("user:1", "Alice");
pipeline.set("user:2", "Bob");
pipeline.exec();

✅ Golang

pipe := rdb.Pipeline()
pipe.Set(ctx, "user:1", "Alice", 0)
pipe.Set(ctx, "user:2", "Bob", 0)
pipe.Exec(ctx)

🔹 Benefit: Reduces network round-trips, improving performance significantly.

4️⃣ Set Redis Memory Limits & Eviction Policies

When Redis memory is full, it needs a strategy to remove old data.

A. Setting a Max Memory Limit

CONFIG SET maxmemory 500mb

B. Choosing an Eviction Policy

CONFIG SET maxmemory-policy allkeys-lru

Policy	Description
noeviction	Returns an error when full (not recommended for caching)
allkeys-lru	Removes least recently used (LRU) keys
volatile-lru	Removes LRU keys only if they have an expiration
allkeys-lfu	Removes least frequently used (LFU) keys

✅ Best Choice for High Traffic: allkeys-lfu or allkeys-lru.

5️⃣ Avoid Expensive Operations

Some Redis commands can slow down performance due to high CPU usage.

Expensive Command	Alternative
`KEYS *` (scans all keys)	`SCAN` (iterates efficiently)
`LRANGE mylist 0 -1` (fetch full list)	`LRANGE mylist 0 10` (limit range)
`HGETALL user:1`	`HMGET user:1 name email` (fetch specific fields)

✅ Example: Using SCAN Instead of KEYS

SCAN 0 MATCH user:* COUNT 100

6️⃣ Use Redis Clustering for Horizontal Scaling

When handling millions of requests per second, use Redis Cluster for sharding.

✅ Start a Redis Cluster with 3 Nodes

docker network create redis-cluster

docker run --rm --net redis-cluster -p 7000:7000 -d redis --port 7000 --cluster-enabled yes
docker run --rm --net redis-cluster -p 7001:7001 -d redis --port 7001 --cluster-enabled yes
docker run --rm --net redis-cluster -p 7002:7002 -d redis --port 7002 --cluster-enabled yes

redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 --cluster-replicas 1

✅ Connect to Redis Cluster in Node.js

const Redis = require("ioredis");
const cluster = new Redis.Cluster([
    { host: "127.0.0.1", port: 7000 },
    { host: "127.0.0.1", port: 7001 },
    { host: "127.0.0.1", port: 7002 }
]);

7️⃣ Enable Redis Persistence for Reliability

Redis stores data in-memory, so persistence ensures no data loss.

✅ AOF (Append-Only File) Mode

CONFIG SET appendonly yes

📌 Benefit: Logs every write operation.

✅ RDB (Snapshot) Mode

CONFIG SET save "900 1 300 10"

📌 Benefit: Takes periodic snapshots.

🚀 Best Strategy: Use both AOF and RDB together.

8️⃣ Monitor Redis Performance

✅ Using Redis CLI

INFO MEMORY      # Check memory usage
INFO CPU         # Check CPU usage
MONITOR          # Live Redis command tracking
SLOWLOG GET 10   # Find slow queries

✅ Using RedisInsight

docker run --name redisinsight -p 8001:8001 -d redislabs/redisinsight

Open http://localhost:8001 for visual monitoring.

✅ Using Prometheus & Grafana

Install Redis Exporter:

docker run -d --name redis_exporter -p 9121:9121 oliver006/redis_exporter

Connect it to Grafana for visual analytics.

✅ Next Steps

Now that you’ve optimized Redis performance, we can explore:

Building an AI-powered recommendation system using Redis
Using Redis with GraphQL for real-time subscriptions
Securing Redis against attacks

🚀 Part 12: Building an AI-Powered Recommendation System with Redis

Redis can power real-time recommendation systems using vector search, collaborative filtering, and personalized ranking.

1️⃣ Why Use Redis for AI-Powered Recommendations?

Feature	Benefit
Fast Vector Search	Supports similarity search for personalized recommendations.
Real-time Updates	Instantly updates rankings based on user interactions.
Scalability	Handles millions of users efficiently.
Hybrid Storage	Stores structured & unstructured data (JSON, vectors, scores).

2️⃣ Techniques for AI Recommendations

1️⃣ Content-Based Filtering: Recommends similar items based on item attributes (e.g., movies with similar genres).
2️⃣ Collaborative Filtering: Suggests items based on what similar users like.
3️⃣ Hybrid Approach: Combines both for better accuracy.
4️⃣ Vector Similarity Search: Uses RedisSearch for AI-powered matching.

3️⃣ Setting Up Redis for AI Recommendations

We’ll use Redis with vector search for fast similarity matching.

A. Install Redis Stack (With Vector Search)

docker run -d --name redis-stack -p 6379:6379 redislabs/redis-stack

This includes RedisJSON, RedisSearch, and RedisAI.

✅ Verify Installation

redis-cli

MODULE LIST  # Ensure RedisSearch is enabled

4️⃣ Storing & Searching Recommendations with Redis

A. Define a Product Schema with Vectors

We’ll use RedisJSON and vector embeddings for product recommendations.

📌 Add Products to Redis with AI Features

HSET product:1 name "Wireless Headphones" category "Electronics" features "0.8,0.2,0.5"
HSET product:2 name "Gaming Laptop" category "Computers" features "0.9,0.7,0.1"
HSET product:3 name "Bluetooth Speaker" category "Electronics" features "0.85,0.3,0.6"

B. Search for Similar Products (Vector Similarity)

1️⃣ Create a Search Index:

FT.CREATE idx_products ON HASH PREFIX 1 product: SCHEMA name TEXT category TEXT features VECTOR FLAT 3 DIM 3 DISTANCE_METRIC COSINE

2️⃣ Find Similar Items (Cosine Similarity)

FT.SEARCH idx_products "*=>[KNN 2 @features $vec AS score]" PARAMS 2 vec "0.85,0.3,0.6"

✅ Returns the 2 closest matches to the input vector.

5️⃣ Implementing Recommendations in Node.js

A. Install Dependencies

npm install ioredis

B. Add Products to Redis

const Redis = require("ioredis");
const redis = new Redis();

async function addProducts() {
    await redis.hset("product:1", "name", "Wireless Headphones", "category", "Electronics", "features", "0.8,0.2,0.5");
    await redis.hset("product:2", "name", "Gaming Laptop", "category", "Computers", "features", "0.9,0.7,0.1");
    await redis.hset("product:3", "name", "Bluetooth Speaker", "category", "Electronics", "features", "0.85,0.3,0.6");
    console.log("Products added!");
}

addProducts();

C. Find Similar Products Using Vector Search

async function findSimilarProducts(vector) {
    const response = await redis.call(
        "FT.SEARCH",
        "idx_products",
        "*=>[KNN 2 @features $vec AS score]",
        "PARAMS", "2", "vec", vector
    );
    console.log("Similar Products:", response);
}

findSimilarProducts("0.85,0.3,0.6");

✅ Returns the most relevant product recommendations instantly!

6️⃣ Implementing Recommendations in Golang

A. Install Redis Client

go get github.com/redis/go-redis/v9

B. Add Products to Redis

package main

import (
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

func main() {
	rdb.HSet(ctx, "product:1", "name", "Wireless Headphones", "category", "Electronics", "features", "0.8,0.2,0.5")
	rdb.HSet(ctx, "product:2", "name", "Gaming Laptop", "category", "Computers", "features", "0.9,0.7,0.1")
	rdb.HSet(ctx, "product:3", "name", "Bluetooth Speaker", "category", "Electronics", "features", "0.85,0.3,0.6")
	fmt.Println("Products added!")
}

C. Find Similar Products

func findSimilarProducts(vector string) {
	res, _ := rdb.Do(ctx, "FT.SEARCH", "idx_products", "*=>[KNN 2 @features $vec AS score]", "PARAMS", "2", "vec", vector).Result()
	fmt.Println("Similar Products:", res)
}

findSimilarProducts("0.85,0.3,0.6")

✅ Returns the most relevant products in milliseconds!

7️⃣ Enhancing Recommendations with User Interactions

1️⃣ Store user clicks:

ZINCRBY user:recommendations 1 "product:1"

2️⃣ Fetch top recommended products for a user:

ZRANGE user:recommendations 0 4 WITHSCORES

✅ Dynamically adapts to user behavior!

8️⃣ Scaling AI Recommendations

✅ Use Redis Cluster for distributing workload:

docker run -d --name redis-cluster -p 7000:7000 redis --cluster-enabled yes

✅ Use Redis Streams for real-time event processing:

XADD user_events * user_id 123 event "viewed_product" product_id 1

✅ Use RedisAI for ML models inside Redis:

AI.MODELSTORE my_model ONNX CPU BLOB my_model.onnx

✅ Next Steps

Now that you’ve built an AI-powered recommendation system, we can explore:

Using Redis with GraphQL for real-time subscriptions
Securing Redis against cyber threats
Building a real-time chat system with Redis

🚀 Part 13: Using Redis with GraphQL for Real-Time Subscriptions

Redis is a powerful real-time data store that can be combined with GraphQL Subscriptions to build real-time APIs for chat apps, live notifications, stock price updates, and more.

1️⃣ Why Use Redis for GraphQL Subscriptions?

✅ Scalability: Multiple GraphQL servers can listen for updates.
✅ Low Latency: Instant data delivery using Pub/Sub.
✅ Event-Driven: Works seamlessly with microservices.
✅ Handles High Traffic: Supports millions of concurrent connections.

2️⃣ How GraphQL Subscriptions Work with Redis

Clients subscribe to real-time data updates.
Redis Pub/Sub sends updates across multiple GraphQL servers.
Clients receive instant updates via WebSockets.

3️⃣ Setting Up Redis for GraphQL Subscriptions

A. Install Required Packages

For Node.js (Apollo Server + Redis):

npm install express graphql @apollo/server graphql-ws ioredis

For Golang (GQLGen + Redis):

go get github.com/99designs/gqlgen
go get github.com/redis/go-redis/v9

4️⃣ Implementing GraphQL Subscriptions in Node.js

A. Set Up a GraphQL Server

Create server.js:

const { ApolloServer } = require("@apollo/server");
const { startStandaloneServer } = require("@apollo/server/standalone");
const Redis = require("ioredis");
const { createServer } = require("http");
const { makeExecutableSchema } = require("@graphql-tools/schema");
const { WebSocketServer } = require("ws");
const { useServer } = require("graphql-ws/lib/use/ws");

const pubsub = new Redis();
const sub = new Redis();

const typeDefs = `
  type Message {
    id: ID!
    content: String!
  }

  type Query {
    hello: String
  }

  type Subscription {
    messageAdded: Message
  }

  type Mutation {
    addMessage(content: String!): Message
  }
`;

const resolvers = {
  Query: {
    hello: () => "Hello from GraphQL + Redis!",
  },
  Mutation: {
    addMessage: async (_, { content }) => {
      const message = { id: Date.now().toString(), content };
      await pubsub.publish("MESSAGE_ADDED", JSON.stringify(message));
      return message;
    },
  },
  Subscription: {
    messageAdded: {
      subscribe: async function* () {
        while (true) {
          const message = await sub.subscribe("MESSAGE_ADDED");
          yield { messageAdded: JSON.parse(message) };
        }
      },
    },
  },
};

const schema = makeExecutableSchema({ typeDefs, resolvers });

const httpServer = createServer();
const wsServer = new WebSocketServer({ server: httpServer });

useServer({ schema }, wsServer);

const server = new ApolloServer({ schema });

startStandaloneServer(server, { listen: { port: 4000 } }).then(({ url }) => {
  console.log(`🚀 GraphQL server running at ${url}`);
});

✅ Run GraphQL Server

node server.js

B. Test Real-Time Subscriptions

1️⃣ Open GraphQL Playground
Visit: http://localhost:4000

2️⃣ Subscribe to Messages

subscription {
  messageAdded {
    id
    content
  }
}

3️⃣ Publish a New Message

mutation {
  addMessage(content: "Hello from Redis and GraphQL!") {
    id
    content
  }
}

✅ Real-time updates appear instantly!

5️⃣ Implementing GraphQL Subscriptions in Golang

A. Generate a GraphQL Server

Run:

gqlgen init

This creates a GraphQL server with a basic setup.

B. Update `schema.graphqls`

type Message {
  id: ID!
  content: String!
}

type Query {
  hello: String!
}

type Mutation {
  addMessage(content: String!): Message!
}

type Subscription {
  messageAdded: Message!
}

C. Implement Redis Pub/Sub in `resolvers.go`

package graph

import (
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"
)

var ctx = context.Background()
var rdb = redis.NewClient(&redis.Options{Addr: "localhost:6379"})

type Resolver struct{}

func (r *Resolver) Mutation_addMessage(ctx context.Context, content string) (*Message, error) {
	message := &Message{ID: "1", Content: content}
	rdb.Publish(ctx, "MESSAGE_ADDED", content)
	return message, nil
}

func (r *Resolver) Subscription_messageAdded(ctx context.Context) (<-chan *Message, error) {
	ch := make(chan *Message)
	pubsub := rdb.Subscribe(ctx, "MESSAGE_ADDED")

	go func() {
		for msg := range pubsub.Channel() {
			ch <- &Message{ID: "1", Content: msg.Payload}
		}
	}()
	return ch, nil
}

✅ Start the GraphQL Server

go run server.go

✅ Test Real-Time Subscriptions using GraphQL Playground!

6️⃣ Scaling GraphQL Subscriptions

A. Use Redis for Multi-Server Scaling

When deploying multiple GraphQL instances:

Use Redis Cluster to distribute events.
Clients subscribe via WebSockets.

✅ Enable Redis Cluster

docker run -d --name redis-cluster -p 7000:7000 redis --cluster-enabled yes

7️⃣ Optimizing Performance for Real-Time Data

✅ Use Connection Pooling

const redis = new Redis({ maxRetriesPerRequest: null, enableAutoPipelining: true });

✅ Limit Subscription Load

subscription {
  messageAdded @rateLimit(limit: 5, duration: "1m")
}

✅ Use Redis Streams Instead of Pub/Sub

XADD messages * content "Hello, Redis Streams!"
XREAD BLOCK 5000 STREAMS messages $

✅ Next Steps

Now that you’ve built GraphQL subscriptions with Redis, we can explore:

Securing Redis Against Cyber Threats
Building a Real-Time Chat System with Redis
Creating a Stock Price Tracker with Redis