Redis 8 was only part of the optimisation

The Redis 8 upgrade helped, but most of the gain came from storing less, batching more and deleting requests.

Updated 2 min read
  • Engineering
  • Rust
  • Redis

After the RabbitMQ rewrite I profiled the next part of our pricing path. Redis showed up in the traces, but Redis itself wasn't the slow part. We were storing more data than the hot path needed and asking for it too many times.

We were already moving to Redis 8, so I used the migration as an excuse to fix both.

Storing less

The first improvement was the least exciting and probably the most important: thinner records. Some objects carried fields through the system only because they had always been there; pricing never read them. I split the data around actual access patterns and moved suitable structured entities to Redis JSON. Smaller documents meant less serialisation, less network traffic and less work during scans.

What Redis 8 changed

Redis 8 has better missing-field queries through INDEXMISSING and ismissing(), which let me remove one workaround. I also moved a slowly changing reference entity from PostgreSQL into persistent Redis with a search index. It changes rarely, can be restored safely and benefits from flexible in-memory selection. Transactional state stayed where it was.

Batching round-trips

Then I counted round-trips. Several paths made many tiny reads and writes that were fast individually and expensive together. I replaced them with MGET, MSET and pipelines where the shape allowed it. Static reference data now loads in one pipeline and stays in a bounded client cache. Independent searches run concurrently instead of one after another.

In code the change is small. A simplified version of loading reference data:

Rust
// Before: one round-trip per key.
for id in &ids {
    let doc: Option<String> = conn.get(format!("ref:{id}")).await?;
    // ...
}

// After: the whole batch in one round-trip.
let mut pipe = redis::pipe();
for id in &ids {
    pipe.cmd("JSON.GET").arg(format!("ref:{id}")).arg("$");
}
let docs: Vec<Option<String>> = pipe.query_async(&mut conn).await?;

Removing a fan-out

The biggest design win was deleting a fan-out completely. After adding the search index, one path that called three endpoints could answer the same question with one query. Two network trips, and all the work behind them, disappeared.

Results

The final result was roughly two times lower Redis latency, a 20% faster price reindex and another improvement of about 20% in the messaging-heavy flow. Alerts became less frequent and the environment steadier.

Redis 8 helped. Most of the gain came from older advice: store only what the path needs, batch the operations that belong together and remove a request entirely whenever possible.

Loading…