Redis pipelines: batch commands to reduce network round trips

The round-trip problem

Each Redis command incurs a network round trip. Sending 100 individual commands adds 100x network latency. Pipelining batches them into one TCP write and one read.

Node.js pipeline

const pipeline = client.pipeline();
for (let i = 0; i < 1000; i++) {
  pipeline.set(`key:${i}`, `value:${i}`);
}
const results = await pipeline.exec();
// 1000 SETs in a single network round trip

Python pipeline

pipe = client.pipeline(transaction=False)
for i in range(1000):
    pipe.set(f'key:{i}', f'value:{i}')
results = pipe.execute()

Pipeline vs transaction

A pipeline is a client-side network optimisation. Commands are still executed one by one on the server, interleaved with other clients. A MULTI/EXEC transaction guarantees isolation from other clients during execution. You can combine both: wrap a pipeline in MULTI/EXEC to get batching AND atomicity.

Use pipelines for bulk imports, batch reads, or any sequence of independent commands. A pipeline with 100 commands typically runs 10-30x faster than 100 sequential commands over a network connection.