Introduction

Welcome to the Xylem documentation!

Xylem is a high-performance and modular traffic generator and measurement tool designed for RPC workloads. It provides a flexible framework for benchmarking and load testing distributed systems with different combinations of application protocols (e.g., Redis, HTTP, Memcached) and transport protocols (e.g., TCP, UDP, Unix Domain Socket).

Key Features

• Multi-Protocol Support: Built-in support for Redis, HTTP, Memcached, Masstree, and xylem-echo protocols
• Flexible Transport Layer: Support for TCP, UDP, and Unix Domain Sockets
• High Performance: Efficient event-driven architecture for generating high loads
• Reproducible: Configuration-first design using TOML profiles ensures reproducibility
• Detailed Metrics: Latency measurements and statistics using various sketch algorithms
• Multi-Threaded: Support for thread affinity and multi-threaded workload generation

Use Cases

Xylem is ideal for:

• Performance Benchmarking: Measure the performance of your RPC services under various load conditions
• Load Testing: Generate realistic traffic patterns to test system behavior under stress
• Latency Analysis: Collect detailed latency statistics to identify performance bottlenecks
• Protocol Testing: Validate protocol implementations across different transport layers
• Capacity Planning: Determine the maximum throughput and optimal configuration for your services

Project Status

Xylem is actively developed and maintained. The project is licensed under MIT OR Apache-2.0.

Getting Help

• GitHub Issues: Report bugs or request features at github.com/minhuw/xylem/issues
• Source Code: View the source code at github.com/minhuw/xylem

Next Steps

• Follow the Quick Start guide
• Learn about CLI Reference
• Explore the Architecture to understand Xylem’s design

Quick Start

This guide will help you run your first benchmark with Xylem.

Prerequisites

• Rust 1.70 or later
• A target service to benchmark (e.g., Redis, HTTP server, Memcached)

Build Xylem

# Clone the repository
git clone https://github.com/minhuw/xylem.git
cd xylem

# Build in release mode
cargo build --release

# The binary will be at target/release/xylem

Running a Redis Benchmark

1. Start a Redis Server

First, ensure you have a Redis server running. You can use Docker for easy setup:

docker run -d -p 6379:6379 redis:latest

Or use Docker Compose with the provided test configuration:

docker compose -f tests/redis/docker-compose.yml up -d

By default, Redis listens on localhost:6379.

2. Run the Benchmark

Use one of the included example profiles:

./target/release/xylem -P tests/redis/redis-get-zipfian.toml

This profile runs a Redis GET benchmark with a Zipfian key distribution.

3. Understanding the Output

Xylem will display statistics about the benchmark, including:

• Latency percentiles (p50, p95, p99, p99.9, etc.)
• Throughput (requests per second)
• Error rates
• Per-thread statistics

Configuration-First Design

Xylem uses TOML configuration files (called “profiles”) to define experiments. This ensures reproducibility and simplifies complex workload specifications.

Basic Syntax

xylem -P <profile.toml>

Example Profiles

Xylem includes example profiles in the profiles/ directory:

# Run a Redis GET benchmark with Zipfian distribution
xylem -P tests/redis/redis-get-zipfian.toml

# Run an HTTP load test
xylem -P profiles/http-spike.toml

# Run a Memcached benchmark
xylem -P profiles/memcached-ramp.toml

Customizing the Benchmark

You can override configuration values using the --set flag with dot notation:

# Change target address
./target/release/xylem -P tests/redis/redis-get-zipfian.toml \
  --set target.address=192.168.1.100:6379

# Change experiment duration
./target/release/xylem -P tests/redis/redis-get-zipfian.toml \
  --set experiment.duration=120s

# Change multiple parameters
./target/release/xylem -P tests/redis/redis-get-zipfian.toml \
  --set experiment.duration=60s \
  --set experiment.seed=42 \
  --set workload.keys.n=1000000

Creating a Custom Profile

Create your own TOML profile file:

# my-benchmark.toml

[experiment]
duration = "30s"
seed = 123

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

[workload]
[workload.keys]
strategy = "zipfian"
n = 10000
theta = 0.99
value_size = 100

[[traffic_groups]]
name = "main"
protocol = "redis"
threads = [0, 1, 2, 3]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "closed-loop"

Run with:

./target/release/xylem -P my-benchmark.toml

Profile File Structure

A typical profile file includes:

# Experiment configuration
[experiment]
duration = "60s"
seed = 123

# Target service configuration
[target]
protocol = "redis"
address = "127.0.0.1:6379"

# Workload configuration
[workload]
# ... workload parameters

# Traffic groups (thread assignment and rate control)
[[traffic_groups]]
name = "group-1"
threads = [0, 1, 2, 3]
# ... rate control parameters

Logging

Control logging verbosity:

# Debug level logging
xylem -P profiles/redis.toml --log-level debug

# Using RUST_LOG environment variable
RUST_LOG=debug xylem -P profiles/redis.toml

Next Steps

• Explore CLI Reference for all available options
• Learn about Configuration file format
• Check out example profiles in the profiles/ directory

CLI Reference

Complete reference for Xylem command-line interface.

Xylem uses a config-first design with TOML profile files. This ensures reproducibility and simplifies complex workload specifications.

Basic Usage

xylem -P tests/redis/redis-get-zipfian.toml

Global Options

--version

Display version information.

xylem --version

--help

Display help information.

xylem --help

-P, --profile <FILE>

Path to TOML profile configuration file.

xylem -P tests/redis/redis-bench.toml

--set <KEY=VALUE>

Override any configuration value using dot notation. Can be specified multiple times.

xylem -P profiles/redis.toml --set target.address=192.168.1.100:6379
xylem -P profiles/http.toml --set experiment.duration=120s --set experiment.seed=12345

Examples:

• --set target.address=127.0.0.1:6379
• --set experiment.duration=60s
• --set experiment.seed=999
• --set target.protocol=memcached-binary
• --set workload.keys.n=1000000
• --set traffic_groups.0.threads=[0,1,2,3]
• --set output.file=/tmp/results.json

-l, --log-level <LEVEL>

Set log level (trace, debug, info, warn, error).

Default: info

xylem -P profiles/redis.toml --log-level debug

Subcommands

completions <SHELL>

Generate shell completion scripts.

Supported shells:

• bash
• zsh
• fish
• powershell
• elvish

Examples:

# Bash
xylem completions bash > ~/.local/share/bash-completion/completions/xylem

# Zsh
xylem completions zsh > ~/.zsh/completions/_xylem

# Fish
xylem completions fish > ~/.config/fish/completions/xylem.fish

schema

Generate JSON Schema for configuration files.

xylem schema > config-schema.json

Configuration Overrides

The --set flag uses dot notation to override any configuration value:

# Override target address
xylem -P profiles/redis.toml --set target.address=localhost:6379

# Override experiment parameters
xylem -P profiles/bench.toml --set experiment.duration=300s --set experiment.seed=42

# Override workload settings
xylem -P profiles/redis.toml --set workload.keys.n=1000000

# Override traffic group settings
xylem -P profiles/multi.toml --set traffic_groups.0.threads=[0,1,2,3]

# Add new traffic group
xylem -P profiles/base.toml --set 'traffic_groups.+={name="new-group",threads=[4,5]}'

Profile Files

Xylem requires a TOML profile file that defines the experiment configuration. See the profiles/ directory for example configurations.

Example profile structure:

[experiment]
duration = "60s"
seed = 123

[target]
protocol = "redis"
address = "127.0.0.1:6379"

[workload]
# Workload configuration...

[[traffic_groups]]
name = "group-1"
threads = [0, 1, 2, 3]

Environment Variables

Xylem respects the following environment variables:

• RUST_LOG - Set logging level (e.g., debug, info, warn, error)

RUST_LOG=debug xylem -P profiles/redis.toml

See Also

• Configuration - Detailed configuration file format
• Configuration Schema - Configuration schema reference

Configuration

Xylem uses TOML configuration files (called “profiles”) for defining experiments.

Configuration File Structure

A profile is a TOML document with the following top-level sections:

[experiment]
duration = "60s"
seed = 123

[target]
protocol = "redis"
address = "127.0.0.1:6379"

[workload]
# Workload parameters

[[traffic_groups]]
name = "main"
threads = [0, 1, 2, 3]
# Rate control parameters

Sections

The configuration is divided into several sections:

• Workload Configuration - Define workload patterns, duration, and key distributions
• Transport Configuration - Configure network transport options
• Protocol Configuration - Protocol-specific settings

Basic Example

[experiment]
duration = "60s"
seed = 42

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

[workload.keys]
strategy = "zipfian"
n = 10000
theta = 0.99
value_size = 100

[[traffic_groups]]
name = "group-1"
threads = [0, 1, 2, 3]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "closed-loop"

[output]
format = "json"
file = "results.json"

Configuration Overrides

You can override any configuration value using the --set flag with dot notation:

# Override target address
xylem -P profile.toml --set target.address=192.168.1.100:6379

# Override experiment duration
xylem -P profile.toml --set experiment.duration=120s

# Override multiple parameters
xylem -P profile.toml --set experiment.duration=60s --set experiment.seed=999

Loading Configuration

Use the -P or --profile flag to load a profile file:

xylem -P tests/redis/redis-get-zipfian.toml

See Also

• Workload Configuration
• Transport Configuration
• Protocol Configuration
• Configuration Schema

Workload Configuration

The workload configuration is where you define how Xylem generates load against your system. This is where theory meets practice - where you translate your understanding of production traffic into a reproducible benchmark. A well-crafted workload configuration captures the essential characteristics of real user behavior: which keys get accessed, how often different operations occur, and how data sizes vary across your application.

The Structure of a Workload

Every workload in Xylem consists of three fundamental components that work together to simulate realistic traffic:

Key distribution determines which keys your benchmark accesses. Will you hit every key equally (uniform random), focus on a hot set (Zipfian), or model temporal locality where recent keys are hot (Gaussian)? This choice fundamentally shapes your cache hit rates and system behavior.

Operations configuration controls what your benchmark does with those keys. Real applications don’t just perform one operation - they mix reads and writes, batch operations, and specialized commands in patterns that reflect business logic. You can model a read-heavy cache (90% GET), a balanced session store (70% GET, 30% SET), or even custom workloads using any Redis command.

Value sizes determine how much data each operation handles. Production systems rarely use fixed-size values - small cache keys, medium session data, large objects all coexist. Xylem lets you model this complexity, even configuring different sizes for different operations to match your actual data patterns.

Together, these three components create a workload that mirrors reality. Let’s explore each in depth.

Understanding Key Distributions

The way your application accesses keys has profound implications for performance. A cache with perfect hit rates under sequential access might struggle with random patterns. A system optimized for uniform load might behave differently when faced with hot keys. Xylem provides several distribution strategies, each modeling distinct real-world patterns.

Sequential Keys: The Baseline

Sequential access is the simplest pattern - keys are accessed in order:

[workload.keys]
strategy = "sequential"
start = 0
max = 100000
value_size = 128

Every benchmark accesses keys 0, 1, 2, and so on up to 99,999. This pattern represents the best-case scenario for most systems: perfect predictability, excellent cache locality, and minimal memory pressure. While unrealistic for production, sequential access gives you a performance ceiling - the best your system can possibly do.

Use sequential keys when establishing baseline performance, comparing different configurations, or debugging issues. The predictability makes problems easier to isolate.

Uniform Random: Pure Chaos

At the opposite extreme, uniform random access treats every key equally:

[workload.keys]
strategy = "random"
max = 1000000
value_size = 128

Each request has an equal chance of accessing any key from 0 to 999,999. This represents the worst case for caching: no locality, maximum memory pressure, and frequent cache misses. If sequential access shows you your ceiling, random access shows you your floor.

Random distributions are perfect for stress testing. They reveal capacity limits, expose memory management issues, and help you understand behavior under the worst possible access patterns. Use this when you need to establish how your system degrades under pressure.

Round-Robin: Predictable Cycling

Round-robin access cycles through keys in order, wrapping back to the beginning:

[workload.keys]
strategy = "round-robin"
max = 100000
value_size = 128

This pattern accesses keys 0, 1, 2, … 99,999, then immediately jumps back to 0. It’s less common in production but useful when you need predictable, repeating patterns - perhaps for testing cache warm-up behavior or validating eviction policies.

Zipfian Distribution: The Power Law

Real-world access patterns rarely distribute evenly. Some keys - popular content, active user accounts, frequently accessed configuration - receive far more traffic than others. Zipfian distribution models this fundamental characteristic of production systems:

[workload.keys]
strategy = "zipfian"
exponent = 0.99
max = 1000000
value_size = 128

The exponent controls how skewed the distribution is. At 0.99 (typical for real workloads), the pattern is dramatically skewed: the top 1% of keys might receive 80% of requests, while the bottom 50% collectively receive only a few percent. This mirrors what you see in production - think of popular YouTube videos, trending tweets, or frequently accessed products.

Zipfian is your go-to distribution for realistic cache testing. It reveals how well your system handles hot keys, how effective your caching strategy is, and whether you can maintain performance when traffic concentrates on a small key set. The pattern remains stable over time - if key 1000 is popular now, it stays popular throughout the benchmark.

Gaussian Distribution: Temporal Locality

Sometimes hotness isn’t about persistent popularity - it’s about recency. Session stores, time-series databases, and log aggregators often exhibit temporal locality: recent data is hot, older data is cold. Gaussian distribution captures this pattern beautifully:

[workload.keys]
strategy = "gaussian"
mean_pct = 0.5      # Center of the hot zone
std_dev_pct = 0.1   # How concentrated
max = 10000
value_size = 128

The percentages make this intuitive. With mean_pct = 0.5, the hot spot centers at 50% of your keyspace - key 5000 in this example. The std_dev_pct = 0.1 means the standard deviation is 10% of the keyspace (1000 keys). Using the standard rules for normal distributions:

• Roughly 68% of accesses hit the center ±1000 keys (4000-6000)
• About 95% hit the center ±2000 keys (3000-7000)
• Nearly all requests fall within ±3000 keys (2000-8000)

This creates a “warm zone” that moves through your keyspace. It’s perfect for modeling recently written data being immediately read (think recent log entries), active user sessions clustering in a time window, or rolling analytics windows where you repeatedly access recent periods.

The key difference from Zipfian: Gaussian models temporal hotness (recent data is hot), while Zipfian models persistent hotness (popular data stays popular). Choose based on whether your hot set is stable (Zipfian) or shifts over time (Gaussian).

Configuring Operations: What Your Benchmark Does

Real applications don’t perform just one operation. They mix reads and writes in patterns that reflect business logic. A content delivery network might be 95% reads, while a logging system might be 80% writes. Xylem lets you model these patterns precisely.

Single Operation for Simplicity

When you’re establishing baselines or focusing on a specific operation’s performance, use a fixed operation:

[workload.operations]
strategy = "fixed"
operation = "get"

This configuration generates only GET operations. It’s simple, predictable, and useful when you want to isolate and measure one aspect of your system. Change to "set" for write testing, or "incr" for counter workloads.

Weighted Operations for Realism

Production systems mix operations in characteristic ratios. Model these patterns with weighted configurations:

[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7  # 70% reads

[[workload.operations.commands]]
name = "set"
weight = 0.3  # 30% writes

The weights define the probability of each operation. Xylem normalizes them automatically, so weight = 0.7 and weight = 70 work identically. Choose whatever feels natural - percentages often read more clearly.

Common patterns you’ll encounter:

Heavy cache workloads (CDN, API response caching):

GET: 0.9-0.95, SET: 0.05-0.1

Session stores (authentication, user state):

GET: 0.7, SET: 0.3

Counter systems (rate limiting, metrics):

GET: 0.5, INCR: 0.5

Write-heavy systems (logging, event streams):

GET: 0.2, SET: 0.8

Batch Operations with MGET

Many applications fetch multiple keys atomically for efficiency. Model this with MGET operations:

[[workload.operations.commands]]
name = "mget"
weight = 0.2
count = 10  # Ten keys per operation

Each MGET request fetches 10 consecutive keys, simulating how your application might retrieve a user’s session along with related data, or fetch a time series of recent measurements. This helps you understand pipeline efficiency and batch operation performance.

Testing Replication with WAIT

Running Redis with replication? The WAIT command helps you measure replication lag under load:

[[workload.operations.commands]]
name = "wait"
weight = 0.1
num_replicas = 2      # Wait for 2 replicas
timeout_ms = 1000     # Give up after 1 second

Include WAIT in your mix to understand consistency guarantees. How often does replication keep up? What’s the latency cost of waiting for replicas? WAIT operations answer these questions.

Custom Commands for Advanced Use Cases

Redis supports hundreds of commands for lists, sets, sorted sets, hashes, and more. Custom templates let you benchmark any operation:

[[workload.operations.commands]]
name = "custom"
weight = 0.1
template = "HSET sessions:__key__ data __data__"

The template system provides three placeholders:

• __key__ - Current key from your distribution
• __data__ - Random bytes sized per your value_size config
• __value_size__ - Numeric size, useful for scores or counts

Examples that model real workloads:

Add timestamped events to a sorted set:

template = "ZADD events __value_size__ event:__key__"

Update session data with TTL:

template = "SETEX session:__key__ 1800 __data__"

Push to an activity log:

template = "LPUSH user:__key__:activity __data__"

Track active users:

template = "SADD active_now user:__key__"

Modeling Value Size Variation

Real data doesn’t come in uniform chunks. Cache keys might be 64 bytes, session tokens 256 bytes, and user profiles 2KB. Different operations handle different sizes. Xylem gives you sophisticated tools to model this reality.

Fixed Sizes: The Known Quantity

When every value has the same size - or when you want consistency for benchmarking - use fixed sizes:

[workload.value_size]
strategy = "fixed"
size = 128

Every operation uses exactly 128 bytes. This is your baseline: consistent, repeatable, and predictable. Use fixed sizes when comparing configurations or establishing capacity limits.

Uniform Distribution: Exploring a Range

Don’t know your exact size distribution but want to test across a range? Uniform distribution gives every size equal probability:

[workload.value_size]
strategy = "uniform"
min = 64
max = 4096

Requests will use sizes anywhere from 64 bytes to 4KB, spread evenly. A request might use 64 bytes, the next 3876 bytes, another 512 bytes - every size in the range has equal probability. This is useful for stress testing memory allocation and fragmentation.

Normal Distribution: Natural Clustering

Most real data clusters around a typical size. Session data averages 512 bytes with variation. Cache entries typically run 256 bytes but occasionally spike. Normal distribution models this natural clustering:

[workload.value_size]
strategy = "normal"
mean = 512.0
std_dev = 128.0
min = 64      # Floor
max = 4096    # Ceiling

The mean (512 bytes) is your typical value size. The standard deviation (128 bytes) controls spread. Using standard normal distribution rules:

• 68% of values fall within 384-640 bytes (±1σ)
• 95% fall within 256-768 bytes (±2σ)
• 99.7% fall within 128-896 bytes (±3σ)

The min/max bounds prevent extremes. Without them, the normal distribution could occasionally generate tiny (2-byte) or huge (50KB) values. The clamping keeps values practical while preserving the clustering behavior.

Per-Command Sizes: Maximum Realism

Your most powerful tool is configuring sizes per operation type. Reads often fetch small keys or IDs, while writes store larger data:

[workload.value_size]
strategy = "per_command"
default = { strategy = "fixed", size = 256 }

[workload.value_size.commands.get]
distribution = "fixed"
size = 64  # GETs fetch small cache keys

[workload.value_size.commands.set]
distribution = "uniform"
min = 128
max = 1024  # SETs write varied data

Notice the terminology shift: per-command configs use distribution to distinguish them from the top-level strategy. Each command can have its own distribution with its own parameters.

The default configuration handles operations you haven’t explicitly configured. If your workload includes INCR but you haven’t specified a size, it uses the default.

Real-world examples:

Session store (small IDs, larger session data):

GET: fixed 64 bytes
SET: normal mean=512, σ=128

Content cache (tiny keys, variable content):

GET: fixed 32 bytes
SET: uniform 1KB-100KB

Analytics (small queries, large result sets):

GET: normal mean=2KB, σ=512
SET: uniform 10KB-1MB

Time-Varying Load Patterns

Beyond key distributions, operations, and sizes, you can vary the overall request rate over time. This is your macro-level control, letting you model daily cycles, traffic spikes, or gradual ramp-ups.

Constant Load

The simplest pattern maintains steady throughput:

[workload.pattern]
type = "constant"
rate = 10000.0  # Requests per second

Use constant patterns for baseline testing and capacity planning.

Gradual Ramp-Up

Simulate scaling up or warming caches:

[workload.pattern]
type = "ramp"
start_rate = 1000.0
end_rate = 10000.0

Traffic increases linearly from 1K to 10K requests per second over your experiment duration.

Traffic Spikes

Model sudden load increases:

[workload.pattern]
type = "spike"
base_rate = 5000.0
spike_rate = 20000.0
spike_start = "10s"
spike_duration = "5s"

Normal traffic at 5K RPS suddenly jumps to 20K RPS at the 10-second mark, holds for 5 seconds, then returns to baseline. Perfect for testing autoscaling, circuit breakers, and graceful degradation.

Cyclical Patterns

Model daily traffic cycles:

[workload.pattern]
type = "sinusoidal"
min_rate = 5000.0
max_rate = 15000.0
period = "60s"

Traffic oscillates smoothly between 5K and 15K RPS, completing one full cycle every 60 seconds.

Bringing It All Together

Let’s build a complete workload that demonstrates these concepts. Imagine you’re testing a session store that exhibits temporal locality (recent sessions are hot), mixed operations (reads dominate but writes are significant), and varied data sizes (small lookups, larger session data):

[workload]
# Temporal locality - recently created sessions get most traffic
[workload.keys]
strategy = "gaussian"
mean_pct = 0.6       # Hot spot toward recent sessions
std_dev_pct = 0.15   # Moderately concentrated
max = 10000
value_size = 512

# Read-heavy but significant writes
[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7

[[workload.operations.commands]]
name = "set"
weight = 0.3

# Small reads (session ID lookups), varied writes (full session data)
[workload.value_size]
strategy = "per_command"
default = { strategy = "fixed", size = 256 }

[workload.value_size.commands.get]
distribution = "fixed"
size = 64

[workload.value_size.commands.set]
distribution = "uniform"
min = 128
max = 1024

# Constant baseline load
[workload.pattern]
type = "constant"
rate = 10000.0

This configuration captures essential characteristics: temporal hotness (Gaussian keys), realistic operation mix (70/30 read-write), and varied sizes matching actual data patterns. Running this gives you insights you can trust.

Practical Wisdom

Ensuring Reproducible Results

Randomness is essential for realistic workloads, but you need reproducible results. Set a seed:

[experiment]
seed = 42  # Any consistent value works

With the same seed, Xylem generates identical sequences of keys and sizes across runs. You can share configurations with colleagues knowing they’ll see exactly the same workload pattern.

Choosing Distributions Wisely

Your distribution choice fundamentally shapes results:

Gaussian when you have temporal locality:

• Session stores (recent sessions hot)
• Time-series databases (recent data accessed)
• Rolling analytics (current window active)

Zipfian when you have persistent hot keys:

• Content caching (popular items stay popular)
• User data (power users dominate traffic)
• E-commerce (bestsellers get most views)

Random when you need worst-case analysis:

• Capacity planning (maximum memory pressure)
• Stress testing (minimum cache effectiveness)
• Establishing performance floors

Sequential when you need predictability:

• Baseline measurements
• Configuration comparison
• Debugging and development

Matching Real Traffic Patterns

Study your production metrics before configuring:

1. Measure your read-write ratio from application logs
2. Sample actual value sizes from your database
3. Identify hot keys through cache hit rate analysis
4. Understand temporal patterns in your access logs

Then translate these observations into configuration. A workload that mirrors production gives you results you can trust.

Starting Simple, Adding Complexity

Begin with simple configurations:

• Fixed sizes
• Single operation type
• Sequential keys

Establish baselines, then add complexity:

• Add operation variety
• Introduce size variation
• Switch to realistic distributions

Each addition reveals new insights. Compare results to understand what each aspect contributes.

See Also

• Redis Protocol - Detailed Redis operation guide
• Transport Configuration - Network settings
• Protocol Configuration - Protocol selection
• Configuration Schema - Complete reference

Transport Configuration

Transport is explicitly specified in the [target] section along with the connection address.

Available Transports

• tcp - TCP/IP connections
• udp - UDP connections
• unix - Unix domain sockets

Configuration

TCP (hostname:port)

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

Unix Domain Socket (filesystem path)

[target]
protocol = "redis"
transport = "unix"
address = "/var/run/redis/redis.sock"

UDP (hostname:port)

UDP support depends on the protocol implementation.

[target]
protocol = "memcached-binary"
transport = "udp"
address = "localhost:11211"

See the Architecture - Transport Layer for implementation details.

Protocol Configuration

Configure the protocol in the [target] section of your TOML profile.

Specifying Protocol

[target]
protocol = "redis"  # or "http", "memcached-binary", "memcached-ascii", "masstree", "xylem-echo"
address = "localhost:6379"

Supported Protocols

• redis - Redis Serialization Protocol (RESP)
• http - HTTP/1.1
• memcached-binary - Memcached binary protocol
• memcached-ascii - Memcached text protocol
• masstree - Masstree protocol
• xylem-echo - Echo protocol (testing/development only)

See the Architecture - Protocol Layer for implementation details.

Protocols

Xylem supports multiple application protocols for different types of RPC workloads.

Supported Protocols

• Redis - Redis protocol for key-value operations
• HTTP - HTTP/1.1 protocol for web services
• Memcached - Memcached protocol for caching

Protocol Selection

Specify the protocol in your TOML profile configuration:

[target]
protocol = "redis"
address = "localhost:6379"

Protocol Architecture

All protocols in Xylem implement a common interface that provides:

1. Request Generation - Convert logical operations to wire format
2. Response Parsing - Parse responses from the server
3. State Management - Track request-response correlation
4. Error Handling - Detect and report protocol errors

See Also

• Configuration

Benchmarking Redis with Xylem

When you’re benchmarking Redis, you’re speaking RESP—the Redis Serialization Protocol. It’s the language Redis uses to understand your commands and send back responses. This guide walks you through how RESP works, what commands Xylem knows how to speak, and how to craft realistic Redis workloads for your benchmarks.

Understanding RESP

RESP is elegantly simple. Commands look almost like what you’d type at a Redis CLI, but they’re formatted in a way that machines can parse efficiently. Everything is text-based—printable ASCII with CRLF line endings—which makes debugging easy. You can literally read RESP messages in a network capture.

But don’t let the text format fool you. RESP is binary-safe. When you send a value, you tell Redis exactly how many bytes to expect, so you can store JPEGs, protocol buffers, or any binary blob without escaping special characters. Redis doesn’t care what’s in your data; it just reads the exact number of bytes you promised and sends it back when you ask for it.

The communication pattern is straightforward: you send a command, Redis sends back a response. One command, one response. But here’s the clever part: you don’t have to wait for the response before sending the next command. This is called pipelining, and it’s how you get serious throughput. Send a batch of GET commands back-to-back, and Redis will send back a batch of responses in the same order. No request IDs needed—FIFO ordering per connection keeps everything straight.

Each command stands alone. Redis doesn’t maintain conversation state (except for transactions and pub/sub, which are special cases). This statelessness is what makes Redis fast and scalable.

The Wire Format

RESP speaks in five types, each starting with a special character:

Simple strings (+) are for short status messages. When you SET a key, Redis responds with +OK\r\n. That’s it—a plus sign, the message, and a newline.

Errors (-) look similar but start with a minus. -ERR unknown command\r\n tells you something went wrong. The distinction between simple strings and errors lets clients handle success and failure differently without parsing the message content.

Integers (:) are for numeric results. When you INCR a counter, Redis might send back :42\r\n—that’s the new count. The colon says “interpret what follows as a number.”

Bulk strings ($) are where it gets interesting. These can hold any binary data of any size. The format is $<length>\r\n<data>\r\n. So $5\r\nhello\r\n is a 5-byte string containing “hello”. The length-prefix is what makes RESP binary-safe—Redis knows exactly how many bytes to read, regardless of what’s in them. A null value is $-1\r\n, which is how Redis says “this key doesn’t exist.”

Arrays (*) let you send or receive multiple values. The format is *<count>\r\n followed by that many elements. Each element can be any RESP type—strings, integers, even nested arrays. This is how commands work: GET key:1234 becomes *2\r\n$3\r\nGET\r\n$8\r\nkey:1234\r\n—an array of two bulk strings.

RESP3 adds more types: true nulls (_\r\n), booleans (#t\r\n or #f\r\n), doubles for floating-point numbers (,3.14\r\n), maps (%<count>\r\n) for key-value pairs, sets (~<count>\r\n) for unique collections, and a few others. These semantic types make client libraries simpler because they don’t have to guess whether a bulk string is a number or text.

Commands on the Wire

Every Redis command is an array of bulk strings. Even simple ones. GET key:1234 becomes:

*2\r\n           # Array with 2 elements
$3\r\nGET\r\n    # First element: "GET" (3 bytes)
$8\r\nkey:1234\r\n  # Second element: "key:1234" (8 bytes)

SET looks similar but with three elements:

*3\r\n                    # Array with 3 elements
$3\r\nSET\r\n            # Command
$8\r\nkey:1234\r\n       # Key
$128\r\n<128 bytes>\r\n   # Value

Responses match what the command returns. A successful GET sends back the value as a bulk string. A missing key sends $-1\r\n. SET responds with +OK\r\n. INCR returns an integer like :43\r\n.

Pipelining: The Secret to Throughput

Here’s where Redis shines. Instead of this:

1. Send GET key1
2. Wait for response
3. Send GET key2
4. Wait for response

You do this:

1. Send GET key1, GET key2, GET key3
2. Read three responses

The latency savings are dramatic. If each round-trip takes 1ms, the first approach takes 2ms for two GETs. Pipelining takes just over 1ms total. With 10 keys, you go from 10ms to ~1ms. The more you pipeline, the closer you get to wire speed.

Redis guarantees responses come back in the same order you sent requests, per connection. You don’t need to tag requests with IDs—just keep track of how many you’ve sent and match them up with responses as they arrive. This is what Xylem does automatically with the max_pending_per_connection setting.

The Commands

Let’s talk about what these commands actually do and when you’d use them in benchmarks.

GET retrieves a value. It’s your bread-and-butter read operation. Benchmark GET to measure read latency, cache hit rates, and how Redis handles concurrent readers. When the key doesn’t exist, you get back a null ($-1\r\n).

SET stores a value. This is your write operation. Redis overwrites any existing value, so SET is idempotent—run it twice and you get the same result. Benchmark SET to test write throughput, persistence overhead (if you have AOF or RDB enabled), and replication lag.

SETEX combines SET with an expiration time. SETEX session:abc123 1800 <data> stores the session and tells Redis to delete it in 30 minutes. This is atomic—no race between SET and EXPIRE. Use it to benchmark TTL accuracy, eviction behavior, and how expiring keys affect memory management.

INCR atomically increments a counter. If the key doesn’t exist, Redis treats it as zero and increments to one. This is crucial for testing atomic operations, counters (like rate limiters or page view trackers), and how Redis handles numeric operations under concurrency.

MGET fetches multiple keys in one command: MGET key:1 key:2 key:3. The response is an array of values (or nulls for missing keys). Use this to benchmark batch operations and see if your network round-trip time dominates your latency.

WAIT blocks until previous writes replicate to N replicas or a timeout expires: WAIT 2 1000 waits for 2 replicas with a 1-second timeout. The response is how many replicas acknowledged. This lets you benchmark replication lag and consistency guarantees.

SETRANGE and GETRANGE work on offsets within a value. SETRANGE key:1000 5 hello overwrites 5 bytes starting at offset 5. GETRANGE key:1000 5 -1 reads from offset 5 to the end. These are useful for large-value workloads where you only modify or read part of the data—think partial updates to binary structures or reading substrings from log entries.

AUTH authenticates your connection. With simple password auth, it’s AUTH mypassword. With Redis 6.0+ ACL, it’s AUTH username password. You’ll need this to benchmark production-like setups where Redis isn’t publicly accessible.

SELECT switches databases. Redis has 16 logical databases by default (numbered 0-15). SELECT 5 moves to database 5. Keys in different databases are isolated. Use this to test multi-tenant scenarios or benchmark how database switching affects performance.

HELLO negotiates the protocol version. HELLO 2 sticks with RESP2. HELLO 3 upgrades to RESP3. The response includes server version, supported features, and other metadata. Benchmark this if you care about RESP3 adoption or client compatibility.

CLUSTER SLOTS returns the cluster topology: which slots are assigned to which nodes. The response is a nested array with start slot, end slot, and node addresses for each range. Use this to benchmark topology discovery and test how clients handle cluster reconfiguration.

Custom Commands via Templates

Xylem’s command templates are your escape hatch for anything not built-in. Want to test sorted sets? Write:

[[workload.operations.commands]]
name = "custom"
template = "ZADD leaderboard __value_size__ player:__key__"

Xylem replaces __key__ with the current key, __value_size__ with the value size, and __data__ with generated data. The template becomes a proper RESP array. You can benchmark HSET, LPUSH, SADD, GEOADD—anything Redis supports.

Building Realistic Workloads

Real applications don’t just GET and SET uniformly distributed keys. They have patterns: hot keys, read-heavy ratios, varied value sizes, bursts of traffic. Xylem lets you model all of this.

Command Mixes

Start simple. A pure GET workload tests read performance:

[workload.operations]
strategy = "fixed"
operation = "get"

But production is rarely pure reads. Model a cache with 70% reads and 30% writes:

[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7

[[workload.operations.commands]]
name = "set"
weight = 0.3

Xylem picks commands randomly based on their weights. Over time, you’ll see 70% GETs and 30% SETs.

Add INCR for counters, MGET for batch reads, and WAIT to test replication:

[[workload.operations.commands]]
name = "incr"
weight = 0.1

[[workload.operations.commands]]
name = "mget"
weight = 0.05
count = 10  # Fetch 10 keys per MGET

[[workload.operations.commands]]
name = "wait"
weight = 0.02
num_replicas = 2
timeout_ms = 1000

Now you’re testing a realistic mix: mostly GETs, some SETs, occasional counter increments, batch reads, and periodic replication checks.

Key Distributions

Uniform random keys are easy to understand but unrealistic. Real workloads have hot keys—some data gets accessed way more than others.

Zipfian distribution models this. With a Zipfian exponent of 0.99, a tiny fraction of keys get most of the traffic:

[workload.keys]
strategy = "zipfian"
exponent = 0.99
max = 1000000

This is perfect for CDN caching (some content is always popular), user sessions (active users generate more requests), or product catalogs (bestsellers dominate).

Gaussian (normal) distribution clusters keys around a mean. This models temporal locality—recent data is hot, older data cools off:

[workload.keys]
strategy = "gaussian"
mean_pct = 0.5  # Center at 50% of key space
std_dev_pct = 0.1  # 10% spread
max = 10000

Use this for time-series data, sliding windows, or any scenario where “recent” matters.

Sequential access walks through keys in order. Good for range scans or import/export workloads:

[workload.keys]
strategy = "sequential"
start = 0
max = 100000

Random is truly uniform. Every key has equal probability. This is your baseline for understanding Redis’s raw performance without hot keys skewing results.

Value Sizes

Fixed-size values are clean for benchmarking:

[workload.value_size]
strategy = "fixed"
size = 128  # All values are 128 bytes

But production has variation. Some cache entries are tiny (user preferences), others are huge (HTML fragments). Model this with distributions:

[workload.value_size]
strategy = "normal"
mean = 512.0
std_dev = 128.0
min = 64
max = 4096

Or make value sizes command-specific. Small GETs, large SETs:

[workload.value_size]
strategy = "per_command"

[workload.value_size.commands.get]
distribution = "fixed"
size = 64

[workload.value_size.commands.set]
distribution = "uniform"
min = 256
max = 2048

This models scenarios like caching API responses (small reads) and storing rendered pages (large writes).

Cluster Benchmarking

When you’re testing Redis Cluster, you’re really testing how well your client handles distributed systems. Redis Cluster shards data across nodes using hash slots. There are 16,384 slots, and every key hashes to one of them. Each node owns a range of slots. When you send a command to the wrong node, Redis tells you where to go with a MOVED or ASK redirect.

Xylem handles this for you. You tell it about your cluster topology—which nodes own which slots—and it routes every request to the right node from the start. No trial and error, no unnecessary redirects.

Here’s the setup:

#![allow(unused)]
fn main() {
use xylem_protocols::*;

// Create a cluster-aware protocol
let selector = Box::new(FixedCommandSelector::new(RedisOp::Get));
let mut protocol = RedisClusterProtocol::new(selector);

// Register your nodes
protocol.register_connection("127.0.0.1:7000".parse()?, 0);
protocol.register_connection("127.0.0.1:7001".parse()?, 1);
protocol.register_connection("127.0.0.1:7002".parse()?, 2);

// Define the topology
let mut ranges = vec![];
for i in 0..3 {
    let (start, end) = calculate_slot_range(i, 3)?;
    let master = format!("127.0.0.1:{}", 7000 + i).parse()?;
    ranges.push(SlotRange { start, end, master, replicas: vec![] });
}
protocol.update_topology(ClusterTopology::from_slot_ranges(ranges));

// Now every request goes to the right node
let (request, req_id) = protocol.generate_request(0, key, value_size);
}

The slot calculation is CRC16-XMODEM of the key. Xylem does a binary search over slot ranges to find the owner—O(log N) in the number of ranges, but with typical cluster sizes (3-10 nodes), this is effectively free.

Hash Tags for Multi-Key Operations

Sometimes you need multiple keys on the same node. MGET won’t work across nodes—Redis can’t fetch {key:1, key:2, key:3} if they’re on different shards. Hash tags solve this.

When you use curly braces in a key name, Redis only hashes what’s inside the braces:

#![allow(unused)]
fn main() {
let key1 = "{user:1000}.profile";
let key2 = "{user:1000}.settings";
let key3 = "{user:1000}.preferences";
}

All three hash to the same slot because they share {user:1000}. Now you can MGET them together, or use MULTI/EXEC for atomic multi-key operations.

Handling Redirects

Even with perfect topology knowledge, clusters reshard. Slots migrate between nodes during scale-up, rebalancing, or failover. When you hit a moving slot, Redis sends a redirect.

MOVED means the slot permanently moved. Update your topology and retry. MOVED is a signal to refresh your cluster map.

ASK means the slot is temporarily migrating. Retry on the new node (with an ASKING command first), but don’t update your topology—the migration might not complete.

Xylem detects both. It parses the redirect response, extracts the target node address, and returns this info so you can decide: update topology, retry immediately, log the event, whatever makes sense for your benchmark.

Common Workload Patterns

Let me paint some pictures of real-world workloads and how to model them.

CDN Edge Cache: 95% GETs, 5% SETs. Zipfian key distribution (popular content dominates). Small reads (64B for metadata), medium writes (512B average, normal distribution for HTML snippets and JSON). High throughput, moderate pipelining. Watch for cache hit rates and P99 latency on those few misses that go to origin.

Session Store: 70% GETs, 30% SETs. Gaussian distribution (recent sessions are hot). Small uniform values (256-512B for session tokens and user state). Low latency is critical—sessions are in the critical path. Use SETEX with realistic TTLs (30-60 minutes) and measure how eviction affects memory and performance.

Rate Limiter: 50% GET (check current count), 50% INCR (increment count). Random or round-robin keys (different users/IPs). Tiny values (just counters). Extremely high throughput, short TTLs (60 seconds for sliding windows). Focus on atomic operation latency and throughput under contention.

Write-Heavy Logging: 80% SET, 20% GET. Sequential keys (time-ordered logs). Uniform value sizes (log entry length is predictable). High write throughput with pipelining. Test persistence overhead (AOF fsync), replication lag, and memory growth.

E-Commerce Product Catalog: 90% GET (browsing), 10% SET (inventory updates). Zipfian (bestsellers are hot). Medium values (1-4KB for product details with images). Add MGET for “customers who bought this also bought” features. Benchmark cache stampede behavior when popular items update.

Wrapping Up

Redis benchmarking is about understanding your workload and translating it into RESP commands. Xylem gives you the tools: command mixes, key distributions, value sizes, pipelining control, cluster routing. The trick is using them to model what your production system actually does.

Start simple. Benchmark pure GETs and SETs with uniform keys to understand Redis’s baseline. Then add complexity: hot keys with Zipfian distribution, mixed command ratios, varied value sizes. Pipeline aggressively to see maximum throughput. Test failure modes: what happens during resharding, replication lag, eviction under memory pressure?

Your benchmarks should tell a story. Not just “Redis can do 100K QPS,” but “our session store can handle 10K users with P99 latency under 2ms, and we can lose a cluster node without dropping requests.” That’s the insight that lets you ship with confidence.

See Also

• Workload Configuration - Detailed configuration guide
• Configuration Schema - Complete reference
• Redis Documentation - Official Redis protocol documentation
• Redis Cluster Tutorial - Official cluster guide

HTTP Protocol

TODO: This section is under development. HTTP protocol documentation will be added once the configuration format is validated.

Overview

Xylem supports HTTP/1.1 protocol for benchmarking web services and APIs.

Supported Methods

• GET
• POST
• PUT
• DELETE
• HEAD
• PATCH

Check back later for detailed configuration examples and usage guidance.

Memcached Protocol

Note: This page is TODO. Configuration details need validation against actual implementation.

Benchmarking Masstree with Xylem

When you’re benchmarking Masstree, you’re speaking MessagePack over TCP—a binary serialization format that’s more compact than JSON but still type-safe. Masstree is a research database system designed for high-performance key-value storage with snapshot isolation and multi-version concurrency control. This guide covers the protocol mechanics, supported operations, and how to build workloads that test MVCC behavior and transactional semantics.

Understanding the Protocol

Masstree’s protocol is built on MessagePack, a binary format that’s more compact than text-based protocols like RESP. Everything is strongly typed—strings are strings, integers are integers, maps are maps. There’s no ambiguity about whether “42” is text or a number.

The communication pattern is straightforward: send a request array, get back a response array. Each request carries a sequence number that matches the response, enabling pipelining and response matching even when they arrive out of order (though Masstree typically preserves order per connection).

Masstree uses numeric command codes instead of text strings. GET is 2, REPLACE is 8, HANDSHAKE is 14. Responses use the command code plus one: GET response is 3, REPLACE response is 9. This makes parsing efficient without string comparisons.

Connection Handshake

Every connection must handshake before sending operations. This negotiates connection parameters: which CPU core the connection should use (for thread pinning), maximum key length, and protocol version.

The handshake request looks like:

#![allow(unused)]
fn main() {
[0, 14, {"core": -1, "maxkeylen": 255}]
}

That’s an array with three elements:

1. Sequence number 0 (handshakes always use seq 0)
2. Command code 14 (CMD_HANDSHAKE)
3. A map with parameters: core=-1 (auto-assign), maxkeylen=255

The server responds with:

#![allow(unused)]
fn main() {
[0, 15, true, 42, "Masstree-0.1"]
}

That’s:

1. Sequence number 0
2. Response code 15 (CMD_HANDSHAKE + 1)
3. Success boolean
4. Maximum sequence number (server tells you when to wrap around)
5. Server version string

Xylem handles this automatically for each connection. You never see the handshake—it just works.

Message Format

Every Masstree message is a MessagePack array. The structure depends on the command:

GET request: [seq, 2, key_string]

• Three elements: sequence number, command 2, the key

GET response: [seq, 3, value_string]

• Three elements: sequence number, response 3, the value (or error)

SET (REPLACE) request: [seq, 8, key_string, value_string]

• Four elements: sequence, command 8, key, value

SET (REPLACE) response: [seq, 9, result_code]

• Three elements: sequence, response 9, result (Inserted=1, Updated=2)

PUT request: [seq, 6, key_string, col_idx, value, col_idx, value, ...]

• Variable length: sequence, command 6, key, then pairs of column indices and values

REMOVE request: [seq, 10, key_string]

• Three elements: sequence, command 10, key

SCAN request: [seq, 4, firstkey_string, count, field_idx, field_idx, ...]

• Variable length: sequence, command 4, starting key, count, optional field indices

SCAN response: [seq, 5, key1, value1, key2, value2, ...]

• Variable length: sequence, response 5, then alternating keys and values

MessagePack handles the binary encoding. Strings are length-prefixed (so they’re binary-safe like RESP bulk strings), integers use the most compact representation (1-5 bytes depending on size), and arrays/maps have efficient headers. This makes the wire format more compact than RESP for numeric-heavy workloads.

Sequence Numbers and Pipelining

Each request has a 16-bit sequence number. The client picks the sequence, the server echoes it back. This enables pipelining—send multiple requests without waiting, then match responses by sequence number.

Xylem manages sequences per-connection automatically. It starts at 1 (0 is reserved for handshake), increments on each request, and wraps around at the server’s max (typically 2^16-1).

Unlike Redis where responses come back in FIFO order, Masstree can reorder responses since they’re tagged by sequence. Single-connection operations usually preserve order, but the protocol allows out-of-order completion for high concurrency scenarios.

The Operations

Each operation serves specific benchmarking purposes.

GET retrieves a value by key. It’s your basic read operation. Benchmark GET to measure read latency, snapshot isolation behavior, and how Masstree handles concurrent readers. If the key doesn’t exist, you get a NotFound result code (-2).

SET (REPLACE) stores or updates a value. Masstree calls it REPLACE because it replaces any existing value atomically. This isn’t an upsert in the SQL sense—it’s a blind write that doesn’t check the old value. Benchmark SET to test write throughput, MVCC overhead, and how concurrent writes interact with readers.

PUT updates specific columns within a value. Instead of reading the entire record, modifying a field, and writing it back (read-modify-write), you send only the column updates. For example, updating a user’s last login timestamp without touching their profile data. The format is [seq, 6, key, col0, val0, col1, val1, ...]. Use this to benchmark partial updates and column-based storage performance.

REMOVE deletes a key. Unlike some systems, Masstree removal is immediate (within the current snapshot)—it doesn’t wait for garbage collection. Benchmark this to test deletion throughput and how removed keys affect memory and performance.

SCAN performs range queries. You specify a starting key (“user:1000”), a count, and optionally which fields to retrieve. The response alternates keys and values: [seq, 5, "user:1000", {data}, "user:1001", {data}, ...]. This tests ordered access patterns like leaderboards, time-series queries, and scenarios where sorted iteration matters.

CHECKPOINT triggers a database snapshot. Masstree persists state by checkpointing to disk periodically. Calling CHECKPOINT explicitly forces this, which is useful for testing recovery time, checkpoint overhead, and consistency guarantees. The response is immediate, but the actual checkpoint happens asynchronously.

Result Codes

Masstree responses include result codes that tell you what happened:

• NotFound (-2): GET couldn’t find the key
• Retry (-1): Transient error, try again
• OutOfDate (0): Version conflict (rare in basic workloads)
• Inserted (1): SET created a new key
• Updated (2): SET replaced an existing value
• Found (3): GET succeeded
• ScanDone (4): SCAN completed

These codes matter for correctness testing. If you’re doing a read-your-writes test, you want to verify that SET returns Inserted or Updated, then GET returns Found with the same value.

Building Realistic Workloads

Real database workloads exhibit patterns: read-heavy vs write-heavy ratios, hot keys vs uniform access, point queries vs range scans. Xylem provides configuration options to model these patterns.

Operation Mixes

Start with pure operations. A GET-only workload tests read performance under snapshot isolation:

[protocol]
type = "masstree"
operation = "get"

But production databases handle mixed workloads. Model a typical application with 60% reads, 30% updates, 10% deletes:

[protocol]
type = "masstree"
operation = "get"  # Default operation

# Define a weighted mix
[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.6

[[workload.operations.commands]]
name = "set"
weight = 0.3

[[workload.operations.commands]]
name = "remove"
weight = 0.1

Add column-based updates to model partial record modifications:

[[workload.operations.commands]]
name = "put"
weight = 0.1
# Update columns 0 and 2 (e.g., timestamp and counter)
columns = [[0, "2024-11-12T10:30:00Z"], [2, "42"]]

Include range queries for analytics workloads:

[[workload.operations.commands]]
name = "scan"
weight = 0.05
firstkey = "user:0"
count = 100
fields = [0, 1]  # Return only fields 0 and 1

Trigger periodic checkpoints to test persistence overhead:

[[workload.operations.commands]]
name = "checkpoint"
weight = 0.001  # Once per thousand operations

Now you’re testing a realistic mix: mostly reads, some writes, occasional range scans, and periodic checkpoints.

Key Distributions

Uniform random keys rarely reflect production workloads. Real systems exhibit skew—some records are hot, others are cold.

Zipfian distribution models skewed access. With exponent 0.99, a small fraction of keys receive most of the traffic:

[workload.keys]
strategy = "zipfian"
exponent = 0.99
max = 1000000

This models user databases where active users dominate, e-commerce where bestsellers get most queries, or social networks where popular content drives traffic.

Gaussian (normal) distribution clusters keys around a center, modeling temporal locality where recent records are hot:

[workload.keys]
strategy = "gaussian"
mean_pct = 0.5  # Center of key space
std_dev_pct = 0.1
max = 10000

Use this for time-series data where recent timestamps matter, or any sliding-window workload.

Sequential access walks through keys in order. Perfect for testing SCAN performance or bulk imports:

[workload.keys]
strategy = "sequential"
start = 0
max = 100000

This is also how you test worst-case behavior in trees—sequential inserts can stress rebalancing.

Value Sizes

Fixed-size values simplify analysis:

[workload.value_size]
strategy = "fixed"
size = 256  # All values are 256 bytes

But real databases have variation. Some records are small (user preferences), others are large (document content):

[workload.value_size]
strategy = "normal"
mean = 1024.0
std_dev = 256.0
min = 128
max = 8192

Or make sizes operation-specific. Small GETs (hot metadata), large SETs (batch updates):

[workload.value_size]
strategy = "per_command"

[workload.value_size.commands.get]
distribution = "fixed"
size = 128

[workload.value_size.commands.set]
distribution = "uniform"
min = 512
max = 4096

This models caching patterns where reads are metadata lookups and writes are full document stores.

MVCC and Snapshot Isolation

Masstree’s key differentiator is MVCC. Every read sees a consistent snapshot—a point-in-time view of the database. Even if concurrent transactions are modifying data, reads see the version that existed when the transaction started.

GET operations never block on writes, and writes never block reads. This provides true concurrency without read locks. The tradeoff is version maintenance: Masstree maintains multiple versions of each value, and garbage collection cleans up old versions periodically.

Benchmarking MVCC Workloads

To test snapshot isolation, run concurrent readers and writers on the same keys:

[runtime]
num_threads = 16
connections_per_thread = 10

[workload.operations]
strategy = "weighted"

[[workload.operations.commands]]
name = "get"
weight = 0.7

[[workload.operations.commands]]
name = "set"
weight = 0.3

[workload.keys]
strategy = "zipfian"
exponent = 1.2  # Very hot keys
max = 1000

This creates contention on hot keys. Measure:

• GET latency distribution: Does P99 stay low even under write pressure?
• SET throughput: How many concurrent updates can Masstree handle?
• Version chain length: Are old versions accumulating?

You should see GET latencies remain stable even as SET rate increases, because readers don’t block. If P99 GET latency spikes, you might be hitting garbage collection pauses or version chain traversal overhead.

Read-Your-Writes Testing

Testing read-your-writes consistency verifies MVCC correctness. Write a value, then immediately read it back:

#![allow(unused)]
fn main() {
// Write
protocol.generate_request(conn_id, key, value_size);  // SET
// Read
protocol.generate_request(conn_id, key, 0);  // GET

// Verify response matches written value
}

With correct MVCC implementation, the GET always sees the SET’s value, even if other concurrent transactions are modifying the same key.

Common Workload Patterns

Here are typical workload patterns and how to model them.

User Session Store: 70% GET (check sessions), 20% SET (create/update), 10% REMOVE (logout/expire). Gaussian distribution (active users cluster), small uniform values (256-512B for session tokens). Use PUT to update just the “last_seen” timestamp without rewriting the entire session. Measure P99 latency and verify MVCC keeps reads fast during write spikes.

E-Commerce Inventory: 80% GET (check stock), 15% PUT (update counts), 5% SCAN (query ranges). Zipfian (popular products dominate), fixed value sizes (structured inventory records). Use PUT with column 0 for stock count. Test throughput under concurrent updates to the same SKU—MVCC should prevent read stalls.

Time-Series Metrics: 30% GET (recent values), 60% SET (new datapoints), 10% SCAN (range queries). Sequential or Gaussian keys (recent timestamps are hot), fixed sizes (metric tuples). High write rate with periodic SCAN to fetch time windows. Measure checkpoint overhead and verify SCAN performance with large result sets.

Analytics Query Engine: 20% GET (metadata), 10% SET (cache results), 70% SCAN (aggregations). Uniform distribution (analytics queries hit many keys), variable value sizes (small metadata, large aggregated results). Focus on SCAN throughput and latency, especially with large counts (1000+ records per SCAN).

Transactional Workload (YCSB-style): Mix of GET, SET, REMOVE with Zipfian distribution, hot keys creating contention. This tests MVCC’s core value proposition: can you sustain high throughput with strong isolation guarantees? Measure throughput vs latency tradeoff and compare with Redis’s single-threaded model.

Comparing Masstree to Redis

Choosing between Masstree and Redis for benchmarking depends on your consistency requirements.

Use Masstree when:

• You need snapshot isolation and MVCC semantics
• Your workload has concurrent readers and writers on the same keys
• You want to test column-based updates (partial record modifications)
• Range queries (SCAN) are a significant part of your workload
• You’re benchmarking transaction processing systems

Use Redis when:

• You need raw speed above all else (single-threaded execution)
• Your workload is primarily read-heavy with occasional writes
• You want to test caching layers where eventual consistency is fine
• You need pub/sub, Lua scripting, or Redis modules
• You’re benchmarking web session stores, rate limiters, or CDN caches

Key differences:

Masstree trades peak throughput for stronger consistency guarantees. Applications requiring “read committed” or “snapshot isolation” semantics should benchmark against Masstree rather than Redis.

Advanced Patterns

Range Query Tuning

SCAN performance depends on how much data you’re fetching:

# Small scans (10 records)
[[workload.operations.commands]]
name = "scan"
firstkey = "user:0"
count = 10
fields = [0, 1]  # Only essential fields

# Large scans (1000 records)
[[workload.operations.commands]]
name = "scan"
firstkey = "metrics:2024-11-01"
count = 1000
fields = []  # All fields

Measure:

• Latency vs count: Does P99 grow linearly or worse?
• Throughput impact: Do large SCANs starve other operations?
• Network overhead: Are you transfer-bound with large counts?

Tuning SCAN is about balancing batch size (larger = more efficient) vs latency (larger = longer per-operation time).

Checkpoint Overhead

Checkpoints serialize the database to disk, consuming time and I/O resources:

# Frequent checkpoints (high durability, low throughput)
[[workload.operations.commands]]
name = "checkpoint"
weight = 0.01  # 1% of operations

# Rare checkpoints (high throughput, lower durability)
[[workload.operations.commands]]
name = "checkpoint"
weight = 0.0001  # 0.01% of operations

Measure:

• Latency during checkpoint: Do other operations slow down?
• Throughput degradation: What’s the sustained write rate with frequent checkpoints?
• Recovery time: How long to load a checkpoint on restart?

This helps you understand the durability vs performance tradeoff.

Column-Based Update Efficiency

PUT operations are more efficient than read-modify-write cycles:

# Inefficient: GET + SET (read full record, modify, write back)
# This requires two network round-trips and full value transfer

# Efficient: PUT (update specific columns)
[[workload.operations.commands]]
name = "put"
columns = [[0, "new_value"]]  # Only update column 0

Benchmark the difference:

• Latency: PUT should be ~2x faster (one round-trip vs two)
• Bandwidth: PUT uses less network (only changed columns vs full record)
• Contention: PUT reduces lock time (Masstree only locks affected columns)

For structured data with frequent partial updates, PUT is the right model.

Wrapping Up

Masstree benchmarking focuses on MVCC behavior and transactional semantics. Xylem provides the configuration primitives: operation mixes, key distributions, value sizes, column-based updates, range queries. The goal is modeling your actual database workload.

Start with baseline measurements. Benchmark pure GET and SET with uniform keys to understand Masstree’s baseline performance. Then add realistic complexity: hot keys with Zipfian distribution, mixed operations with production ratios, column updates with PUT, range queries with SCAN. Pipeline aggressively since Masstree handles concurrency well, and watch for MVCC overhead—version chains, garbage collection, checkpoint pauses.

Your benchmarks should provide concrete insights. Not just “Masstree can do 50K QPS,” but “our transactional system can handle 10K concurrent users with snapshot isolation, P99 latency under 5ms, and checkpoint every 10 seconds without stalling operations.” This data determines whether MVCC overhead is justified for your use case.

See Also

• Workload Configuration - Detailed configuration guide
• Configuration Schema - Complete reference
• Masstree Paper - Original research paper
• MessagePack Format - Wire format specification

Xylem Echo Protocol

Note: This is a testing and development protocol, not intended for production use.

TODO: This page requires validation of TOML configuration format and protocol-specific details.

The xylem-echo protocol is used for testing and development purposes. Full documentation will be added once the TOML configuration format is validated.

See Also

• Protocol Configuration
• Architecture: Protocol Layer

Transports

Xylem supports multiple transport layers for network communication.

Supported Transports

• TCP - Reliable byte stream transport
• UDP - Unreliable datagram transport
• Unix Domain Sockets - Local inter-process communication

Transport Selection

Specify the transport explicitly in your TOML profile configuration:

[target]
protocol = "redis"
transport = "tcp"
address = "localhost:6379"

# or, for a Unix socket:
[target]
protocol = "redis"
transport = "unix"
address = "/var/run/redis.sock"

See Also

• Transport Configuration
• Architecture

TCP Transport

Note: This page is TODO. Configuration details need validation against actual implementation.

UDP Transport

Note: This page is TODO. Configuration details need validation against actual implementation.

Unix Domain Sockets

Note: This page is TODO. Configuration details need validation against actual implementation.

Output Formats

TODO: This section is under development. Output format configuration and examples will be added once the implementation is validated.

Planned Support

Xylem will support multiple output formats for results and metrics:

• Text (human-readable)
• JSON (machine-readable)
• CSV (for analysis)

Check back later for detailed documentation on output configuration.

Architecture Overview

Xylem is designed with a modular, layered architecture that separates concerns and enables extensibility.

High-Level Architecture

┌─────────────────────────────────────────┐
│           CLI Interface                 │
│            (xylem-cli)                  │
└─────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────┐
│         Core Engine                     │
│       (xylem-core)                      │
│  - Workload Management                  │
│  - Statistics Collection                │
│  - Threading & Event Loop               │
└─────────────────────────────────────────┘
         │                    │
         ▼                    ▼
┌─────────────────┐  ┌──────────────────┐
│  Protocols - X  │  │  Transports - Y  │
│(xylem-protocols)│  │ (xylem-transport)│
│  - Redis        │  │   - TCP          │
│  - HTTP         │  │   - UDP          │
│  - Memcached    │  │   - Unix Socket  │
│  - Masstree     │  │                  │
│  - xylem-echo   │  │                  │
└─────────────────┘  └──────────────────┘

Design Principles

Xylem’s architecture follows several core design principles. The system employs a modular design where each component has a well-defined responsibility: the CLI layer handles user interaction and TOML configuration parsing, the core engine orchestrates workload generation and statistics collection, protocol implementations handle message encoding/decoding, and the transport layer manages network communication.

The architecture enables composability through clean interfaces. Protocols and transports can be combined freely - Redis can run over TCP, UDP, or Unix domain sockets; HTTP over TCP or Unix sockets; and Memcached over any supported transport. This flexibility allows Xylem to adapt to different testing scenarios without code modifications.

Key Components

Core Engine

The core engine handles workload generation, connection management, statistics collection, and event loop orchestration.

Protocol Layer

Implements application protocols through request encoding, response parsing, and protocol state management.

Transport Layer

Handles network communication including connection establishment, data transmission, and error handling.

Data Flow

The system processes requests through a defined pipeline. Users provide workload configuration via TOML profile files. The core engine reads the configuration during initialization to instantiate the appropriate protocols and transports. During execution, the event loop generates requests according to the configured pattern and collects responses as they arrive. Statistics are gathered throughout the run, tracking latency, throughput, and error rates. Upon completion, results are formatted according to the output configuration and presented to the user.

See Also

• Protocol Layer
• Transport Layer

Protocol Layer

The protocol layer (xylem-protocols) implements application-level protocols for RPC workload generation and measurement.

Protocol Trait

All protocol implementations conform to the Protocol trait interface:

#![allow(unused)]
fn main() {
pub trait Protocol: Send {
    type RequestId: Eq + Hash + Clone + Copy + Debug;
    
    /// Generate a request with an ID
    fn generate_request(
        &mut self,
        conn_id: usize,
        key: u64,
        value_size: usize,
    ) -> (Vec<u8>, Self::RequestId);
    
    /// Parse a response and return the request ID
    fn parse_response(
        &mut self,
        conn_id: usize,
        data: &[u8],
    ) -> Result<(usize, Option<Self::RequestId>)>;
    
    /// Protocol name
    fn name(&self) -> &'static str;
    
    /// Reset protocol state
    fn reset(&mut self);
}
}

The trait provides a uniform interface for protocol implementations, enabling the core engine to interact with any protocol through a consistent API.

Protocol Design

Protocol implementations are designed to minimize state and memory overhead. Each implementation encodes requests in the appropriate wire format and parses responses into a form suitable for latency measurement. The RequestId mechanism enables request-response correlation, which is required for accurate latency measurement in scenarios involving pipelining or out-of-order responses.

Supported Protocols

The following protocol implementations are available:

• Redis - RESP (Redis Serialization Protocol) for key-value operations
• HTTP - HTTP/1.1 for web service testing
• Memcached - Binary and text protocol variants for cache testing

Detailed documentation for each protocol is available in the Protocols section of the User Guide.

Request-Response Correlation

The protocol layer maintains state necessary for correlating responses with their originating requests. This is required for protocols supporting pipelining or scenarios where multiple requests are in flight concurrently. Each protocol defines an appropriate RequestId type for its correlation semantics.

See Also

• Protocol implementations in User Guide
• Architecture Overview
• Transport Layer

Transport Layer

The transport layer (xylem-transport) provides network communication primitives for Xylem.

Transport Trait

All transport implementations conform to the Transport trait interface:

#![allow(unused)]
fn main() {
pub trait Transport: Send {
    /// Connect to the target
    fn connect(&mut self) -> Result<()>;
    
    /// Send data
    fn send(&mut self, data: &[u8]) -> Result<usize>;
    
    /// Receive data
    fn recv(&mut self, buf: &mut [u8]) -> Result<usize>;
    
    /// Close the connection
    fn close(&mut self) -> Result<()>;
    
    /// Check if connected
    fn is_connected(&self) -> bool;
}
}

The trait provides a uniform interface for transport mechanisms, enabling the core engine to perform network operations independently of the underlying transport implementation.

Transport Design

Transport implementations are lightweight wrappers around operating system network primitives. Each implementation handles protocol-specific communication details while presenting a consistent interface to higher layers. The design prioritizes efficiency and correctness, with explicit error handling and resource management.

Supported Transports

The following transport implementations are available:

• TCP - Connection-oriented byte streams using TCP sockets
• UDP - Connectionless datagram communication
• Unix Domain Sockets - Local inter-process communication

Detailed documentation for each transport is available in the Transports section of the User Guide.

Non-Blocking I/O

All transport implementations support non-blocking operation for integration with the event-driven architecture. The core engine uses mio for I/O multiplexing, enabling a single thread to manage multiple concurrent connections. Transports register file descriptors with the event loop and respond to readiness notifications.

Connection Management

The transport layer provides primitives for connection lifecycle management. The core engine implements higher-level functionality such as connection pooling and reconnection strategies. This separation maintains transport implementation simplicity while allowing sophisticated connection management policies in the core.

See Also

• Transport implementations in User Guide
• Architecture Overview
• Protocol Layer

Configuration Schema

Xylem uses TOML configuration files to define experiments. This reference documents the complete schema for experiment profiles.

Overview

A complete Xylem configuration file contains the following sections:

[experiment]      # Experiment metadata and duration
[target]          # Target service address, protocol, and transport
[workload]        # Workload generation parameters
[[traffic_groups]] # One or more traffic generation groups
[output]          # Output format and destination

Experiment Section

The [experiment] section defines metadata and experiment-wide settings.

[experiment]
name = "redis-bench"
description = "Redis benchmark with latency/throughput agent separation"
duration = "30s"
seed = 42

Fields

• name (string, required): Name of the experiment
• description (string, optional): Description of the experiment
• duration (string, required): Duration in format “Ns”, “Nm”, “Nh” (seconds, minutes, hours)
• seed (integer, optional): Random seed for reproducibility

Target Section

The [target] section specifies the target service to benchmark.

[target]
address = "127.0.0.1:6379"
protocol = "redis"
transport = "tcp"

Fields

• address (string, required): Target address. Format depends on transport:
  • TCP/UDP: "host:port" or "ip:port"
  • Unix socket: "/path/to/socket"
• protocol (string, required): Application protocol. Supported values:
  • "redis"
  • "http"
  • "memcached-binary"
  • "memcached-ascii"
  • "masstree"
  • "xylem-echo" (testing only)
• transport (string, required): Transport layer. Supported values:
  • "tcp"
  • "udp"
  • "unix"

Workload Section

The [workload] section defines the workload pattern and key distribution.

[workload.keys]
strategy = "zipfian"
n = 1000000
theta = 0.99
value_size = 64

[workload.pattern]
type = "constant"
rate = 50000.0

Fields

[workload.keys] - Key distribution parameters:

• strategy (string, required): Distribution strategy
  • "uniform": Uniform distribution
  • "zipfian": Zipfian distribution (power law)
  • "random": Random distribution
• n (integer, required for uniform/zipfian): Key space size (total number of keys)
• max (integer, required for random): Maximum key value
• theta (float, required for zipfian): Skew parameter (0.0 to 1.0)
  • 0.99 = high skew (typical for caches)
  • 0.5 = moderate skew
• value_size (integer, required): Size of values in bytes

[workload.pattern] - Traffic pattern:

• type (string, required): Pattern type
  • "constant": Constant rate
  • (other types may be supported)
• rate (float, required): Target request rate in requests/second

Traffic Groups

Traffic groups define how workload is distributed across threads and connections. You can define multiple [[traffic_groups]] sections.

[[traffic_groups]]
name = "latency-agent"
protocol = "redis"
threads = [0]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "poisson"
rate = 100.0

Fields

• name (string, required): Name for this traffic group
• protocol (string, optional): Override protocol for this group (defaults to [target] protocol)
• threads (array of integers, required): Thread IDs to use for this group
• connections_per_thread (integer, required): Number of connections per thread
• max_pending_per_connection (integer, required): Maximum pending requests per connection
  • 1 = no pipelining (accurate latency measurement)
  • Higher values = pipelining enabled

Sampling Policy

[traffic_groups.sampling_policy]:

• type (string, required):
  • "unlimited": Sample every request (100% sampling)
  • "limited": Sample a fraction of requests
• rate (float, required for limited): Sampling rate (0.0 to 1.0)
  • 0.01 = 1% sampling
  • 0.1 = 10% sampling

Traffic Policy

[traffic_groups.policy]:

• type (string, required):
  • "poisson": Poisson arrival process (open-loop)
  • "closed-loop": Closed-loop (send as fast as possible)
• rate (float, required for poisson): Request rate per connection in requests/second

Output Section

The [output] section configures where and how results are written.

[output]
format = "json"
file = "/tmp/results.json"

Fields

• format (string, required): Output format
  • "json": JSON format
  • (other formats may be supported)
• file (string, required): Output file path

Complete Example

[experiment]
name = "redis-bench"
description = "Redis benchmark"
duration = "30s"
seed = 42

[target]
address = "127.0.0.1:6379"
protocol = "redis"
transport = "tcp"

[workload.keys]
strategy = "zipfian"
n = 1000000
theta = 0.99
value_size = 64

[workload.pattern]
type = "constant"
rate = 50000.0

[[traffic_groups]]
name = "latency-agent"
protocol = "redis"
threads = [0]
connections_per_thread = 10
max_pending_per_connection = 1

[traffic_groups.sampling_policy]
type = "unlimited"

[traffic_groups.policy]
type = "poisson"
rate = 100.0

[[traffic_groups]]
name = "throughput-agent"
protocol = "redis"
threads = [1, 2, 3]
connections_per_thread = 25
max_pending_per_connection = 32

[traffic_groups.sampling_policy]
type = "limited"
rate = 0.01

[traffic_groups.policy]
type = "closed-loop"

[output]
format = "json"
file = "/tmp/redis-bench-results.json"

See Also

• Configuration Guide
• CLI Reference
• Example Profiles