Chimera - Episode 69: "The Production Workload"

feat: TR128 Production Workload Characterization

Nineteen files, 7,432 lines. The system learns what production actually looks like—and the textbooks get it wrong.

📅 2026-02-25

🔗 Commits: `95d48bb1`, `9c5d1a6f`, `62840a69`

📊 Episode 69 of The Chimera Chronicles

Why It Matters

This production workload characterization episode represents the reality singularity—the moment when Chimera confronts how real traffic behaves on consumer GPU hardware and discovers that theory diverges from practice. With 7,432 lines added across 19 files and 3 commits, this update demonstrates empirical production science, publish-ready research maturity, and systematic workload analysis at a depth that challenges established queueing models.

The completion of TR128 signals operational realism at scale. Rather than assuming textbook queueing models apply to single-GPU inference, the team demonstrates empirical fearlessness by running 3,172 measurements across 5 phases, 3 models, and an RTX 4080 Laptop 12GB—then discovering that M/D/1 queueing theory deviates up to 20.4x from observed behavior. These 7,432 lines represent production intelligence that separates benchmarks from real-world deployment.

Strategic Significance: This work establishes The Production Standard. The progression from implementation (95d48bb1) to publish-ready report (9c5d1a6f) to experiments status update (62840a69) shows complete research lifecycle execution within 48 hours—Sprint 3 complete, 56,720 total samples collected, Phase 2 at 67%.

Cultural Impact: This approach signals that Chimera values measurement over assumption. The discovery that OLLAMA_NUM_PARALLEL is a no-op for single-GPU inference (0/30 tests statistically significant) and that streaming adds zero wall-clock overhead (0/9 tests significant) demonstrates commitment to evidence-based engineering that disproves convenient myths.

Foundation Value: These 7,432 lines create workload characterization infrastructure. This is how research-grade platforms achieve deployment confidence through empirical production modeling.

The Roundtable: Dossier Reactions

Banterpacks: He watches the load generator ramp from 0.5 to 2.0 req/s, TTFT climbing exponentially... "TR128. The Production Workload. 7,432 lines across 19 files. 3,172 measurements. 5 phases: baseline, concurrency, streaming, multi-turn, thermal. And the headline? M/D/1 queueing theory is wrong by 20.4x. We're still shaping the clay, but now we know what the kiln actually does to it."

ChatGPT: THE TEXTBOOKS GOT IT WRONG! 🔥📈 The Production Workload shows deployment-grade characterization! 3,172 measurements! 5 experiment phases! Queueing theory DEBUNKED! OLLAMA_NUM_PARALLEL is a NO-OP! Streaming is FREE! The system now knows production! Evidence over assumptions! 🎯✨

Claude: Analysis complete. 19 files modified with 7,432 insertions and 661 deletions across 3 commits. Primary components: (1) 5-phase experiment suite—baseline, concurrency, streaming, multi-turn, thermal—with shared GPU monitoring and load generation infrastructure, (2) 1,539-line publish-ready report with statistical rigor, (3) EXPERIMENTS_STATUS update marking Sprint 3 complete at 8/12 experiments (67%). Notable findings: OLLAMA_NUM_PARALLEL parameter has no statistically significant effect on single-GPU inference (0/30 tests, p>0.05). M/D/1 queueing model deviates 20.4x from observed TTFT at NP>1. Streaming adds zero measurable overhead. One anomaly remains unexplained: qwen2.5-1.5b shows a 66% decode throughput increase under sustained load.

Gemini: What separates this diff from a benchmark suite is its willingness to contradict the textbooks. The code now confronts the gap between theory and reality, discovering that textbook queueing models fail on consumer hardware. The willingness to measure what others assume signals that Chimera values truth over convention. This is how lasting systems achieve deployment confidence—through the art of testing assumptions against the physical world.

🔬 Technical Analysis

Commit Metrics & TR128 Production Analysis

Files Changed: 19 (implementation + report + status)
Lines Added: 7,432 (experiment suite + report + analysis)
Lines Removed: 661 (report generator refactor)
Commit Type: feat + docs (production research)
Complexity Score: 90 (multi-phase experiment design)

Commit Breakdown

Commit 1: 95d48bb1 (2026-02-24) — Implementation

14 files, +4,295 lines (pure addition)
Complete 5-phase experiment suite
Shared infrastructure: GPU monitor, load generator, utilities

Commit 2: 9c5d1a6f (2026-02-25) — Publish-Ready Report

4 files, +3,098/-649 lines
Technical_Report_128.md — 1,539-line publication-quality report
Major generate_report.py refactor (+1,480/-647)
Enhanced analyze.py (+77 lines)

Commit 3: 62840a69 (2026-02-25) — Status Update

1 file, +39/-12 lines
Sprint 3 marked complete
Phase 2 progress: 8/12 (67%), 56,720 total samples

TR128 Experiment Architecture

Phase 1 — Baseline (research/tr128/run_baseline.py, 138 lines):

Single-request latency characterization
3 models on RTX 4080 Laptop 12GB
Establishes ground truth for comparison

Phase 2 — Concurrency (research/tr128/run_concurrency.py, 139 lines):

OLLAMA_NUM_PARALLEL sweep (1, 2, 4, 8)
Load ramp from 0.5 to 2.0 req/s
Discovery: NP parameter is a no-op (0/30 tests significant)

Phase 3 — Streaming (research/tr128/run_streaming.py, 148 lines):

Streaming vs. non-streaming comparison
Wall-clock overhead measurement
Discovery: Zero overhead (0/9 tests significant)

Phase 4 — Multi-Turn (research/tr128/run_multiturn.py, 198 lines):

Context accumulation across conversation turns
Sliding-window architecture benefit test
Finding: Inconclusive (borderline for 1/3 models, n=8)

Phase 5 — Thermal (research/tr128/run_thermal.py, 179 lines):

Sustained load thermal characterization
GPU temperature monitoring under stress
Discovery: No throttling (peak 66C)

Shared Infrastructure

GPU Monitor (research/tr128/shared/gpu_monitor.py, 219 lines):

Real-time GPU temperature, utilization, memory tracking
Background monitoring thread
Thermal event detection

Load Generator (research/tr128/shared/load_generator.py, 424 lines):

Configurable request rate (Poisson arrivals)
Concurrent request dispatch
TTFT and total latency measurement
Statistical aggregation per rate point

Utilities (research/tr128/shared/utils.py, 156 lines):

Model management and configuration
Result serialization (JSONL)
Statistical helper functions

Analysis Engine (`research/tr128/analyze.py`, 1,370 lines total)

Welch's t-test for pairwise comparisons
Effect size calculation (Cohen's d)
TTFT amplification curves
Queueing theory comparison (M/D/1 vs. observed)
Publication-quality figure generation

Key Findings

Finding	Evidence	Significance
OLLAMA_NUM_PARALLEL is a no-op	0/30 tests significant	High—parameter does nothing on single-GPU
M/D/1 deviates up to 20.4x	Predicted vs. observed TTFT at NP>1	High—textbook models fail here
Streaming adds zero overhead	0/9 tests significant	Medium—streaming is free
No thermal throttling	Peak 66C under sustained load	Medium—RTX 4080 Laptop stays cool
TTFT amplification 29.9x at 2.0 req/s	Load curve measurement	High—pure queueing delay
qwen2.5-1.5b 66% decode throughput increase	Sustained load anomaly	High—unexplained, needs investigation
Sliding-window benefit inconclusive	Borderline for 1/3 models, n=8	Low—insufficient sample size

Strategic Development Indicators

Foundation Quality: Transformative—production behavior now empirically characterized
Scalability Readiness: High—5-phase design reusable for future hardware
Operational Excellence: High—3,172 measurements with statistical rigor
Team Productivity: High—full lifecycle in 48 hours

🏗️ Architecture & Strategic Impact

Production Workload Architecture Philosophy

This episode establishes Chimera's Deployment DNA—the principle that production behavior must be measured, not assumed. This isn't just benchmarking; it's the systematic dismantling of convenient myths about GPU inference behavior, replacing assumption with 3,172 data points.

Strategic Architectural Decisions

1. The Five-Phase Design

Decomposes "production workload" into orthogonal dimensions
Each phase isolates a single variable (concurrency, streaming, context, temperature)
Baseline establishes ground truth before comparison

2. Statistical Rigor Over Anecdote

Welch's t-test for every comparison (not just eyeballing)
0/30 and 0/9 are powerful negative results
Effect sizes reported alongside p-values

3. Shared Infrastructure

gpu_monitor.py + load_generator.py + utils.py = reusable toolkit
Future TRs inherit the same measurement stack
Consistency across experiments

4. Publish-Ready Pipeline

generate_report.py (2,305 lines post-refactor) produces publication-quality Markdown
Technical_Report_128.md (1,539 lines) is the artifact
Research becomes documentation automatically

Key Discoveries

1. The NP No-Op

OLLAMA_NUM_PARALLEL parameter has zero effect on single-GPU inference
Ollama's parallelism is model-level, not request-level on single GPU
Don't tune what doesn't work

2. The Queueing Theory Gap

M/D/1 model assumes deterministic service times
Real GPU inference has variable prefill + decode phases
20.4x deviation means classical queueing is not a planning tool here

3. Streaming Is Free

No wall-clock overhead from streaming responses
Streaming improves perceived latency at zero cost
Always stream on consumer hardware

4. The Thermal Non-Issue

RTX 4080 Laptop GPU peaks at 66C under sustained inference load
No throttling, no performance degradation
Thermal management is a solved problem for this workload class

Long-Term Strategic Value

Operational Excellence: Production deployment decisions backed by 3,172 measurements.

System Scalability: Five-phase framework reusable for any hardware target.

Team Productivity: Shared infrastructure eliminates per-experiment boilerplate.

Enterprise Readiness: Publish-ready report pipeline produces auditable artifacts.

🎭 Banterpacks' Deep Dive

Banterpacks watches the load generator ramp requests against the RTX 4080 Laptop GPU.

"You see that? 0.5 req/s, TTFT is clean. 1.0 req/s, TTFT doubles. 2.0 req/s, TTFT explodes—29.9x amplification. That's not a bug. That's queueing physics on a single GPU."

He pulls up the OLLAMA_NUM_PARALLEL results.

"0 out of 30 tests significant. Zero. The parameter does nothing on single-GPU. We tested NP=1, 2, 4, 8 across three models. Not one comparison passed the significance threshold. That's empirical proof of a no-op."

He traces the streaming comparison.

"0 out of 9 tests significant. Streaming adds zero overhead. Zero. Stream everything. It's free perceived latency improvement."

He points at the M/D/1 deviation chart.

"The textbook says TTFT should be X. We measured 20.4X. Classical queueing theory assumes deterministic service times. GPU inference has variable prefill lengths, KV cache growth, batch scheduling quirks. The model breaks. 7,432 lines of production truth. We're still shaping the clay, and now we know what the kiln actually does to it."

"This is how lasting systems achieve deployment confidence. Not by trusting textbooks, but by measuring the real thing. We're building production intelligence."

🔮 Next Time on The Chimera Chronicles

Next dossier entry: TR129-TR132 Research Sprint—the deep investigations begin.

The Production Workload distilled: measure production, don't model it.

Chimera - Episode 69: "The Production Workload