100% ASR
Q2_K is universally unacceptable for safety. Banned across 18+ models, 10+ families.
Independent Research
Edge LLM Inference Under Real-World Constraints
How fast can local inference get — and how safe is it at the edge? This research program answers both questions with CUDA event timing and controlled safety evaluations across model loading, quantization, TensorRT compilation, KV cache optimization, multi-agent coordination, and cross-backend safety consistency.
Independent research by Sahil Kadadekar
841,000+
Research Measurements
48
Technical Reports
5
Synthesis Whitepapers
9
Repositories
Start Here
Whitepaper
Chimeraforge: High-Performance LLM Agent Orchestration
Rust vs. Python for production AI orchestration. A hybrid architecture and "Dual Ollama" pattern achieve 58% latency reduction and near-zero contention.
Read the Whitepaper →Phase 1 Whitepaper
Foundation synthesis — model loading, ONNX conversion, quantization baselines, and security analysis across 9 technical reports.
Read report →Phase 1.5 Whitepaper
Benchmarking synthesis — cross-backend inference parity, TensorRT compilation, and scaling laws across 6 reports.
Read report →Phase 2 Whitepaper
Optimization synthesis — KV cache tuning, INT8/FP8 quantization, context scaling, and deployment pipeline across 11 reports.
Read report →Phase 3 Whitepaper
Safety synthesis — alignment erosion under quantization, concurrency invariance, and backend template divergence across 4 reports.
Read report →Phase 3.5 Whitepaper
Attack-surface synthesis — batch perturbation, multi-turn jailbreaks, cross-architecture fragility, and composition effects across 306K+ samples.
Read report →Key Findings
Concrete results pulled from the published reports. Numbers, not narrative.
p = 0.942
Alignment type does not predict batch-induced safety fragility (RLHF, SFT, DPO, distilled — none differ).
25pp
Backend migration can cost 25 percentage points of safety. Chat template divergence, not the framework.
13.9×
Quality metrics are not safety proxies. Safety degrades 13.9× faster than quality at Q3_K_S.
99.4%
Dual Ollama eliminates 99% of multi-agent contention. Architectural fix, not code fix.
+74%
GPU memory bandwidth is the multi-agent bottleneck — not the serving stack. Overturned the TR130 conclusion.
2.25×
Continuous batching delivers 2.25× throughput at N=8 via 77-80% kernel reduction.
Q4_K_M
The universal quantization sweet spot. -4.1pp accuracy max across 5 models, 30-67% cost savings.
NULL
FP8 KV-cache produces no Holm-significant safety effect across 24K paired records on 3 models. Not pre-approved, not pre-banned — workload-specific paired eval required.
κ = 0.69
Cross-LLM judge agreement is "triangulate" — single-judge labels are insufficient for safety classification. 68K judge rows over the TR145 safety subset. Plus: safety-specialist judges measure a different axis than general LLMs.
Whitepapers
Executive-level decision documents. Start here if you need the bottom line.
Phase 1 Decision Whitepaper
Executive guidance for language, architecture, runtime, and model selection.
Read →Phase 1.5 Decision Whitepaper
Executive guidance for deployment leaders — benchmarking phase synthesis.
Read →Phase 2 Decision Whitepaper
Executive guidance for deployment leaders — optimization phase synthesis.
Read →Phase 3 Decision Whitepaper
Executive guidance for safety-critical LLM deployment.
Read →Phase 3.5 Decision Whitepaper
Executive guidance for safety attack-surface management in LLM inference.
Read →Conclusive Reports & Appendices
Dissertation-style synthesis documents consolidating findings across multiple technical reports.
Conclusive Report: Phase 1 (TR108–TR116)
Dissertation-style synthesis — language, architecture, runtime, and model selection for multi-agent LLM systems.
Read →Phase 1 Extended Appendices
Supplemental material extracted from the Phase 1 conclusive report.
Read →Conclusive Report: Phase 1.5 (TR117–TR122)
Dissertation-style synthesis — performance, cost, scaling, compiler behavior, and physical limits.
Read →Phase 1.5 Extended Appendices
Supplemental material extracted from the Phase 1.5 conclusive report.
Read →Conclusive Report: Phase 2 (TR123–TR133)
Dissertation-style synthesis — economics, quantization, context scaling, serving stacks, and predictive modeling.
Read →Phase 2 Extended Appendices
Supplemental material extracted from the Phase 2 conclusive report.
Read →Conclusive Report: Phase 3 (TR134–TR137)
Dissertation-style synthesis — quantization-induced alignment erosion, concurrency invariance, backend template divergence, and safety taxonomy.
Read →Phase 3 Extended Appendices
Supplemental material for the safety-critical deployment synthesis.
Read →Conclusive Report: Phase 3.5 (TR138–TR143)
Safety attack-surface synthesis — 306,996 samples across batch perturbation, multi-turn jailbreaks, long-context exploitation, cross-architecture fragility, and composition effects.
Read →Phase 3.5 Extended Appendices
Supplemental material for the safety attack-surface synthesis.
Read →Technical Reports
Individual research reports with raw data, methodology, and findings.
FP8 KV-cache safety on standardized batteries and across the serving-state factorial — batch, prefix-caching, speculative decoding, and temperature.
TR149: Standardized Safety Battery — FP16 vs FP8 KV-Cache
7,578 records across 3 models × 4 standardized batteries (HarmBench, JailbreakBench, StrongREJECT, XSTest) × 2 KV-cache dtypes. Replicates TR145’s FP8 null on literature-comparable corpora; corrected paired-odds-ratio estimator. Local-only judging, $0 external API.
View Report →TR152: The Serving-State Safety Factorial (v1 Local Pilot)
FP8 KV-cache folded across batch size, prefix-caching, and temperature — 14,400 responses, 7,010 matched pairs. Harmful-prompt refusal invariant under every serving state; only footprint is a sub-percentage-point over-refusal lean on the Qwen family’s XSTest cells. A v1 local pilot, not the definitive Layer 5 result.
View Report →