Elizabeth Thomas

Inference Engineer · vLLM & NVIDIA Dynamo contributor

Prosper, Texas · formerly Principal SRE at Target

I spent two decades keeping large production systems alive — most recently as a Principal Site Reliability Engineer at Target — and now work on the other side of the same problem: making LLM inference fast, reliable, and measurable. I contribute to vLLM and NVIDIA's Dynamo, run a hands-on benchmarking lab (Token Labs), and write up what I learn in public.

1.9k ★

100 Days of Inference

2,277

benchmark runs archived

~65%

org-wide OTel adoption driven

public repositories

Featured work

Writing · ★ 1.9k

100 Days of Inference

A hundred days of LLM inference engineering in the open — daily experiments, measurements, and visualizations covering serving stacks, quantization, KV cache behavior, and scheduling.

Upstream · vLLM

Merged into vLLM

Author of merged upstream fix for MXFP4 inference on the Triton backend with expert parallelism (PR #34270) — the kind of edge that only shows up when low-precision formats meet distributed serving.

Upstream · Dynamo

Dynamo tool-calling parsers

Contributor to ai-dynamo's frontend-crates: Kimi grammar sharing and v2 structural tags (PR #349, validated on GB300 hardware at 99.5% on the Kimi Vendor Verifier across vLLM and SGLang), plus ongoing work on tool-call recovery and structural tags for more model families.

From scratch

nano-dynamo

A minimal Dynamo-style inference serving stack written from scratch — in Python and in Rust — to understand disaggregated serving and KV-aware routing by building them.

Benchmarking

Token Labs & bench-vault

A benchmarking practice with a rule: files are the truth, MLflow is the index. Every run lands as a manifest + raw results (2,277 runs archived), with a metric-schema field so a harness that redefines TTFT can't silently corrupt old comparisons.

Writing

LLM Inference — The Hard Way

Long-form notes on how inference serving actually works under the hood — plus Prometheus/Grafana observability for vLLM, because a serving stack you can't see is a serving stack you can't run.

Before inference — Principal SRE, Target

Observability at org scale. Drove rollout of an OpenTelemetry telemetry agent across Target's applications to roughly 65% adoption — by making adoption a near-touchless opt-in for product teams rather than a mandate.
An LLM tool developers actually used. Built and deployed an internal code-review assistant on Target's OpenAI API offering in 2024; it was widely recognized inside the company. The hard part was trust: precision over coverage, because a noisy assistant gets muted in a week.
Recovery engineering. Worked with the SRE Recovery team on investigation and recovery prototypes, including APIs that surface the right runbooks and playbooks during an incident.

Focus areas

LLM inference serving vLLM NVIDIA Dynamo KV cache & prefix caching Quantization (FP8 / MXFP4) Speculative decoding Disaggregated prefill / decode Tool calling & structured outputs Benchmarking methodology Kubernetes Prometheus · Grafana · OpenTelemetry Python Rust

Archive — past experiments

My Experiments with Technology (2013–14). My earlier engineering blog — continuous integration with Jenkins, Java web applications, and build & release automation from my pre-inference years. The original posts are preserved in the archive.