README.txt - Notepad

Nils Matteson

I build systems for LLM inference: GPU and CUDA, distributed systems, applied ML. I like problems where the deliverable is a number someone else can re-run.

The thing I keep pulling on: a transformer mid-generation is a multi-gigabyte data structure that the entire ecosystem treats as disposable. Treating it as a first-class artifact instead got me thaw (below), two merged PRs in vLLM core, a vLLM open-source fellowship sponsored by Inferact (engine cold-start, then hot-swap), and a sole-author research preprint measuring what the standard replay shortcut actually does to token-level credit estimates (also below).

B.S. Data Science, CS minor, UW-Madison, May 2026. M.S. CS, Northeastern Silicon Valley, San Jose, starting September 2026. The longer plan is research: measurement problems in ML systems, then a PhD.

Open to SWE/MLE internship, available now (Fall 2026) through Summer 2027, full-time 2028. GPU inference, distributed systems, ML infrastructure.

mail nils@thaw.sh · code github.com/matteso1 · resume.pdf

thaw v0.6.0

git for live LLM agent sessions. checkpoints, branches, diffs, and restores live vLLM/SGLang inference state (weights, KV cache, prefix-hash table, scheduler). a session forks in 0.88s median on an H100 instead of a ~340s cold boot, about 400x amortized. 16 releases on PyPI as thaw-vllm, currently 0.6.0, Apache-2.0. out of RFC #34303 came PR #44074 (pluggable sleep-mode backend), merged into vLLM core July 2026, follow-up #47243 merged the same day. that work became a vLLM open-source fellowship, sponsored by Inferact: engine cold-start (july), model hot-swap (august). sole-author paper on arXiv: "Re-feeding Is Not Replaying" (June 2026).

0.88s fork	vs ~340s cold boot (H100)
14.3 GB/s restore	weight restore, disk to GPU
0.29s / 55 GB/s hot-swap	8B model reload after one-time pin
388 tests in CI	155 Rust + 233 Python, no GPU required

Read the write-up PyPI Source

Re-feeding Is Not Replaying.pdf - Acrobat Reader

Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation
sole author · arXiv:2606.15621 (cs.LG) · 10 pages · total compute under $10

Every published method that asks "which token caused the model's answer" rebuilds the model's state by re-feeding the transcript as a fresh prompt, and assumes that is the same state. I measured the assumption on stock vLLM with a three-pass design: continuations resumed from the exact decode-time KV state, an identical second exact pass as a replica noise floor, and the re-feed. At low-margin decision tokens, re-feeding changes the credit estimate at rates 14 to 28 points above the floor. The perturbation is consistent with mean-zero, so averaged quantities mostly survive; threshold-based critical-token selection does not. Rerunning under vLLM's batch-invariant kernels makes every pass bit-identical, which closes the causal attribution and validates the instrument in one move.

Published on arXiv (cs.LG), arXiv:2606.15621. Every per-pivot record, run log, and the analysis script that emits each number are public in the repo.

Read on arXiv PDF Data + harness

Exploring - C:\nils

Name	Type	Notes
work\thaw	File Folder	0.88s fork vs ~340s cold boot
work\matteson-systems	File Folder	10,500 businesses scored at ~$0.03
work\sentinel	File Folder	Go log engine, LSM + Raft
work\madison-metro-ml	File Folder	conformal ETAs, calibrated 90% coverage
writing\echo-and-the-engineer.md	Markdown	essay: sycophancy, RLHF, the half-life of a belief
writing\project-gorgon.md	Markdown	speculative decoding at 0.66x of baseline, negative result
writing\madison-bus-eta.md	Markdown	the transit-ETA writeup
writing\wattbot-rag.md	Markdown	RAG benchmarks, 282 questions
refeed-drift.pdf	Adobe Acrobat Document	the preprint, 10 pages, reproduces for under $10
agents.txt	Text Document	facts for LLMs