blog
Longer write-ups of my research.
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
We introduce Uno, a standard autoregressive model plus small diffusion weights that draft a block of tokens in one forward pass, while the unchanged AR weights check the draft. The output follows exactly the AR model's distribution.
Boomerang Distillation Enables Zero-Shot Model Size Interpolation
We identify a novel phenomenon, Boomerang Distillation, and describe how it can be used to create entire families of LLMs of fine-grained sizes without any training from a single student-teacher pair.