← Home

Blog

Plain-language write-ups of my projects and papers.

• Language modeling with 30–86 bytes of context memory per token Updated 29 October 2026. Results from 155M–963M models, 30–86 bytes of cache per token, and the distinction between long-context capacity and far recall.
• Self-challenging language models: endless verifiable problems at the edge of ability Early self-play results: better task generation, a collapsed run, and the untested question of transferable solver improvement.
• Can one diffusion model tell where it is unsure? Work in progress. Using one diffusion model's spread, instead of an ensemble, as uncertainty for offline RL.
• Teaching a model to summarize from raw text alone Work in progress. Reinforcement learning for summarization with no human labels.
• Does a diffusion model remember where it started? Bounds for diffusion sampling from one fixed starting point.
• Giving an RL agent a hunch about its world ALH, ICML 2024. Why value-based RL is slow to exploit, and a cheap fix.
• Summarizing many documents with a model trained on one Multi-Channel Attention, ECML PKDD 2024. Linear cost, no retraining.