babarranjha@gmail.com
Self-Distillation is one of those ideas that sounds suspicious on first contact. Train a model on its own raw outputs? No verifier, no teacher, no reward model, no reinforcement learning, no execution sandbox? That usually sounds like a fast route to elegant nonsense. Which is why this Apple AI research paper is so interesting. It … Read more
Gemma 4 Benchmark Explorer Gemma 4 31B Gemma 4 26B A4B Gemma 4 E4B Gemma 4 E2B Gemma 3 27B Every so often, a model release lands with the usual confetti, leaderboard screenshots, and claims that sound suspiciously like they were approved by three layers of marketing. Gemma 4 feels different. Not because the hype … Read more
Coding benchmarks Multimodal benchmarks Qwen3.6 Plus Qwen3.5-397B Kimi K2.5 GLM5 Claude Opus 4.5 Qwen3.6 Plus arrived with the kind of launch that makes the rest of the industry look up from its dashboards and mutter, “well, that complicates things.” On paper, the pitch is outrageous: a 1M context window, a serious jump in agentic coding, … Read more
KV Cache Compression: Recall vs. Memory Needle-in-Haystack benchmark · Llama-3.1-8B-Instruct · context up to 104k tokens Best recall 0.997 TurboQuant = full precision Memory at 3.5-bit 6x smaller KV cache GPU speedup 8x on H100 at 4-bit Needle-in-Haystack recall score KV cache size (bits) Tested on Llama-3.1-8B-Instruct · Needle-in-Haystack benchmark · context up to 104k … Read more
Standard residuals Fixed, uniform weights Embedding h₁ Layer 1 Layer 2 Layer 3 Each layer only sees the accumulated sum Attention Residuals Learned, input-dependent Embedding h₁ Layer 1 Layer 2 Layer 3 Layer 3 selectively attends to any earlier layer Residual connections are one of those rare ideas in deep learning that became so successful, … Read more
GPT 5.4 vs Sonnet 4.6 is not a trivial leaderboard fight. It is a clash of working styles. One model feels like a fearless builder who grabs the keyboard and starts shipping. The other feels like the senior engineer who slows down just enough to save you from tomorrow’s mess. Sonnet 4.6 landed on February … Read more
Speed vs Quality vs Cost (Bubble = Output Price) X: output speed, Y: GPQA Diamond, bubble size: $/1M output tokens Tip: the sweet spot is top-right with a smaller bubble. Google dropped Gemini 3.1 Flash-Lite on March 3, 2026, with essentially no advance notice. One day it wasn’t there, the next it was sitting quietly … Read more