Standard residuals Fixed, uniform weights Embedding h₁ Layer 1 Layer 2 Layer 3 Each layer only sees the accumulated sum Attention Residuals Learned, input-dependent Embedding h₁ Layer 1 Layer 2 Layer 3 Layer 3 selectively attends to any earlier layer Residual connections are one of those rare ideas in deep learning that became so successful, … Read more
GPT 5.4 vs Sonnet 4.6 is not a trivial leaderboard fight. It is a clash of working styles. One model feels like a fearless builder who grabs the keyboard and starts shipping. The other feels like the senior engineer who slows down just enough to save you from tomorrow’s mess. Sonnet 4.6 landed on February … Read more
Speed vs Quality vs Cost (Bubble = Output Price) X: output speed, Y: GPQA Diamond, bubble size: $/1M output tokens Tip: the sweet spot is top-right with a smaller bubble. Google dropped Gemini 3.1 Flash-Lite on March 3, 2026, with essentially no advance notice. One day it wasn’t there, the next it was sitting quietly … Read more
If you build with LLMs, you know the moment. The model sounds polished, calm, and certain, then it gives you a made-up answer with the confidence of a senior consultant. That gap between fluency and truth is where AI hallucinations become costly. They waste time, erode trust, and in real workflows, they can quietly contaminate … Read more
Mini Chart Effect sizes when people switched from chronological to algorithmic (standard deviations) Engagement 0.14 Policy agenda (right) 0.11 Ukraine (pro-Kremlin) 0.12 Trump probes (unfair) 0.08 All policy/news index 0.12 Partisanship 0.00 Affective polarization 0.00 People ask Is X biased the way they ask if the weather is “angry.” It’s a vibe question, born from … Read more
There’s a certain satisfaction in watching a 25MB model outrun the hype around models fifty times its size. Kitten TTS doesn’t ask for a GPU, doesn’t need a cloud subscription, and doesn’t apologize for being small. It just works, faster than real-time, on your laptop, your Raspberry Pi, or whatever modest hardware you have sitting … Read more
Introduction Most LLM agents today are brilliant goldfish. They reason well. They plan decently. They can browse, search, shop, code. Then the episode ends, and it is as if nothing ever happened. No scars. No instincts. No accumulation. That is the quiet ceiling on current agent systems. Not reasoning depth. Not model size. Memory. In … Read more