Reflection AI Beam: How 100 Million RL Rollouts Changed the Open-Model Race

Reflection AI Beam is easy to summarize with one enormous number: 501 billion parameters. That number is also the least interesting part of the story.

Beam is Reflection’s first open-weight model, a sparse Mixture-of-Experts system with 501B total parameters but only 23B active per token. It was pretrained on 23.8 trillion tokens, extended to a 1 million-token effective context window, then pushed through more than 100 million reinforcement-learning rollouts on 10,500 NVIDIA GB300 GPUs. Reflection is positioning it primarily for coding, reasoning, and agentic work.

The real question is whether all that post-training compute bought something more valuable than another respectable benchmark table. Reflection’s argument is that Beam reinforcement learning improved the amount of useful capability the model can extract from each generated token.

That is a much more interesting claim, and a much harder one to prove.

1. What Is Reflection AI Beam?

Beam is currently an early-access, text-only model designed around software engineering, reasoning, tool use, and longer agentic workflows. Its 501B parameter count describes the full model, while sparse routing activates roughly 23B parameters for each token.

The 1M-token effective context comes from Beam’s midtraining stage, which mixed long code repositories, long-horizon tasks, and high-quality long-form documents. That is separate from the RL campaign, where rollout context reached up to 256K tokens.

Calling Beam a “23B model” would therefore be misleading. Calling it a conventional 501B dense model would be wrong too. Its design sits between those intuitions.

2. Beam AI Pricing, API Access, And Release Status

There is no public Beam AI pricing to compare yet. That is important because Reflection repeatedly emphasizes inference efficiency, but lower estimated compute does not automatically reveal what developers will actually pay.

The sensible pricing table, for now, is a status table rather than a collection of invented dollar figures.

Reflection says the weights will ship with documentation and tooling for running, evaluating, and fine-tuning the model, plus integrations with open-source libraries and distribution partners. The source material does not provide a public per-token price, so any current “$X per million tokens” estimate would be speculation.

For builders, this means the most interesting commercial question remains unanswered: does Beam’s claimed compute efficiency translate into meaningfully cheaper production inference?

3. Beam 501B Architecture: How Can Only 23B Parameters Be Active?

Beam uses a sparse Mixture-of-Experts, or MoE, architecture.

Instead of running every parameter for every token, an MoE model contains multiple specialized expert blocks and routes each token through a subset of them. The full 501B parameters provide model capacity, while about 23B participate in the computation for a particular token.

That distinction explains how a very large model can have a smaller generation-compute footprint than its headline parameter count suggests.

It does not mean you only need enough memory to store 23B parameters. The experts that are inactive for one token may be required for the next one, so the broader model still needs to be available to the inference system.

Reflection also describes several less marketable but technically important architecture choices: interleaved local and global attention, fine-grained routed experts, load-balancing mechanisms, controlled residual scaling, SandwichNorm, attention gating, and FP32 residual accumulation. Its goal was not merely good pretraining loss, but a numerically stable base that could survive unusually heavy downstream RL.

That matters because Beam’s architecture was effectively built backward from the post-training workload Reflection wanted to run.

4. From 23.8 Trillion Tokens To 100 Million RL Rollouts

Beam’s development has two distinct scaling stories.

First came conventional pretraining. Reflection used 23.8 trillion curated tokens spanning web material, code, technical documents, scientific content, and licensed datasets. The company says its pipeline rejected roughly 95% of raw internet tokens while retaining material that simpler filters would have thrown away.

The full pretraining run completed in under four weeks on 6,144 NVIDIA GB300 NVL72 GPUs.

Then came the unusual part.

Reflection devoted 10,500 GB300 GPUs to four weeks of high-compute RL, producing more than 100 million rollouts. Training and grading involved roughly 1.3 billion sandbox instances and close to one million coding, STEM, terminal, tool-use, and agentic environments. Rollouts could reach 256K tokens.

A rollout is essentially one attempt by a model to interact with a task or environment. For an agent, that can involve reasoning, writing code, running commands, inspecting results, correcting mistakes, and trying again.

More rollouts therefore mean more opportunities to explore strategies and learn from outcomes. But scale alone isn’t enough. If the environments are trivial, broken, exploitable, or badly graded, more RL simply produces more bad training data.

Reflection says data quality became a direct bottleneck during Beam training, with weaker task pools causing capability plateaus until the environments were improved.

5. What Did 100 Million RL Rollouts Actually Change?

The headline is not simply that Beam completed 100 million attempts. It is that Reflection reports continued benchmark improvement as RL compute increased, without an observed capability plateau during the run.

One particularly useful result concerns reasoning length.

Reflection trained Beam with a controllable penalty for unnecessary tokens. Early in RL, performance improved while responses became shorter. In other words, the model appeared to learn better strategies rather than merely thinking for longer.

Later, as more demanding agentic skills developed, completion length rose again at higher reasoning settings and performance continued improving. Beam exposes this tradeoff through a reasoning-effort control.

That creates a more nuanced picture than “more tokens equals more intelligence.” The target is a better capability-per-token frontier.

Reflection also reports transfer between tasks. During one training phase containing reasoning, software-engineering, and terminal environments but no browsing tasks, browsing performance still improved. Beam reportedly learned to search the web, query other language models, and use OCR services when tools were available.

That is potentially important. If reproducible, large-scale RL may be teaching reusable agent behaviors rather than benchmark-specific tricks.

It does not prove that 100 million rollouts are an optimal recipe for every LLM. It shows that Reflection’s own evaluation suite continued responding to additional RL compute.

6. How Beam Makes Asynchronous Reinforcement Learning Work

Running 100 million rollouts creates an awkward engineering problem: the model generating an experience may no longer be the model being trained by the time that experience arrives.

Reflection used asynchronous policy gradients. Inference workers kept producing trajectories while training workers updated the policy independently.

That creates policy staleness.

Imagine an agent starts a long coding job using model version 400. While it works, training advances through versions 401, 402, and beyond. When that trajectory finally returns, parts of it were generated by an older policy.

Beam reportedly remained stable even when training data was as much as 107 weight versions, roughly a day, behind the current policy.

Reflection handled this partly by tagging tokens with the policy version that generated them. Its infrastructure averaged about 110,000 concurrent rollouts, pushed updated weights to inference workers in roughly 12 seconds median, and dynamically varied inference-to-training GPU ratios between 3.9:1 and 5.4:1.

The system also supported up to 170,000 concurrent sandboxes and processed more than one billion sandbox-creation requests.

This may ultimately be Beam’s most consequential contribution. Future agentic systems need to learn from long, expensive interactions. Waiting for every trajectory to finish before training resumes wastes extraordinary amounts of compute. Stable asynchronous reinforcement learning for LLMs is one way around that bottleneck.

7. Reflection AI Beam Benchmarks: How Good Is It Really?

Beam is strong, but the complete benchmark picture is less dramatic than a simple “frontier open model” label suggests.

Below is the full comparison supplied in the research material. NR means no result was reported. Benchmark scores come from different evaluation sources and should not be treated as perfectly controlled head-to-head measurements.

The table makes one thing obvious. Reflection AI Beam is not the universal raw-capability champion.

8. Beam Vs GLM, DeepSeek, Kimi, And Qwen

Newer competitors beat Beam on several individual tests.

DeepSeek V4.1 Flash leads DeepSWE and Terminal Bench in the supplied results. Kimi K3 leads SWE Bench Pro v2-Hard, HLE, BrowseComp, and AA-LCR. Qwen 3.8 Max leads tau3, IFBench, and LongBench among models with reported scores. GLM 5.3 is substantially ahead on SciCode.

Beam does look excellent against Inkling and Nemotron 3 Ultra across many of the overlapping evaluations, and scores such as 80.9 on SWEBench Verified and 90.5 on GPQA Diamond are strong.

That changes how the model should be judged.

The case for Beam is not “Reflection built the smartest open model.” The stronger case is “Reflection built a model near the upper end of open-model capability while activating far fewer parameters per generated token than some much larger competitors.”

That brings us to the claim that matters most.

9. Is Beam Really Three To Four Times More Efficient?

Reflection says Beam can achieve reasoning performance comparable to GLM-5.2 while using three to four times less estimated inference compute.

But “inference compute” has a specific meaning here.

Reflection approximates generation compute using:

FLOPs ≈ 2 × active parameters × generated tokens

For MoE systems, it uses the number of parameters activated per token rather than total model size.

That is reasonable for comparing generation-side forward-pass work, but Reflection explicitly says the estimate excludes prompt prefill, context-dependent attention operations, and serving overhead.

So a claim of three to four times less estimated generation compute does not automatically mean:

• three to four times lower API prices
• three to four times lower electricity use
• three to four times lower end-to-end latency
• three to four times higher production throughput

Those depend on hardware utilization, batching, memory movement, context length, quantization, routing efficiency, KV-cache costs, and the serving stack.

This distinction turns Reflection’s efficiency result from a marketing slogan into a useful technical claim. Beam appears to move the capability-versus-generation-compute frontier. Whether it moves the capability-versus-dollar frontier by the same amount remains unknown.

10. Reflection AI Beam Hardware Requirements: Can You Run It Locally?

This is where “23B active” can cause the most confusion.

MoE sparsity reduces how many parameters are computed with per token. It does much less to reduce how many parameters must be stored.

Using simple weight-only arithmetic for 501 billion parameters:

Those numbers exclude KV cache, activations, routing buffers, framework overhead, and other runtime memory. A 1M-token context can make memory requirements even more demanding.

So Beam hardware requirements are not comparable to a normal 23B dense model. Even an idealized 4-bit checkpoint is roughly a quarter-terabyte before runtime overhead.

In practical terms, serious self-hosting will almost certainly mean multi-accelerator or distributed inference rather than an ordinary consumer GPU. Exactly how well aggressive quantization works, and what hardware configurations Reflection officially supports, cannot be known until the weights and serving stack arrive.

The useful rule is simple: MoE dramatically lowers compute per token, not necessarily the amount of memory needed to keep the model available.

11. What Reflection AI Beam Proves, And What Remains Unverified

Beam gives the open-model ecosystem several concrete things to examine.

Reflection has described a 501B sparse model with 23B active parameters, a massive pretraining run, a 100M-plus-rollout RL campaign, and engineering for asynchronous agent training at unusual scale. It also plans an Apache 2.0 release rather than keeping the model permanently behind a closed endpoint.

The more ambitious claims still need independent testing.

Benchmark reproducibility matters. So do real API costs, wall-clock latency, sustained serving throughput, long-context behavior, quantized quality, tool reliability, and hardware requirements outside Reflection’s infrastructure.

The 100 million rollout result deserves the same distinction. Reflection observed continued gains on its own evaluation suite. That is evidence that reinforcement learning remained productive at this scale. It is not yet evidence that every frontier model should spend its next training budget the same way.

That caveat does not make Beam less interesting. It makes the experiment more interesting.

12. Why Beam Matters Beyond Another 500B Model

Reflection AI Beam matters because it shifts attention from one familiar scaling question to another.

For years, much of the industry asked how far capability could be pushed by adding pretraining data, parameters, and compute. Beam asks what happens when enormous compute budgets are spent after pretraining, letting models repeatedly act, fail, adapt, and learn inside difficult environments.

The early answer from Reflection is that the RL curve still had room to run.

If that result survives independent evaluation, capability per generated token could become as important as raw benchmark score. A model that reaches nearly the same answer with less active computation and fewer unnecessary reasoning tokens may be more useful than a nominal benchmark winner that burns far more compute getting there.

For developers, the next step is straightforward: wait for the weights, API pricing, technical report, and reproducible serving data before accepting the efficiency headline at face value.

For everyone else, the important number may not be 501 billion after all. It may be 100 million.

Binary Verse AI will be tracking the Reflection AI Beam open-weight release, independent benchmarks, real-world inference costs, and self-hosting results as they arrive. If you care about what AI models actually deliver rather than just launch-day scores, follow BinaryVerseAI.com for the deeper technical breakdowns.