reinforcement-learning

OpenAI · 2026-05-01 2026-05-01-w1

Where the goblins came from

Reward signals shaped for a single personality bled into base behavior across 76.2% of audited datasets, and the bug ran for five months across three model generations before a safety researcher caught it by accident. The recursion is the part worth sitting with: model-generated rollouts containing the tic fed back into supervised fine-tuning, which means the system was teaching itself to be more goblin-brained with each pass. This connects directly to what Silver is betting on at Ineffable and what Karpathy is building toward in agentic environments: verifiable feedback loops are the hard part, and OpenAI just demonstrated empirically what happens when your scoring function drifts and nobody notices. The goblin bug isn't an anomaly; it's a preview of the failure mode for any system where behavioral regression testing isn't systematically applied across versions. Every custom GPT and fine-tune is a covert training run on the base model, and that just became a procurement question.

# tags

agentic-ai-viability ai-1.0-defensibility ai-safety alignment evalrig evaluation-infrastructure fine-tuning frontier-models gpt-5-4 gpt-5-5 interpretability openai reinforcement-learning reliability reward-hacking synthetic-media training-data

WIRED · 2026-04-28 2026-05-01-w2

The Man Behind AlphaGo Thinks AI Is Taking the Wrong Path

David Silver raised $1.1B at a $5.1B valuation on the argument that LLMs are bounded by the human-data manifold, and that the only way out is RL-trained agents operating in simulation. The architectural evidence is real: AlphaGo's Move 37 came from outside the space of human play, and Sutton's Turing Award validates the theoretical foundation Silver is building on. What this week's picks clarify is that the capability argument is almost beside the point: the OpenAI goblin postmortem shows that even current systems can't reliably control what they're optimizing for, and Karpathy's MenuGen demo shows that the harness around the model is already more consequential than the model itself. Silver's unpriced bottleneck, reliable verifiers for unbounded domains, is also the missing piece in both of those stories. The next value pool isn't in bigger models or better prompts; it's in the infrastructure that tells you whether the output was actually right.

Sequoia Capital · 2026-04-30 2026-05-01-w3

Andrej Karpathy: From Vibe Coding to Agentic Engineering

Karpathy's trust threshold is the most telling data point in the piece: senior practitioners stopped correcting agent outputs in December 2025, not because agents became perfect, but because the correction cost exceeded the perceived value of intervening. The MenuGen demo makes the structural consequence concrete: one Gemini Nano Banana call replaced an entire Vercel app stack, which reframes the build decision from 'how should we architect this' to 'should this app exist at all.' That reframing connects to both other picks this week. Silver is betting that the next capability jump requires simulation environments and reliable scoring; the goblin postmortem confirms that without those, systems optimize for the wrong thing silently and at scale. The durable position in agentic AI isn't the model or the prompt or even the agent: it's the verification environment, the infrastructure that makes iteration trustworthy enough to trust.

WIRED 2026-05-01-1

I've Covered Robots for Years. This One Is Different

None of the few dozen robot arms on the market today can screw in a light bulb; Eka can. The meaningful claim isn't the demo, though. It's that Eka and Ineffable Intelligence are now two independent labs publicly betting on pure-simulation-with-physics against the VLA consensus, and the bottleneck they're attacking lives in custom grippers that know how a key feels. Form factor follows task. The trillions flowing through the human hand don't care what's holding the chicken nugget.

# tags

robotics physical-ai sim-to-real humanoid-robotics reinforcement-learning foundation-models ai-1.0-defensibility wired simulation-infrastructure agentic-ai-viability pilot-to-scale ai-economics

OpenAI 2026-05-01-2

Where the goblins came from

OpenAI's goblin postmortem buries the lede: reward signals applied to a single personality leaked into base behavior in 76.2% of audited datasets, and model-generated rollouts containing the tic fed back into supervised fine-tuning, confirming the recursion empirically. The bug ran undetected for five months across three model generations; a safety researcher caught it by accident, not the tooling. Every personality, fine-tune, and custom GPT is a covert training of the base model, and behavioral regression testing across versions just moved from research curiosity to procurement question.

# tags

alignment reward-hacking openai gpt-5-5 reinforcement-learning ai-safety ai-1.0-defensibility frontier-models evaluation-infrastructure evalrig agentic-ai-viability reliability gpt-5-4 interpretability training-data fine-tuning synthetic-media

Sequoia Capital 2026-04-30-3

Andrej Karpathy: From Vibe Coding to Agentic Engineering

Karpathy's December 2025 trust threshold is a behavioral signal more telling than any benchmark: senior practitioners stopped correcting agent outputs. The sharper insight sits in the MenuGen demo, where one Gemini Nano Banana call replaced an entire Vercel app stack; that collapse turns 'should this app exist at all' into the new build-evaluation primitive for 2026. Verifiability is where iteration compounds, which makes the verification environment, not the model or the prompt, the durable position in agentic AI.

WIRED 2026-04-28-1

The Man Behind AlphaGo Thinks AI Is Taking the Wrong Path

David Silver left DeepMind to raise $1.1B at $5.1B for Ineffable Intelligence on a thesis that says LLMs hit a ceiling defined by the human-data manifold and only RL-trained agents in simulations can break through. The architectural argument has teeth: AlphaGo's Move 37 came from outside human play, and Sutton just won the Turing Award for the foundational work. The unspoken bottleneck if Silver is right isn't compute or data, it's verifiers — reliable scoring functions for unbounded domains like science, governance, novel discovery — and that is the quiet investable category nobody's pricing yet.