AI 101: The State of Reinforcement Learning in 2025

Reinforcement learning in 2025: RLVR surprising findings, GRPO vs PPO, RLHF vs RLAIF, agentic RL, robotics advances, and Karpathy's critique explained

Read the full version on Turing Post

Reinforcement Learning (RL) – a topic that’s constantly present in 2025. Ever since DeepSeek-R1 kicked off the year with a major reasoning breakthrough, researchers and developers have increasingly turned to RL-based training as a way to strengthen reasoning and agentic behavior. We saw hundreds of research on RL everywhere: in LLMs, VLMs coding domain, agentic systems, robotics, and beyond.

Unlike supervised learning (depends on labeled datasets), or unsupervised learning (searches for patterns without guidance), RL is built on interaction. The system learns by doing, by nudging the environment and watching how it reacts. Rewards, penalties, surprises, and mistakes shape the policy more than any static dataset ever could. In other words, it pushes models closer to real-world behavior.

But is reinforcement learning really as strong as it looks right now? This year also surfaced a set of concerns about the strategy.

Today we’re zooming out to see what RL achieved in 2025, where it ran into walls, and what momentum it brings into 2026. Let’s begin the full year-in-review.

Reinforcement Learning is a lot worse than I think the average person thinks. Reinforcement Learning is terrible. It just so happens that everything that we had before in much worse.

Andrej Karpathy

RL moved beyond alignment

For several years, Reinforcement Learning from Human Feedback (RLHF) has been the standard way to align language models with human preferences. Humans compare model outputs, a reward model learns these preferences, and reinforcement learning optimizes the model to produce more desirable responses.

But human feedback is expensive and difficult to scale.

As a result, 2025 saw rapid adoption of Reinforcement Learning from AI Feedback (RLAIF), where powerful AI models or specialized evaluators replace human judges. This dramatically reduces cost while allowing much larger-scale training, although it introduces new risks: evaluator bias, feedback loops, and imperfect alignment with human values.

RLVR became the biggest story of the year

One of the year’s most influential ideas was Reinforcement Learning with Verifiable Rewards (RLVR). RLVR rewards outputs that can be verified automatically through unit tests, math proofs, execution results, or logical constraints. This approach powered many modern reasoning models, including DeepSeek-R1.

Rather than rewarding subjective preferences, RLVR rewards objective correctness.

It also triggered one of the biggest research debates of 2025.

One group of researchers argued that RLVR does not create fundamentally new reasoning abilities while others found that RLVR genuinely improves reasoning and helps models discover better solution paths.

The discussion is still ongoing, but both sides agree on one thing: evaluation has become one of the hardest problems in modern AI.

Reinforcement learning also received criticism

Despite its success, reinforcement learning is far from universally accepted.

Andrej Karpathy summarized the criticism:

“Reinforcement Learning is terrible. It just so happens that everything we had before was much worse.”

His concern is not that RL fails, but that it rewards every step in a successful reasoning path instead of learning which parts truly led to the solution.

Policy optimization evolved rapidly

2025 also became the year of new RL optimization algorithms — newer methods were designed specifically for reasoning models.

The most influential was Group Relative Policy Optimization (GRPO), introduced by DeepSeek. Instead of estimating rewards for every individual action, it compares multiple candidate solutions together, producing a more stable training signal for complex reasoning tasks.

Other approaches, including GSPO, DAPO, BAPO, and ARPO, focused on improving long reasoning chains, stabilizing training, or enabling better tool use in autonomous agents.

RL expanded far beyond language models

Perhaps the biggest surprise of 2025 was how broadly reinforcement learning spread.

Coding assistants, math models, GUI agents, robotics also gained momentum learning with RL. So it is becoming a general optimization layer across AI systems.

So what does 2026 have in store for RL?