CUDA Agent: ByteDance Taught AI to Write Faster GPU Code Without a Single Programmer

CUDA Agent: ByteDance Taught AI to Write Faster GPU Code Without a Single Programmer

Ondřej Barták
Ondřej Barták
Entrepreneur and Programmer
4. 3. 2026
4 minutes reading · 42 views
CUDA Agent: ByteDance Taught AI to Write Faster GPU Code Without a Single Programmer

Optimizing GPU code is one of the most difficult disciplines in software engineering. It is not enough to write code that works. It also has to be fast. And that is a completely different game. Most developers are satisfied when their code passes the tests. But that is not enough for CUDA kernels. A correct result and a fast result are two different things. And this is exactly the problem researchers from ByteDance Seed and Tsinghua University decided to tackle. The result? A system called CUDA Agent that outperforms standard compilers and the world's leading language models.

What is CUDA Agent?

CUDA Agent is a large-scale reinforcement learning system designed to automatically generate and optimize CUDA kernels for deep learning. It sounds technical, but the basic idea is simple: it is an AI agent that learned to write fast GPU code by being rewarded for actual speed on hardware, not merely for producing correct output.

Most previous approaches worked differently. Generate code, check whether it compiles, and fix any errors. Done. But GPU performance depends on factors that no correctness test can reveal: memory access, compute unit occupancy, shared memory, or memory bank conflicts. You can only see these in a profiler.

CUDA Agent changes the reward criterion. Instead of asking, "Did the code compile?" it asks, "Is the code faster than torch.compile?" And by at least 5%.

The results leave the competition far behind

The results from testing on the KernelBench benchmark (250 GPU kernels divided into three difficulty levels) are impressive. CUDA Agent achieved an overall speedup of 2.11x compared with torch.compile and was faster than this standard PyTorch compiler in 96.8% of cases. By comparison, Claude Opus 4.5 achieved a speedup of 1.46x and Gemini 3 Pro 1.42x. At the most difficult third level, which involves composite fused kernels such as entire ResNet blocks, CUDA Agent outperformed both models by 40 percentage points in the share of cases faster than torch.compile (90% vs. 50% and 52%).

Even more interesting is the comparison with the base model. Seed 1.6 without reinforcement learning training achieved only 0.69x — meaning it was slower than torch.compile. After training, the same architecture jumped to 2.11x. That is a threefold improvement purely due to the training method.

KernelBench results - The base model (Seed 1.6) starts at 0.69x - it is actually slower than torch.compile. After RL training, the same architecture achieves a speedup of 2.11x. That is a 3x improvement in effective speedup of
KernelBench results - The base model (Seed 1.6) starts at 0.69x - it is actually slower than torch.compile.

How the agent thinks and works

The agent operates in a ReAct-style loop: it analyzes performance, identifies bottlenecks, writes a custom CUDA kernel, compiles it, tests it, and then continues optimizing. It can perform up to 200 steps per task within a context window of 131,000 tokens.

Optimization takes place across three layers. First come the algorithmic changes with the greatest impact: kernel fusion, shared memory, and coalesced memory access. Next come hardware utilization optimizations: vectorized data loading, warp primitives, and occupancy tuning. Finally, there is fine-tuning: mixed precision, loop unrolling, and avoiding memory conflicts.

To prevent the agent from cheating, the system includes five safeguards: protected profiling scripts, a ban on fallback PyTorch calls, correctness verification on five different inputs, synchronized performance measurement, and no internet access.

Training that nearly failed

This is where the most interesting part of the story begins. The Seed 1.6 base model is a 230-billion-parameter sparse MoE model with 23 billion active parameters. CUDA code accounts for less than 0.01% of its pretraining data. This mismatch nearly doomed the entire project. Without special multi-stage warm-up, training collapsed as early as step 17. The cause: CUDA tokens have a probability of around 10⁻⁹ in the model, which caused sampling ratios to explode in the PPO (proximal policy optimization) algorithm.

The solution came in four stages: first, a one-time PPO warm-up on 6,000 synthetic operators; then trajectory filtering and supervised fine-tuning; followed by critic pretraining for stable value estimates; and finally full agentic reinforcement learning. Every stage is essential. Omitting the agentic loop reduces the share of faster kernels from 96.8% to just 14.1%.

What is publicly available and what remains behind closed doors

The research team released a research paper, a project page, and the CUDA-Agent-Ops-6K training dataset on Hugging Face. This dataset contains 6,000 synthetically generated operators with thorough contamination checks against the test data.

However, the weights of the trained model itself are not publicly available. The Seed 1.6 base model is accessible through ByteDance's API, but the full CUDA Agent remains behind closed doors. ByteDance also maintains a smaller project called cudaLLM, built on the Qwen3-8B model, which is available under the Apache 2.0 license and offers a more accessible alternative for researchers with limited computing resources.

The question hanging in the air is: when will ByteDance open up the model? The method has been proven. The results are compelling. And if an AI agent can consistently outperform standard compilers on real hardware, it could fundamentally change how deep learning compute libraries are created. That would be a genuine leap forward for the entire field.

Advertisement

Content created with help from UpTier.

SEO and GEO on autopilot. UpTier’s multi-agent systems write and optimize content for search engines and AI answers.

Discover UpTier ↗

Category:AI
Did you enjoy this article?
Discover more interesting posts on our blog
Back to blog

Related posts

Google makes Tasks free for Gemini 4 Hydrogen usersGoogle makes Tasks free for Gemini 4 Hydrogen users
Tasks is becoming available for free to hundreds of millions of Gemini 4 Hydrogen users. Google says it can call businesses, book appointments and complete purchases on their behalf.
1 min read
18. 11. 2026
OpenAI introduces GPT-6.2 Astra and Sol for professional and everyday workOpenAI introduces GPT-6.2 Astra and Sol for professional and everyday work
OpenAI has introduced two models: Astra for complex professional tasks and Sol for everyday work. Both are available in ChatGPT Work, Codex and through the API.
3 min read
17. 11. 2026
Tencent releases Hy4 Flash with image support and a context window of up to 1 million tokensTencent releases Hy4 Flash with image support and a context window of up to 1 million tokens
The lighter Hy4 Flash model targets chat, documents and routine code edits. It accepts text and images, supports tool use and, according to Tencent, matched Claude Haiku 5.5 in tests of everyday tasks.
1 min read
16. 11. 2026
Přihlaste se k odběru našeho newsletteru
Zůstaňte informováni o nejnovějších příspěvcích, exkluzivních nabídkách, a aktualizacích.
CodedTrip

Operated by CodedTrip LLC, USA.

YouTube
TikTok