Thousands of decisions
Navigation, puzzles, team building, resources and turn-based tactics, chained across dozens of hours with no reset button.
New
Pokémon Black harness: two screens, one agent
Long-horizon AI agents, proven one badge at a time.
Charmi builds harnesses that let frontier models play real Pokémon games from start to finish. We read game memory, give the model typed tools, and verify every step against the game's own flags. Then we stream the run, live.
The Champions Harness broadcast, designed in Figma, with a retained game frame. Team panels show sample data.
120KYouTube subscribers
10.7KTwitch followers
10Pokémon games with a harness
5platforms, Game Boy to GameCube
530+commits since June 2026
Why games
The most honest benchmark for an agent is a game it can't fake.
Most benchmarks end after a few turns. A Pokémon campaign runs for thousands of decisions, and the cartridge keeps score.
Ground truth in memory
Badges, story flags and Pokédex entries live in RAM. Our checkpoints read them directly, so progress is a fact, never a model's claim.
Every decision on screen
Runs stream with the model's reasoning beside the game. Failures are visible, debuggable, and genuinely fun to watch.
How it works
Observe. Decide. Act. Verify.
Every harness runs the same loop. The model makes the decisions. Deterministic code does the precise, mechanical work and checks the result.
01
Observe
A bridge inside the emulator streams memory. Party, bag, map, dialogue, menus and battles become one structured observation.
map Route 119 party 6/6 · HP 61/74 badges 5/8 dialog none
02
Decide
The model gets the current objective, the observation, and only the tools that make sense in this scene.
“Weather Institute is north. Heal first, then walk.”
03
Act
One typed tool call becomes bounded controller input: a path walked with A*, a dialogue advanced, a move picked.
walk_to(x=18, y=9) → 14 tiles · 3 batches
04
Verify
The harness reads the game again. Position, HP, items and story flags confirm what happened before the next turn.
✓ position == (18, 9) ✓ in_battle == false
Product
Everything an agent needs to finish the game.
One foundation, refined across ten games, from 8-bit Johto to the GameCube's Orre.
Verified progress
Progress the game can prove.
Each campaign is a chain of checkpoints, and each checkpoint is backed by a predicate on game memory. A model saying “done” never completes an objective.
- Native flags, scene variables, inventory and Pokédex ownership
- The current objective and checkpoint count stay on screen
- Loading an older save recalculates progress from the game
56checkpoints in Polished Crystal
144in Emerald
361objectives in XD
Operator control
A control room for every run.
Run sessions, steer the agent when it matters, and take the controller back in one click. This is the Champions control room for live ranked doubles.
- Casual or ranked doubles, a set number of battles or Infinity
- Pause, stop now, or stop after the current battle finishes
- Choose the decision model and recovery per session
Model-agnostic
Bring any model. Keep the same game.
The harness owns the controller, the memory and the checks. Swap the model without touching the game, and compare runs on equal footing.
- Claude through MCP or native tool use
- OpenAI Codex, xAI Grok, or any OpenAI-compatible endpoint
- Local models on our own DGX Spark cluster, with opt-in fallback
Typed tools
Up to 78 per game: A* walking, map routing, dialogue, battles, catching, HMs and PC transfers.
MCP adapters
The same tool surface over MCP, so any MCP client can drive a game directly.
Safe by construction
Bounded controller input only. No memory writes, cheats, resets or save edits in the model's tools.
Fingerprinted games
Every bridge checks the exact ROM revision before a run. Hacks and other versions get their own profile.
Broadcast at 1440p
A transparent 2560 × 1440 OBS overlay with exact capture geometry for each console.
Evidence, published
Dated validation records separate live checks from fixtures. We say what isn't verified yet.
Harnesses
Ten games. Five platforms.
One family.
Each harness is built for its game's memory layout, mechanics and story, and shares the same loop, controls and broadcast.
Pokémon Emerald
Littleroot to the Hall of Fame, with double battles, cable cars, currents and Victory Road handled natively.
- 518 maps
- 144 checkpoints
- 64 tools
Pokémon Crystal
New Bark Town to Mt. Silver, with pathfinding, map travel, HM use, PC transfers and puzzle tools.
- 57 milestones
- 16 badges
- 2 regions
Pokémon FireRed
Pallet Town to the League. Silph Co., Cinnabar quizzes, spin floors and boulders get dedicated helpers.
- 425 maps
- 99 checkpoints
- 67 tools
Pokémon Radical Red 4.1
Modern mechanics in a GBA shell: Fairy typing, Mega Evolution, Z-Moves, Dynamax and 120 TMs.
- 1,348 species
- 98 checkpoints
- 78 tools
Pokémon Polished Crystal
Its own memory layout: regional forms, abilities, natures, EVs and a 20-box Pokémon database.
- 605 maps
- 393 Pokédex
- 56 checkpoints
Pokémon XD: Gale of Darkness
RAM-only play in a pinned Dolphin build, with a registry that tracks every Shadow Pokémon.
- 361 objectives
- 83 Shadows
- 0 screenshots
Pokémon Black
Native dual-screen control with touch input, a Unova control room and a 4× DS window for OBS.
- 2 screens
- 15 milestones
- 20-tile walks
Pokémon Champions
Competitive doubles: plans both allies under the battle clock and checks the screen before each input.
- RAM or OCR
- Paired plans
- 1440p overlay
Pokémon Platinum
Our first DS port: DeSmuME in-process, touch input, Gen 4 decryption and a local vision model.
- Gen 4 codec
- Touch input
- Sinnoh goals
Pokémon Ruby
The first harness: a Lua server inside mGBA, a 14-tool Python layer, an MCP server, a reference Claude tool-use loop and a live dashboard. Every harness since grew from it.
Harness repositories are private while we prepare public releases. Every harness needs your own legally obtained game. We never distribute ROMs.
Made to watch
A broadcast for every run.
Each harness ships a public broadcast with the model's live notes, a private trainer's desk, and a transparent OBS overlay that wraps the game capture.
Emerald · Sapphire, Ruby and Emerald color cycle with animated party cards.
Real broadcast interfaces with gameplay placed where OBS puts the game capture. Panels show sample data.
Public
Broadcast
Party, objective, badges and the model's latest note, for anyone with the link.
Owner
Trainer's desk
Model, reasoning and fallback settings, run controls and checkpoint restores.
OBS
Overlay
A transparent 2560 × 1440 browser source that sits on top of the game capture.
Community
Built in public for 130K+ Pokémon fans.
Charmi started as a Spanish-language Pokémon channel. Today that audience is where the harnesses go live, so every run is watched by people who know these games inside out.
Research & infrastructure
Our own inference. Ready for Claude.
Long runs need models that never sleep. We operate our own GPU cluster, open-source what we learn, and speak the protocols Claude already knows.
Two-node NVIDIA DGX Spark cluster
Live telemetry, allowlisted model deployment, and a public recipe that serves Qwen3.8-27B with DFlash2 speculative decoding on a single Spark.
- Code generation
- 56.32 tok/s
- Native context
- 262K tokens
- Concurrent requests
- 10
- Weights · KV cache
- NVFP4 · FP8
MCP · tool use
Ready for Claude from day one
The first harness shipped with a reference Claude tool-use loop and an MCP server. The harness walks, battles and verifies. Claude decides.
# one turn of the loop
resp = client.messages.create(
model="claude-opus-5-5",
tools=TOOLS_SCHEMA, messages=messages)
for block in resp.content:
if block.type == "tool_use":
harness.call_tool(block.name, block.input)What's next
A Claude season
Full campaigns played by Claude, streamed live, with every tool call and checkpoint on screen.
An open, verified leaderboard
Cross-model results scored by the game itself: checkpoints per hour, tool efficiency and unassisted recoveries.
One harness core
A shared engine across mGBA, DeSmuME and Dolphin, with public MCP servers for anyone's own cartridge.
FAQ
Questions, answered.
Anything else? Write to contact@charmi.us.
Does the AI play completely on its own?
That's the goal, and we're upfront about where we are. Our Emerald run reached the Hall of Fame with human help along the way. Every harness keeps a dated validation record, and none claims a flawless unattended playthrough yet.
Which models can play?
Any model with tool calling: Claude through MCP or native tool use, OpenAI Codex, xAI Grok, and local OpenAI-compatible servers such as our DGX Spark cluster. Fallback between providers is opt-in, per run.
Do the harnesses cheat?
No. The model's tools only produce bounded controller input, the same buttons a player presses. Memory writes, cheats, resets and save edits are outside its reach.
Do you distribute games or ROMs?
Never. Every harness requires your own legally obtained copy and checks its fingerprint before a run. ROMs, saves and credentials are excluded from our repositories.
Is Charmi affiliated with Nintendo or The Pokémon Company?
No. Charmi L.L.C. is an independent company. Pokémon and all related names are trademarks of Nintendo, Creatures Inc. and GAME FREAK inc.
Let's build the benchmark agents can't fake.
Labs, researchers and partners: we'd love to hear from you.