MaxEnt-GRPO

This documentation now treats the upstream OAT stack as the canonical training surface for the repository. The active path is the README-flash understand-r1-zero baseline plus the local listwise maxent-explorer overlay on top of it.

Older TRL/Hydra orchestration and pre-canonical launchers are still retained under archive/trl/, but they are no longer presented as the default way to train in this repo.

Get Started

Getting Started

• Overview
• Canonical Runtime
• Quickstart
• What Is Archived
• Quick Links

Guides

Guides

• Method Identity
  • Current 1.5B Math Presets
  • Source of Truth
  • Family-Specific Code
  • Recommended Convention
• Training Architecture
  • Flow Diagram
  • Stage Breakdown
• OAT Upstream DR.GRPO
  • Working Baseline
  • Explorer Overlay
  • Safety Guardrails
  • Implementation Notes
  • Guarantees And Limits
• Training
  • Canonical Launchers
  • Shared Runtime
  • Objective Routing
  • What Changed
  • Archive Location
• Runtime Setup
  • Canonical OAT Runtime
  • Why This Matters
  • Active Launchers
  • Environment Notes
  • Archived Runtime Surface
• Evaluation
  • Benchmarks
  • Launching Jobs
  • Notes
• CLI Usage
  • Command Routing
  • Recipes and Overrides
  • Validation
  • Examples
• Recipes
  • Recipe Layout
  • Loading and Validation
  • Dataset Mixtures
  • Math GRPO (Qwen 1.5B)
  • Paired Recipes (GRPO vs MaxEnt)
  • Tips
  • Hydra recipes

Reference

Reference

• API Reference
  • Top-level Modules
  • Configuration
  • CLI
  • Core
  • Rewards
  • Training
  • Utilities

Project Notes

• Active launchers live in ops/ and are OAT-only.

• Retired launchers live in archive/trl/.

• The runtime audit entrypoint is tools/audit_oat_setup.py.