maxent_grpo.training.controller_optimizer

Meta-optimizer orchestration for controller updates.

Functions

_no_grad_context(torch_mod)

Return a torch.no_grad context when available, else a nullcontext.

Classes

ControllerMetaManager(cfg, weighting)

Manage meta-controller optimizer state and cadence.

class maxent_grpo.training.controller_optimizer.ControllerMetaManager(cfg, weighting)

Bases: object

Manage meta-controller optimizer state and cadence.

Parameters:

• cfg (GRPOConfig)

• weighting (WeightingSettings)

should_run(global_step)

Parameters:

global_step (int)

Return type:

bool

make_backprop_fn()

Return a callback that computes gradients via autograd.

Return type:

Callable[[int], ControllerGradients | None] | None

apply_gradients(gradients, *, lr_scale)

Apply controller updates based on the configured method.

Parameters:

• gradients (ControllerGradients | None)

• lr_scale (float)

Return type:

None