Config Variants
Common launch-script edits and building the docs site
This page collects the edits you make most often and documents how to build and deploy this docs site. All run config lives in the launch script's top-of-file variables (or env overrides) — see Inputs & Configuration.
Common edits
| Goal | Edit |
|---|---|
| Switch policy model | MODEL_PATH (re-check gen_tp divides num_key_value_heads) |
| Different K8s cluster | K8S_KUBECONFIG / K8S_NAMESPACE |
| Switch to Docker mode | HARBOR_ENVIRONMENT_IMPORT_PATH=...:RemoteDockerEnvironment + DOCKER_HOST (see Backends) |
| Reuse a prebuilt venv | VENV_PATH (mind the editable-install gotcha) |
| Bump rollout parallelism | NUM_WORKERS (16 cold-start; raise for steady state) |
| Change the algorithm | adv_estimator (ppo/grpo) + policy_loss_mode (gspo) |
| Different scaffold | HARBOR_AGENT_IMPORT_PATH |
| Opt out of wandb | WANDB_MODE=disabled |
The default profile runs GRPO advantages with a GSPO policy loss
(adv_estimator=grpo, policy_loss_mode=gspo), lr 1e-6, batch
64 × 8 = 512 trials/step, context 40k prompt + 68k response, on Qwen3-30B
with vLLM TP=4.
wandb
Never hardcode the key in the launch script. export WANDB_API_KEY=... in your
shell before launch (or set WANDB_MODE=disabled).
Build & deploy the docs site
This documentation is a fumadocs (Next.js) static
export, deployed to a dedicated Cloudflare Pages project (live-rl-docs).
cd docs
export NVM_DIR="$HOME/.nvm"; [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"; nvm use 22
npm install # first time
npm run dev # local preview at http://localhost:3000
npm run build # static export to out/
bash deploy_cloudflare_pages.sh # build + deploy to live-rl-docsThe deploy script asserts Node >= 20 (via nvm), reuses the Cloudflare credentials
(CLOUDFLARE_API_TOKEN + CLOUDFLARE_ACCOUNT_ID from .env.cf or
~/.config/rl_dashboard_cloudflare.env), and publishes out/. Override the
project with PROJECT_NAME=... (or DOCS_PROJECT_NAME=... in the env file).
To add a page: drop an .mdx file under content/docs/ with title +
description frontmatter, and add its slug to the folder's meta.json pages
array.