Sanity Check & Live Promotion Policy

# Sanity Check & Live Promotion Policy

## Two tiers

### Full Sanity Check (strategy changes, sweep optimizations, new features)

Run: `scripts/sanity_check.sh <bot_config_name>`

All three gates MUST pass:

1. **Hand tests** — All hand replay tests pass (`cargo test --release -p hand_replay`)
2. **Self-play audit** — 1000 hands vs mixed chump/flock table:
   - No panics or fatal errors
   - Guardrail fires reviewed (low count acceptable, investigate patterns)
   - Both candidate instances profitable
3. **Performance vs field** — Positive BB/100 (3 seeds × 1000 hands each) against:
   - Gen 2 (`cash_nl_gen2`)
   - Gen 3 (`cash_nl`)
   - Gen 4.5 (`cash_nl_g45`)
   - Gen 4.7 (`cash_nl_g47`)
   - Flock (`flock_bot`)
   - Chump (`chump_bot`)
   - Mixed field (all of the above together)

### Hotfix Sanity Check (urgent live-play bug fixes)

Required before deploying a hotfix:

1. **Hand tests** — All existing hand replay tests pass
2. **New regression test** — At least one new hand replay test covering the observed issue
3. **Self-play audit** — 1000 hands self-play, no panics, no guardrail anomalies

Full performance sweep NOT required for hotfixes — the fix should be targeted and narrow.

## Continuous Improvement

- Hand test cases are continuously expanded — add new regression hands for any live-play issue discovered
- If a hotfix passes but later causes regressions, add a full sanity check to the next regular update

## Script Details

`scripts/sanity_check.sh` implements the full check. For hotfixes, run steps 1-2 manually:

```bash
# Step 1: Hand tests
cargo test --release -p hand_replay

# Step 2: Add a new hand test for the issue, then re-run

# Step 3: Self-play audit
RUST_LOG=warn nice -n 19 ionice -c 3 ./target/release/holdem_bots --bot-dir configs/bots <selfplay_config.toml>
# Check: grep for "guardrail\|panic\|ERROR" in output
```


id: d632cea3d15f4323bbac7eda687f2467
parent_id: 2c8da247905946c3aa19eb4936e16323
created_time: 2026-07-07T11:44:40.189Z
updated_time: 2026-07-07T11:44:40.189Z
is_conflict: 0
latitude: 0.00000000
longitude: 0.00000000
altitude: 0.0000
author: 
source_url: 
is_todo: 0
todo_due: 0
todo_completed: 0
source: joplin-desktop
source_application: net.cozic.joplin-desktop
application_data: 
order: 1783424680190
user_created_time: 2026-07-07T11:44:40.189Z
user_updated_time: 2026-07-07T11:44:40.189Z
encryption_cipher_text: 
encryption_applied: 0
markup_language: 1
is_shared: 0
share_id: 
conflict_original_id: 
master_key_id: 
user_data: 
deleted_time: 0
type_: 1