id: 86daf6dfd4304999b5908c4a899ddb61
parent_id: 27630096c4a148ca8da7a3e72a4bd59e
item_type: 1
item_id: 986f40b025fb454ab643b22d60cc5369
item_updated_time: 1782586573400
title_diff: "[{\"diffs\":[[0,\"06-2\"],[-1,\"6: KEHU Port, Hand Analysis, Sweeps\"],[1,\"7: Pot Odds Fix, Curve Sweeps, Call Mismatch, Gen 5 Design\"]],\"start1\":23,\"start2\":23,\"length1\":39,\"length2\":62}]"
body_diff: "[{\"diffs\":[[0,\"06-2\"],[-1,\"6: KEHU Port, Hand Analysis, Sweeps, Gen 5\\\n\\\n## 1. Live Hand Analysis (Gatling Gun Table)\\\n\\\n### Session Overview\\\n- **Duration**: ~2 hours (17:00–19:00), 500/1000 blinds, 9-max\\\n- **Hands played**: ~40, stack 799K → 1,271K (+472K / +473 BB)\\\n- **Observer data**: 14 hands tracked, 12 player profiles + table snapshot saved to disk\\\n\\\n### Key Findings\\\n\\\n**Cold Start Problem (4 hands wasted):**\\\nObserver reset to Gen3 defaults for first 4 hands after restart. Profile persistence saves to disk but `cold_start` threshold doesn't check for loaded data — counts from zero each restart. **Needs fix**: check if profile files exist on disk and set `hands_tracked` from loaded data.\\\n\\\n**Passive Postflop Play:**\\\nNearly every first-in postflop decision was CHECK_DEFAULT. Bot rarely bets for value or applies pressure. Main weakness — too much checking, even with made hands (e.g., Ac5d top pair checked all 3 streets).\\\n\\\n**Notable Decisions:**\\\n- **JsKd (trips) — WON 600K pot.** Called 60K turn shove with JJJ+K on As Jd 6h Jc. Risky (facing AJ/AA = full house) but got there. Main profit source.\\\n- **AdQd — fold to 3-bet.** Reasonable tight fold.\\\n- **Ac5d (top pair) — checked all 3 streets.** Should have value-bet river.\\\n- **TcTs on Q-high board — called 16K into 48K.** Marginal but OK.\\\n\\\n**Table Stats:** VPIP 0.63, PFR 0.30, AF 0.94, F2B 0.51. Loose-passive field.\\\n\\\n**Profile Persistence Working:**\\\n12 player profiles saved to `profiles/cash_nl_500_1000/`. Display name fix deployed (save_player takes explicit key param, profile.name is display-only). Player files use Torn ID as key.\\\n\\\n---\\\n\\\n## 2. KEHU20BBNeqV3 Port (g2_ke_hu_nash)\\\n\\\n### Implementation\\\n- **Ported from Java** `KEHU20BBNeqV3.java` (Kill Everyone HU Nash equilibrium tables)\\\n- **19 CSV tables** (169 hands each) embedded as Rust static constants in `ke_hu_tables.rs` (3,272 lines auto-generated from CSVs)\\\n- **Strategy**: `holdem_bots/src/sng/ke_hu_nash.rs` — full decision tree for 15BB and 20BB stack depths\\\n- **State tracking**: Uses `Mutex<KehuHandState>` for per-hand commitments (shove_to_reraise, max_3bet_to_call, oppo_limped_from_sb), reset via `hand_number` detection\\\n- **Hand index**: `ke_hand_index(r0, r1, suited)` → 0-168 (0=22, 168=AA)\\\n- **Registered** as `g2_ke_hu_nash` (19th strategy in registry)\\\n- **Config**: `max_stack_bb = 25.0` (configurable via TOML)\\\n\\\n### Chain Placement\\\n**CRITICAL**: KEHU must be placed BEFORE `g1_short_stack_preflop` in the chain, because `g1_short_stack_preflop` intercepts all ≤25BB preflop decisions and KEHU would never fire. KEHU is more specific (HU only), so it goes first.\\\n\\\nChain order: `g2_ke_hu_nash` → `g1_short_stack_preflop` → `ng_preflop`/`g4_adaptive_preflop` → postflop\\\n\\\nFor SNG: `g1_sage_endgame` (≤14BB) → `g2_ke_hu_nash` (15-25BB) → `g1_short_stack_preflop_sng`\\\n\\\n### Configs Updated (7 files)\\\nAll Gen 2+ configs updated:\\\n- cash_nl.toml, cash_nl_gen2.toml, cash_nl_g4.toml, cash_nl_g4_live.toml\\\n- cash_nl_loose.toml, cash_nl_normal.toml\\\n- sng_gen2.toml\\\n\\\n### Tests\\\n- 9 unit tests pass (hand index, applicability, pot odds, cum, config)\\\n- Full suite: 454 tests pass (426 prior + 9 KEHU + 19 Gen5)\\\n- Registry test updated to expect 20 strategies\\\n\\\n### HU Tournament Results (3 of 6 matchups completed)\\\n2-seater HU, 20BB stacks, 5000 hands × 5 seeds:\\\n\\\n| Matchup | KEHU Bot | No-KEHU Bot | Diff (chips) | BB/hand |\\\n|---------|----------|-------------|-------------|---------|\\\n| Gen 2 (g2_ke vs g2_no_ke) | +17,458 | -17,458 | 34,916 | **+3.49** |\\\n| Gen 3 (g3_ke vs g3_no_ke) | +22,285 | -22,311 | 44,596 | **+4.46** |\\\n| Gen 4 (g4_ke vs g4_no_ke) | +14,833 | -14,833 | 29,666 | **+2.97** |\\\n\\\n**KEHU provides massive HU improvement across all generations.** Gen 3 benefits most.\\\n\\\nCross-generation matchups didn't complete — tournament process crashed during 4th matchup. Results from 3 completed matchups are conclusive.\\\n\\\n---\\\n\\\n## 3. Narrow Sweep 2 — Final Results\\\n\\\n9 variants × 5 seeds × 50k hands, 6-max vs mixed field (2 flock + 2 chump + 1 test):\\\n\\\n| Variant | Gen4 BB/100 | Delta vs Baseline | Notes |\\\n|---------|-------------|-------------------|-------|\\\n| baseline (ts=0.0, no narrow) | 407.79 | — | Reference |\\\n| narrow_only (ts=0.0) | 387.75 | −20.04 | Narrowing without ts HURTS |\\\n| narrow_ts03 | 436.70 | +28.91 | |\\\n| narrow_ts05 | 461.54 | +53.75 | |\\\n| **narrow_ts07** | **488.56** | **+80.77** | **BEST (+19.8%)** |\\\n| scurve_narrow_ts05 | 452.86 | +45.07 | S-curve < linear |\\\n| scurve_narrow_ts03 | 426.76 | +18.97 | |\\\n| narrow_s05_ts05 | 455.89 | +48.10 | Narrowing strength 0.5 |\\\n| narrow_s08_ts05 | 459.54 | +51.75 | Narrowing strength 0.8 |\\\n\\\n**Range Prediction Accuracy:** Linear variants = 100% hit rate. S-curve variants = 86.6% hit rate (13.4% miss — some opponents fall outside predicted range).\\\n\\\n### Key Findings\\\n1. **Threshold scaling is monotonically beneficial** — no plateau at 0.7\\\n2. **Linear ranges > S-curve** at same ts level (and 100% vs 86.6% hit rate)\\\n3. **Narrowing alone hurts** without ts compensation (−20 BB/100)\\\n4. **Narrowing strength doesn't matter much** (s05=455.89, s08=459.54, s1.0=461.54 at ts05)\\\n5. **Best config: narrow_ts07** (+80.77 BB/100 over baseline = +19.8%)\\\n\\\n---\\\n\\\n## 4. Gen 4 Deep Sweep (Running Overnight)\\\n\\\n`scripts/gen4_deep_sweep.sh` — 17 variants × 5 seeds × 50k hands, started 22:27, ETA ~12:00 next day.\\\n\\\nExploring:\\\n- Threshold scaling 0.8/0.9/1.0 (extending the winning trend past 0.7)\\\n- Narrowing strength 0.8/1.2/1.5\\\n- Narrowing center offset ±0.1\\\n- S-curve steepness 0.3/0.7/1.0/1.5\\\n- S-curve floor 0.0/0.05/0.10\\\n- Best combinations (scurve+narrow+high ts)\\\n\\\nResults in `/tmp/gen4_deep_sweep/`. Log: `/tmp/gen4_deep_sweep/sweep_log.txt`.\\\n\\\n---\\\n\\\n## 5. Gen 5 RL Framework (Started)\\\n\\\n### Implemented\\\n- **Feature extraction** (280 dims): card/equity (5), hand category (9), draws (6), board texture (10), betting/situation (25), hero state (5) = **55 active features**\\\n- **Action space** (11 discrete): fold/check/call/bet_1_3..bet_2x/raise_2.5x/raise_pot/all_in, with legality masking\\\n- **Transition recorder**: JSONL format for offline training\\\n- **RL strategy**: `Strategy` trait impl, teacher delegation mode\\\n- **Python training script**: `scripts/train_gen5.py` — PokerPolicy MLP (280→512→512→256→Actor(11)+Value(1))\\\n- **Data collection config**: `configs/bots/gen5_collect.toml`\\\n- **19 unit tests** all pass\\\n\\\n### Pending (Questions for User)\\\n1. **ML framework**: candle-rs vs ONNX vs tch-rs for Rust inference?\\\n2. **Observer access**: How to pass Gen 4 Observer to RL strategy?\\\n3. **Training approach**: HU first or 6-max from start?\\\n4. **Reward signal**: per-hand BB with equity adjustment?\\\n5. **Self-play pool composition**?\\\n\\\nSee Gen 5 Implementation Progress note for details.\\\n\\\n---\\\n\\\n## 6. Files Changed This Session\\\n\\\n### New Files\\\n- `holdem_bots/src/sng/ke_hu_tables.rs` — 19 embedded Nash equilibrium tables (3,272 lines)\\\n- `holdem_bots/src/sng/ke_hu_nash.rs` — KEHU strategy implementation + tests\\\n- `holdem_bots/src/gen5/mod.rs` — Gen 5 module declaration\\\n- `holdem_bots/src/gen5/actions.rs` — 11-action discrete space + legality masking\\\n- `holdem_bots/src/gen5/features.rs` — 280-dim feature extraction\\\n- `holdem_bots/src/gen5/recorder.rs` — JSONL transition recorder\\\n- `holdem_bots/src/gen5/rl_strategy.rs` — RL strategy (Strategy trait impl)\\\n- `configs/bots/_hu/*.toml` — 6 HU test configs\\\n- `configs/bots/gen5_collect.toml` — Imitation learning data collection config\\\n- `scripts/hu_tournament.sh` — HU cross-generation tournament\\\n- `scripts/gen4_deep_sweep.sh` — Gen 4 deep parameter sweep\\\n- `scripts/train_gen5.py` — Python imitation learning training script\\\n\\\n### Modified Files\\\n- `holdem_bots/src/lib.rs` — Added gen5 module\\\n- `holdem_bots/src/sng/mod.rs` — Added ke_hu_nash + ke_hu_tables modules\\\n- `holdem_bots/src/assembly/registrations.rs` — Registered g2_ke_hu_nash + gen5_rl (20 strategies)\\\n- `configs/bots/cash_nl*.toml` (7 files) — Added g2_ke_hu_nash before g1_short_stack_preflop\\\n- `configs/bots/sng_gen2.toml` — Added g2_ke_hu_nash after g1_sage_endgame\"],[1,\"7: Pot Odds Fix, Curve Sweeps, Call Mismatch Fix, Gen 5 Design\\\n\\\n## 1. Stack Loss Root Cause: Pot Odds Blind Spot in `should_raise`\\\n\\\n### Bug\\\n`should_raise()` in `rollout_postflop_gen3.rs` did NOT use pot odds, raise sizing, or SPR. The bot would happily shove all-in with medium hands facing bets, regardless of SPR or pot odds.\\\n\\\n### Fix\\\nAdded three safety mechanisms to `should_raise()`:\\\n1. **Large bet penalty**: When facing a bet >60% pot, threshold is increased proportionally\\\n2. **SPR penalty**: When SPR < 3 and committing >50% of stack, threshold raised by 0.05\\\n3. **Hard equity gate**: If hs < 0.60 and the raise would commit >100% of pot, threshold ≥ 0.65\\\n\\\n### Regression Tests\\\n- `hand_replay/tests/hand_tests.rs`: 2 new tests (18 total, all pass)\\\n- **66 underpair shove bug**: Bot no longer shoves AllIn with pocket 66 on A-high river\\\n- **77 underpair overplay**: Bot doesn't raise flop with underpair 77 on J-high board\\\n- Test data files: `hand_replay/tests/data/66_underpair_shove_bug.txt`, `77_underpair_overplay.txt`\\\n\\\n## 2. Curve Tuning Sweeps (3 Rounds Complete)\\\n\\\n### Round 1 — Discovery Sweep (22 params)\\\nBaseline: flock=928.72, chump=1326.82, combined=2255.55\\\n\\\nTop winners (both opponents improve, marked ***):\\\n| Param | Value | Combined Δ |\\\n|---|---|---|\\\n| bet_turn_base | 0.55 | +458.94 *** |\\\n| raise_base | 0.80 | +70.68 *** |\\\n| raise_w_protection | 0.35 | +60.64 *** |\\\n| scurve_floor | 0.05 | +58.84 *** |\\\n| bet_flop_base | 0.30 | +54.26 *** |\\\n| bluff_hs_max | 0.05 | +52.97 *** |\\\n| call_w_pot_odds | 0.78 | +46.23 *** |\\\n\\\n### Round 2 — Narrowing & Curve Refinement\\\nBaseline (post-R1): flock=1061.19, chump=1563.21\\\n\\\nTop winners:\\\n| Param | Value | Combined Δ |\\\n|---|---|---|\\\n| narrow_center_offset | 0.2 | +46.18 *** |\\\n| range_looseness | 0.05 | +20.72 *** |\\\n| scurve_floor | 0.03 | +16.25 *** |\\\n| raise_w_protection | 0.40 | +6.37 *** |\\\n\\\n### Round 3 — Final Tuning\\\nBaseline (post-R2): flock=1125.79, chump=1584.53\\\n\\\nTop winners:\\\n| Param | Value | Combined Δ |\\\n|---|---|---|\\\n| range_tightness_scale | 0.8 | +30.50 *** |\\\n| strong_hands_fraction | 0.30 | +18.10 *** |\\\n| semi_bluff_max_bet_ratio | 0.30 | +15.80 *** |\\\n| call_w_draw | 0.75 | +15.61 *** |\\\n\\\n### Applied to `cash_nl_g4_live.toml`\\\nAll *** winners applied. Current config values:\\\n```\\\nbet_turn_base=0.58, bet_flop_base=0.30, raise_base=0.80\\\nraise_w_protection=0.35, scurve_floor=0.05, scurve_steepness_mult=0.3\\\nbluff_hs_max=0.05, semi_bluff_max_bet_ratio=0.30\\\nrange_tightness_scale=0.8, strong_hands_fraction=0.30\\\nnarrow_center_offset=0.2, range_looseness=0.05\\\ncall_w_draw=0.75, call_ppot_base=0.25\\\n```\\\n\\\n## 3. Call Mismatch Bug Fixed\\\n\\\n### Root Cause\\\n`reconcile_missing_blinds()` only ran at decision time (waiting_for_action). But call/raise events from other players arrive earlier. When the SB's blind wasn't delivered as an explicit event, `wagered_this_round=0`, so the server thought the SB owed the full BB instead of BB-SB.\\\n\\\n62 call mismatches in the log, all caused by this timing issue.\\\n\\\n### Fix\\\nCall `reconcile_missing_blinds()` at the start of `apply_action()` for all **non-blind** actions (Call, Bet, Raise, Fold, Check). Skip for blind actions themselves to avoid synthesizing the other blind prematurely.\\\n\\\n### Tests\\\n- 21 live_server tests pass (was 20, added `reconcile_blinds_before_call_avoids_mismatch`)\\\n- The \\\"Raise did not exceed current_bet\\\" warnings are **partial all-ins** — working correctly, no fix needed\\\n\\\n## 4. Gen 5 Architecture Note Updated\\\n\\\nComprehensive update to Joplin note `bac569872f9149acbfa4832eb553312c` answering all design questions:\\\n\\\n- **Range prediction NN vs current approach**: Start with current (feed model as features), add range NN in Phase 2 if needed\\\n- **Record all hands**: YES — essential for training data\\\n- **Preflop + postflop**: Same unified network handles both\\\n- **RL in live play**: Not sufficient alone — need offline training first\\\n- **Training curriculum**: Gen 1→2→3→4 with statistical advancement criteria\\\n- **Table hopper**: Phase 2 feature, needs performance tracking\\\n- **DOM snapshots**: Add to userscript for debugging + training data\\\n\\\n## 5. Replay Engine\\\n\\\n`hand_replay` crate with 18 tests:\\\n- Parses PokerStars-format hand histories\\\n- Replays against any registered bot type (including config-loaded bots)\\\n- `scripts/convert_hand.py` converts live server logs to replay format\\\n- Regression tests prevent known bugs from recurring\\\n\\\n## 6. Files Changed This Session\\\n\\\n### Modified\\\n- `holdem_bots/src/cash/rollout_postflop_gen3.rs` — Pot odds fix in `should_raise()`\\\n- `live_server/src/table_state.rs` — Blind reconciliation before non-blind actions\\\n- `hand_replay/tests/hand_tests.rs` — 2 new regression tests\\\n- `configs/bots/cash_nl_g4_live.toml` — All sweep winners applied\\\n- `configs/bots/cash_nl_normal.toml` — Sweep winners applied\\\n- `configs/bots/cash_nl_loose.toml` — Sweep winners applied\\\n\\\n### New\\\n- `hand_replay/tests/data/66_underpair_shove_bug.txt` — Regression test data\\\n- `hand_replay/tests/data/77_underpair_overplay.txt` — Regression test data\\\n- `scripts/overnight_validation.sh` — Validation sweep script\\\n\\\n## 7. Overnight Work\\\n\\\n- Validation sweep running: 30 sims × 20k hands (g4_live vs chump/flock, g2 vs chump)\\\n- Gen 5 Joplin note fully updated with design decisions\\\n- Next session: wire Gen5 recorder into live server, bridge PlayerModel→OpponentFeatures, start imitation learning data collection\"]],\"start1\":25,\"start2\":25,\"length1\":7952,\"length2\":5360}]"
metadata_diff: {"new":{},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2026-06-27T18:57:39.067Z
created_time: 2026-06-27T18:57:39.067Z
type_: 13