id: 7a36a41fdb8c4746a6cae910e97f4c00
parent_id: 
item_type: 1
item_id: de72ff3ebcbd45689c988e50038e80b0
item_updated_time: 1782797515635
title_diff: "[{\"diffs\":[[1,\"Gen 4 — Adaptive Opponent Modelling (State of the Art)\"]],\"start1\":0,\"start2\":0,\"length1\":0,\"length2\":54}]"
body_diff: "[{\"diffs\":[[1,\"# Gen 4 — Adaptive Opponent Modelling\\\n\\\n> **Current state, 2026-06-30. Current production strategy.** Wraps Gen 3 with a dynamic observer that tracks real opponent statistics (VPIP, PFR, AF, bet frequencies) via EMA, then uses those stats to build more accurate ranges and adapt thresholds.\\\n\\\n## Role in the Framework\\\n\\\nGen 4 is the **live production strategy**. It doesn't replace Gen 3's decision logic — it feeds Gen 3 better inputs:\\\n1. **Better ranges**: Observer tracks opponent actions → predicts tighter/more accurate ranges\\\n2. **Adaptive thresholds**: Table VPIP/AF observations override Gen 3 config defaults\\\n3. **Range narrowing**: Per-action within a hand, the opponent's range is narrowed based on their specific actions (bet, call, raise, check)\\\n\\\n## Preflop Maneuver\\\n\\\n| Priority | Strategy | When | Adaptivity |\\\n|---|---|---|---|\\\n| 1 | `g2_ke_hu_nash` | HU, 15-25 BB | Nash equilibrium (no adaptation) |\\\n| 2 | `g1_short_stack_preflop` | ≤25 BB | Push/fold chart (no adaptation) |\\\n| 3 | `g4_adaptive_preflop` | >25 BB | NG preflop + **VPIP-based looseness** (wired) |\\\n\\\n### Adaptive Preflop (`g4_adaptive_preflop`)\\\nWraps `ng_preflop` with table-VPIP-aware looseness. The looseness factor scales open-raise and limp thresholds (clamped 0.5–2.0):\\\n- Table VPIP 0.15 (tight) → looseness 0.85 (we play tighter)\\\n- Table VPIP 0.30 (normal) → looseness 1.0 (standard)\\\n- Table VPIP 0.50 (loose) → looseness 1.15 (we play slightly looser)\\\n- Table VPIP 0.70 (very loose) → looseness 1.30\\\n\\\n## Postflop Maneuver\\\n\\\n### Adaptive Postflop (`g4_adaptive_postflop`)\\\n\\\n**Decision flow:**\\\n1. **Self-observe** (simulation mode): replay this hand's actions into the observer to keep it in sync\\\n2. **Check observer warmth**: if `table.hands_observed < 5`, fall back to Gen 3 defaults (cold start)\\\n3. **For each opponent seat**:\\\n   - `observer.effective_range_params(seat)` → range width/strength (three-tier blend)\\\n   - `observer.effective_action_freqs(seat)` → bet/raise/call frequencies\\\n   - Build `NarrowContext` from observed frequencies + this hand's actions\\\n   - `range_builder::build_opponent_range_v2()` → weighted `HandRange`\\\n4. **Store prediction** for showdown accuracy tracking\\\n5. **Compute config overrides** from table aggregate stats (VPIP, AF)\\\n6. **Build overridden Gen3Config** with table-adaptive thresholds\\\n7. **Delegate to Gen 3**: `gen3.compute_equity_with_ranges(ctx, &ranges)` → equity against predicted ranges\\\n8. **Gen 3 decides**: `gen3.get_action_with_equity(ctx, &equity)` → final action\\\n\\\n### The Observer\\\n\\\nTracks per-opponent statistics using **exponential moving averages** (EMA):\\\n\\\n```rust\\\nstruct PlayerStats {\\\n    vpip: EmaStat,       // Voluntary preflop entry rate\\\n    pfr: EmaStat,        // Preflop raise rate\\\n    af: EmaStat,         // Postflop aggression factor (bet+raise / call)\\\n    fold_to_bet: EmaStat,\\\n    fold_to_cbet: EmaStat,\\\n    cbet_freq: EmaStat,  // Continuation bet frequency\\\n    threebet: EmaStat,   // 3-bet frequency\\\n    wtsd: EmaStat,       // Went-to-showdown rate\\\n    bet_freq: EmaStat,   // First-in postflop bet rate\\\n    raise_freq: EmaStat, // Facing-bet raise rate\\\n    call_freq: EmaStat,  // Facing-bet call rate\\\n    hands_observed: usize,\\\n}\\\n```\\\n\\\n### Three-Tier Blend\\\n\\\n- **<5 hands observed**: pure table aggregate\\\n- **5-50 hands**: linear blend (gradually trust player-specific data)\\\n- **≥50 hands**: pure player stats\\\n\\\n### Range Prediction Accuracy\\\n\\\nAt showdown, the observer checks whether the opponent's actual hole cards were within the predicted range. Current accuracy: **~88% hit rate** across all streets. Accuracy is uniform across flop/turn/river.\\\n\\\n### Profile Persistence\\\n\\\nProfiles saved to `profiles/{format}_{sb}_{bb}/` (table + per-player JSON). Loaded on startup via `update_table_info()` and `on_hand_start()`. Survives server restarts.\\\n\\\n## Configs\\\n\\\n| Config | Chain | Purpose |\\\n|---|---|---|\\\n| `cash_nl_g4_live.toml` | ke_hu → short_stack → g4_adaptive_preflop → g4_adaptive_postflop | **Live production** |\\\n| `cash_nl_g4.toml` | same chain, default thresholds | Gen 4 baseline (non-tuned) |\\\n| `cash_nl_g4_loose.toml` | same chain, looser starting thresholds | Calling-station tables |\\\n| `cash_nl_g4_tight.toml` | same chain, tighter starting thresholds | Skilled tables |\\\n\\\n### Current Live Tuning (post-discount sweep, 2026-06-29)\\\n\\\n| Parameter | Value | Rationale |\\\n|---|---|---|\\\n| `bet_turn_base` | 0.40 | +251 combined Δ (was 0.58) |\\\n| `bet_flop_base` | 0.15 | +79 combined Δ (was 0.30) |\\\n| `scurve_steepness_mult` | 0.3 | +45 combined (curve sweep R1) |\\\n| `bluff_hs_max` | 0.05 | +53 combined (curve sweep R1) |\\\n| `narrow_center_offset` | 0.2 | +46 combined (curve sweep R2) |\\\n| `strong_hands_fraction` | 0.30 | +18 combined (curve sweep R3) |\\\n| `call_w_draw` | 0.75 | +16 combined (curve sweep R3) |\\\n\\\n### 60k-Hand Validation (2026-06-29)\\\n\\\n- G4Live: **+132.9/hand** vs chump field (was +100.5 pre-fix)\\\n- G4Live: **+0.95/hand** vs gen field (was +0.52 pre-fix)\\\n\\\n### Key Sweep Findings (consolidated)\\\n\\\n- **Threshold scaling is the strongest lever** — weight map parameters have only ±2/hand impact when thresholds are fixed. The opponent model is already well-calibrated (~88%).\\\n- **Linear ranges > S-curve** — S-curve consistently underperforms linear at the same threshold scaling level.\\\n- **Narrowing strength has zero effect** — 0.8× = 1.0× = 1.2× = 1.5× all give identical results.\\\n- **Center offset helps slightly** — +0.1 to +0.3 shifts ranges wider (less aggressive narrowing).\\\n- **Gen 4 dominates Gen 3 by ~70%** vs chumps (148 vs 86 profit/hand).\\\n- **Loose/tight config spread is small** for Gen 4 (2 pts) vs Gen 3 (17 pts) — the adaptive layer absorbs table-type differences.\\\n\\\n## Key Files\\\n\\\n| File | Content |\\\n|---|---|\\\n| `holdem_bots/src/gen4/adaptive.rs` | AdaptivePostflopGen4 strategy |\\\n| `holdem_bots/src/gen4/adaptive_preflop.rs` | AdaptivePreflopGen4 strategy (looseness wired) |\\\n| `holdem_bots/src/gen4/observer.rs` | Observer (EMA tracking, event callbacks) |\\\n| `holdem_bots/src/gen4/profile.rs` | PlayerStats, PlayerProfile, TableProfile, three-tier blend |\\\n| `holdem_bots/src/gen4/range_builder.rs` | Range construction (S-curve + narrowing) |\\\n| `holdem_bots/src/gen4/storage.rs` | ProfileStore (JSON persistence) |\\\n| `holdem_bots/src/gen4/accuracy.rs` | Showdown-based range accuracy tracking |\\\n| `holdem_bots/src/gen4/model_registry.rs` | Model snapshot registry |\\\n\\\n## Open Issues\\\n\\\n1. **Passive postflop play**: Bot still under-bets first-in postflop situations. The post-discount sweep (`bet_flop_base=0.15`, `bet_turn_base=0.40`) improved aggression, but the bot could still value-bet more frequently. This is the main remaining weakness — likely needs Gen 5 (learned betting frequencies) to fully address.\\\n\\\n2. **Range prediction misses**: 88% hit rate means 12% of shown hands fall outside the predicted range. Misses are concentrated against extreme opponents (very tight or very loose). Further improvement likely needs a learned range model (Gen 5 Phase 2).\\\n\\\n3. **No automated sweep pipeline**: The GUI sweep tool can sweep 84 parameters but requires manual winner selection. An automated sweep → apply loop would accelerate tuning.\\\n\\\n## Where This Goes Next\\\n\\\nGen 5 replaces the heuristic decision logic with a **neural network** trained via imitation learning (DAgger) from Gen 4 teacher decisions. The observer continues to provide input features, but range prediction and action selection become learned rather than hand-tuned.\"]],\"start1\":0,\"start2\":0,\"length1\":0,\"length2\":7423}]"
metadata_diff: {"new":{"id":"de72ff3ebcbd45689c988e50038e80b0","parent_id":"e13f1845de9b4b6392ad866354fbd562","latitude":"0.00000000","longitude":"0.00000000","altitude":"0.0000","author":"","source_url":"","is_todo":0,"todo_due":0,"todo_completed":0,"source":"joplin-desktop","source_application":"net.cozic.joplin-desktop","application_data":"","order":1782623817134,"markup_language":1,"is_shared":0,"share_id":"","conflict_original_id":"","master_key_id":"","user_data":"","deleted_time":0},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2026-06-30T05:37:45.258Z
created_time: 2026-06-30T05:37:45.258Z
type_: 13