id: e6db3832305f43f58758368f3371b7b5
parent_id: 
item_type: 1
item_id: 226d902e8ddd4baebd546b6f2fe2613b
item_updated_time: 1782797680215
title_diff: "[{\"diffs\":[[1,\"Range Prediction Accuracy Tracking — Implementation & Tournament Results\"]],\"start1\":0,\"start2\":0,\"length1\":0,\"length2\":72}]"
body_diff: "[{\"diffs\":[[1,\"# Range Prediction Accuracy Tracking — Implementation & Tournament Results\\\n\\\n## Date: 2026-06-25\\\n\\\n## What Was Built\\\n\\\n### Showdown-based accuracy tracking\\\n- Added `showdown_event` to the `Strategy` trait (default no-op), forwarded through `ModularStrategy` and `StrategyBot`\\\n- Gen4's `AdaptivePostflopGen4` implements `showdown_event` → routes revealed cards to `Observer::on_showdown`\\\n- Observer stores per-opponent `HandRange` predictions at decision time, checks them against actual cards at showdown\\\n- Thread-local `AccuracyData` collector accumulates records across all Gen4 bots in the simulation\\\n- Summary printed after simulation in `main.rs`\\\n\\\n### Metrics tracked\\\n- **Hit rate**: fraction of shown hands with weight > 0 in predicted range\\\n- **Average weight**: mean predicted weight of actual hands (higher = better prediction)\\\n- **Weight distribution**: high (≥0.8) / mid (0.3-0.8) / low (0-0.3) / zero\\\n- **By confidence**: low (<0.3) / mid (0.3-0.7) / high (≥0.7) observer confidence\\\n- **By opponent VPIP**: tight (<0.25) / medium (0.25-0.45) / loose (≥0.45)\\\n- **By street**: flop / turn / river\\\n\\\n### Files modified\\\n- `holdem_bots/src/common/modular.rs` — Strategy trait + ModularStrategy showdown forwarding\\\n- `holdem_bots/src/strategy_bot.rs` — Forward showdown_event to strategy\\\n- `holdem_bots/src/gen4/observer.rs` — Prediction storage + accuracy checking in on_showdown\\\n- `holdem_bots/src/gen4/adaptive.rs` — Store predictions during get_action, implement showdown_event\\\n- `holdem_bots/src/gen4/accuracy.rs` (new) — Thread-local collector + summary reporting\\\n- `holdem_bots/src/gen4/mod.rs` — Export accuracy module\\\n- `holdem_bots/src/main.rs` — Print accuracy summary after cash simulation\\\n- `scripts/range_eval.sh` (new) — Overnight evaluation script\\\n\\\n## Tournament Results (10 seeds × 100k hands, old binary)\\\n\\\n| Bot Type | Avg Profit/Hand | BB/100 | StdDev |\\\n|----------|-----------------|--------|--------|\\\n| Chump    | -235.25 | -1176.2 | 2.44 |\\\n| Flock    | +55.96 | +279.8 | 1.47 |\\\n| Gen2     | +39.27 | +196.3 | 1.08 |\\\n| Gen3     | +60.06 | +300.3 | 0.95 |\\\n| **Gen4** | **+79.95** | **+399.8** | **1.63** |\\\n\\\n**Gen4 vs Gen3: +99.5 BB/100 improvement** (33% relative). Range prediction is clearly valuable.\\\n\\\n## Smoke Test Accuracy Data (500 hands, chump opponents only)\\\n\\\n```\\\nTotal showdowns checked: 768\\\nIn-range hit rate: 100.0% (768/768)\\\nAverage predicted weight: 0.624\\\n\\\nWeight distribution: high(≥0.8)=444 (58%) | mid=8 (1%) | low=316 (41%) | zero=0\\\n\\\nBy observer confidence:\\\n  Low  (<0.3):  100.0% / 0.549 / 70\\\n  Mid  (0.3-0.7): 100.0% / 0.613 / 109\\\n  High (≥0.7): 100.0% / 0.635 / 589\\\n```\\\n\\\n**Observation**: avg_weight increases with confidence (0.549 → 0.635), showing adaptation works.\\\n\\\n## Overnight Run\\\n\\\nLaunched `scripts/range_eval.sh`: 10 seeds × 50k hands with accuracy tracking.\\\nFull tournament lineup (2×Chump + 2×Flock + 2×Gen2 + 2×Gen3 + 2×Gen4).\\\n\\\nExpected: accuracy data across different opponent types (tight Flock vs loose Chump vs medium Gen2/Gen3).\\\n\\\n## Next Steps\\\n- Analyze overnight accuracy data: are ranges predicted well for different opponent types?\\\n- Tune range parameters based on accuracy findings\\\n- Consider gradient range tails (instead of hard cutoff) for better prediction\"]],\"start1\":0,\"start2\":0,\"length1\":0,\"length2\":3221}]"
metadata_diff: {"new":{"id":"226d902e8ddd4baebd546b6f2fe2613b","parent_id":"abc167de888d41bf9391c373e90dec8d","latitude":"0.00000000","longitude":"0.00000000","altitude":"0.0000","author":"","source_url":"","is_todo":0,"todo_due":0,"todo_completed":0,"source":"joplin-desktop","source_application":"net.cozic.joplin-desktop","application_data":"","order":1782415068086,"user_updated_time":1782415068086,"markup_language":1,"is_shared":0,"share_id":"","conflict_original_id":"","master_key_id":"","user_data":"","deleted_time":1782797680215},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2026-06-30T05:37:46.384Z
created_time: 2026-06-30T05:37:46.384Z
type_: 13