Gen 5 Implementation Progress — 2026-06-26

# Gen 5 Implementation Progress — 2026-06-27

## Status: Full framework + neural network + training pipeline working

## Decision: candle-rs (pure Rust ML)

Chosen by user. No Python dependency. Train and inference all in Rust.

## What's Implemented

### Module Structure (`holdem_bots/src/gen5/`)
- `mod.rs` — module declaration + re-exports
- `actions.rs` — 11-action discrete space with legality masking (7 tests)
- `features.rs` — 280-dim feature vector extraction (5 tests)
- `recorder.rs` — JSONL transition recorder (3 tests)
- `rl_strategy.rs` — `Strategy` trait impl with model inference (3 tests)
- `network.rs` — **candle-nn MLP + Trainer** behind `gen5_nn` feature (4 tests)

### Neural Network (`network.rs`)
```
Input(280) → Linear(512) → ReLU
           → Linear(512) → ReLU
           → Linear(256) → ReLU
           → Actor head: Linear(11)  [with legal action masking]
           → Value head: Linear(1)
```
- `PokerPolicy::predict(features, legal_mask)` → `(action, probs, value)` for inference
- `PokerPolicy::forward(features, legal_mask)` → `(logits, value)` for batch training
- `Trainer::new(lr, device)` — creates optimizer + model
- `Trainer::train_batch(samples)` — cross-entropy loss + AdamW backward step
- `Trainer::evaluate(samples)` — accuracy on validation set
- `Trainer::save(path)` — safetensors format
- `save_model()` / `load_model()` — standalone save/load

### RL Strategy Model Integration
- `model_infer()` uses `PokerPolicy::predict()` when `gen5_nn` feature is enabled
- Falls back to teacher strategy when model not loaded or `use_teacher=true`
- Graceful error handling: inference errors fall back to teacher

### Feature Vector (280 dims, 215 active)
| Section | Dims | Status |
|---------|------|--------|
| Card/Equity | 5 | ✅ hs, ppot, npot, nutpot, rpot |
| Hand category | 9 | ✅ one-hot (HighCard→StraightFlush) |
| Draw flags | 6 | ✅ flush_draw, OESD, gutshot, paired, monotone, connected |
| Board texture | 10 | ✅ board_count, wetness, max_suit, straight_conn, high_card |
| Betting/Situation | 25 | ✅ pot_bb, to_call_bb, SPR, stack_bb, position, street, counts |
| Hero state | 5 | ✅ wagered_bb, aggressor flags, facing bet |
| Opponent models | 60 | ⬜ PENDING (need Observer access) |
| Action history | 160 | ✅ last 8 actions (player, street, action_type, amounts) |

### Data Collection & Training
- `configs/bots/gen5_collect.toml` — bot config for imitation learning data collection
- `scripts/train_gen5.py` — Python training script (backup, candle is primary)
- Rust Trainer ready for JSONL → train → safetensors pipeline

## Tests
- Without candle: 454 tests pass
- With candle (--features gen5_nn): 458 tests pass (454 + 4 network tests)
- Release binary built with `--features gen5_nn`

## Build Commands
```bash
# Normal build (no neural network):
cargo build --release -p holdem_bots

# With neural network support:
cargo build --release -p holdem_bots --features gen5_nn

# Run tests with neural network:
cargo test -p holdem_bots --features gen5_nn
```

## What's NOT Implemented Yet

### 1. Opponent Model Features (60 dims)
Need Observer access in RL strategy. Open question for user.

### 2. JSONL Training Driver
Need a command-line tool or script that:
1. Reads JSONL transition file
2. Batches samples
3. Runs Trainer::train_batch in a loop
4. Evaluates on validation set
5. Saves best model

The Trainer is ready, just needs the driver loop.

### 3. Self-play Data Generation
Need to run simulations with `gen5_collect.toml` to generate training data.

## Remaining Open Questions for User

1. **Observer access pattern:** How should RL strategy access Gen 4 Observer?
   - Option A: Pass `Arc<Mutex<Observer>>` via config parameter
   - Option B: RL strategy creates its own Observer
   - **Recommendation:** Option A

2. **Training approach:** HU first (5× faster) or 6-max?
   - **Recommendation:** HU first

3. **Reward signal:** Per-hand BB with all-in equity adjustment?
   - **Recommendation:** Yes

4. **Self-play pool:** Mixed (Gen 1-4 + checkpoints) or pure self-play?
   - **Recommendation:** Mixed pool

id: 486b52d6fe994be1ab6d74735bfe322f
parent_id: 5a06903f7db44bfcb4c8c8a9cf0d2326
created_time: 2026-06-26T20:27:44.599Z
updated_time: 2026-06-28T05:18:04.873Z
is_conflict: 0
latitude: 0.00000000
longitude: 0.00000000
altitude: 0.0000
author: 
source_url: 
is_todo: 0
todo_due: 0
todo_completed: 0
source: joplin-desktop
source_application: net.cozic.joplin-desktop
application_data: 
order: 1782505664599
user_created_time: 2026-06-26T20:27:44.599Z
user_updated_time: 2026-06-28T05:14:34.721Z
encryption_cipher_text: 
encryption_applied: 0
markup_language: 1
is_shared: 0
share_id: 
conflict_original_id: 
master_key_id: 
user_data: 
deleted_time: 1782623884873
type_: 1