id: c815852224e9491ebc3e0b07b34d671a
parent_id: 
item_type: 1
item_id: 6794e0bb60ae4da68720868546ddf628
item_updated_time: 1784277453309
title_diff: "[{\"diffs\":[[1,\"Prototype: Web Traffic Interception (TamperMonkey + Rust)\"]],\"start1\":0,\"start2\":0,\"length1\":0,\"length2\":57}]"
body_diff: "[{\"diffs\":[[1,\"# Prototype: Web Traffic Interception System\\\n\\\n**Date:** 2026-06-08  \\\n**Context:** First prototype for interfacing poker bot with a website  \\\n**Stack:** Firefox + TamperMonkey → Rust (Axum) backend → SQLite  \\\n\\\n---\\\n\\\n## Architecture\\\n\\\n```\\\n[Firefox + TamperMonkey] --HTTP/WS--> [Rust Server (Axum)] --> [SQLite]\\\n```\\\n\\\n### TamperMonkey (Client)\\\n- Intercepts XHR/Fetch requests and overrides prototypes\\\n- Extracts page content via DOM observation\\\n- Sends data to backend via `GM_xmlhttpRequest` (bypasses CORS)\\\n- Batching + debouncing for performance\\\n- Offline retry queue for reliability\\\n- Configurable URL filters (include/exclude patterns)\\\n\\\n### Rust Backend (Axum)\\\n- REST endpoints: `/api/traffic`, `/api/traffic/batch`, `/api/content`\\\n- WebSocket endpoint: `/ws` for real-time streaming\\\n- SQLite storage via `sqlx`\\\n- CORS middleware (permissive for dev)\\\n- Key dependencies: `axum 0.7`, `tokio`, `sqlx`, `serde`, `dashmap`, `tower-http`\\\n\\\n---\\\n\\\n## Project Structure\\\n\\\n```\\\ntraffic-collector/\\\n├── backend/\\\n│   ├── Cargo.toml\\\n│   ├── src/\\\n│   │   ├── main.rs          # Server bootstrap, CORS, tracing\\\n│   │   ├── handlers.rs      # REST + WS route handlers\\\n│   │   ├── models.rs        # TrafficEntry, ContentSnapshot, ResourceType\\\n│   │   └── db.rs            # SQLite init, insert, query\\\n│   └── migrations/\\\n│       └── 001_init.sql\\\n├── userscript/\\\n│   └── traffic-collector.user.js\\\n└── docker-compose.yml\\\n```\\\n\\\n---\\\n\\\n## Key Data Models\\\n\\\n**TrafficEntry:** `id`, `session_id`, `url`, `method`, `status`, `request_headers`, `response_headers`, `request_body`, `response_body`, `content_type`, `size_bytes`, `duration_ms`, `timestamp`, `page_url`, `resource_type`\\\n\\\n**ContentSnapshot:** `id`, `session_id`, `page_url`, `title`, `content`, `meta_description`, `meta_keywords`, `timestamp`\\\n\\\n---\\\n\\\n## TamperMonkey Userscript Grants\\\n\\\n```javascript\\\n// @grant  GM_xmlhttpRequest     // Cross-origin requests (bypasses CORS)\\\n// @grant  GM_getValue            // Persistent storage\\\n// @grant  GM_setValue\\\n// @grant  GM_addValueChangeListener\\\n// @grant  GM_websocket           // WebSocket from userscript\\\n// @connect localhost              // Allow connections to local server\\\n// @run-at document-start          // Hook before page scripts load\\\n```\\\n\\\n---\\\n\\\n## Firefox Advantages\\\n- More permissive `GM_xmlhttpRequest` than Chrome\\\n- `about:debugging#/runtime/this-firefox` for TamperMonkey inspection\\\n- Better privacy model\\\n- `GM.registerMenuCommand` for UI controls (toggle capture, flush buffer, show stats)\\\n\\\n---\\\n\\\n## JetBrains Setup\\\n- **RustRover** for Rust backend development\\\n- **WebStorm** (optional) for userscript editing\\\n- Run config with env vars: `DATABASE_URL`, `RUST_LOG`, `BIND_ADDR`\\\n- File template for `.user.js` in Settings → Editor → File and Code Templates\\\n\\\n---\\\n\\\n## Running\\\n\\\n```bash\\\n# Start backend\\\ncd backend\\\nRUST_LOG=debug DATABASE_URL=sqlite:./data/traffic.db cargo run\\\n\\\n# Verify\\\ncurl http://localhost:3000/health\\\n# → {\\\"success\\\":true,\\\"data\\\":\\\"healthy\\\",\\\"error\\\":null}\\\n\\\n# Install userscript: TamperMonkey Dashboard → Utilities → Import .user.js\\\n```\\\n\\\n---\\\n\\\n## Verification Checklist\\\n\\\n| Component | Check |\\\n|---|---|\\\n| Rust backend compiles | `cargo build` ✓ |\\\n| Server starts | `curl localhost:3000/health` ✓ |\\\n| TamperMonkey installed | Firefox addon ✓ |\\\n| Userscript enabled | TM Dashboard ✓ |\\\n| Traffic captured | Browser Console `[TrafficCollector]` logs ✓ |\\\n| Data stored | `sqlite3 data/traffic.db \\\"SELECT COUNT(*) FROM traffic\\\"` ✓ |\\\n\\\n---\\\n\\\n## Next Steps for Poker Integration\\\n- Define target site URL patterns in `@match` and `FILTER.include`\\\n- Parse poker-specific data structures (hand histories, player actions, table state)\\\n- Add real-time processing pipeline for decision-making\\\n- Extend backend with poker logic modules (connect to existing super-marvin engine)\"]],\"start1\":0,\"start2\":0,\"length1\":0,\"length2\":3761}]"
metadata_diff: {"new":{"id":"6794e0bb60ae4da68720868546ddf628","parent_id":"2c8da247905946c3aa19eb4936e16323","latitude":"0.00000000","longitude":"0.00000000","altitude":"0.0000","author":"","source_url":"","is_todo":0,"todo_due":0,"todo_completed":0,"source":"joplin-desktop","source_application":"net.cozic.joplin-desktop","application_data":"","order":1780904577892,"user_updated_time":1780904577892,"markup_language":1,"is_shared":0,"share_id":"","conflict_original_id":"","master_key_id":"","user_data":"","deleted_time":1784277453309},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2026-07-17T08:46:47.502Z
created_time: 2026-07-17T08:46:47.502Z
type_: 13