backend-infra-engineer: Release v0.3.9-hotfix7 snapshot
This commit is contained in:
scawful
230
docs/internal/plans/ai-infra-improvements.md
Normal file
230
docs/internal/plans/ai-infra-improvements.md
Normal file
@@ -0,0 +1,230 @@
|
||||
# AI Infrastructure Improvements Plan
|
||||
|
||||
**Branch:** `feature/ai-infra-improvements`
|
||||
**Created:** 2025-11-21
|
||||
**Status:** Planning
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the gaps in yaze's AI infrastructure (gRPC services, MCP integration) and the planned improvements to make yaze-mcp fully functional for AI-assisted SNES ROM hacking.
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### yaze-mcp Tools (28 Emulator + 10 ROM = 38 total)
|
||||
|
||||
| Category | Tools | Status |
|
||||
|----------|-------|--------|
|
||||
| Emulator Lifecycle | pause, resume, reset | Working |
|
||||
| Execution Control | step_instruction, run_to_breakpoint | Working |
|
||||
| Execution Control | step_over, step_out | Partial (step_over falls back to single-step) |
|
||||
| Memory | read_memory, write_memory, get_game_state | Working |
|
||||
| Breakpoints | add/remove/list/toggle | Working (execute only during emulation) |
|
||||
| Watchpoints | add/remove/list/history | NOT WIRED (only called from RPC handlers) |
|
||||
| Disassembly | get_disassembly | Partial (returns OPCODE_XX placeholders) |
|
||||
| Tracing | get_execution_trace | BROKEN (returns FAILED_PRECONDITION) |
|
||||
| Symbols | load/resolve/get_symbol_at | DISABLED (ASAR integration disabled) |
|
||||
| ROM Basic | get_rom_info, read_rom_bytes | Working |
|
||||
| ROM Advanced | overworld/dungeon/sprite reads | NOT IMPLEMENTED (stubs return errors) |
|
||||
| ROM Versioning | create/list snapshots | Working |
|
||||
| Input | press_buttons | Working |
|
||||
|
||||
### Critical Gaps
|
||||
|
||||
1. **gRPC Server Not Started** - `AgentControlServer` exists but is never instantiated
|
||||
2. **Watchpoints Bypass Emulation** - Only triggered by RPC read/write, not CPU bus activity
|
||||
3. **Disassembly Uses Placeholders** - No proper 65816 disassembler integration
|
||||
4. **Execution Trace Not Buffered** - No ring buffer for instruction history
|
||||
5. **Symbols Disabled** - ASAR integration commented out
|
||||
6. **ROM Domain RPCs Stubbed** - Overworld/dungeon/sprite return "not yet implemented"
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: gRPC Server Hosting (Priority: Critical)
|
||||
|
||||
**Goal:** Stand up a unified gRPC server that registers EmulatorService + RomService
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/app/editor/editor_manager.cc` - Start AgentControlServer when YAZE_ENABLE_REMOTE_AUTOMATION
|
||||
- `src/cli/service/agent/agent_control_server.cc` - Register both EmulatorService and RomService
|
||||
- `src/app/service/unified_grpc_server.cc` - Consider merging with AgentControlServer
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Add `StartAgentServer()` method to EditorManager
|
||||
- [ ] Wire startup to `YAZE_ENABLE_REMOTE_AUTOMATION` flag or `--enable-grpc` CLI flag
|
||||
- [ ] Register EmulatorService and RomService on same server
|
||||
- [ ] Add configurable port (default 50051)
|
||||
- [ ] Test with yaze-mcp `check_status`
|
||||
|
||||
### Phase 2: Emulator Debug RPCs (Priority: High)
|
||||
|
||||
**Goal:** Flesh out disassembly, execution trace, and stepping
|
||||
|
||||
**2a. Proper Disassembly**
|
||||
- Use DisassemblyViewer's existing instruction recording
|
||||
- Or integrate a standalone 65816 disassembler (bsnes style)
|
||||
- File: `src/cli/service/agent/emulator_service_impl.cc` lines 661-705
|
||||
|
||||
**2b. Execution Trace Buffer**
|
||||
- Add ring buffer (1000-10000 entries) to DisassemblyViewer
|
||||
- Record: address, opcode, operands, cycle count, register snapshot
|
||||
- File: `src/app/emu/debug/disassembly_viewer.h`
|
||||
|
||||
**2c. StepOver Implementation**
|
||||
- Detect JSR (0x20) and JSL (0x22) opcodes
|
||||
- Set temporary breakpoint at return address (PC + instruction length)
|
||||
- Run until breakpoint hit, then remove temporary BP
|
||||
- File: `src/cli/service/agent/emulator_service_impl.cc` lines 598-607
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Integrate real 65816 disassembly into GetDisassembly RPC
|
||||
- [ ] Add ExecutionTraceBuffer class with ring buffer
|
||||
- [ ] Implement GetExecutionTrace from buffer
|
||||
- [ ] Implement proper StepOver with JSR/JSL detection
|
||||
|
||||
### Phase 3: Breakpoint/Watchpoint Memory Integration (Priority: High)
|
||||
|
||||
**Goal:** Wire memory breakpoints and watchpoints into emulator memory bus
|
||||
|
||||
**Current State:**
|
||||
- `BreakpointManager::ShouldBreakOnExecute()` IS called via CPU callback
|
||||
- `BreakpointManager::ShouldBreakOnMemoryAccess()` IS NOT called during emulation
|
||||
- `WatchpointManager::OnMemoryAccess()` IS NOT called during emulation
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/app/emu/snes.h` - Add read/write callbacks
|
||||
- `src/app/emu/snes.cc` - Invoke breakpoint/watchpoint managers in CpuRead/CpuWrite
|
||||
- `src/app/emu/emulator.cc` - Wire managers to callbacks
|
||||
|
||||
**Implementation:**
|
||||
```cpp
|
||||
// In Snes::CpuRead() or via callback:
|
||||
if (debugging_enabled_) {
|
||||
if (breakpoint_manager_.ShouldBreakOnMemoryAccess(addr, BreakpointManager::AccessType::READ)) {
|
||||
running_ = false;
|
||||
}
|
||||
watchpoint_manager_.OnMemoryAccess(addr, /*is_write=*/false, value);
|
||||
}
|
||||
|
||||
// In Snes::CpuWrite() or via callback:
|
||||
if (debugging_enabled_) {
|
||||
if (breakpoint_manager_.ShouldBreakOnMemoryAccess(addr, BreakpointManager::AccessType::WRITE)) {
|
||||
running_ = false;
|
||||
}
|
||||
watchpoint_manager_.OnMemoryAccess(addr, /*is_write=*/true, value);
|
||||
}
|
||||
```
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Add `on_memory_read_` and `on_memory_write_` callbacks to CPU
|
||||
- [ ] Invoke BreakpointManager from callbacks
|
||||
- [ ] Invoke WatchpointManager from callbacks
|
||||
- [ ] Add MCP tools for watchpoints: `add_watchpoint`, `list_watchpoints`, `get_watchpoint_history`
|
||||
- [ ] Test memory breakpoints and watchpoints with yaze-mcp
|
||||
|
||||
### Phase 4: Symbol Loading & Resolution (Priority: Medium)
|
||||
|
||||
**Goal:** Load ASAR/WLA-DX/CA65 symbol files and enable label resolution
|
||||
|
||||
**Current State:**
|
||||
- EmulatorServiceImpl has stubbed symbol methods returning "not available"
|
||||
- ASAR wrapper exists in `src/core/asar_wrapper.h`
|
||||
|
||||
**Implementation Approach:**
|
||||
1. Create `SymbolTable` class to store symbols (name -> address map)
|
||||
2. Implement parsers for each format:
|
||||
- ASAR: `.sym` files with `label = $XXXXXX` format
|
||||
- WLA-DX: `.sym` files with different format
|
||||
- CA65: `.dbg` or `.map` files
|
||||
- Mesen: `.mlb` label files
|
||||
3. Wire LoadSymbols RPC to parse and populate SymbolTable
|
||||
4. Wire ResolveSymbol/GetSymbolAt to query SymbolTable
|
||||
|
||||
**Files to Create:**
|
||||
- `src/app/emu/debug/symbol_table.h`
|
||||
- `src/app/emu/debug/symbol_table.cc`
|
||||
- `src/app/emu/debug/symbol_parser.h` - Format parsers
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Design SymbolTable class (bidirectional lookup)
|
||||
- [ ] Implement ASAR .sym parser
|
||||
- [ ] Implement WLA-DX parser
|
||||
- [ ] Wire to EmulatorServiceImpl
|
||||
- [ ] Test with Oracle of Secrets symbols
|
||||
|
||||
### Phase 5: ROM Domain RPCs (Priority: Medium)
|
||||
|
||||
**Goal:** Implement overworld/dungeon/sprite read/write RPCs
|
||||
|
||||
**Current State:**
|
||||
- All domain RPCs return "not yet implemented"
|
||||
- ROM class has raw access, but not structured zelda3 data
|
||||
|
||||
**Implementation:**
|
||||
- Leverage `zelda3::Overworld`, `zelda3::Dungeon`, `zelda3::Sprite` classes
|
||||
- Need to instantiate these in RomServiceImpl or get from shared state
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/app/net/rom_service_impl.cc` - Implement ReadOverworldMap, ReadDungeonRoom, ReadSprite
|
||||
- Proto messages already defined in `rom_service.proto`
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Add zelda3::Overworld access to RomServiceImpl
|
||||
- [ ] Implement ReadOverworldMap (tile16 data for 160 maps)
|
||||
- [ ] Implement WriteOverworldTile
|
||||
- [ ] Add zelda3::Dungeon access
|
||||
- [ ] Implement ReadDungeonRoom (tile16 data for 296 rooms)
|
||||
- [ ] Implement WriteDungeonTile
|
||||
- [ ] Implement ReadSprite
|
||||
|
||||
### Phase 6: yaze-mcp Enhancements (Priority: Low)
|
||||
|
||||
**Goal:** Improve MCP error handling and add missing tools
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Add timeout/retry logic based on gRPC status codes
|
||||
- [ ] Add clearer error messages for unimplemented RPCs
|
||||
- [ ] Add watchpoint tools to server.py
|
||||
- [ ] Document required build preset and port
|
||||
- [ ] Add connection health monitoring
|
||||
|
||||
## File Reference
|
||||
|
||||
### Emulator Service
|
||||
- **Header:** `src/cli/service/agent/emulator_service_impl.h`
|
||||
- **Implementation:** `src/cli/service/agent/emulator_service_impl.cc` (822 lines)
|
||||
- **Server:** `src/cli/service/agent/agent_control_server.cc`
|
||||
|
||||
### ROM Service
|
||||
- **Header:** `src/app/net/rom_service_impl.h`
|
||||
- **Implementation:** `src/app/net/rom_service_impl.cc`
|
||||
- **Version Manager:** `src/app/net/rom_version_manager.h`
|
||||
|
||||
### Debug Managers
|
||||
- **Breakpoints:** `src/app/emu/debug/breakpoint_manager.h|cc`
|
||||
- **Watchpoints:** `src/app/emu/debug/watchpoint_manager.h|cc`
|
||||
- **Disassembly:** `src/app/emu/debug/disassembly_viewer.h`
|
||||
|
||||
### Emulator Core
|
||||
- **Emulator:** `src/app/emu/emulator.h|cc`
|
||||
- **SNES:** `src/app/emu/snes.h|cc`
|
||||
- **CPU:** `src/app/emu/cpu/cpu.h`
|
||||
|
||||
### MCP Server
|
||||
- **Location:** `/Users/scawful/Code/yaze-mcp/server.py`
|
||||
- **Proto Stubs:** `/Users/scawful/Code/yaze-mcp/protos/`
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. **yaze-mcp `check_status`** connects and returns full emulator state
|
||||
2. **Memory breakpoints** pause emulation on WRAM/SRAM access
|
||||
3. **Watchpoints** track and log all memory accesses in specified ranges
|
||||
4. **`get_disassembly`** returns proper 65816 mnemonics
|
||||
5. **`get_execution_trace`** returns last N instructions executed
|
||||
6. **Symbol loading** works with ASAR output from Oracle of Secrets
|
||||
7. **ROM domain RPCs** return structured overworld/dungeon/sprite data
|
||||
|
||||
## Notes
|
||||
|
||||
- Consider performance impact of memory access callbacks (may need optimization)
|
||||
- May want debug mode toggle to enable/disable expensive instrumentation
|
||||
- Future: Canvas automation service for GUI automation via MCP
|
||||
Reference in New Issue
Block a user