backend-infra-engineer: Post v0.3.9-hotfix7 snapshot (build cleanup)
This commit is contained in:
scawful
75
docs/internal/agents/README.md
Normal file
75
docs/internal/agents/README.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Agent Coordination Hub
|
||||
|
||||
Welcome to the `yaze` internal agent workspace. This directory contains the rules, roles, and records for AI agents contributing to the project.
|
||||
|
||||
## Quick Start
|
||||
1. **Identify Your Role**: Check [personas.md](./personas.md) to choose the correct Agent ID.
|
||||
2. **Load Your Prompt**: Open `.claude/agents/<agent-id>.md` for your persona’s system prompt (available to all agents).
|
||||
3. **Follow the Protocol**: Read [AGENTS.md](../../../AGENTS.md) for rules on communication, task logging, and handoffs.
|
||||
4. **Check Status**: Review the [coordination-board.md](./coordination-board.md) for active tasks and blockers.
|
||||
5. **Keep docs lean**: Use [doc-hygiene.md](./doc-hygiene.md) and avoid creating duplicate specs; archive idle docs.
|
||||
|
||||
## Documentation Index
|
||||
|
||||
### Core Coordination (Keep Visible)
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [AGENTS.md](../../../AGENTS.md) | **The Core Protocol.** Read this first. Defines how to work. |
|
||||
| [personas.md](./personas.md) | **Who is Who.** Canonical list of Agent IDs and their scopes. |
|
||||
| [.claude/agents/*](../../../.claude/agents) | **System Prompts.** Persona-specific prompts for all agents. |
|
||||
| [coordination-board.md](./coordination-board.md) | **The Live Board.** Shared state, active tasks, and requests. |
|
||||
| [collaboration-framework.md](./collaboration-framework.md) | **Team Organization.** Architecture vs Automation team structure and protocols. |
|
||||
|
||||
### Active Projects
|
||||
| File | Purpose | Status |
|
||||
|------|---------|--------|
|
||||
| [initiative-v040.md](./initiative-v040.md) | v0.4.0 development plan with 5 parallel workstreams | ACTIVE |
|
||||
| [handoff-sidebar-menubar-sessions.md](./handoff-sidebar-menubar-sessions.md) | UI systems architecture reference | Active Reference |
|
||||
| [wasm-development-guide.md](./wasm-development-guide.md) | WASM build and development guide | UPDATED |
|
||||
| [wasm-antigravity-playbook.md](./wasm-antigravity-playbook.md) | WASM with Gemini integration and debugging | UPDATED |
|
||||
| [web-port-handoff.md](./web-port-handoff.md) | WASM web build status and blockers | IN_PROGRESS |
|
||||
|
||||
### Documentation Hygiene
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [doc-hygiene.md](./doc-hygiene.md) | Rules to keep specs/notes lean and archived on time. |
|
||||
|
||||
## Tools
|
||||
Agents have built-in CLI capabilities to assist with this workflow:
|
||||
* `z3ed agent todo`: Manage your personal task list.
|
||||
* `z3ed agent handoff`: Generate context bundles for the next agent.
|
||||
* `z3ed agent chat`: (If enabled) Inter-agent communication channel.
|
||||
|
||||
## Picking the right agent
|
||||
- **ai-infra-architect**: AI/agent infra, MCP/gRPC, z3ed tooling, model plumbing.
|
||||
- **backend-infra-engineer**: Build/packaging/toolchains, CI reliability, release plumbing.
|
||||
- **imgui-frontend-engineer**: ImGui/editor UI, renderer/backends, canvas/docking UX.
|
||||
- **snes-emulator-expert**: Emulator core (CPU/APU/PPU), performance/accuracy/debugging.
|
||||
- **zelda3-hacking-expert**: Gameplay/ROM logic, data formats, hacking workflows.
|
||||
- **test-infrastructure-expert**: Test harnesses, CTest/gMock infra, flake/bloat triage.
|
||||
- **docs-janitor**: Docs/process hygiene, onboarding, checklists.
|
||||
Pick the persona that owns the dominant surface of your task; cross-surface work should add both owners to the board entry.
|
||||
|
||||
## Build & Test
|
||||
Default builds now share `build/` (native) and `build-wasm/` (WASM). If you need isolation, set `YAZE_BUILD_DIR` or create a local `CMakeUserPresets.json`.
|
||||
* **Build**: `./scripts/agent_build.sh [target]` (Default target: `yaze`)
|
||||
* **Test**: `./scripts/agent_build.sh yaze_test && ./build/bin/yaze_test`
|
||||
* **Directory**: `build/` by default (override via `YAZE_BUILD_DIR`).
|
||||
|
||||
## Archive
|
||||
|
||||
This directory maintains a **lean, active documentation set**. Historical or reference-only documents are organized in `archive/`:
|
||||
|
||||
### Archived Categories
|
||||
- **large-ref-docs/** (148KB) - Large reference documents (40KB+): dungeon rendering, ZSOW, design specs
|
||||
- **foundation-docs-old/** (68KB) - Older foundational docs (October) on agent architecture and CLI design
|
||||
- **utility-tools/** (52KB) - Tool documentation: filesystem, dev-assist, modularity, AI dev tools
|
||||
- **wasm-planning-2025/** (64KB) - Earlier WASM planning iterations
|
||||
- **testing-docs-2025/** (112KB) - Test infrastructure and CI documentation archives
|
||||
- **gemini-session-2025-11-23/** (40KB) - Gemini-specific session context and task planning
|
||||
- **plans-2025-11/** (56KB) - Completed or archived feature/project plans
|
||||
- **legacy-2025-11/** (28KB) - Completed initiatives and coordination improvements
|
||||
- **session-handoffs/** (24KB) - Agent handoff and onboarding documents
|
||||
- **reports/** (24KB) - Audit reports and documentation assessments
|
||||
|
||||
**Target: 10-15 active files in root directory. Archive completed work within 1-2 weeks of completion.**
|
||||
182
docs/internal/agents/archive/docs-cleanup-2025-11-27.md
Normal file
182
docs/internal/agents/archive/docs-cleanup-2025-11-27.md
Normal file
@@ -0,0 +1,182 @@
|
||||
# Documentation Cleanup - November 27, 2025
|
||||
|
||||
## Summary
|
||||
|
||||
Comprehensive review and update of YAZE documentation, focusing on public-facing docs, web app support, and organizational cleanup.
|
||||
|
||||
## Changes Made
|
||||
|
||||
### 1. Web App Documentation
|
||||
|
||||
**Created: `docs/public/usage/web-app.md`**
|
||||
- Comprehensive guide for the WASM web application
|
||||
- Clearly marked as **Preview** status (not production-ready)
|
||||
- Detailed feature status table showing incomplete editors
|
||||
- Browser requirements and compatibility
|
||||
- Performance tips and troubleshooting
|
||||
- Comparison table: Web vs Desktop
|
||||
- Developer tools and API references
|
||||
- Deployment instructions
|
||||
- Privacy and storage information
|
||||
|
||||
**Key Points:**
|
||||
- ⚠️ Emphasized preview/experimental status throughout
|
||||
- Listed editor completeness accurately (Preview/Incomplete vs Working)
|
||||
- Recommended desktop build for serious ROM hacking
|
||||
- Linked to internal technical docs for developers
|
||||
|
||||
### 2. Main README Updates
|
||||
|
||||
**Updated: `README.md`**
|
||||
- Added web preview mention in highlights section
|
||||
- Added "Web App (Preview)" to Applications & Workflows
|
||||
- Clearly linked to web-app.md guide
|
||||
- Maintained focus on desktop as primary platform
|
||||
|
||||
### 3. Public Docs Index
|
||||
|
||||
**Updated: `docs/public/index.md`**
|
||||
- Added Web App (Preview) to Usage Guides section
|
||||
- Placed at top for visibility
|
||||
|
||||
### 4. Directory Organization
|
||||
|
||||
**Moved technical implementation docs to internal:**
|
||||
- `docs/web/drag-drop-rom-loading.md` → `docs/internal/web-drag-drop-implementation.md`
|
||||
- `docs/wasm/patch_export.md` → `docs/internal/wasm-patch-export-implementation.md`
|
||||
- Removed empty `docs/web/` and `docs/wasm/` directories
|
||||
|
||||
**Organized format documentation:**
|
||||
Moved to `docs/public/reference/` for better discoverability:
|
||||
- `SAVE_STATE_FORMAT.md`
|
||||
- `SNES_COMPRESSION.md`
|
||||
- `SNES_GRAPHICS.md`
|
||||
- `SYMBOL_FORMAT.md`
|
||||
- `ZSM_FORMAT.md`
|
||||
|
||||
**Updated: `docs/public/reference/rom-reference.md`**
|
||||
- Added "Additional Format Documentation" section
|
||||
- Linked to all format specification docs
|
||||
- Updated last modified date to November 27, 2025
|
||||
|
||||
### 5. Documentation Accuracy
|
||||
|
||||
**Updated: `docs/public/build/platform-compatibility.md`**
|
||||
- Updated "Last Updated" from October 9, 2025 to November 27, 2025
|
||||
|
||||
**Reviewed for accuracy:**
|
||||
- ✅ `docs/public/build/quick-reference.md` - Accurate
|
||||
- ✅ `docs/public/build/build-from-source.md` - Accurate
|
||||
- ✅ `docs/public/build/presets.md` - Accurate
|
||||
- ✅ `docs/public/developer/architecture.md` - Accurate (updated Nov 2025)
|
||||
- ✅ `docs/public/developer/testing-quick-start.md` - Accurate
|
||||
|
||||
### 6. Coordination Board
|
||||
|
||||
**Updated: `docs/internal/agents/coordination-board.md`**
|
||||
- Added entry for docs-janitor work session
|
||||
- Marked status as COMPLETE
|
||||
- Listed all changes made
|
||||
|
||||
## File Structure After Cleanup
|
||||
|
||||
```
|
||||
docs/
|
||||
├── public/
|
||||
│ ├── build/ [5 docs - build system]
|
||||
│ ├── deployment/ [1 doc - collaboration server]
|
||||
│ ├── developer/ [18 docs - developer guides]
|
||||
│ ├── examples/ [1 doc - code examples]
|
||||
│ ├── guides/ [1 doc - z3ed workflows]
|
||||
│ ├── overview/ [1 doc - getting started]
|
||||
│ ├── reference/ [8 docs - ROM & format specs] ⭐ IMPROVED
|
||||
│ │ ├── rom-reference.md
|
||||
│ │ ├── SAVE_STATE_FORMAT.md ⬅️ MOVED HERE
|
||||
│ │ ├── SNES_COMPRESSION.md ⬅️ MOVED HERE
|
||||
│ │ ├── SNES_GRAPHICS.md ⬅️ MOVED HERE
|
||||
│ │ ├── SYMBOL_FORMAT.md ⬅️ MOVED HERE
|
||||
│ │ └── ZSM_FORMAT.md ⬅️ MOVED HERE
|
||||
│ ├── usage/ [4 docs including web-app] ⭐ NEW
|
||||
│ │ ├── web-app.md ⬅️ NEW
|
||||
│ │ ├── dungeon-editor.md
|
||||
│ │ ├── overworld-loading.md
|
||||
│ │ └── z3ed-cli.md
|
||||
│ ├── index.md
|
||||
│ └── README.md
|
||||
├── internal/
|
||||
│ ├── agents/ [Agent coordination & playbooks]
|
||||
│ ├── architecture/ [System architecture docs]
|
||||
│ ├── blueprints/ [Refactoring plans]
|
||||
│ ├── plans/ [Implementation plans]
|
||||
│ ├── reports/ [Investigation reports]
|
||||
│ ├── roadmaps/ [Feature roadmaps]
|
||||
│ ├── testing/ [Test infrastructure]
|
||||
│ ├── web-drag-drop-implementation.md ⬅️ MOVED HERE
|
||||
│ ├── wasm-patch-export-implementation.md ⬅️ MOVED HERE
|
||||
│ └── [other internal docs]
|
||||
├── examples/ [Code examples]
|
||||
├── GIGALEAK_INTEGRATION.md
|
||||
└── index.md
|
||||
```
|
||||
|
||||
## Removed Directories
|
||||
|
||||
- ❌ `docs/web/` - consolidated into internal
|
||||
- ❌ `docs/wasm/` - consolidated into internal
|
||||
|
||||
## Documentation Principles Applied
|
||||
|
||||
1. **Public vs Internal Separation**
|
||||
- Public: User-facing, stable, external developers
|
||||
- Internal: AI agents, implementation details, planning
|
||||
|
||||
2. **Accuracy & Honesty**
|
||||
- Web app clearly marked as preview/experimental
|
||||
- Editor status accurately reflects incomplete state
|
||||
- Recommended desktop for production work
|
||||
|
||||
3. **Organization**
|
||||
- Format docs in reference section for easy discovery
|
||||
- Technical implementation in internal for developers
|
||||
- Clear navigation through index files
|
||||
|
||||
4. **Currency**
|
||||
- Updated "Last Modified" dates
|
||||
- Removed outdated content
|
||||
- Consolidated duplicate information
|
||||
|
||||
## Impact
|
||||
|
||||
### For Users
|
||||
- ✅ Clear understanding that web app is preview
|
||||
- ✅ Easy access to format documentation
|
||||
- ✅ Better organized public docs
|
||||
- ✅ Honest feature status
|
||||
|
||||
### For Developers
|
||||
- ✅ Technical docs in predictable locations
|
||||
- ✅ Format specs easy to find in reference/
|
||||
- ✅ Implementation details separated from user guides
|
||||
- ✅ Clear documentation hierarchy
|
||||
|
||||
### For AI Agents
|
||||
- ✅ Updated coordination board with session
|
||||
- ✅ Clear doc hygiene maintained
|
||||
- ✅ No doc sprawl in root directories
|
||||
|
||||
## Follow-up Actions
|
||||
|
||||
None required. Documentation is now:
|
||||
- Organized
|
||||
- Accurate
|
||||
- Complete for web app preview
|
||||
- Properly separated (public vs internal)
|
||||
- Up to date
|
||||
|
||||
## Agent
|
||||
|
||||
**Agent ID:** docs-janitor
|
||||
**Session Date:** November 27, 2025
|
||||
**Duration:** Single session
|
||||
**Status:** Complete
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
# Draw Routine Fixes - Phase 2
|
||||
|
||||
## Status
|
||||
**Owner:** ai-dungeon-specialist
|
||||
**Created:** 2025-12-07
|
||||
**Status:** Partial Complete - Build Verified
|
||||
|
||||
## Summary
|
||||
|
||||
This document tracks fixes for specific dungeon object draw routines and outline sizing issues identified during testing. These issues complement the Phase 1 diagonal/edge/spacing fixes.
|
||||
|
||||
## Issues to Fix
|
||||
|
||||
### Issue 1: Block 0x5E Draw Routine Inverted
|
||||
**Object:** 0x5E (RoomDraw_RightwardsBlock2x2spaced2_1to16)
|
||||
**Problem:** Draw routine is inverted for the simple block pattern
|
||||
**ASM Reference:** bank_01.asm line 363
|
||||
**Fix Required:** Review tile ordering and column-major vs row-major layout
|
||||
|
||||
### Issue 2: Vertical Pegs Wrong Outline Shape
|
||||
**Objects:** 0x95 (DownwardsPots2x2), 0x96 (DownwardsHammerPegs2x2)
|
||||
**Problem:** Selection outline shows 2x2 square but should be vertical single row (1 tile wide)
|
||||
**ASM Reference:** RoomDraw_DownwardsPots2x2_1to16, RoomDraw_DownwardsHammerPegs2x2_1to16
|
||||
**Analysis:** The pots/pegs are 2x2 objects that repeat vertically with 2-row spacing (0x100). However, user reports they should display as 1-tile-wide vertical strips. Need to verify actual drawn dimensions from ASM.
|
||||
**Fix Required:** Update dimension calculations in CalculateObjectDimensions and ObjectDimensionTable
|
||||
|
||||
### Issue 3: Thick Rail Horizontal/Vertical Draw Issues
|
||||
**Objects:** 0x5D (RightwardsBigRail1x3), 0x88 (DownwardsBigRail3x1)
|
||||
**Problem:** Repeats far left edge rather than inner parts of the thick rail
|
||||
**ASM Reference:** RoomDraw_RightwardsBigRail1x3_1to16plus5, RoomDraw_DownwardsBigRail3x1_1to16plus5
|
||||
**Analysis:** ASM shows:
|
||||
- First draws a 2x2 block, then advances tile pointer by 8
|
||||
- Then draws middle section (1x3 tiles per iteration)
|
||||
- Finally draws end cap (2 tiles)
|
||||
**Fix Required:** Fix draw routine to draw: left cap → middle repeating → right cap
|
||||
|
||||
### Issue 4: Large Decor Outline Too Small, Draw Routine Repeats Incorrectly
|
||||
**Problem:** Large decoration objects have outlines that are too small and draw routines that repeat when they shouldn't
|
||||
**Fix Required:** Identify specific objects and verify repetition count logic (size+1 vs size)
|
||||
|
||||
### Issue 5: Ceiling 0xC0 Doesn't Draw Properly
|
||||
**Object:** 0xC0 (RoomDraw_4x4BlocksIn4x4SuperSquare)
|
||||
**Problem:** Object doesn't draw at all or draws incorrectly
|
||||
**ASM Reference:** bank_01.asm lines 1779-1831
|
||||
**Analysis:** Uses complex 4x4 super-square pattern with 8 buffer pointers ($BF through $D4). Draws 4x4 blocks in a grid pattern based on size parameters B2 and B4.
|
||||
**Fix Required:** Implement proper super-square drawing routine
|
||||
|
||||
### Issue 6: 0xF99 Chest Outline Correct But Draw Routine Repeats
|
||||
**Object:** 0xF99 (RoomDraw_Chest - Type 3 chest)
|
||||
**Problem:** Outline is correct but draw routine repeats when it shouldn't
|
||||
**ASM Reference:** RoomDraw_Chest at bank_01.asm line 4707
|
||||
**Analysis:** Chest should only draw once (single 2x2 pattern), not repeat based on size
|
||||
**Fix Required:** Remove size-based repetition from chest draw routine
|
||||
|
||||
### Issue 7: Pit Edges Outlines Too Thin
|
||||
**Problem:** Pit edge outlines shouldn't be single tile thin based on direction
|
||||
**Objects:** Various pit edge objects
|
||||
**Fix Required:** Update pit edge dimension calculations to include proper width based on direction
|
||||
|
||||
### Issue 8: 0x3D Tall Torches Wrong Top Half Graphics
|
||||
**Object:** 0x3D (mapped to RoomDraw_RightwardsPillar2x4spaced4_1to16)
|
||||
**Problem:** Bottom half draws correctly but top half with fire draws pegs instead
|
||||
**ASM Reference:** bank_01.asm line 330 - uses RoomDraw_RightwardsPillar2x4spaced4_1to16 (same as 0x39)
|
||||
**Analysis:** Object 0x3D and 0x39 share the same draw routine but use different tile data. Need to verify tile data offset is correct.
|
||||
**Fix Required:** Verify tile data loading for 0x3D or create dedicated draw routine
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Completed Fixes
|
||||
1. **Block 0x5E** ✅ - Fixed tile ordering in DrawRightwardsBlock2x2spaced2_1to16 to use column-major order
|
||||
2. **Thick Rails 0x5D/0x88** ✅ - Rewrote DrawRightwardsBigRail and DrawDownwardsBigRail to correctly draw cap-middle-cap pattern
|
||||
3. **Chest 0xF99** ✅ - Fixed chest to draw single 2x2 pattern without size-based repetition
|
||||
4. **Large Decor 0xFEB** ✅ - Created Single4x4 routine (routine 113) - no repetition, correct 32x32 outline
|
||||
5. **Water Grate 0xFED** ✅ - Created Single4x3 routine (routine 114) - no repetition, correct 32x24 outline
|
||||
6. **Big Chest 0xFB1** ✅ - Mapped to Single4x3 routine (routine 114) - no repetition
|
||||
7. **Blue Rupees 0xF92** ✅ - Created DrawRupeeFloor routine (routine 115) - special 6x8 pattern with gaps
|
||||
|
||||
### Pending Investigation
|
||||
8. **Rails 0x022** - Uses RoomDraw_RightwardsHasEdge1x1_1to16_plus3 - needs internal rail drawing
|
||||
9. **Ceiling 0xC0** - Draw4x4BlocksIn4x4SuperSquare may have tile loading issue
|
||||
10. **Vertical Pegs 0x95/0x96** - Current dimensions look correct. May be UI display issue.
|
||||
11. **Pit Edges** - Need specific object IDs to investigate
|
||||
12. **Torches 0x3D** - May be ROM tile data issue (verify data at 0x807A)
|
||||
|
||||
## New Routines Added
|
||||
- Routine 113: DrawSingle4x4 - Single 4x4 block, no repetition
|
||||
- Routine 114: DrawSingle4x3 - Single 4x3 block, no repetition
|
||||
- Routine 115: DrawRupeeFloor - Special 6x8 pattern with gaps at rows 2 and 5
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `src/zelda3/dungeon/object_drawer.cc`
|
||||
- `src/zelda3/dungeon/object_drawer.h`
|
||||
- `docs/internal/agents/draw_routine_fixes_phase2.md`
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
# Draw Routine Fixes - Handoff Document
|
||||
|
||||
**Status:** Handoff
|
||||
**Owner:** draw-routine-engineer
|
||||
**Created:** 2025-12-07
|
||||
**Last Session:** 2025-12-07
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
This session made significant progress on dungeon object draw routines. Many fixes were completed, but some objects still have minor issues requiring further investigation.
|
||||
|
||||
---
|
||||
|
||||
## Completed Fixes ✅
|
||||
|
||||
| Object ID | Name | Fix Applied |
|
||||
|-----------|------|-------------|
|
||||
| 0x5E | Block | Fixed column-major tile ordering |
|
||||
| 0x5D/0x88 | Thick Rails | Fixed cap-middle-cap pattern |
|
||||
| 0xF99 | Chest | Changed to DrawSingle2x2 (no size-based repeat) |
|
||||
| 0xFB1 | Big Chest | Changed to DrawSingle4x3 (no repeat) |
|
||||
| 0xF92 | Blue Rupees | Implemented DrawRupeeFloor per ASM (3x8 pattern) |
|
||||
| 0xFED | Water Grate | Changed to DrawSingle4x3 (no repeat) |
|
||||
| 0x3A | Wall Decors | Fixed spacing from 6 to 8 tiles |
|
||||
| 0x39/0x3D | Pillars | Fixed spacing from 6 to 4 tiles |
|
||||
| 0xFEB | Large Decor | Now 64x64 (4x4 tile16s = 8x8 tile8s) |
|
||||
| 0x138-0x13B | Spiral Stairs | Fixed to 4x3 pattern per ASM |
|
||||
| 0xC0/0xC2 | SuperSquare Ceilings | Dimensions now use size parameter |
|
||||
| 0xFE6 | Pit | Uses DrawActual4x4 (32x32, no repeat) |
|
||||
| 0x55-0x56 | Wall Torches | Fixed to 1x8 column with 12-tile spacing |
|
||||
| 0x22 | Small Rails | Now CORNER+MIDDLE*count+END pattern |
|
||||
| 0x23-0x2E | Carpet Trim | Now CORNER+MIDDLE*count+END pattern |
|
||||
|
||||
---
|
||||
|
||||
## Known Issues - Need Further Work ⚠️
|
||||
|
||||
### 1. Horizontal vs Vertical Rails Asymmetry
|
||||
**Objects:** 0x22 (horizontal) vs vertical counterparts
|
||||
**Issue:** Horizontal rails now work with CORNER+MIDDLE+END pattern, but vertical rails may not be updated to match.
|
||||
**Files to check:**
|
||||
- `DrawDownwardsHasEdge1x1_*` routines
|
||||
- Look for routines mapped to 0x8A-0x8E (vertical equivalents)
|
||||
|
||||
### 2. Diagonal Ceiling Outlines (0xA0-0xA3)
|
||||
**Issue:** Draw routine is correct (triangular fill), but outline dimensions still too large for selection purposes.
|
||||
**Current state:**
|
||||
- Draw: `count = (size & 0x0F) + 4`
|
||||
- Outline: `count = (size & 0x0F) + 2` (still too big)
|
||||
**Suggestion:** May need special handling in selection code rather than dimension calculation, since triangles only fill half the bounding box.
|
||||
|
||||
### 3. Torch Object 0x3D
|
||||
**Issue:** Top half draws pegs instead of fire graphics.
|
||||
**Likely cause:** ROM tile data issue - tiles at offset 0x807A may be incorrect or tile loading is wrong.
|
||||
**Uses same routine as pillars (0x39).**
|
||||
|
||||
### 4. Vertical Pegs 0x95/0x96
|
||||
**Issue:** Outline appears square (2x2) but should be vertical single row.
|
||||
**May be UI/selection display issue rather than draw routine.**
|
||||
|
||||
---
|
||||
|
||||
## Pending Tasks 📋
|
||||
|
||||
### High Priority
|
||||
1. **0xDC Chest Platform** - Complex routine with multiple segments, not currently working
|
||||
2. **Staircases audit** - Objects 0x12D-0x137, 0x21B-0x221, 0x226-0x229, 0x233
|
||||
|
||||
### Medium Priority
|
||||
3. **Vertical rail patterns** - Match horizontal rail fixes
|
||||
4. **Pit edges** - Need specific object IDs to investigate
|
||||
|
||||
### Low Priority
|
||||
5. **Diagonal ceiling selection** - May need UI-level fix
|
||||
|
||||
---
|
||||
|
||||
## Key Formulas Reference
|
||||
|
||||
### Size Calculations (from ASM)
|
||||
|
||||
| Pattern | Formula |
|
||||
|---------|---------|
|
||||
| GetSize_1to16 | `count = (size & 0x0F) + 1` |
|
||||
| GetSize_1to16_timesA | `count = (size & 0x0F + 1) * A` |
|
||||
| GetSize_1to15or26 | `count = size; if 0, count = 26` |
|
||||
| GetSize_1to15or32 | `count = size; if 0, count = 32` |
|
||||
| SuperSquare | `size_x = (size & 0x0F) + 1; size_y = ((size >> 4) & 0x0F) + 1` |
|
||||
|
||||
### Rail Pattern Structure
|
||||
```
|
||||
[CORNER tile 0] -> [MIDDLE tile 1 × count] -> [END tile 2]
|
||||
```
|
||||
|
||||
### Tile Ordering
|
||||
Most routines use **COLUMN-MAJOR** order: tiles advance down each column, then right.
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `src/zelda3/dungeon/object_drawer.cc` - Main draw routines and mappings
|
||||
- `src/zelda3/dungeon/object_drawer.h` - Added DrawActual4x4 declaration
|
||||
- `src/zelda3/dungeon/draw_routines/special_routines.cc` - Spiral stairs fix
|
||||
- `docs/internal/agents/draw_routine_tracker.md` - Tracking document
|
||||
|
||||
---
|
||||
|
||||
## Testing Notes
|
||||
|
||||
- Log file at `logs/dungeon-object-draw.log` shows object draw data
|
||||
- Most testing was done on limited room set - need broader room testing
|
||||
- Use grep on log file to find specific object draws: `grep "obj=0xXX" logs/dungeon-object-draw.log`
|
||||
|
||||
---
|
||||
|
||||
## Next Steps for Continuation
|
||||
|
||||
1. Test the rail fixes in rooms with both horizontal and vertical rails
|
||||
2. Find and fix the vertical rail equivalents (likely routines 23-28 or similar)
|
||||
3. Investigate 0xDC chest platform ASM in detail
|
||||
4. Consider UI-level fix for diagonal ceiling selection bounds
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
# Gemini Pro 3 Build Setup Guide
|
||||
|
||||
Quick iteration setup for maximizing Gemini's effectiveness on YAZE.
|
||||
|
||||
## One-Time Setup
|
||||
|
||||
```bash
|
||||
# Use the dedicated Gemini build script (recommended)
|
||||
./scripts/gemini_build.sh
|
||||
|
||||
# Or manually configure with the Gemini preset
|
||||
cmake --preset mac-gemini
|
||||
|
||||
# Verify setup succeeded
|
||||
ls build_gemini/compile_commands.json
|
||||
```
|
||||
|
||||
## Fast Rebuild Cycle (~30s-2min)
|
||||
|
||||
```bash
|
||||
# Recommended: Use the build script
|
||||
./scripts/gemini_build.sh # Build yaze (default)
|
||||
./scripts/gemini_build.sh yaze_test # Build tests
|
||||
./scripts/gemini_build.sh --fresh # Clean reconfigure
|
||||
|
||||
# Or use cmake directly
|
||||
cmake --build build_gemini -j8 --target yaze
|
||||
|
||||
# Rebuild editor library (for src/app/editor/ changes)
|
||||
cmake --build build_gemini -j8 --target yaze_editor
|
||||
|
||||
# Rebuild zelda3 library (for src/zelda3/ changes)
|
||||
cmake --build build_gemini -j8 --target yaze_lib
|
||||
```
|
||||
|
||||
## Quick Validation (~2-3min)
|
||||
|
||||
```bash
|
||||
# Run stable tests (unit + integration, no ROM required)
|
||||
ctest --test-dir build_gemini -L stable -j4 --output-on-failure
|
||||
|
||||
# Run GUI smoke tests only (~1min)
|
||||
ctest --test-dir build_gemini -L gui --output-on-failure
|
||||
|
||||
# Run specific test by name
|
||||
ctest --test-dir build_gemini -R "OverworldRegression" --output-on-failure
|
||||
```
|
||||
|
||||
## Full Validation (when confident)
|
||||
|
||||
```bash
|
||||
# All tests
|
||||
ctest --test-dir build_gemini --output-on-failure -j4
|
||||
```
|
||||
|
||||
## Format Check (before committing)
|
||||
|
||||
```bash
|
||||
# Check format without modifying
|
||||
cmake --build build_gemini --target format-check
|
||||
|
||||
# Auto-format all code
|
||||
cmake --build build_gemini --target format
|
||||
```
|
||||
|
||||
## Key Directories
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `src/app/editor/overworld/` | Overworld editor modules (8 files) |
|
||||
| `src/zelda3/overworld/` | Overworld data models |
|
||||
| `src/zelda3/dungeon/` | Dungeon data models |
|
||||
| `src/app/editor/dungeon/` | Dungeon editor components |
|
||||
| `test/unit/zelda3/` | Unit tests for zelda3 logic |
|
||||
| `test/e2e/` | GUI E2E tests |
|
||||
|
||||
## Iteration Strategy
|
||||
|
||||
1. **Make changes** to target files
|
||||
2. **Rebuild** with `cmake --build build_gemini -j8 --target yaze`
|
||||
3. **Test** with `ctest --test-dir build_gemini -L stable -j4`
|
||||
4. **Repeat** until all tests pass
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Build fails with "target not found"
|
||||
```bash
|
||||
# Reconfigure (CMakeLists.txt changed)
|
||||
./scripts/gemini_build.sh --fresh
|
||||
# Or: cmake --preset mac-gemini --fresh
|
||||
```
|
||||
|
||||
### Tests timeout
|
||||
```bash
|
||||
# Run with longer timeout
|
||||
ctest --test-dir build_gemini -L stable --timeout 300
|
||||
```
|
||||
|
||||
### Need to see test output
|
||||
```bash
|
||||
ctest --test-dir build_gemini -L stable -V # Verbose
|
||||
```
|
||||
@@ -0,0 +1,245 @@
|
||||
# Gemini Pro 3 Antigravity - YAZE Development Session
|
||||
|
||||
## Context
|
||||
|
||||
You are working on **YAZE** (Yet Another Zelda3 Editor), a C++23 cross-platform ROM editor for The Legend of Zelda: A Link to the Past.
|
||||
|
||||
**Your previous session accomplished:**
|
||||
- Fixed ASM version check regression using `OverworldVersionHelper` abstraction
|
||||
- Improved texture queueing in `Tile16Editor`
|
||||
- The insight: `0xFF >= 3` evaluates true, incorrectly treating vanilla ROMs as v3
|
||||
|
||||
**Reference documentation available:**
|
||||
- `docs/internal/agents/gemini-overworld-system-reference.md` - Overworld architecture
|
||||
- `docs/internal/agents/gemini-dungeon-system-reference.md` - Dungeon architecture
|
||||
- `docs/internal/agents/gemini-build-setup.md` - Build commands
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Setup build directory (use dedicated dir, not user's build/)
|
||||
cmake --preset mac-dbg -B build_gemini
|
||||
|
||||
# 2. Build editor
|
||||
cmake --build build_gemini -j8 --target yaze
|
||||
|
||||
# 3. Run stable tests
|
||||
ctest --test-dir build_gemini -L stable -j4 --output-on-failure
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task Categories
|
||||
|
||||
### Category A: Overworld Editor Gaps
|
||||
|
||||
#### A1. Texture Queueing TODOs (6 locations)
|
||||
**File:** `src/app/editor/overworld/overworld_editor.cc`
|
||||
**Lines:** 1392, 1397, 1740, 1809, 1819, 1962
|
||||
|
||||
These commented-out Renderer calls need to be converted to the Arena queue system:
|
||||
|
||||
```cpp
|
||||
// BEFORE (blocking - commented out)
|
||||
// Renderer::Get().RenderBitmap(&bitmap_);
|
||||
|
||||
// AFTER (non-blocking)
|
||||
gfx::Arena::Get().QueueTextureCommand(gfx::TextureCommand{
|
||||
.operation = gfx::TextureOperation::kCreate,
|
||||
.bitmap = &bitmap_,
|
||||
.priority = gfx::TexturePriority::kHigh
|
||||
});
|
||||
```
|
||||
|
||||
#### A2. Unimplemented Editor Methods
|
||||
**File:** `src/app/editor/overworld/overworld_editor.h` lines 82-87
|
||||
|
||||
| Method | Status | Complexity |
|
||||
|--------|--------|------------|
|
||||
| `Undo()` | Returns `UnimplementedError` | Medium |
|
||||
| `Redo()` | Returns `UnimplementedError` | Medium |
|
||||
| `Cut()` | Returns `UnimplementedError` | Simple |
|
||||
| `Find()` | Returns `UnimplementedError` | Medium |
|
||||
|
||||
#### A3. Entity Popup Static Variable Bug
|
||||
**File:** `src/app/editor/overworld/entity.cc`
|
||||
|
||||
Multiple popups use `static` variables that persist across calls, causing state contamination:
|
||||
|
||||
```cpp
|
||||
// CURRENT (BUG)
|
||||
bool DrawExitEditorPopup() {
|
||||
static bool set_done = false; // Persists! Wrong entity data shown
|
||||
static int doorType = ...;
|
||||
}
|
||||
|
||||
// FIX: Use local variables or proper state management
|
||||
bool DrawExitEditorPopup(ExitEditorState& state) {
|
||||
// state is passed in, not static
|
||||
}
|
||||
```
|
||||
|
||||
#### A4. Exit Editor Unconnected UI
|
||||
**File:** `src/app/editor/overworld/entity.cc` lines 216-264
|
||||
|
||||
UI elements exist but aren't connected to data:
|
||||
- Door type editing (Wooden, Bombable, Sanctuary, Palace)
|
||||
- Door X/Y position
|
||||
- Center X/Y, Link posture, sprite/BG GFX, palette
|
||||
|
||||
---
|
||||
|
||||
### Category B: Dungeon Object Rendering
|
||||
|
||||
#### B1. BothBG Dual-Layer Stubs (4 locations)
|
||||
**File:** `src/zelda3/dungeon/object_drawer.cc`
|
||||
|
||||
These routines should draw to BOTH BG1 and BG2 but only accept one buffer:
|
||||
|
||||
| Line | Routine | Status |
|
||||
|------|---------|--------|
|
||||
| 375-381 | `DrawRightwards2x4spaced4_1to16_BothBG` | STUB |
|
||||
| 437-442 | `DrawDiagonalAcute_1to16_BothBG` | STUB |
|
||||
| 444-449 | `DrawDiagonalGrave_1to16_BothBG` | STUB |
|
||||
| 755-761 | `DrawDownwards4x2_1to16_BothBG` | STUB |
|
||||
|
||||
**Fix:** Change signature to accept both buffers:
|
||||
```cpp
|
||||
// BEFORE
|
||||
void DrawRightwards2x4spaced4_1to16_BothBG(
|
||||
const RoomObject& obj, gfx::BackgroundBuffer& bg, ...);
|
||||
|
||||
// AFTER
|
||||
void DrawRightwards2x4spaced4_1to16_BothBG(
|
||||
const RoomObject& obj,
|
||||
gfx::BackgroundBuffer& bg1,
|
||||
gfx::BackgroundBuffer& bg2, ...);
|
||||
```
|
||||
|
||||
#### B2. Diagonal Routines Unclear Logic
|
||||
**File:** `src/zelda3/dungeon/object_drawer.cc` lines 401-435
|
||||
|
||||
Issues with `DrawDiagonalAcute_1to16` and `DrawDiagonalGrave_1to16`:
|
||||
- Hardcoded `size + 6` and 5 iterations (why?)
|
||||
- Coordinate formula `obj.y_ + (i - s)` can produce negative Y
|
||||
- No bounds checking
|
||||
- Only uses 4 tiles from larger span
|
||||
|
||||
#### B3. CustomDraw and DoorSwitcherer Stubs
|
||||
**File:** `src/zelda3/dungeon/object_drawer.cc`
|
||||
- Lines 524-532: `CustomDraw` - only draws first tile
|
||||
- Lines 566-575: `DrawDoorSwitcherer` - only draws first tile
|
||||
|
||||
#### B4. Unknown Object Names (30+ items)
|
||||
**File:** `src/zelda3/dungeon/room_object.h`
|
||||
|
||||
See `gemini-dungeon-system-reference.md` section 7 for full list of objects needing in-game verification.
|
||||
|
||||
---
|
||||
|
||||
### Category C: Already Complete (Verification Only)
|
||||
|
||||
#### C1. Tile16Editor Undo/Redo - COMPLETE
|
||||
**File:** `src/app/editor/overworld/tile16_editor.cc`
|
||||
- `SaveUndoState()` at lines 547, 1476, 1511, 1546, 1586, 1620
|
||||
- `Undo()` / `Redo()` at lines 1707-1760
|
||||
- Ctrl+Z/Ctrl+Y at lines 224-231
|
||||
- UI button at line 1101
|
||||
|
||||
**No work needed** - just verify it works.
|
||||
|
||||
#### C2. Entity Deletion Pattern - CORRECT
|
||||
**File:** `src/app/editor/overworld/entity.cc` line 319
|
||||
|
||||
The TODO comment is misleading. The `deleted` flag pattern IS correct for ROM editors:
|
||||
- Entities at fixed ROM offsets can't be "removed"
|
||||
- `entity_operations.cc` reuses deleted slots
|
||||
- Just clarify the comment if desired
|
||||
|
||||
---
|
||||
|
||||
## Prioritized Task List
|
||||
|
||||
### Phase 1: High Impact (45-60 min)
|
||||
1. **A1** - Texture queueing (6 TODOs) - Prevents UI freezes
|
||||
2. **B1** - BothBG dual-layer stubs (4 routines) - Completes dungeon rendering
|
||||
|
||||
### Phase 2: Medium Impact (30-45 min)
|
||||
3. **A3** - Entity popup static variable bug - Fixes data corruption
|
||||
4. **B2** - Diagonal routine logic - Fixes rendering artifacts
|
||||
|
||||
### Phase 3: Polish (30+ min)
|
||||
5. **A2** - Implement Undo/Redo for OverworldEditor
|
||||
6. **A4** - Connect exit editor UI to data
|
||||
7. **B3** - Implement CustomDraw/DoorSwitcherer
|
||||
|
||||
### Stretch Goals
|
||||
8. **B4** - Verify unknown object names (requires game testing)
|
||||
9. E2E cinematic tests (see `docs/internal/testing/dungeon-gui-test-design.md`)
|
||||
|
||||
---
|
||||
|
||||
## Code Patterns
|
||||
|
||||
### Texture Queue (Use This!)
|
||||
```cpp
|
||||
gfx::Arena::Get().QueueTextureCommand(gfx::TextureCommand{
|
||||
.operation = gfx::TextureOperation::kCreate, // or kUpdate
|
||||
.bitmap = &bitmap_,
|
||||
.priority = gfx::TexturePriority::kHigh
|
||||
});
|
||||
```
|
||||
|
||||
### Version-Aware Code
|
||||
```cpp
|
||||
auto version = OverworldVersionHelper::GetVersion(*rom_);
|
||||
if (OverworldVersionHelper::SupportsAreaEnum(version)) {
|
||||
// v3+ only
|
||||
}
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
```cpp
|
||||
absl::Status MyFunction() {
|
||||
ASSIGN_OR_RETURN(auto data, LoadData());
|
||||
RETURN_IF_ERROR(ProcessData(data));
|
||||
return absl::OkStatus();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validation
|
||||
|
||||
```bash
|
||||
# After each change
|
||||
cmake --build build_gemini -j8 --target yaze
|
||||
ctest --test-dir build_gemini -L stable -j4 --output-on-failure
|
||||
|
||||
# Before finishing
|
||||
cmake --build build_gemini --target format-check
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
- [ ] `grep "TODO.*texture" src/app/editor/overworld/overworld_editor.cc` returns nothing
|
||||
- [ ] BothBG routines accept both buffer parameters
|
||||
- [ ] Static variable bug in entity popups fixed
|
||||
- [ ] `ctest -L stable` passes 100%
|
||||
- [ ] Code formatted
|
||||
|
||||
---
|
||||
|
||||
## File Quick Reference
|
||||
|
||||
| System | Key Files |
|
||||
|--------|-----------|
|
||||
| Overworld Editor | `src/app/editor/overworld/overworld_editor.cc` (3,204 lines) |
|
||||
| Entity UI | `src/app/editor/overworld/entity.cc` (491 lines) |
|
||||
| Tile16 Editor | `src/app/editor/overworld/tile16_editor.cc` (2,584 lines) |
|
||||
| Object Drawer | `src/zelda3/dungeon/object_drawer.cc` (972 lines) |
|
||||
| Room Object | `src/zelda3/dungeon/room_object.h` (633 lines) |
|
||||
@@ -0,0 +1,376 @@
|
||||
# YAZE Overworld System - Complete Technical Reference
|
||||
|
||||
Comprehensive reference for AI agents working on the YAZE Overworld editing system.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
### File Structure
|
||||
```
|
||||
src/zelda3/overworld/
|
||||
├── overworld.h/cc # Main Overworld class (2,500+ lines)
|
||||
├── overworld_map.h/cc # Individual map (OverworldMap class)
|
||||
├── overworld_version_helper.h # Version detection & feature gates
|
||||
├── overworld_item.h/cc # Item entities
|
||||
├── overworld_entrance.h # Entrance/hole entities
|
||||
├── overworld_exit.h # Exit entities
|
||||
|
||||
src/app/editor/overworld/
|
||||
├── overworld_editor.h/cc # Main editor (3,204 lines)
|
||||
├── map_properties.cc # MapPropertiesSystem (1,759 lines)
|
||||
├── tile16_editor.h/cc # Tile16Editor (2,584 lines)
|
||||
├── entity.cc # Entity UI popups (491 lines)
|
||||
├── entity_operations.cc # Entity CRUD helpers (239 lines)
|
||||
├── overworld_entity_renderer.cc # Entity drawing (151 lines)
|
||||
├── scratch_space.cc # Tile storage utilities (444 lines)
|
||||
```
|
||||
|
||||
### Data Model Hierarchy
|
||||
```
|
||||
Rom (raw data)
|
||||
└── Overworld (coordinator)
|
||||
├── tiles16_[] (3,752-4,096 Tile16 definitions)
|
||||
├── tiles32_unique_[] (up to 8,864 Tile32 definitions)
|
||||
├── overworld_maps_[160] (individual map screens)
|
||||
│ └── OverworldMap
|
||||
│ ├── area_graphics_, area_palette_
|
||||
│ ├── bitmap_data_[] (256x256 rendered pixels)
|
||||
│ └── current_gfx_[] (graphics buffer)
|
||||
├── all_entrances_[] (~129 entrance points)
|
||||
├── all_holes_[] (~19 hole entrances)
|
||||
├── all_exits_[] (~79 exit points)
|
||||
├── all_items_[] (collectible items)
|
||||
└── all_sprites_[3][] (sprites per game state)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. ZSCustomOverworld Version System
|
||||
|
||||
### Version Detection
|
||||
```cpp
|
||||
// ROM address for version byte
|
||||
constexpr int OverworldCustomASMHasBeenApplied = 0x140145;
|
||||
|
||||
// Version values
|
||||
0xFF = Vanilla ROM (unpatched)
|
||||
0x01 = ZSCustomOverworld v1
|
||||
0x02 = ZSCustomOverworld v2
|
||||
0x03 = ZSCustomOverworld v3+
|
||||
```
|
||||
|
||||
### Feature Matrix
|
||||
|
||||
| Feature | Vanilla | v1 | v2 | v3 |
|
||||
|---------|---------|----|----|-----|
|
||||
| Basic map editing | Y | Y | Y | Y |
|
||||
| Large maps (2x2) | Y | Y | Y | Y |
|
||||
| Expanded pointers (0x130000+) | - | Y | Y | Y |
|
||||
| Custom BG colors | - | - | Y | Y |
|
||||
| Main palette per area | - | - | Y | Y |
|
||||
| Wide areas (2x1) | - | - | - | Y |
|
||||
| Tall areas (1x2) | - | - | - | Y |
|
||||
| Custom tile GFX (8 per map) | - | - | - | Y |
|
||||
| Animated GFX | - | - | - | Y |
|
||||
| Subscreen overlays | - | - | - | Y |
|
||||
|
||||
### OverworldVersionHelper API
|
||||
```cpp
|
||||
// src/zelda3/overworld/overworld_version_helper.h
|
||||
|
||||
enum class OverworldVersion { kVanilla=0, kZSCustomV1=1, kZSCustomV2=2, kZSCustomV3=3 };
|
||||
enum class AreaSizeEnum { SmallArea=0, LargeArea=1, WideArea=2, TallArea=3 };
|
||||
|
||||
// Detection
|
||||
static OverworldVersion GetVersion(const Rom& rom);
|
||||
static uint8_t GetAsmVersion(const Rom& rom);
|
||||
|
||||
// Feature gates (use these, not raw version checks!)
|
||||
static bool SupportsAreaEnum(OverworldVersion v); // v3 only
|
||||
static bool SupportsExpandedSpace(OverworldVersion v); // v1+
|
||||
static bool SupportsCustomBGColors(OverworldVersion v);// v2+
|
||||
static bool SupportsCustomTileGFX(OverworldVersion v); // v3 only
|
||||
static bool SupportsAnimatedGFX(OverworldVersion v); // v3 only
|
||||
static bool SupportsSubscreenOverlay(OverworldVersion v); // v3 only
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Tile System Architecture
|
||||
|
||||
### Tile Hierarchy
|
||||
```
|
||||
Tile8 (8x8 pixels) - Base SNES tile
|
||||
↓
|
||||
Tile16 (16x16 pixels) - 2x2 grid of Tile8s
|
||||
↓
|
||||
Tile32 (32x32 pixels) - 2x2 grid of Tile16s
|
||||
↓
|
||||
Map Screen (256x256 pixels) - 8x8 grid of Tile32s
|
||||
```
|
||||
|
||||
### Tile16 Structure
|
||||
```cpp
|
||||
// src/app/gfx/types/snes_tile.h
|
||||
|
||||
class TileInfo {
|
||||
uint16_t id_; // 9-bit tile8 ID (0-511)
|
||||
uint8_t palette_; // 3-bit palette (0-7)
|
||||
bool over_; // Priority flag
|
||||
bool vertical_mirror_; // Y-flip
|
||||
bool horizontal_mirror_; // X-flip
|
||||
};
|
||||
|
||||
class Tile16 {
|
||||
TileInfo tile0_; // Top-left
|
||||
TileInfo tile1_; // Top-right
|
||||
TileInfo tile2_; // Bottom-left
|
||||
TileInfo tile3_; // Bottom-right
|
||||
};
|
||||
|
||||
// ROM storage: 8 bytes per Tile16 at 0x78000 + (ID * 8)
|
||||
// Total: 4,096 Tile16s (0x0000-0x0FFF)
|
||||
```
|
||||
|
||||
### Tile16Editor Features (COMPLETE)
|
||||
The Tile16Editor at `tile16_editor.cc` is **fully implemented** with:
|
||||
|
||||
- **Undo/Redo System** (lines 1681-1760)
|
||||
- `SaveUndoState()` - captures current state
|
||||
- `Undo()` / `Redo()` - restore states
|
||||
- Ctrl+Z / Ctrl+Y keyboard shortcuts
|
||||
- 50-state stack with rate limiting
|
||||
|
||||
- **Clipboard Operations**
|
||||
- Copy/Paste Tile16s
|
||||
- 4 scratch space slots
|
||||
|
||||
- **Editing Features**
|
||||
- Tile8 composition into Tile16
|
||||
- Flip horizontal/vertical
|
||||
- Palette cycling (0-7)
|
||||
- Fill with single Tile8
|
||||
|
||||
---
|
||||
|
||||
## 4. Map Organization
|
||||
|
||||
### Index Scheme
|
||||
```
|
||||
Index 0x00-0x3F: Light World (64 maps, 8x8 grid)
|
||||
Index 0x40-0x7F: Dark World (64 maps, 8x8 grid)
|
||||
Index 0x80-0x9F: Special World (32 maps, 8x4 grid)
|
||||
|
||||
Total: 160 maps
|
||||
|
||||
Grid position: X = index % 8, Y = index / 8
|
||||
World position: X * 512 pixels, Y * 512 pixels
|
||||
```
|
||||
|
||||
### Multi-Area Maps
|
||||
```cpp
|
||||
enum class AreaSizeEnum {
|
||||
SmallArea = 0, // 1x1 screen (standard)
|
||||
LargeArea = 1, // 2x2 screens (4 quadrants)
|
||||
WideArea = 2, // 2x1 screens (v3 only)
|
||||
TallArea = 3, // 1x2 screens (v3 only)
|
||||
};
|
||||
|
||||
// IMPORTANT: Always use ConfigureMultiAreaMap() for size changes!
|
||||
// Never set area_size_ directly - it handles parent IDs and ROM persistence
|
||||
absl::Status Overworld::ConfigureMultiAreaMap(int parent_index, AreaSizeEnum size);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Entity System
|
||||
|
||||
### Entity Types
|
||||
| Type | Storage | Count | ROM Address |
|
||||
|------|---------|-------|-------------|
|
||||
| Entrances | `all_entrances_` | ~129 | 0xDB96F+ |
|
||||
| Holes | `all_holes_` | ~19 | 0xDB800+ |
|
||||
| Exits | `all_exits_` | ~79 | 0x15D8A+ |
|
||||
| Items | `all_items_` | Variable | 0xDC2F9+ |
|
||||
| Sprites | `all_sprites_[3]` | Variable | 0x4C881+ |
|
||||
|
||||
### Entity Deletion Pattern
|
||||
Entities use a `deleted` flag pattern - this is **CORRECT** for ROM editors:
|
||||
```cpp
|
||||
// Entities live at fixed ROM offsets, cannot be truly "removed"
|
||||
// Setting deleted = true marks them as inactive
|
||||
// entity_operations.cc reuses deleted slots for new entities
|
||||
item.deleted = true; // Proper pattern
|
||||
|
||||
// Renderer skips deleted entities (overworld_entity_renderer.cc)
|
||||
if (!item.deleted) { /* render */ }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Graphics Loading Pipeline
|
||||
|
||||
### Load Sequence
|
||||
```
|
||||
1. Overworld::Load(rom)
|
||||
└── LoadOverworldMaps()
|
||||
└── For each map (0-159):
|
||||
└── OverworldMap::ctor()
|
||||
├── LoadAreaInfo()
|
||||
└── LoadCustomOverworldData() [v3]
|
||||
|
||||
2. On map access: EnsureMapBuilt(map_index)
|
||||
└── BuildMap()
|
||||
├── LoadAreaGraphics()
|
||||
├── BuildTileset()
|
||||
├── BuildTiles16Gfx()
|
||||
├── LoadPalette()
|
||||
├── LoadOverlay()
|
||||
└── BuildBitmap()
|
||||
```
|
||||
|
||||
### Texture Queue System
|
||||
Use deferred texture loading via `gfx::Arena`:
|
||||
```cpp
|
||||
// CORRECT: Non-blocking, uses queue
|
||||
gfx::Arena::Get().QueueTextureCommand(gfx::TextureCommand{
|
||||
.operation = gfx::TextureOperation::kCreate,
|
||||
.bitmap = &some_bitmap_,
|
||||
.priority = gfx::TexturePriority::kHigh
|
||||
});
|
||||
|
||||
// WRONG: Blocking, causes UI freeze
|
||||
Renderer::Get().RenderBitmap(&some_bitmap_);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. ROM Addresses (Key Locations)
|
||||
|
||||
### Vanilla Addresses
|
||||
```cpp
|
||||
// Tile data
|
||||
kTile16Ptr = 0x78000 // Tile16 definitions
|
||||
kOverworldMapSize = 0x12844 // Map size bytes
|
||||
|
||||
// Graphics & Palettes
|
||||
kAreaGfxIdPtr = 0x7C9C // Area graphics IDs
|
||||
kOverworldMapPaletteGroup = 0x7D1C // Palette IDs
|
||||
|
||||
// Entities
|
||||
kOverworldEntranceMap = 0xDB96F // Entrance data
|
||||
kOverworldExitRooms = 0x15D8A // Exit room IDs
|
||||
kOverworldItemPointers = 0xDC2F9 // Item pointers
|
||||
```
|
||||
|
||||
### Expanded Addresses (v1+)
|
||||
```cpp
|
||||
// Custom data at 0x140000+
|
||||
OverworldCustomASMHasBeenApplied = 0x140145 // Version byte
|
||||
OverworldCustomAreaSpecificBGPalette = 0x140000 // BG colors (160*2)
|
||||
OverworldCustomMainPaletteArray = 0x140160 // Main palettes (160)
|
||||
OverworldCustomAnimatedGFXArray = 0x1402A0 // Animated GFX (160)
|
||||
OverworldCustomTileGFXGroupArray = 0x140480 // Tile GFX (160*8)
|
||||
OverworldCustomSubscreenOverlayArray = 0x140340 // Overlays (160*2)
|
||||
kOverworldMapParentIdExpanded = 0x140998 // Parent IDs (160)
|
||||
kOverworldMessagesExpanded = 0x1417F8 // Messages (160*2)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Known Gaps in OverworldEditor
|
||||
|
||||
### Critical: Texture Queueing TODOs (6 locations)
|
||||
```cpp
|
||||
// overworld_editor.cc - these Renderer calls need to be converted:
|
||||
Line 1392: // TODO: Queue texture for later rendering
|
||||
Line 1397: // TODO: Queue texture for later rendering
|
||||
Line 1740: // TODO: Queue texture for later rendering
|
||||
Line 1809: // TODO: Queue texture for later rendering
|
||||
Line 1819: // TODO: Queue texture for later rendering
|
||||
Line 1962: // TODO: Queue texture for later rendering
|
||||
```
|
||||
|
||||
### Unimplemented Core Methods
|
||||
```cpp
|
||||
// overworld_editor.h lines 82-87
|
||||
Undo() → Returns UnimplementedError
|
||||
Redo() → Returns UnimplementedError
|
||||
Cut() → Returns UnimplementedError
|
||||
Find() → Returns UnimplementedError
|
||||
```
|
||||
|
||||
### Entity Popup Static Variable Bug
|
||||
```cpp
|
||||
// entity.cc - Multiple popups use static variables that persist
|
||||
// Causes state contamination when editing multiple entities
|
||||
bool DrawExitEditorPopup() {
|
||||
static bool set_done = false; // BUG: persists across calls
|
||||
static int doorType = ...; // BUG: wrong entity's data shown
|
||||
}
|
||||
```
|
||||
|
||||
### Exit Editor Unimplemented Features
|
||||
```cpp
|
||||
// entity.cc lines 216-264
|
||||
// UI exists but not connected to data:
|
||||
- Door type editing (Wooden, Bombable, Sanctuary, Palace)
|
||||
- Door X/Y position
|
||||
- Center X/Y, Link posture, sprite/BG GFX, palette
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Code Patterns to Follow
|
||||
|
||||
### Graphics Refresh
|
||||
```cpp
|
||||
// 1. Update model
|
||||
map.SetProperty(new_value);
|
||||
|
||||
// 2. Reload from ROM
|
||||
map.LoadAreaGraphics();
|
||||
|
||||
// 3. Queue texture update (NOT RenderBitmap!)
|
||||
gfx::Arena::Get().QueueTextureCommand(gfx::TextureCommand{
|
||||
.operation = gfx::TextureOperation::kUpdate,
|
||||
.bitmap = &map_bitmap_,
|
||||
.priority = gfx::TexturePriority::kHigh
|
||||
});
|
||||
```
|
||||
|
||||
### Version-Aware Code
|
||||
```cpp
|
||||
auto version = OverworldVersionHelper::GetVersion(*rom_);
|
||||
|
||||
// Use semantic helpers, not raw version checks
|
||||
if (OverworldVersionHelper::SupportsAreaEnum(version)) {
|
||||
// v3+ only code
|
||||
}
|
||||
```
|
||||
|
||||
### Entity Rendering Colors (0.85f alpha)
|
||||
```cpp
|
||||
ImVec4 entrance_color(1.0f, 0.85f, 0.0f, 0.85f); // Yellow-gold
|
||||
ImVec4 exit_color(0.0f, 1.0f, 1.0f, 0.85f); // Cyan
|
||||
ImVec4 item_color(1.0f, 0.0f, 0.0f, 0.85f); // Red
|
||||
ImVec4 sprite_color(1.0f, 0.0f, 1.0f, 0.85f); // Magenta
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Testing
|
||||
|
||||
### Run Overworld Tests
|
||||
```bash
|
||||
# Unit tests
|
||||
ctest --test-dir build -R "Overworld" -V
|
||||
|
||||
# Regression tests
|
||||
ctest --test-dir build -R "OverworldRegression" -V
|
||||
```
|
||||
|
||||
### Test Files
|
||||
- `test/unit/zelda3/overworld_test.cc` - Core tests
|
||||
- `test/unit/zelda3/overworld_regression_test.cc` - Version helper tests
|
||||
@@ -0,0 +1,217 @@
|
||||
# Gemini Pro 3 Task Checklist
|
||||
|
||||
Prioritized checklist of all identified work items for YAZE development.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Critical Rendering Issues (HIGH PRIORITY)
|
||||
|
||||
### Task A1: Texture Queueing TODOs
|
||||
**File:** `src/app/editor/overworld/overworld_editor.cc`
|
||||
**Impact:** Prevents UI freezes, enables proper texture loading
|
||||
**Time Estimate:** 25 min
|
||||
|
||||
- [ ] Line 1392: Convert commented Renderer call to Arena queue
|
||||
- [ ] Line 1397: Convert commented Renderer call to Arena queue
|
||||
- [ ] Line 1740: Convert commented Renderer call to Arena queue
|
||||
- [ ] Line 1809: Convert commented Renderer call to Arena queue
|
||||
- [ ] Line 1819: Convert commented Renderer call to Arena queue
|
||||
- [ ] Line 1962: Convert commented Renderer call to Arena queue
|
||||
- [ ] Verify: `grep "TODO.*texture" overworld_editor.cc` returns nothing
|
||||
- [ ] Test: Run stable tests, no UI freezes on map load
|
||||
|
||||
---
|
||||
|
||||
### Task B1: BothBG Dual-Layer Stubs
|
||||
**File:** `src/zelda3/dungeon/object_drawer.cc`
|
||||
**Impact:** Completes dungeon object rendering for dual-layer objects
|
||||
**Time Estimate:** 30 min
|
||||
|
||||
- [ ] Line 375-381: `DrawRightwards2x4spaced4_1to16_BothBG`
|
||||
- Change signature to accept both `bg1` and `bg2`
|
||||
- Call underlying routine for both buffers
|
||||
- [ ] Line 437-442: `DrawDiagonalAcute_1to16_BothBG`
|
||||
- Change signature to accept both buffers
|
||||
- [ ] Line 444-449: `DrawDiagonalGrave_1to16_BothBG`
|
||||
- Change signature to accept both buffers
|
||||
- [ ] Line 755-761: `DrawDownwards4x2_1to16_BothBG`
|
||||
- Change signature to accept both buffers
|
||||
- [ ] Update `DrawObject()` call sites to pass both buffers for BothBG routines
|
||||
- [ ] Test: Dungeon rooms with dual-layer objects render correctly
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Bug Fixes (MEDIUM PRIORITY)
|
||||
|
||||
### Task A3: Entity Popup Static Variable Bug
|
||||
**File:** `src/app/editor/overworld/entity.cc`
|
||||
**Impact:** Fixes data corruption when editing multiple entities
|
||||
**Time Estimate:** 20 min
|
||||
|
||||
- [ ] `DrawExitEditorPopup()` (line 152+):
|
||||
- Remove `static bool set_done`
|
||||
- Remove other static variables
|
||||
- Pass state via parameter or use popup ID
|
||||
- [ ] `DrawItemEditorPopup()` (line 320+):
|
||||
- Remove static variables
|
||||
- [ ] Other popup functions:
|
||||
- Audit for static state
|
||||
- [ ] Test: Edit multiple exits/items in sequence, verify correct data
|
||||
|
||||
---
|
||||
|
||||
### Task B2: Diagonal Routine Logic
|
||||
**File:** `src/zelda3/dungeon/object_drawer.cc` lines 401-435
|
||||
**Impact:** Fixes rendering artifacts for diagonal objects
|
||||
**Time Estimate:** 30 min
|
||||
|
||||
- [ ] `DrawDiagonalAcute_1to16`:
|
||||
- Verify/document the `size + 6` constant
|
||||
- Add bounds checking for negative Y coordinates
|
||||
- Handle edge cases at canvas boundaries
|
||||
- [ ] `DrawDiagonalGrave_1to16`:
|
||||
- Same fixes as acute version
|
||||
- [ ] Test: Place diagonal objects in dungeon editor, verify appearance
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Feature Completion (POLISH)
|
||||
|
||||
### Task A2: OverworldEditor Undo/Redo
|
||||
**File:** `src/app/editor/overworld/overworld_editor.h` lines 82-87
|
||||
**Impact:** Enables undo/redo for overworld edits
|
||||
**Time Estimate:** 45 min
|
||||
|
||||
- [ ] Design undo state structure:
|
||||
```cpp
|
||||
struct OverworldUndoState {
|
||||
int map_index;
|
||||
std::vector<uint16_t> tile_data;
|
||||
// Entity changes?
|
||||
};
|
||||
```
|
||||
- [ ] Add `undo_stack_` and `redo_stack_` members
|
||||
- [ ] Implement `Undo()` method
|
||||
- [ ] Implement `Redo()` method
|
||||
- [ ] Wire up Ctrl+Z / Ctrl+Y shortcuts
|
||||
- [ ] Test: Make edits, undo, redo - verify state restoration
|
||||
|
||||
---
|
||||
|
||||
### Task A4: Exit Editor UI Connection
|
||||
**File:** `src/app/editor/overworld/entity.cc` lines 216-264
|
||||
**Impact:** Makes exit editor fully functional
|
||||
**Time Estimate:** 30 min
|
||||
|
||||
- [ ] Connect door type radio buttons to `exit.door_type_`
|
||||
- [ ] Connect door X/Y inputs to exit data
|
||||
- [ ] Connect Center X/Y to exit scroll data
|
||||
- [ ] Connect palette/GFX fields to exit properties
|
||||
- [ ] Test: Edit exit properties, save ROM, verify changes persisted
|
||||
|
||||
---
|
||||
|
||||
### Task B3: CustomDraw and DoorSwitcherer
|
||||
**File:** `src/zelda3/dungeon/object_drawer.cc`
|
||||
**Impact:** Completes special object rendering
|
||||
**Time Estimate:** 30 min
|
||||
|
||||
- [ ] `CustomDraw` (lines 524-532):
|
||||
- Research expected behavior from ZScream/game
|
||||
- Implement proper drawing logic
|
||||
- [ ] `DrawDoorSwitcherer` (lines 566-575):
|
||||
- Research door switching animation/logic
|
||||
- Implement proper drawing
|
||||
- [ ] Test: Place custom objects, verify appearance
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Stretch Goals
|
||||
|
||||
### Task B4: Object Name Verification
|
||||
**File:** `src/zelda3/dungeon/room_object.h`
|
||||
**Impact:** Improves editor usability with proper names
|
||||
**Time Estimate:** 2+ hours (requires game testing)
|
||||
|
||||
See `gemini-dungeon-system-reference.md` section 7 for full list.
|
||||
|
||||
High-priority unknowns:
|
||||
- [ ] Line 234: Object 0x35 "WEIRD DOOR" - investigate
|
||||
- [ ] Lines 392-395: Objects 0xDE-0xE1 "Moving wall flag" - WTF IS THIS?
|
||||
- [ ] Lines 350-353: Diagonal layer 2 mask B objects - verify
|
||||
- [ ] Multiple "Unknown" objects in Type 2 and Type 3 ranges
|
||||
|
||||
---
|
||||
|
||||
### Task E2E: Cinematic Tests
|
||||
**Reference:** `docs/internal/testing/dungeon-gui-test-design.md`
|
||||
**Impact:** Visual regression testing, demo capability
|
||||
**Time Estimate:** 45+ min
|
||||
|
||||
- [ ] Create screenshot capture utility
|
||||
- [ ] Implement basic cinematic test sequence
|
||||
- [ ] Add visual diff comparison
|
||||
- [ ] Document test workflow
|
||||
|
||||
---
|
||||
|
||||
## Already Complete (Verification Only)
|
||||
|
||||
### Tile16Editor Undo/Redo
|
||||
**File:** `src/app/editor/overworld/tile16_editor.cc`
|
||||
**Status:** FULLY IMPLEMENTED
|
||||
|
||||
- [x] `SaveUndoState()` implemented
|
||||
- [x] `Undo()` / `Redo()` implemented
|
||||
- [x] Ctrl+Z / Ctrl+Y shortcuts working
|
||||
- [x] UI button at line 1101
|
||||
- [x] Stack management with limits
|
||||
|
||||
**Action:** Verify it works, no changes needed.
|
||||
|
||||
---
|
||||
|
||||
### Entity Deletion Pattern
|
||||
**File:** `src/app/editor/overworld/entity.cc` line 319
|
||||
**Status:** CORRECT (misleading TODO)
|
||||
|
||||
The `deleted` flag pattern IS correct for ROM editors:
|
||||
- Entities at fixed ROM offsets
|
||||
- `entity_operations.cc` reuses deleted slots
|
||||
- Renderer skips deleted entities
|
||||
|
||||
**Action:** Optionally clarify the TODO comment.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference: File Locations
|
||||
|
||||
| Task | Primary File | Line Numbers |
|
||||
|------|--------------|--------------|
|
||||
| A1 | overworld_editor.cc | 1392, 1397, 1740, 1809, 1819, 1962 |
|
||||
| A2 | overworld_editor.h | 82-87 |
|
||||
| A3 | entity.cc | 152+, 320+ |
|
||||
| A4 | entity.cc | 216-264 |
|
||||
| B1 | object_drawer.cc | 375, 437, 444, 755 |
|
||||
| B2 | object_drawer.cc | 401-435 |
|
||||
| B3 | object_drawer.cc | 524-532, 566-575 |
|
||||
| B4 | room_object.h | Multiple (see section 7 of dungeon ref) |
|
||||
|
||||
---
|
||||
|
||||
## Validation Commands
|
||||
|
||||
```bash
|
||||
# Build
|
||||
cmake --build build_gemini -j8 --target yaze
|
||||
|
||||
# Test
|
||||
ctest --test-dir build_gemini -L stable -j4 --output-on-failure
|
||||
|
||||
# Format check
|
||||
cmake --build build_gemini --target format-check
|
||||
|
||||
# Specific test
|
||||
ctest --test-dir build_gemini -R "Overworld" -V
|
||||
ctest --test-dir build_gemini -R "Dungeon" -V
|
||||
```
|
||||
@@ -0,0 +1,165 @@
|
||||
# Gemini 3 Pro Prompt: Overworld Regression Fix and Improvements
|
||||
|
||||
## Context
|
||||
|
||||
You are working on **yaze** (Yet Another Zelda3 Editor), a C++23 ROM editor for The Legend of Zelda: A Link to the Past. A regression has been introduced that breaks loading of custom ROMs like "Oracle of Secrets" ROM hack.
|
||||
|
||||
## Primary Bug: ASM Version Check Inconsistency
|
||||
|
||||
### Root Cause Analysis
|
||||
|
||||
The recent refactoring introduced `OverworldVersionHelper` for centralized ROM version detection, but **not all code paths were updated to use it consistently**. Specifically:
|
||||
|
||||
**In `src/zelda3/overworld/overworld.cc:71`:**
|
||||
```cpp
|
||||
uint8_t asm_version = (*rom_)[OverworldCustomASMHasBeenApplied];
|
||||
if (asm_version >= 3) { // BUG: 0xFF (255) is >= 3!
|
||||
AssignMapSizes(overworld_maps_);
|
||||
} else {
|
||||
FetchLargeMaps(); // Vanilla ROMs need this path
|
||||
}
|
||||
```
|
||||
|
||||
**The bug**: `asm_version >= 3` evaluates to `true` for vanilla ROMs where `asm_version == 0xFF` (255), causing vanilla ROMs and custom ROMs without ZScream ASM patches to incorrectly call `AssignMapSizes()` instead of `FetchLargeMaps()`.
|
||||
|
||||
**Other places correctly check**:
|
||||
```cpp
|
||||
if (asm_version >= 3 && asm_version != 0xFF) { ... } // Correct
|
||||
```
|
||||
|
||||
### Inconsistent Locations Found
|
||||
|
||||
Search results showing mixed patterns:
|
||||
- `overworld.cc:71` - **BUG**: `if (asm_version >= 3)` - missing `&& asm_version != 0xFF`
|
||||
- `overworld.cc:449` - **BUG**: `if (expanded_flag != 0x04 || asm_version >= 3)` - missing check
|
||||
- `overworld.cc:506` - **BUG**: similar pattern
|
||||
- `overworld.cc:281` - **CORRECT**: `(asm_version < 3 || asm_version == 0xFF)`
|
||||
- `overworld.cc:373` - **CORRECT**: `if (asm_version >= 3 && asm_version != 0xFF)`
|
||||
- Other files also have inconsistencies
|
||||
|
||||
## Your Task
|
||||
|
||||
### Phase 1: Fix the Regression (CRITICAL)
|
||||
|
||||
1. **Update all ASM version checks** in overworld code to either:
|
||||
- Use `OverworldVersionHelper::GetVersion()` and semantic checks like `SupportsAreaEnum()`, OR
|
||||
- Consistently use `asm_version >= 3 && asm_version != 0xFF` pattern
|
||||
|
||||
2. **Key files to fix**:
|
||||
- `src/zelda3/overworld/overworld.cc`
|
||||
- `src/zelda3/overworld/overworld_map.cc`
|
||||
- `src/zelda3/overworld/overworld_item.cc`
|
||||
|
||||
3. **Priority fixes in `overworld.cc`**:
|
||||
- Line 71: Change to `if (asm_version >= 3 && asm_version != 0xFF)`
|
||||
- Line 449: Add `&& asm_version != 0xFF` check
|
||||
- Line 506: Add `&& asm_version != 0xFF` check
|
||||
- Review all other locations from the grep results
|
||||
|
||||
### Phase 2: Standardize Version Checking (Recommended)
|
||||
|
||||
Replace all raw `asm_version` checks with `OverworldVersionHelper`:
|
||||
|
||||
**Instead of:**
|
||||
```cpp
|
||||
uint8_t asm_version = (*rom_)[OverworldCustomASMHasBeenApplied];
|
||||
if (asm_version >= 3 && asm_version != 0xFF) {
|
||||
```
|
||||
|
||||
**Use:**
|
||||
```cpp
|
||||
auto version = OverworldVersionHelper::GetVersion(*rom_);
|
||||
if (OverworldVersionHelper::SupportsAreaEnum(version)) {
|
||||
```
|
||||
|
||||
This centralizes the logic and prevents future inconsistencies.
|
||||
|
||||
### Phase 3: Add Unit Tests
|
||||
|
||||
Create tests in `test/unit/zelda3/overworld_test.cc` to verify:
|
||||
1. Vanilla ROM (0xFF) uses `FetchLargeMaps()` path
|
||||
2. ZScream v3 ROM (0x03) uses `AssignMapSizes()` path
|
||||
3. Custom ROMs with other values behave correctly
|
||||
|
||||
## Key Files Reference
|
||||
|
||||
```
|
||||
src/zelda3/overworld/
|
||||
├── overworld.cc # Main loading logic
|
||||
├── overworld.h
|
||||
├── overworld_map.cc # Individual map handling
|
||||
├── overworld_map.h
|
||||
├── overworld_item.cc # Item loading
|
||||
├── overworld_item.h
|
||||
├── overworld_entrance.h # Entrance/Exit data
|
||||
├── overworld_exit.cc
|
||||
├── overworld_exit.h
|
||||
├── overworld_version_helper.h # Version detection helper
|
||||
```
|
||||
|
||||
## OverworldVersionHelper API
|
||||
|
||||
```cpp
|
||||
enum class OverworldVersion {
|
||||
kVanilla = 0, // 0xFF in ROM - no ZScream ASM
|
||||
kZSCustomV1 = 1,
|
||||
kZSCustomV2 = 2,
|
||||
kZSCustomV3 = 3 // Area enum system
|
||||
};
|
||||
|
||||
class OverworldVersionHelper {
|
||||
static OverworldVersion GetVersion(const Rom& rom);
|
||||
static bool SupportsAreaEnum(OverworldVersion v); // v3 only
|
||||
static bool SupportsExpandedSpace(OverworldVersion v); // v1+
|
||||
static bool SupportsCustomBGColors(OverworldVersion v); // v2+
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
## Commits That Introduced the Regression
|
||||
|
||||
1. `1e39df88a3` - "refactor: enhance overworld entity properties and version handling"
|
||||
- Introduced `OverworldVersionHelper`
|
||||
- 15 files changed, +546 -282 lines
|
||||
|
||||
2. `5894809aaf` - "refactor: improve overworld map version handling and code organization"
|
||||
- Updated `OverworldMap` to use version helper
|
||||
- 4 files changed, +145 -115 lines
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
```bash
|
||||
# Configure
|
||||
cmake --preset mac-dbg
|
||||
|
||||
# Build
|
||||
cmake --build build --target yaze -j8
|
||||
|
||||
# Run unit tests
|
||||
ctest --test-dir build -L stable -R overworld
|
||||
|
||||
# Run the app to test loading
|
||||
./build/bin/Debug/yaze.app/Contents/MacOS/yaze --rom_file=/path/to/oracle_of_secrets.sfc
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. Oracle of Secrets ROM loads correctly in the Overworld Editor
|
||||
2. Vanilla ALTTP ROMs continue to work
|
||||
3. ZScream v3 patched ROMs continue to work
|
||||
4. All existing unit tests pass
|
||||
5. No new compiler warnings
|
||||
|
||||
## Additional Context
|
||||
|
||||
- The editor supports multiple ROM types: Vanilla, ZScream v1/v2/v3 patched ROMs, and custom hacks
|
||||
- `OverworldCustomASMHasBeenApplied` address (0x130000) stores the version byte
|
||||
- 0xFF = vanilla (no patches), 1-2 = legacy ZScream, 3 = current ZScream
|
||||
- "Oracle of Secrets" is a popular ROM hack that may use 0xFF or a custom value
|
||||
|
||||
## Code Quality Requirements
|
||||
|
||||
- Follow Google C++ Style Guide
|
||||
- Use `absl::Status` for error handling
|
||||
- Run clang-format before committing
|
||||
- Update CLAUDE.md coordination board when done
|
||||
@@ -0,0 +1,384 @@
|
||||
# Gemini 3 Pro - v0.4.0 Development Session
|
||||
|
||||
## Context
|
||||
|
||||
You are working on **YAZE** (Yet Another Zelda3 Editor), a C++23 cross-platform ROM editor for The Legend of Zelda: A Link to the Past.
|
||||
|
||||
**Current Situation:**
|
||||
- **v0.3.9**: CI/CD hotfix running (another agent handling - DO NOT TOUCH)
|
||||
- **v0.4.0**: Your focus - "Editor Stability & OOS Support"
|
||||
- **Goal**: Unblock Oracle of Secrets (OOS) ROM hack development
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Use dedicated build directory (DO NOT use build/ or build_agent/)
|
||||
cmake --preset mac-dbg -B build_gemini
|
||||
cmake --build build_gemini -j8 --target yaze
|
||||
|
||||
# Run stable tests
|
||||
ctest --test-dir build_gemini -L stable -j4 --output-on-failure
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Completed Work
|
||||
|
||||
### Priority 2: Message Editor - Expanded BIN Export [COMPLETE]
|
||||
|
||||
**Completed by Gemini 3 Pro:**
|
||||
|
||||
1. **JSON Export Implementation** (`message_data.h/cc`):
|
||||
- Added `SerializeMessagesToJson()` - Converts `MessageData` vector to JSON array
|
||||
- Added `ExportMessagesToJson()` - Writes JSON file with error handling
|
||||
- Uses nlohmann/json library with 2-space pretty printing
|
||||
|
||||
2. **File Dialog Integration** (`message_editor.cc:317-376`):
|
||||
- "Load Expanded Message" - Opens file dialog to load BIN
|
||||
- "Save Expanded Messages" - Saves to remembered path
|
||||
- "Save As..." - Prompts for new path
|
||||
- "Export to JSON" - JSON export with file dialog
|
||||
|
||||
3. **Path Persistence**:
|
||||
- `expanded_message_path_` member stores last used path
|
||||
- Reuses path for subsequent saves
|
||||
|
||||
4. **SaveExpandedMessages() Implementation** (`message_editor.cc:521-571`):
|
||||
- Uses `expanded_message_bin_` member for ROM-like storage
|
||||
- Handles buffer expansion for new messages
|
||||
- Writes terminator byte (0xFF) after data
|
||||
|
||||
### Priority 1: Dungeon Editor - SaveDungeon() [IN PROGRESS]
|
||||
|
||||
**Completed by Gemini 3 Pro:**
|
||||
|
||||
1. **SaveDungeon() Implementation** (`dungeon_editor_system.cc:44-59`):
|
||||
- No longer a stub! Loops through all 296 rooms
|
||||
- Calls `SaveRoomData()` for each room
|
||||
- Tracks dirty state and last save time
|
||||
|
||||
2. **SaveObjects() Implementation** (`room.cc:873-925`):
|
||||
- Properly calculates available space via `CalculateRoomSize()`
|
||||
- Validates encoded size fits before writing
|
||||
- Returns `OutOfRangeError` if data too large
|
||||
|
||||
3. **SaveSprites() Implementation** (`room.cc:927-999`):
|
||||
- Calculates sprite space from pointer table
|
||||
- Handles sortsprites byte
|
||||
- Returns `OutOfRangeError` if data too large
|
||||
|
||||
4. **New Tests** (`test/unit/zelda3/dungeon/dungeon_save_test.cc`):
|
||||
- `SaveObjects_FitsInSpace` - verifies normal save works
|
||||
- `SaveObjects_TooLarge` - verifies overflow detection
|
||||
- `SaveSprites_FitsInSpace` - verifies normal save works
|
||||
- `SaveSprites_TooLarge` - verifies overflow detection
|
||||
|
||||
**Tests Pass:**
|
||||
```bash
|
||||
./build_gemini/bin/Debug/yaze_test_stable --gtest_filter="*DungeonSave*"
|
||||
# [ PASSED ] 4 tests.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Instructions
|
||||
|
||||
### Build and Run Tests
|
||||
|
||||
```bash
|
||||
# Build test target (uses existing build_gemini)
|
||||
cmake --build build_gemini --target yaze_test_stable -j8
|
||||
|
||||
# Run ALL stable tests
|
||||
./build_gemini/bin/Debug/yaze_test_stable
|
||||
|
||||
# Run specific test pattern
|
||||
./build_gemini/bin/Debug/yaze_test_stable --gtest_filter="*DungeonSave*"
|
||||
|
||||
# Run dungeon-related tests
|
||||
./build_gemini/bin/Debug/yaze_test_stable --gtest_filter="*Dungeon*"
|
||||
|
||||
# Run with verbose output
|
||||
./build_gemini/bin/Debug/yaze_test_stable --gtest_filter="*YourTest*" --gtest_print_time=1
|
||||
```
|
||||
|
||||
### Known Test Issues (Pre-existing)
|
||||
|
||||
**FAILING:** `RoomObjectEncodingTest.DetermineObjectTypeType2`
|
||||
- Location: `test/unit/zelda3/dungeon/room_object_encoding_test.cc:29-31`
|
||||
- Issue: `DetermineObjectType()` returns 1 instead of 2 for bytes 0xFC, 0xFD, 0xFF
|
||||
- Status: Pre-existing failure, NOT caused by your changes
|
||||
- Action: Ignore unless you're specifically working on object type detection
|
||||
|
||||
### Test File Locations
|
||||
|
||||
| Test Type | Location | Filter Pattern |
|
||||
|-----------|----------|----------------|
|
||||
| Dungeon Save | `test/unit/zelda3/dungeon/dungeon_save_test.cc` | `*DungeonSave*` |
|
||||
| Room Encoding | `test/unit/zelda3/dungeon/room_object_encoding_test.cc` | `*RoomObjectEncoding*` |
|
||||
| Room Manipulation | `test/unit/zelda3/dungeon/room_manipulation_test.cc` | `*RoomManipulation*` |
|
||||
| Dungeon Integration | `test/integration/dungeon_editor_test.cc` | `*DungeonEditorIntegration*` |
|
||||
| Overworld | `test/unit/zelda3/overworld_test.cc` | `*Overworld*` |
|
||||
|
||||
---
|
||||
|
||||
## Your Priorities (Pick One)
|
||||
|
||||
### Priority 1: Dungeon Editor - Save Infrastructure [COMPLETE] ✅
|
||||
|
||||
**Completed by Gemini 3 Pro:**
|
||||
|
||||
1. **SaveRoomData() Implementation** (`dungeon_editor_system.cc:914-973`):
|
||||
- ✅ Detects if room is currently being edited in UI
|
||||
- ✅ Uses editor's in-memory `Room` object (contains unsaved changes)
|
||||
- ✅ Syncs sprites from `DungeonEditorSystem` to `Room` before saving
|
||||
- ✅ Selectively saves objects only for current room (optimization)
|
||||
|
||||
2. **UI Integration** (`dungeon_editor_v2.cc:244-291`):
|
||||
- ✅ `Save()` method calls `SaveDungeon()` correctly
|
||||
- ✅ Palette saving via `PaletteManager`
|
||||
- ✅ Room objects saved via `Room::SaveObjects()`
|
||||
- ✅ Sprites saved via `DungeonEditorSystem::SaveRoom()`
|
||||
|
||||
3. **Edge Cases Verified:**
|
||||
- ✅ Current room with unsaved changes
|
||||
- ✅ Non-current rooms (sprite-only save)
|
||||
- ✅ Multiple rooms open in tabs
|
||||
- ✅ Empty sprite lists
|
||||
|
||||
**Audit Report:** `zscow_audit_report.md` (artifact)
|
||||
|
||||
**Minor Improvements Recommended:**
|
||||
- Add integration tests for `DungeonEditorSystem` save flow
|
||||
- Remove redundant `SaveObjects()` call in `DungeonEditorV2::Save()`
|
||||
- Document stub methods
|
||||
|
||||
**Test your changes:**
|
||||
```bash
|
||||
./build_gemini/bin/Debug/yaze_test_stable --gtest_filter="*DungeonSave*:*RoomManipulation*"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Priority 3: ZSCOW Audit [COMPLETE] ✅
|
||||
|
||||
**Completed by Gemini 3 Pro:**
|
||||
|
||||
#### 1. Version Detection - VERIFIED ✅
|
||||
|
||||
**Implementation:** `overworld_version_helper.h:51-71`
|
||||
|
||||
| ASM Byte | Version | Status |
|
||||
|----------|---------|--------|
|
||||
| `0xFF` | Vanilla | ✅ Correct |
|
||||
| `0x00` | Vanilla | ✅ **CORRECT** - Expanded ROMs are zero-filled |
|
||||
| `0x01` | v1 | ✅ Verified |
|
||||
| `0x02` | v2 | ⚠️ Not tested (no v2 ROM available) |
|
||||
| `0x03+` | v3 | ✅ Verified |
|
||||
|
||||
**Task 1: Version 0x00 Edge Case - RESOLVED ✅**
|
||||
- **Answer:** Treating `0x00` as vanilla is **CORRECT**
|
||||
- **Rationale:** When vanilla ROM is expanded to 2MB, new space is zero-filled
|
||||
- **Address:** `0x140145` is in expanded space (beyond 1MB)
|
||||
- **Tests Added:** 5 comprehensive tests in `overworld_version_helper_test.cc`
|
||||
- **Bounds Check Added:** Prevents crashes on small ROMs
|
||||
|
||||
**Task 2: Death Mountain GFX TODO - DOCUMENTED ⚠️**
|
||||
|
||||
**Location:** `overworld_map.cc:595-602`
|
||||
|
||||
- **Current logic:** Checks parent ranges `0x03-0x07`, `0x0B-0x0E` (LW), `0x43-0x47`, `0x4B-0x4E` (DW)
|
||||
- **Recommended:** Only check `0x03`, `0x05`, `0x07` (LW) and `0x43`, `0x45`, `0x47` (DW)
|
||||
- **Rationale:** Matches ZScream 3.0.4+ behavior, handles non-large DM areas correctly
|
||||
- **Impact:** Low risk improvement
|
||||
- **Status:** Documented in audit report, not implemented yet
|
||||
|
||||
**Task 3: ConfigureMultiAreaMap - FULLY VERIFIED ✅**
|
||||
|
||||
**Location:** `overworld.cc:267-422`
|
||||
|
||||
- ✅ Vanilla ROM: Correctly rejects Wide/Tall areas
|
||||
- ✅ v1/v2 ROM: Same as vanilla (no area enum support)
|
||||
- ✅ v3 ROM: All 4 sizes work (Small, Large, Wide, Tall)
|
||||
- ✅ Sibling cleanup: Properly resets all quadrants when shrinking Large→Small
|
||||
- ✅ Edge maps: Boundary conditions handled safely
|
||||
|
||||
**Task 4: Special World Hardcoded Cases - VERIFIED ✅**
|
||||
|
||||
**Location:** `overworld_map.cc:202-293`
|
||||
|
||||
- ✅ Triforce room (`0x88`, `0x93`): Special graphics `0x51`, palette `0x00`
|
||||
- ✅ Master Sword area (`0x80`): Special GFX group
|
||||
- ✅ Zora's Domain (`0x81`, `0x82`, `0x89`, `0x8A`): Sprite GFX `0x0E`
|
||||
- ✅ Version-aware logic for v3 vs vanilla/v2
|
||||
|
||||
**Audit Report:** `zscow_audit_report.md` (artifact)
|
||||
|
||||
**Test Results:**
|
||||
```bash
|
||||
./build_gemini/bin/Debug/yaze_test_stable --gtest_filter="OverworldVersionHelperTest*"
|
||||
# [ PASSED ] 5 tests.
|
||||
```
|
||||
|
||||
**Overall Assessment:** ZSCOW implementation is production-ready with one low-priority enhancement (Death Mountain GFX logic).
|
||||
|
||||
---
|
||||
|
||||
### Priority 4: Agent Inspection - Wire Real Data [MEDIUM] - DETAILED
|
||||
|
||||
**Problem:** CLI inspection commands return stub output. Helper functions exist but aren't connected.
|
||||
|
||||
#### Overworld Command Handlers (`src/cli/handlers/game/overworld.cc`)
|
||||
|
||||
| Line | Handler | Status | Helper to Use |
|
||||
|------|---------|--------|---------------|
|
||||
| 33 | `OverworldGetTileCommandHandler` | TODO | Manual ROM read |
|
||||
| 84 | `OverworldSetTileCommandHandler` | TODO | Manual ROM write |
|
||||
| 162 | `OverworldFindTileCommandHandler` | TODO | `FindTileMatches()` |
|
||||
| 325 | `OverworldDescribeMapCommandHandler` | TODO | `BuildMapSummary()` |
|
||||
| 508 | `OverworldListWarpsCommandHandler` | TODO | `CollectWarpEntries()` |
|
||||
| 721 | `OverworldSelectRectCommandHandler` | TODO | N/A (GUI) |
|
||||
| 759 | `OverworldScrollToCommandHandler` | TODO | N/A (GUI) |
|
||||
| 794 | `OverworldSetZoomCommandHandler` | TODO | N/A (GUI) |
|
||||
| 822 | `OverworldGetVisibleRegionCommandHandler` | TODO | N/A (GUI) |
|
||||
|
||||
#### Dungeon Command Handlers (`src/cli/handlers/game/dungeon_commands.cc`)
|
||||
|
||||
| Line | Handler | Status |
|
||||
|------|---------|--------|
|
||||
| 36 | Sprite listing | TODO - need `Room::sprites()` |
|
||||
| 158 | Object listing | TODO - need `Room::objects()` |
|
||||
| 195 | Tile data | TODO - need `Room::floor1()`/`floor2()` |
|
||||
| 237 | Property setting | TODO - need `Room::set_*()` methods |
|
||||
|
||||
#### Available Helper Functions (`overworld_inspect.h`)
|
||||
|
||||
These are fully implemented and ready to use:
|
||||
|
||||
```cpp
|
||||
// Build complete map metadata summary
|
||||
absl::StatusOr<MapSummary> BuildMapSummary(zelda3::Overworld& overworld, int map_id);
|
||||
|
||||
// Find all warps matching query (entrances, exits, holes)
|
||||
absl::StatusOr<std::vector<WarpEntry>> CollectWarpEntries(
|
||||
const zelda3::Overworld& overworld, const WarpQuery& query);
|
||||
|
||||
// Find all occurrences of a tile16 ID
|
||||
absl::StatusOr<std::vector<TileMatch>> FindTileMatches(
|
||||
zelda3::Overworld& overworld, uint16_t tile_id, const TileSearchOptions& options);
|
||||
|
||||
// Get sprites on overworld maps
|
||||
absl::StatusOr<std::vector<OverworldSprite>> CollectOverworldSprites(
|
||||
const zelda3::Overworld& overworld, const SpriteQuery& query);
|
||||
|
||||
// Get entrance details by ID
|
||||
absl::StatusOr<EntranceDetails> GetEntranceDetails(
|
||||
const zelda3::Overworld& overworld, uint8_t entrance_id);
|
||||
|
||||
// Analyze how often a tile is used
|
||||
absl::StatusOr<TileStatistics> AnalyzeTileUsage(
|
||||
zelda3::Overworld& overworld, uint16_t tile_id, const TileSearchOptions& options);
|
||||
```
|
||||
|
||||
#### Example Fix Pattern
|
||||
|
||||
```cpp
|
||||
// BEFORE (broken):
|
||||
absl::Status OverworldFindTileCommandHandler::Execute(
|
||||
CommandContext& context, std::ostream& output) {
|
||||
output << "Placeholder: would find tile..."; // STUB!
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
// AFTER (working):
|
||||
absl::Status OverworldFindTileCommandHandler::Execute(
|
||||
CommandContext& context, std::ostream& output) {
|
||||
auto tile_id_str = context.GetArgument("tile_id");
|
||||
ASSIGN_OR_RETURN(auto tile_id, ParseHexOrDecimal(tile_id_str));
|
||||
|
||||
TileSearchOptions options;
|
||||
if (auto world = context.GetOptionalArgument("world")) {
|
||||
ASSIGN_OR_RETURN(options.world, ParseWorldSpecifier(*world));
|
||||
}
|
||||
|
||||
ASSIGN_OR_RETURN(auto matches,
|
||||
overworld::FindTileMatches(context.overworld(), tile_id, options));
|
||||
|
||||
output << "Found " << matches.size() << " occurrences:\n";
|
||||
for (const auto& match : matches) {
|
||||
output << absl::StrFormat(" Map %d (%s): (%d, %d)\n",
|
||||
match.map_id, WorldName(match.world), match.local_x, match.local_y);
|
||||
}
|
||||
return absl::OkStatus();
|
||||
}
|
||||
```
|
||||
|
||||
#### Priority Order
|
||||
|
||||
1. `OverworldDescribeMapCommandHandler` - Most useful for agents
|
||||
2. `OverworldFindTileCommandHandler` - Common query
|
||||
3. `OverworldListWarpsCommandHandler` - Needed for navigation
|
||||
4. Dungeon sprite/object listing - For dungeon editing support
|
||||
|
||||
---
|
||||
|
||||
## DO NOT TOUCH
|
||||
|
||||
- `.github/workflows/` - CI/CD hotfix in progress
|
||||
- `build/`, `build_agent/`, `build_test/` - User's build directories
|
||||
- `src/util/crash_handler.cc` - Being patched for Windows
|
||||
|
||||
---
|
||||
|
||||
## Code Style
|
||||
|
||||
- Use `absl::Status` and `absl::StatusOr<T>` for error handling
|
||||
- Macros: `RETURN_IF_ERROR()`, `ASSIGN_OR_RETURN()`
|
||||
- Format: `cmake --build build_gemini --target format`
|
||||
- Follow Google C++ Style Guide
|
||||
|
||||
---
|
||||
|
||||
## Reference Documentation
|
||||
|
||||
- **CLAUDE.md** - Project conventions and patterns
|
||||
- **Roadmap:** `docs/internal/roadmaps/2025-11-23-refined-roadmap.md`
|
||||
- **Message Editor Plans:** `docs/internal/plans/message_editor_implementation_roadmap.md`
|
||||
- **Test Guide:** `docs/public/build/quick-reference.md`
|
||||
|
||||
---
|
||||
|
||||
## Recommended Approach
|
||||
|
||||
1. **Pick ONE priority** to focus on
|
||||
2. **Read the relevant files** before making changes
|
||||
3. **Run tests frequently** (`ctest --test-dir build_gemini -L stable`)
|
||||
4. **Commit with clear messages** following conventional commits (`fix:`, `feat:`)
|
||||
5. **Don't touch CI/CD** - that's being handled separately
|
||||
|
||||
---
|
||||
|
||||
## Current State of Uncommitted Work
|
||||
|
||||
The working tree has changes you should be aware of:
|
||||
- `tile16_editor.cc` - Texture queueing improvements
|
||||
- `entity.cc/h` - Sprite movement fixes
|
||||
- `overworld_editor.cc` - Entity rendering
|
||||
- `overworld_map.cc` - Map rendering
|
||||
- `object_drawer.cc/h` - Dungeon objects
|
||||
|
||||
Review these before making overlapping changes.
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
When you're done, one of these should be true:
|
||||
- [x] ~~Dungeon save actually persists changes to ROM~~ **COMPLETE** ✅
|
||||
- [x] ~~Message editor can export expanded BIN files~~ **COMPLETE** ✅
|
||||
- [x] ~~ZSCOW loads correctly for vanilla + v1/v2/v3 ROMs~~ **COMPLETE** ✅
|
||||
- [ ] Agent inspection returns real data
|
||||
|
||||
Good luck!
|
||||
@@ -0,0 +1,482 @@
|
||||
# YAZE Dungeon System - Complete Technical Reference
|
||||
|
||||
Comprehensive reference for AI agents working on the YAZE Dungeon editing system.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
### File Structure
|
||||
```
|
||||
src/zelda3/dungeon/
|
||||
├── dungeon.h/cc # Main Dungeon class
|
||||
├── room.h/cc # Room class (1,337 lines)
|
||||
├── room_object.h/cc # RoomObject encoding (633+249 lines)
|
||||
├── object_drawer.h/cc # Object rendering (210+972 lines)
|
||||
├── object_parser.h/cc # ROM tile parsing (172+387 lines)
|
||||
├── room_entrance.h # Entrance data (367 lines)
|
||||
├── dungeon_rom_addresses.h # ROM address constants (108 lines)
|
||||
|
||||
src/app/editor/dungeon/
|
||||
├── dungeon_editor_v2.h/cc # Main editor (card-based)
|
||||
├── dungeon_room_loader.h/cc # ROM data loading
|
||||
├── dungeon_room_selector.h/cc # Room selection UI
|
||||
├── dungeon_canvas_viewer.h/cc # Canvas rendering
|
||||
├── dungeon_object_selector.h/cc # Object palette
|
||||
├── dungeon_object_interaction.h/cc # Mouse interactions
|
||||
```
|
||||
|
||||
### Data Model
|
||||
```
|
||||
Dungeon
|
||||
└── rooms_[296]
|
||||
└── Room
|
||||
├── tile_objects_[] (RoomObject instances)
|
||||
├── sprites_[]
|
||||
├── chests_in_room_[]
|
||||
├── z3_staircases_[]
|
||||
├── bg1_buffer_ (512x512 pixels)
|
||||
├── bg2_buffer_ (512x512 pixels)
|
||||
└── current_gfx16_[] (16KB graphics)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Room Structure
|
||||
|
||||
### Room Count & Organization
|
||||
- **Total Rooms:** 296 (indices 0x00-0x127)
|
||||
- **Canvas Size:** 512x512 pixels (64x64 tiles)
|
||||
- **Layers:** BG1, BG2, BG3
|
||||
|
||||
### Room Properties
|
||||
```cpp
|
||||
// room.h
|
||||
int room_id_; // Room index (0-295)
|
||||
uint8_t blockset; // Graphics blockset ID
|
||||
uint8_t spriteset; // Sprite set ID
|
||||
uint8_t palette; // Palette ID (0-63)
|
||||
uint8_t layout; // Layout template (0-7)
|
||||
uint8_t floor1, floor2; // Floor graphics (nibbles)
|
||||
uint16_t message_id_; // Associated message
|
||||
|
||||
// Behavioral
|
||||
CollisionKey collision_type; // Collision enum
|
||||
EffectKey effect_type; // Visual effect enum
|
||||
TagKey tag1, tag2; // Special condition tags
|
||||
LayerMergeType layer_merge; // BG1/BG2 blend mode
|
||||
```
|
||||
|
||||
### Layer Merge Types
|
||||
```cpp
|
||||
enum LayerMergeType {
|
||||
LayerMerge00 = 0x00, // Off - Layer 2 invisible
|
||||
LayerMerge01 = 0x01, // Parallax scrolling
|
||||
LayerMerge02 = 0x02, // Dark overlay
|
||||
LayerMerge03 = 0x03, // On top (translucent)
|
||||
LayerMerge04 = 0x04, // Translucent blend
|
||||
LayerMerge05 = 0x05, // Addition blend
|
||||
LayerMerge06 = 0x06, // Normal overlay
|
||||
LayerMerge07 = 0x07, // Transparent
|
||||
LayerMerge08 = 0x08, // Dark room effect
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Object Encoding System
|
||||
|
||||
### 3-Byte Object Format
|
||||
|
||||
Objects are stored as 3 bytes in ROM with three distinct encoding types:
|
||||
|
||||
#### Type 1: Standard Objects (ID 0x00-0xFF)
|
||||
```
|
||||
Byte format: xxxxxxss | yyyyyyss | iiiiiiii
|
||||
b1 b2 b3
|
||||
|
||||
Decoding:
|
||||
x = (b1 & 0xFC) >> 2 // 6 bits (0-63)
|
||||
y = (b2 & 0xFC) >> 2 // 6 bits (0-63)
|
||||
size = ((b1 & 0x03) << 2) | (b2 & 0x03) // 4 bits (0-15)
|
||||
id = b3 // 8 bits
|
||||
```
|
||||
|
||||
#### Type 2: Extended Objects (ID 0x100-0x1FF)
|
||||
```
|
||||
Indicator: b1 >= 0xFC
|
||||
|
||||
Byte format: 111111xx | xxxxyyyy | yyiiiiii
|
||||
b1 b2 b3
|
||||
|
||||
Decoding:
|
||||
id = (b3 & 0x3F) | 0x100
|
||||
x = ((b2 & 0xF0) >> 4) | ((b1 & 0x03) << 4)
|
||||
y = ((b2 & 0x0F) << 2) | ((b3 & 0xC0) >> 6)
|
||||
size = 0 // No size parameter
|
||||
```
|
||||
|
||||
#### Type 3: Rare Objects (ID 0xF00-0xFFF)
|
||||
```
|
||||
Indicator: b3 >= 0xF8
|
||||
|
||||
Byte format: xxxxxxii | yyyyyyii | 11111iii
|
||||
b1 b2 b3
|
||||
|
||||
Decoding:
|
||||
id = (b3 << 4) | 0x80 | ((b2 & 0x03) << 2) | (b1 & 0x03)
|
||||
x = (b1 & 0xFC) >> 2
|
||||
y = (b2 & 0xFC) >> 2
|
||||
size = ((b1 & 0x03) << 2) | (b2 & 0x03)
|
||||
```
|
||||
|
||||
### Object Categories
|
||||
|
||||
| Type | ID Range | Examples |
|
||||
|------|----------|----------|
|
||||
| Type 1 | 0x00-0xFF | Walls, floors, decorations |
|
||||
| Type 2 | 0x100-0x1FF | Corners, stairs, furniture |
|
||||
| Type 3 | 0xF00-0xFFF | Chests, pipes, special objects |
|
||||
|
||||
---
|
||||
|
||||
## 4. ObjectDrawer Rendering System
|
||||
|
||||
### Class Structure
|
||||
```cpp
|
||||
// object_drawer.h
|
||||
class ObjectDrawer {
|
||||
// Entry point
|
||||
absl::Status DrawObject(const RoomObject& object,
|
||||
gfx::BackgroundBuffer& bg1,
|
||||
gfx::BackgroundBuffer& bg2,
|
||||
const gfx::PaletteGroup& palette_group);
|
||||
|
||||
// Data
|
||||
Rom* rom_;
|
||||
const uint8_t* room_gfx_buffer_; // current_gfx16_
|
||||
std::unordered_map<int16_t, int> object_to_routine_map_;
|
||||
std::vector<DrawRoutine> draw_routines_;
|
||||
};
|
||||
```
|
||||
|
||||
### Draw Routine Status
|
||||
|
||||
| # | Routine Name | Status | Lines |
|
||||
|---|--------------|--------|-------|
|
||||
| 0 | DrawRightwards2x2_1to15or32 | COMPLETE | 302-321 |
|
||||
| 1 | DrawRightwards2x4_1to15or26 | COMPLETE | 323-348 |
|
||||
| 2 | DrawRightwards2x4spaced4_1to16 | COMPLETE | 350-373 |
|
||||
| 3 | DrawRightwards2x4spaced4_BothBG | **STUB** | 375-381 |
|
||||
| 4 | DrawRightwards2x2_1to16 | COMPLETE | 383-399 |
|
||||
| 5 | DrawDiagonalAcute_1to16 | **UNCLEAR** | 401-417 |
|
||||
| 6 | DrawDiagonalGrave_1to16 | **UNCLEAR** | 419-435 |
|
||||
| 7 | DrawDownwards2x2_1to15or32 | COMPLETE | 689-708 |
|
||||
| 8 | DrawDownwards4x2_1to15or26 | COMPLETE | 710-753 |
|
||||
| 9 | DrawDownwards4x2_BothBG | **STUB** | 755-761 |
|
||||
| 10 | DrawDownwardsDecor4x2spaced4 | COMPLETE | 763-782 |
|
||||
| 11 | DrawDownwards2x2_1to16 | COMPLETE | 784-799 |
|
||||
| 12 | DrawDownwardsHasEdge1x1 | COMPLETE | 801-813 |
|
||||
| 13 | DrawDownwardsEdge1x1 | COMPLETE | 815-827 |
|
||||
| 14 | DrawDownwardsLeftCorners | COMPLETE | 829-842 |
|
||||
| 15 | DrawDownwardsRightCorners | COMPLETE | 844-857 |
|
||||
| 16 | DrawRightwards4x4_1to16 | COMPLETE | 534-550 |
|
||||
| - | CustomDraw | **STUB** | 524-532 |
|
||||
| - | DrawDoorSwitcherer | **STUB** | 566-575 |
|
||||
|
||||
### INCOMPLETE: BothBG Routines (4 locations)
|
||||
|
||||
These routines should draw to BOTH BG1 and BG2 but currently only call single-layer version:
|
||||
|
||||
```cpp
|
||||
// Line 375-381: DrawRightwards2x4spaced4_1to16_BothBG
|
||||
// Line 437-442: DrawDiagonalAcute_1to16_BothBG
|
||||
// Line 444-449: DrawDiagonalGrave_1to16_BothBG
|
||||
// Line 755-761: DrawDownwards4x2_1to16_BothBG
|
||||
|
||||
// Current (WRONG):
|
||||
void DrawRightwards2x4spaced4_1to16_BothBG(
|
||||
const RoomObject& obj, gfx::BackgroundBuffer& bg, // Only 1 buffer!
|
||||
std::span<const gfx::TileInfo> tiles) {
|
||||
// Just calls single-layer version - misses BG2
|
||||
DrawRightwards2x4spaced4_1to16(obj, bg, tiles);
|
||||
}
|
||||
|
||||
// Should be:
|
||||
void DrawRightwards2x4spaced4_1to16_BothBG(
|
||||
const RoomObject& obj,
|
||||
gfx::BackgroundBuffer& bg1, // Both buffers
|
||||
gfx::BackgroundBuffer& bg2,
|
||||
std::span<const gfx::TileInfo> tiles) {
|
||||
DrawRightwards2x4spaced4_1to16(obj, bg1, tiles);
|
||||
DrawRightwards2x4spaced4_1to16(obj, bg2, tiles);
|
||||
}
|
||||
```
|
||||
|
||||
### UNCLEAR: Diagonal Routines
|
||||
|
||||
```cpp
|
||||
// Lines 401-417, 419-435
|
||||
// Issues:
|
||||
// - Hardcoded +6 and 5 iterations (why?)
|
||||
// - Coordinate formula may produce negative Y
|
||||
// - Only uses 4 tiles from larger span
|
||||
// - No bounds checking
|
||||
|
||||
for (int s = 0; s < size + 6; s++) { // Why +6?
|
||||
for (int i = 0; i < 5; i++) { // Why 5?
|
||||
WriteTile8(bg, obj.x_ + s, obj.y_ + (i - s), tiles[i % 4]);
|
||||
// ^^ (i - s) can be negative when s > i
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Tile Rendering Pipeline
|
||||
|
||||
### WriteTile8() - Tile to Pixel Conversion
|
||||
```cpp
|
||||
// object_drawer.cc lines 863-883
|
||||
void WriteTile8(gfx::BackgroundBuffer& bg, int tile_x, int tile_y,
|
||||
const gfx::TileInfo& tile_info) {
|
||||
// tile coords → pixel coords: tile_x * 8, tile_y * 8
|
||||
DrawTileToBitmap(bitmap, tile_info, tile_x * 8, tile_y * 8, room_gfx_buffer_);
|
||||
}
|
||||
```
|
||||
|
||||
### DrawTileToBitmap() - Pixel Rendering
|
||||
```cpp
|
||||
// object_drawer.cc lines 890-970
|
||||
// Key steps:
|
||||
// 1. Graphics sheet lookup: tile_info.id_ → (sheet_x, sheet_y)
|
||||
// 2. Palette offset: (palette & 0x0F) * 8
|
||||
// 3. Per-pixel with mirroring support
|
||||
// 4. Color 0 = transparent (skipped)
|
||||
|
||||
int tile_sheet_x = (tile_info.id_ % 16) * 8; // 0-127 pixels
|
||||
int tile_sheet_y = (tile_info.id_ / 16) * 8; // 0-127 pixels
|
||||
uint8_t palette_offset = (tile_info.palette_ & 0x0F) * 8;
|
||||
|
||||
for (int py = 0; py < 8; py++) {
|
||||
for (int px = 0; px < 8; px++) {
|
||||
int src_x = tile_info.horizontal_mirror_ ? (7 - px) : px;
|
||||
int src_y = tile_info.vertical_mirror_ ? (7 - py) : py;
|
||||
// Read pixel, apply palette, write to bitmap
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Palette Application (CRITICAL)
|
||||
```cpp
|
||||
// object_drawer.cc lines 71-115
|
||||
// Palette must be applied AFTER drawing, BEFORE SDL sync
|
||||
|
||||
// 1. Draw all objects (writes palette indices 0-255)
|
||||
for (auto& obj : objects) {
|
||||
DrawObject(obj, bg1, bg2, palette_group);
|
||||
}
|
||||
|
||||
// 2. Apply dungeon palette to convert indices → RGB
|
||||
bg1_bmp.SetPalette(dungeon_palette);
|
||||
bg2_bmp.SetPalette(dungeon_palette);
|
||||
|
||||
// 3. Sync to SDL surfaces
|
||||
SDL_LockSurface(bg1_bmp.surface());
|
||||
memcpy(bg1_bmp.surface()->pixels, bg1_bmp.mutable_data().data(), ...);
|
||||
SDL_UnlockSurface(bg1_bmp.surface());
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. ROM Addresses
|
||||
|
||||
### Room Data
|
||||
```cpp
|
||||
kRoomObjectLayoutPointer = 0x882D // Layout pointer table
|
||||
kRoomObjectPointer = 0x874C // Object data pointer
|
||||
kRoomHeaderPointer = 0xB5DD // Room headers (3-byte long)
|
||||
kRoomHeaderPointerBank = 0xB5E7 // Bank byte
|
||||
```
|
||||
|
||||
### Palette & Graphics
|
||||
```cpp
|
||||
kDungeonsMainBgPalettePointers = 0xDEC4B
|
||||
kDungeonsPalettes = 0xDD734
|
||||
kGfxGroupsPointer = 0x6237
|
||||
kTileAddress = 0x1B52 // Main tile graphics
|
||||
kTileAddressFloor = 0x1B5A // Floor tile graphics
|
||||
```
|
||||
|
||||
### Object Subtypes
|
||||
```cpp
|
||||
kRoomObjectSubtype1 = 0x0F8000 // Standard objects
|
||||
kRoomObjectSubtype2 = 0x0F83F0 // Extended objects
|
||||
kRoomObjectSubtype3 = 0x0F84F0 // Rare objects
|
||||
kRoomObjectTileAddress = 0x091B52 // Tile data
|
||||
```
|
||||
|
||||
### Special Objects
|
||||
```cpp
|
||||
kBlocksPointer[1-4] = 0x15AFA-0x15B0F
|
||||
kChestsDataPointer1 = 0xEBFB
|
||||
kTorchData = 0x2736A
|
||||
kPitPointer = 0x394AB
|
||||
kDoorPointers = 0xF83C0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. TODOs in room_object.h (30+ items)
|
||||
|
||||
### Unknown Objects Needing Verification
|
||||
|
||||
| Line | ID | Description |
|
||||
|------|-----|-------------|
|
||||
| 234 | 0x35 | "WEIRD DOOR" - needs investigation |
|
||||
| 252-255 | 0x49-0x4C | "Unknown" Type 1 objects |
|
||||
| 350-353 | 0xC4-0xC7 | "Diagonal layer 2 mask B" - needs verify |
|
||||
| 392-395 | 0xDE-0xE1 | "Moving wall flag" - WTF IS THIS? |
|
||||
| 466-476 | Type 2 | Multiple "Unknown" objects |
|
||||
| 480 | 0x30 | "Intraroom stairs north B" - verify layer |
|
||||
| 486 | 0x36 | "Water ladder (south)" - needs verify |
|
||||
| 512-584 | Type 3 | Multiple "Unknown" objects |
|
||||
|
||||
---
|
||||
|
||||
## 8. DungeonEditorV2 Architecture
|
||||
|
||||
### Card-Based Component System
|
||||
```cpp
|
||||
DungeonEditorV2 (Coordinator)
|
||||
├── DungeonRoomLoader // ROM data loading
|
||||
├── DungeonRoomSelector // Room list/selection
|
||||
├── DungeonCanvasViewer // 512x512 canvas
|
||||
├── DungeonObjectSelector // Object palette
|
||||
├── DungeonObjectInteraction // Mouse handling
|
||||
├── ObjectEditorCard // Property editing
|
||||
└── PaletteEditorWidget // Color editing
|
||||
```
|
||||
|
||||
### Card Types
|
||||
```cpp
|
||||
show_control_panel_ // Room/entrance selection
|
||||
show_room_selector_ // Room list
|
||||
show_room_matrix_ // 16x19 visual layout
|
||||
show_entrances_list_ // Entrance/spawn list
|
||||
show_room_graphics_ // Blockset/palette
|
||||
show_object_editor_ // Object placement
|
||||
show_palette_editor_ // Palette colors
|
||||
show_debug_controls_ // Debug options
|
||||
```
|
||||
|
||||
### Undo/Redo System
|
||||
```cpp
|
||||
// Per-room object snapshots
|
||||
std::unordered_map<int, std::vector<std::vector<RoomObject>>> undo_history_;
|
||||
std::unordered_map<int, std::vector<std::vector<RoomObject>>> redo_history_;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Room Loading Flow
|
||||
|
||||
```
|
||||
LoadRoomFromRom(room_id)
|
||||
│
|
||||
├── Resolve room header pointer (0xB5DD + room_id * 3)
|
||||
│
|
||||
├── Parse header bytes:
|
||||
│ ├── BG2 type, collision, light flag
|
||||
│ ├── Palette, blockset, spriteset
|
||||
│ ├── Effect type, tags
|
||||
│ └── Staircase data
|
||||
│
|
||||
├── Load graphics sheets (16 blocks)
|
||||
│
|
||||
└── LoadObjects()
|
||||
│
|
||||
├── Read floor/layout header (2 bytes)
|
||||
│
|
||||
├── Parse object stream:
|
||||
│ ├── 3 bytes per object
|
||||
│ ├── 0xFF 0xFF = layer boundary
|
||||
│ └── 0xF0 0xFF = door section
|
||||
│
|
||||
└── Handle special objects:
|
||||
├── Staircases
|
||||
├── Chests
|
||||
├── Doors
|
||||
├── Torches
|
||||
└── Blocks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Rendering Pipeline
|
||||
|
||||
```
|
||||
1. LoadRoomGraphics()
|
||||
└── Build graphics sheet list from blockset
|
||||
|
||||
2. CopyRoomGraphicsToBuffer()
|
||||
└── Copy ROM sheets → current_gfx16_[]
|
||||
|
||||
3. RenderRoomGraphics()
|
||||
├── Check dirty flags
|
||||
├── LoadLayoutTilesToBuffer()
|
||||
├── Draw floor to bg1/bg2 buffers
|
||||
└── RenderObjectsToBackground()
|
||||
└── ObjectDrawer::DrawObjectList()
|
||||
|
||||
4. Present (Canvas Viewer)
|
||||
├── Process deferred texture queue
|
||||
├── Create/update GPU textures
|
||||
└── Render to ImGui canvas
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Known Issues Summary
|
||||
|
||||
### BothBG Support (4 stubs)
|
||||
- Line 380: `DrawRightwards2x4spaced4_1to16_BothBG`
|
||||
- Line 441: `DrawDiagonalAcute_1to16_BothBG`
|
||||
- Line 448: `DrawDiagonalGrave_1to16_BothBG`
|
||||
- Line 760: `DrawDownwards4x2_1to16_BothBG`
|
||||
|
||||
**Fix:** Change signature to accept both `bg1` and `bg2` buffers.
|
||||
|
||||
### Diagonal Logic (2 routines)
|
||||
- Lines 401-435: Hardcoded constants, potential negative coords
|
||||
- **Needs:** Game verification or ZScream reference
|
||||
|
||||
### Custom/Door Stubs (2 routines)
|
||||
- Line 524-532: `CustomDraw` - only draws first tile
|
||||
- Line 566-575: `DrawDoorSwitcherer` - only draws first tile
|
||||
|
||||
### Object Names (30+ unknowns)
|
||||
- Multiple objects need in-game verification
|
||||
- See section 7 for full list
|
||||
|
||||
---
|
||||
|
||||
## 12. Testing
|
||||
|
||||
### Run Dungeon Tests
|
||||
```bash
|
||||
# Unit tests
|
||||
ctest --test-dir build -R "dungeon\|Dungeon" -V
|
||||
|
||||
# E2E tests
|
||||
ctest --test-dir build -R "DungeonEditor" -V
|
||||
```
|
||||
|
||||
### E2E Test Files
|
||||
- `test/e2e/dungeon_editor_smoke_test.cc`
|
||||
- `test/e2e/dungeon_canvas_interaction_test.cc`
|
||||
- `test/e2e/dungeon_layer_rendering_test.cc`
|
||||
- `test/e2e/dungeon_object_drawing_test.cc`
|
||||
|
||||
### Test Design Doc
|
||||
`docs/internal/testing/dungeon-gui-test-design.md` (1000+ lines)
|
||||
File diff suppressed because it is too large
Load Diff
1275
docs/internal/agents/archive/large-ref-docs/gemini-zsow-ref.md
Normal file
1275
docs/internal/agents/archive/large-ref-docs/gemini-zsow-ref.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,224 @@
|
||||
# Gemini Pro 3 Overworld Architecture Reference
|
||||
|
||||
Compact reference for YAZE Overworld/Dungeon systems. Use this to quickly locate code and understand patterns.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overworld Editor Architecture (~8,900 lines across 7 modules)
|
||||
|
||||
### Main Editor
|
||||
| File | Lines | Purpose |
|
||||
|------|-------|---------|
|
||||
| `src/app/editor/overworld/overworld_editor.cc` | 3,204 | Main coordinator, canvas, menus |
|
||||
| `src/app/editor/overworld/overworld_editor.h` | 350 | Class definition |
|
||||
|
||||
### Sub-Modules
|
||||
| File | Lines | Purpose |
|
||||
|------|-------|---------|
|
||||
| `map_properties.cc` | 1,759 | `MapPropertiesSystem` - property panels |
|
||||
| `tile16_editor.cc` | 2,584 | `Tile16Editor` - tile editing popup |
|
||||
| `entity.cc` | 491 | `OverworldEntity` - entity containers |
|
||||
| `entity_operations.cc` | 239 | Entity CRUD helpers |
|
||||
| `overworld_entity_renderer.cc` | 151 | Entity drawing |
|
||||
| `scratch_space.cc` | 444 | Tile16 storage utilities |
|
||||
|
||||
### Data Models
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/zelda3/overworld/overworld.cc` | Main overworld class, loading logic |
|
||||
| `src/zelda3/overworld/overworld_map.cc` | Individual map data |
|
||||
| `src/zelda3/overworld/overworld_item.cc` | Item entities |
|
||||
| `src/zelda3/overworld/overworld_version_helper.h` | Version detection API |
|
||||
|
||||
---
|
||||
|
||||
## 2. Completed Work (Gemini's Previous Session)
|
||||
|
||||
### ASM Version Check Standardization
|
||||
- Replaced raw `asm_version >= 3` with `OverworldVersionHelper::SupportsAreaEnum()`
|
||||
- Fixed critical bug: vanilla ROMs (0xFF) incorrectly treated as v3+
|
||||
- Applied consistently across: `overworld.cc`, `overworld_item.cc`, `overworld_map.cc`
|
||||
|
||||
### Tile16Editor Texture Queueing
|
||||
- Fixed `tile16_editor.cc` lines 37-38, 44-45, 56-57
|
||||
- Pattern: `QueueTextureCommand()` instead of `RenderBitmap()` during init
|
||||
|
||||
### Test Infrastructure
|
||||
- Created `test/unit/zelda3/overworld_regression_test.cc`
|
||||
- Tests `OverworldVersionHelper` logic (passing)
|
||||
|
||||
---
|
||||
|
||||
## 3. Remaining Work Checklist
|
||||
|
||||
### P0 - Must Complete
|
||||
|
||||
#### Texture Queueing TODOs in overworld_editor.cc
|
||||
6 locations still use inline rendering instead of deferred queueing:
|
||||
|
||||
| Line | Current Code | Fix Pattern |
|
||||
|------|--------------|-------------|
|
||||
| 1392 | `// TODO: Queue texture` | Use `QueueTextureCommand()` |
|
||||
| 1397 | `// TODO: Queue texture` | Use `QueueTextureCommand()` |
|
||||
| 1740 | `// TODO: Queue texture` | Use `QueueTextureCommand()` |
|
||||
| 1809 | `// TODO: Queue texture` | Use `QueueTextureCommand()` |
|
||||
| 1819 | `// TODO: Queue texture` | Use `QueueTextureCommand()` |
|
||||
| 1962 | `// TODO: Queue texture` | Use `QueueTextureCommand()` |
|
||||
|
||||
**Pattern to follow** (from tile16_editor.cc):
|
||||
```cpp
|
||||
// BEFORE (blocking)
|
||||
Renderer::Get().RenderBitmap(&some_bitmap_);
|
||||
|
||||
// AFTER (non-blocking)
|
||||
gfx::Arena::Get().QueueTextureCommand(gfx::TextureCommand{
|
||||
.operation = gfx::TextureOperation::kCreate,
|
||||
.bitmap = &some_bitmap_,
|
||||
.priority = gfx::TexturePriority::kHigh
|
||||
});
|
||||
```
|
||||
|
||||
#### Entity Deletion Pattern (entity.cc:319) - WORKING CORRECTLY
|
||||
- **Note:** The TODO comment is misleading. The `deleted` flag pattern IS CORRECT for ROM editors
|
||||
- Entities live at fixed ROM offsets, so marking `deleted = true` is the proper approach
|
||||
- Renderer correctly skips deleted entities (see `overworld_entity_renderer.cc`)
|
||||
- `entity_operations.cc` reuses deleted slots when creating new entities
|
||||
- **No fix needed** - just a cleanup of the misleading TODO comment
|
||||
|
||||
### P1 - Should Complete
|
||||
|
||||
#### Tile16Editor Undo/Redo - ALREADY COMPLETE
|
||||
- Location: `tile16_editor.cc:1681-1760`
|
||||
- `SaveUndoState()` called before all edit operations
|
||||
- `Undo()` and `Redo()` fully implemented with `absl::Status` returns
|
||||
- Ctrl+Z/Ctrl+Y handling at lines 224-231
|
||||
- Stack management with `kMaxUndoStates_` limit
|
||||
- **No additional work needed**
|
||||
|
||||
#### Overworld Regression Test Completion
|
||||
- Location: `test/unit/zelda3/overworld_regression_test.cc`
|
||||
- Current: Only tests version helper
|
||||
- Need: Add more comprehensive mock ROM data
|
||||
|
||||
### P2 - Stretch Goals
|
||||
|
||||
#### Dungeon Downwards Draw Routines
|
||||
- Location: `src/zelda3/dungeon/object_drawer.cc` lines 160-185
|
||||
- Missing implementations marked with stubs
|
||||
- Need: Implement based on game ROM patterns
|
||||
|
||||
#### E2E Cinematic Tests
|
||||
- Design doc: `docs/internal/testing/dungeon-gui-test-design.md`
|
||||
- Framework: ImGuiTestEngine integration ready
|
||||
- Need: Screenshot capture, visual verification
|
||||
|
||||
---
|
||||
|
||||
## 4. Key Patterns
|
||||
|
||||
### Bitmap/Surface Synchronization
|
||||
```cpp
|
||||
// CORRECT: Use set_data() for bulk replacement
|
||||
bitmap.set_data(new_data); // Syncs both data_ and surface_->pixels
|
||||
|
||||
// WRONG: Direct assignment breaks sync
|
||||
bitmap.mutable_data() = new_data; // NEVER DO THIS
|
||||
```
|
||||
|
||||
### Graphics Refresh Order
|
||||
```cpp
|
||||
// 1. Update model
|
||||
map.SetProperty(new_value);
|
||||
|
||||
// 2. Reload from ROM
|
||||
map.LoadAreaGraphics();
|
||||
|
||||
// 3. Force render
|
||||
Renderer::Get().RenderBitmap(&map_bitmap_); // Immediate
|
||||
// OR
|
||||
gfx::Arena::Get().QueueTextureCommand(...); // Deferred (preferred)
|
||||
```
|
||||
|
||||
### Version Helper Usage
|
||||
```cpp
|
||||
#include "zelda3/overworld/overworld_version_helper.h"
|
||||
|
||||
auto version = OverworldVersionHelper::GetVersion(*rom_);
|
||||
|
||||
// Feature gates
|
||||
if (OverworldVersionHelper::SupportsAreaEnum(version)) {
|
||||
// v3+ only features
|
||||
}
|
||||
if (OverworldVersionHelper::SupportsExpandedSpace(version)) {
|
||||
// v1+ features
|
||||
}
|
||||
```
|
||||
|
||||
### Entity Rendering Colors (0.85f alpha)
|
||||
```cpp
|
||||
ImVec4 entrance_color(1.0f, 0.85f, 0.0f, 0.85f); // Bright yellow-gold
|
||||
ImVec4 exit_color(0.0f, 1.0f, 1.0f, 0.85f); // Cyan
|
||||
ImVec4 item_color(1.0f, 0.0f, 0.0f, 0.85f); // Bright red
|
||||
ImVec4 sprite_color(1.0f, 0.0f, 1.0f, 0.85f); // Bright magenta
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. File Quick Reference
|
||||
|
||||
### Overworld Editor Entry Points
|
||||
- `Initialize()` → overworld_editor.cc:64
|
||||
- `Load()` → overworld_editor.cc:150
|
||||
- `Update()` → overworld_editor.cc:203
|
||||
- `DrawFullscreenCanvas()` → overworld_editor.cc:472
|
||||
- `ProcessDeferredTextures()` → overworld_editor.cc:899
|
||||
|
||||
### Tile16Editor Entry Points
|
||||
- `Initialize()` → tile16_editor.cc:30
|
||||
- `Update()` → tile16_editor.cc:100
|
||||
- `DrawTile16Editor()` → tile16_editor.cc:200
|
||||
|
||||
### Entity System Entry Points
|
||||
- `OverworldEntity::Draw()` → entity.cc:50
|
||||
- `DeleteSelectedEntity()` → entity.cc:319
|
||||
- Entity containers: `entrances_`, `exits_`, `items_`, `sprites_`
|
||||
|
||||
### Dungeon System Entry Points
|
||||
- `ObjectDrawer::DrawObject()` → object_drawer.cc:50
|
||||
- Draw routines: lines 100-300
|
||||
- Downwards stubs: lines 160-185
|
||||
|
||||
---
|
||||
|
||||
## 6. Test Commands
|
||||
|
||||
```bash
|
||||
# Run overworld regression test specifically
|
||||
ctest --test-dir build_gemini -R "OverworldRegression" -V
|
||||
|
||||
# Run all zelda3 unit tests
|
||||
ctest --test-dir build_gemini -R "zelda3" -L unit
|
||||
|
||||
# Run GUI E2E tests
|
||||
ctest --test-dir build_gemini -L gui -V
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Validation Criteria
|
||||
|
||||
### For Texture Queueing Fix
|
||||
- [ ] No `// TODO` comments remain at specified lines
|
||||
- [ ] `QueueTextureCommand()` used consistently
|
||||
- [ ] UI doesn't freeze when loading maps
|
||||
- [ ] `ctest -L stable` passes
|
||||
|
||||
### For Entity Deletion Fix
|
||||
- [ ] Items actually removed from vector
|
||||
- [ ] No memory leaks (items properly destroyed)
|
||||
- [ ] Undo can restore deleted items (if implementing undo)
|
||||
|
||||
### For Dungeon Draw Routines
|
||||
- [ ] Downwards objects render correctly
|
||||
- [ ] Layer ordering maintained (BG1 → BG2 → BG3)
|
||||
- [ ] Palette applied correctly after render
|
||||
@@ -0,0 +1,859 @@
|
||||
# Z3ED CLI & Agent API Enhancement Design
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines comprehensive enhancements to the z3ed CLI and agent APIs to significantly improve AI agent interaction with YAZE. The design focuses on enabling better automation, testing, and feature development through a robust command interface, programmatic editor access, and enhanced collaboration features.
|
||||
|
||||
## Current Architecture Analysis
|
||||
|
||||
### Existing Components
|
||||
- **CLI Framework**: ModernCLI with CommandRegistry pattern
|
||||
- **Command Handlers**: 70+ specialized handlers (hex, palette, sprite, music, dialogue, dungeon, overworld, gui, emulator)
|
||||
- **Canvas Automation API**: Programmatic interface for tile operations, selection, and view control
|
||||
- **Network Client**: WebSocket/HTTP fallback for collaborative editing
|
||||
- **HTTP API**: REST endpoints for health, models, and basic operations
|
||||
- **Model Integration**: Ollama and Gemini support through ModelRegistry
|
||||
|
||||
### Key Strengths
|
||||
- Clean command handler abstraction with consistent execution pattern
|
||||
- Canvas automation already supports tile operations and coordinate conversion
|
||||
- Network infrastructure in place for collaboration
|
||||
- Extensible model registry for multiple AI providers
|
||||
|
||||
### Gaps to Address
|
||||
- Limited ROM direct manipulation commands
|
||||
- No session persistence or REPL mode
|
||||
- Minimal test generation capabilities
|
||||
- Limited agent coordination features
|
||||
- No batch operation support for complex workflows
|
||||
- Missing introspection and discovery APIs
|
||||
|
||||
## 1. Z3ED CLI Enhancements
|
||||
|
||||
### 1.1 ROM Operations Commands
|
||||
|
||||
```bash
|
||||
# Direct ROM manipulation
|
||||
z3ed rom read --address <hex> [--length <bytes>] [--format hex|ascii|binary]
|
||||
z3ed rom write --address <hex> --data <hex_string> [--verify]
|
||||
z3ed rom validate [--checksums] [--headers] [--regions]
|
||||
z3ed rom diff --base <rom1> --compare <rom2> [--output patch]
|
||||
z3ed rom patch --input <rom> --patch <ips|bps> --output <patched_rom>
|
||||
z3ed rom export --region <name> --start <hex> --end <hex> --output <file>
|
||||
z3ed rom import --region <name> --address <hex> --input <file>
|
||||
|
||||
# ROM state management
|
||||
z3ed rom snapshot --name <snapshot_name> [--compress]
|
||||
z3ed rom restore --snapshot <name> [--verify]
|
||||
z3ed rom list-snapshots [--details]
|
||||
z3ed rom compare-snapshot --current --snapshot <name>
|
||||
```
|
||||
|
||||
#### Implementation Details
|
||||
```cpp
|
||||
class RomReadCommandHandler : public CommandHandler {
|
||||
protected:
|
||||
absl::Status ValidateArgs(const ArgumentParser& parser) override {
|
||||
RETURN_IF_ERROR(parser.RequireArgs({"address"}));
|
||||
if (auto len = parser.GetInt("length")) {
|
||||
if (*len <= 0 || *len > 0x10000) {
|
||||
return absl::InvalidArgumentError("Length must be 1-65536 bytes");
|
||||
}
|
||||
}
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
absl::Status Execute(Rom* rom, const ArgumentParser& parser,
|
||||
OutputFormatter& formatter) override {
|
||||
uint32_t address = parser.GetHex("address").value();
|
||||
int length = parser.GetInt("length").value_or(16);
|
||||
std::string format = parser.GetString("format").value_or("hex");
|
||||
|
||||
std::vector<uint8_t> data;
|
||||
for (int i = 0; i < length; i++) {
|
||||
data.push_back(rom->ReadByte(address + i));
|
||||
}
|
||||
|
||||
formatter.AddField("address", absl::StrFormat("0x%06X", address));
|
||||
formatter.AddField("data", FormatData(data, format));
|
||||
return absl::OkStatus();
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### 1.2 Editor Automation Commands
|
||||
|
||||
```bash
|
||||
# Dungeon editor automation
|
||||
z3ed editor dungeon place-object --room <id> --type <object_id> --x <x> --y <y>
|
||||
z3ed editor dungeon remove-object --room <id> --object-index <idx>
|
||||
z3ed editor dungeon set-property --room <id> --property <name> --value <val>
|
||||
z3ed editor dungeon list-objects --room <id> [--filter-type <type>]
|
||||
z3ed editor dungeon validate-room --room <id> [--fix-issues]
|
||||
|
||||
# Overworld editor automation
|
||||
z3ed editor overworld set-tile --map <id> --x <x> --y <y> --tile <tile_id>
|
||||
z3ed editor overworld place-entrance --map <id> --x <x> --y <y> --target <room>
|
||||
z3ed editor overworld modify-sprite --map <id> --sprite-index <idx> --property <prop> --value <val>
|
||||
z3ed editor overworld generate-minimap --map <id> --output <file>
|
||||
|
||||
# Graphics editor automation
|
||||
z3ed editor graphics import-sheet --sheet <id> --file <png> [--palette <id>]
|
||||
z3ed editor graphics export-sheet --sheet <id> --output <png>
|
||||
z3ed editor graphics modify-palette --palette <id> --color <idx> --rgb <#RRGGBB>
|
||||
|
||||
# Batch operations
|
||||
z3ed editor batch --script <file> [--dry-run] [--parallel]
|
||||
```
|
||||
|
||||
#### Batch Script Format (JSON)
|
||||
```json
|
||||
{
|
||||
"operations": [
|
||||
{
|
||||
"editor": "dungeon",
|
||||
"action": "place-object",
|
||||
"params": {
|
||||
"room": 1,
|
||||
"type": 0x22,
|
||||
"x": 10,
|
||||
"y": 15
|
||||
}
|
||||
},
|
||||
{
|
||||
"editor": "overworld",
|
||||
"action": "set-tile",
|
||||
"params": {
|
||||
"map": 0x00,
|
||||
"x": 20,
|
||||
"y": 30,
|
||||
"tile": 0x142
|
||||
}
|
||||
}
|
||||
],
|
||||
"options": {
|
||||
"stop_on_error": false,
|
||||
"validate_after": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 1.3 Testing Commands
|
||||
|
||||
```bash
|
||||
# Test execution
|
||||
z3ed test run --category <unit|integration|e2e> [--filter <pattern>]
|
||||
z3ed test validate-feature --feature <name> [--rom <file>]
|
||||
z3ed test generate --target <class|file> --output <test_file>
|
||||
z3ed test coverage --report <html|json|text>
|
||||
|
||||
# Test recording
|
||||
z3ed test record --name <test_name> --start
|
||||
z3ed test record --stop [--save-as <file>]
|
||||
z3ed test playback --file <test_file> [--speed <1-10>]
|
||||
|
||||
# Regression testing
|
||||
z3ed test baseline --create --name <baseline>
|
||||
z3ed test baseline --compare --name <baseline> [--threshold <percent>]
|
||||
```
|
||||
|
||||
### 1.4 Build & Deploy Commands
|
||||
|
||||
```bash
|
||||
# Build management
|
||||
z3ed build --preset <preset> [--verbose] [--parallel <jobs>]
|
||||
z3ed build clean [--all]
|
||||
z3ed build test [--preset <preset>]
|
||||
z3ed build package --platform <win|mac|linux> --output <dir>
|
||||
|
||||
# CI/CD integration
|
||||
z3ed ci status [--workflow <name>]
|
||||
z3ed ci trigger --workflow <name> [--branch <branch>]
|
||||
z3ed ci logs --run-id <id> [--follow]
|
||||
z3ed ci artifacts --run-id <id> --download <path>
|
||||
```
|
||||
|
||||
### 1.5 Query & Introspection Interface
|
||||
|
||||
```bash
|
||||
# System queries
|
||||
z3ed query rom-info [--detailed]
|
||||
z3ed query test-status [--failures-only]
|
||||
z3ed query build-status [--preset <preset>]
|
||||
z3ed query available-commands [--category <cat>] [--format tree|list|json]
|
||||
|
||||
# Data queries
|
||||
z3ed query find-tiles --pattern <hex> [--context <bytes>]
|
||||
z3ed query find-sprites --type <id> [--map <id>]
|
||||
z3ed query find-text --search <string> [--case-sensitive]
|
||||
z3ed query dependencies --entity <type:id>
|
||||
|
||||
# Statistics
|
||||
z3ed query stats --type <rom|dungeon|overworld|sprites>
|
||||
z3ed query usage --command <name> [--since <date>]
|
||||
```
|
||||
|
||||
### 1.6 Interactive REPL Mode
|
||||
|
||||
```bash
|
||||
# Start REPL
|
||||
z3ed repl [--rom <file>] [--history <file>]
|
||||
|
||||
# REPL Features:
|
||||
# - Persistent ROM state across commands
|
||||
# - Command history with arrow keys
|
||||
# - Tab completion for commands and parameters
|
||||
# - Context-aware suggestions
|
||||
# - Session recording/playback
|
||||
# - Variable assignment ($var = command output)
|
||||
# - Pipes and filters (command1 | command2)
|
||||
```
|
||||
|
||||
#### REPL Implementation
|
||||
```cpp
|
||||
class ReplSession {
|
||||
Rom* rom_;
|
||||
std::map<std::string, json> variables_;
|
||||
std::vector<std::string> history_;
|
||||
|
||||
public:
|
||||
absl::Status ProcessLine(const std::string& line) {
|
||||
// Parse for variable assignment
|
||||
if (auto var_match = ParseVariableAssignment(line)) {
|
||||
auto result = ExecuteCommand(var_match->command);
|
||||
variables_[var_match->var_name] = result;
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
// Parse for pipes
|
||||
if (auto pipe_commands = ParsePipe(line)) {
|
||||
json previous_output;
|
||||
for (const auto& cmd : *pipe_commands) {
|
||||
auto expanded = ExpandVariables(cmd, previous_output);
|
||||
previous_output = ExecuteCommand(expanded);
|
||||
}
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
// Simple command
|
||||
return ExecuteCommand(line);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## 2. Agent API Improvements
|
||||
|
||||
### 2.1 Enhanced Canvas Automation API
|
||||
|
||||
```cpp
|
||||
namespace yaze {
|
||||
namespace gui {
|
||||
|
||||
class EnhancedCanvasAutomationAPI : public CanvasAutomationAPI {
|
||||
public:
|
||||
// Object selection by properties
|
||||
struct ObjectQuery {
|
||||
std::optional<int> type_id;
|
||||
std::optional<ImVec2> position_min;
|
||||
std::optional<ImVec2> position_max;
|
||||
std::optional<std::string> name_pattern;
|
||||
std::map<std::string, std::any> properties;
|
||||
};
|
||||
|
||||
std::vector<ObjectHandle> FindObjects(const ObjectQuery& query) const;
|
||||
|
||||
// Batch operations
|
||||
struct BatchOperation {
|
||||
enum Type { MOVE, MODIFY, DELETE, DUPLICATE };
|
||||
Type type;
|
||||
std::vector<ObjectHandle> objects;
|
||||
std::map<std::string, std::any> parameters;
|
||||
};
|
||||
|
||||
absl::Status ExecuteBatch(const std::vector<BatchOperation>& ops);
|
||||
|
||||
// Validation queries
|
||||
bool IsValidPlacement(ObjectHandle obj, ImVec2 position) const;
|
||||
std::vector<std::string> GetPlacementErrors(ObjectHandle obj, ImVec2 pos) const;
|
||||
|
||||
// Event simulation
|
||||
void SimulateClick(ImVec2 position, int button = 0);
|
||||
void SimulateDrag(ImVec2 from, ImVec2 to);
|
||||
void SimulateKeyPress(ImGuiKey key, bool shift = false, bool ctrl = false);
|
||||
void SimulateContextMenu(ImVec2 position);
|
||||
|
||||
// Advanced queries
|
||||
struct CanvasStatistics {
|
||||
int total_objects;
|
||||
std::map<int, int> objects_by_type;
|
||||
float canvas_coverage_percent;
|
||||
ImVec2 bounding_box_min;
|
||||
ImVec2 bounding_box_max;
|
||||
};
|
||||
|
||||
CanvasStatistics GetStatistics() const;
|
||||
|
||||
// Undo/Redo support
|
||||
bool CanUndo() const;
|
||||
bool CanRedo() const;
|
||||
void Undo();
|
||||
void Redo();
|
||||
std::vector<std::string> GetUndoHistory(int count = 10) const;
|
||||
};
|
||||
|
||||
} // namespace gui
|
||||
} // namespace yaze
|
||||
```
|
||||
|
||||
### 2.2 Programmatic Editor Access
|
||||
|
||||
```cpp
|
||||
namespace yaze {
|
||||
namespace app {
|
||||
|
||||
class EditorAutomationAPI {
|
||||
public:
|
||||
// Editor lifecycle
|
||||
absl::Status OpenEditor(EditorType type, const std::string& params = "");
|
||||
absl::Status CloseEditor(EditorHandle handle);
|
||||
std::vector<EditorHandle> GetOpenEditors() const;
|
||||
|
||||
// State snapshots
|
||||
absl::StatusOr<EditorSnapshot> CaptureState(EditorHandle editor);
|
||||
absl::Status RestoreState(EditorHandle editor, const EditorSnapshot& snapshot);
|
||||
absl::Status CompareStates(const EditorSnapshot& s1, const EditorSnapshot& s2);
|
||||
|
||||
// Query current state
|
||||
struct EditorState {
|
||||
EditorType type;
|
||||
std::string name;
|
||||
bool has_unsaved_changes;
|
||||
std::map<std::string, std::any> properties;
|
||||
std::vector<std::string> available_actions;
|
||||
};
|
||||
|
||||
EditorState GetState(EditorHandle editor) const;
|
||||
|
||||
// Execute operations
|
||||
absl::Status ExecuteAction(EditorHandle editor,
|
||||
const std::string& action,
|
||||
const json& parameters);
|
||||
|
||||
// Event subscription
|
||||
using EventCallback = std::function<void(const EditorEvent&)>;
|
||||
|
||||
void Subscribe(EditorHandle editor, EventType type, EventCallback cb);
|
||||
void Unsubscribe(EditorHandle editor, EventType type);
|
||||
|
||||
// Validation
|
||||
absl::Status ValidateEditor(EditorHandle editor);
|
||||
std::vector<ValidationIssue> GetValidationIssues(EditorHandle editor);
|
||||
};
|
||||
|
||||
} // namespace app
|
||||
} // namespace yaze
|
||||
```
|
||||
|
||||
### 2.3 Test Generation API
|
||||
|
||||
```cpp
|
||||
namespace yaze {
|
||||
namespace test {
|
||||
|
||||
class TestGenerationAPI {
|
||||
public:
|
||||
// Record interactions
|
||||
void StartRecording(const std::string& test_name);
|
||||
void StopRecording();
|
||||
void PauseRecording();
|
||||
void ResumeRecording();
|
||||
|
||||
// Generate tests from recordings
|
||||
absl::StatusOr<std::string> GenerateTestCode(
|
||||
const std::string& test_name,
|
||||
TestFramework framework = TestFramework::GTEST);
|
||||
|
||||
// Generate tests from specifications
|
||||
struct TestSpecification {
|
||||
std::string class_under_test;
|
||||
std::vector<std::string> methods_to_test;
|
||||
bool include_edge_cases = true;
|
||||
bool include_error_cases = true;
|
||||
bool generate_mocks = true;
|
||||
};
|
||||
|
||||
absl::StatusOr<std::string> GenerateTests(const TestSpecification& spec);
|
||||
|
||||
// Test fixtures from state
|
||||
absl::StatusOr<std::string> GenerateFixture(EditorHandle editor);
|
||||
|
||||
// Regression test generation
|
||||
absl::StatusOr<std::string> GenerateRegressionTest(
|
||||
const std::string& bug_description,
|
||||
const std::vector<std::string>& repro_steps);
|
||||
|
||||
// Test execution
|
||||
struct TestResult {
|
||||
bool passed;
|
||||
std::string output;
|
||||
double execution_time_ms;
|
||||
std::vector<std::string> failures;
|
||||
};
|
||||
|
||||
absl::StatusOr<TestResult> RunGeneratedTest(const std::string& test_code);
|
||||
};
|
||||
|
||||
} // namespace test
|
||||
} // namespace yaze
|
||||
```
|
||||
|
||||
## 3. Agent UI Enhancements
|
||||
|
||||
### 3.1 Status Dashboard
|
||||
|
||||
```cpp
|
||||
class AgentStatusDashboard : public Panel {
|
||||
public:
|
||||
void Draw() override {
|
||||
// Real-time agent activity
|
||||
DrawAgentActivity();
|
||||
|
||||
// Test execution progress
|
||||
DrawTestProgress();
|
||||
|
||||
// Build/CI status
|
||||
DrawBuildStatus();
|
||||
|
||||
// Recent changes
|
||||
DrawRecentChanges();
|
||||
|
||||
// Performance metrics
|
||||
DrawPerformanceMetrics();
|
||||
}
|
||||
|
||||
private:
|
||||
struct AgentActivity {
|
||||
std::string agent_name;
|
||||
std::string current_task;
|
||||
float progress_percent;
|
||||
std::chrono::steady_clock::time_point started_at;
|
||||
};
|
||||
|
||||
std::vector<AgentActivity> active_agents_;
|
||||
|
||||
void DrawAgentActivity() {
|
||||
ImGui::Text("Active Agents");
|
||||
for (const auto& agent : active_agents_) {
|
||||
ImGui::ProgressBar(agent.progress_percent / 100.0f,
|
||||
ImVec2(-1, 0),
|
||||
agent.current_task.c_str());
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### 3.2 Agent Control Panel
|
||||
|
||||
```cpp
|
||||
class AgentControlPanel : public Panel {
|
||||
public:
|
||||
void Draw() override {
|
||||
// Agent task management
|
||||
if (ImGui::Button("Start New Task")) {
|
||||
ShowTaskDialog();
|
||||
}
|
||||
|
||||
// Active tasks
|
||||
DrawActiveTasks();
|
||||
|
||||
// Agent logs
|
||||
DrawAgentLogs();
|
||||
|
||||
// Manual intervention
|
||||
DrawInterventionControls();
|
||||
|
||||
// Collaboration coordination
|
||||
DrawCollaborationStatus();
|
||||
}
|
||||
|
||||
private:
|
||||
void ShowTaskDialog() {
|
||||
ImGui::OpenPopup("New Agent Task");
|
||||
if (ImGui::BeginPopupModal("New Agent Task")) {
|
||||
static char task_name[256];
|
||||
ImGui::InputText("Task Name", task_name, sizeof(task_name));
|
||||
|
||||
static int selected_agent = 0;
|
||||
ImGui::Combo("Agent", &selected_agent, available_agents_);
|
||||
|
||||
if (ImGui::Button("Start")) {
|
||||
StartAgentTask(task_name, selected_agent);
|
||||
ImGui::CloseCurrentPopup();
|
||||
}
|
||||
ImGui::EndPopup();
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## 4. Network/Collaboration Features
|
||||
|
||||
### 4.1 Multi-Agent Coordination
|
||||
|
||||
```cpp
|
||||
namespace yaze {
|
||||
namespace agent {
|
||||
|
||||
class MultiAgentCoordinator {
|
||||
public:
|
||||
// Agent registration
|
||||
absl::Status RegisterAgent(const AgentInfo& info);
|
||||
absl::Status UnregisterAgent(const std::string& agent_id);
|
||||
|
||||
// Work queue management
|
||||
absl::Status QueueTask(const Task& task);
|
||||
absl::StatusOr<Task> ClaimTask(const std::string& agent_id);
|
||||
absl::Status CompleteTask(const std::string& task_id, const TaskResult& result);
|
||||
|
||||
// Shared state
|
||||
absl::Status UpdateSharedState(const std::string& key, const json& value);
|
||||
absl::StatusOr<json> GetSharedState(const std::string& key);
|
||||
absl::Status SubscribeToState(const std::string& key, StateCallback cb);
|
||||
|
||||
// Conflict resolution
|
||||
enum ConflictStrategy {
|
||||
LAST_WRITE_WINS,
|
||||
MERGE,
|
||||
MANUAL_RESOLUTION,
|
||||
QUEUE_SEQUENTIAL
|
||||
};
|
||||
|
||||
absl::Status SetConflictStrategy(ConflictStrategy strategy);
|
||||
absl::StatusOr<Resolution> ResolveConflict(const Conflict& conflict);
|
||||
|
||||
// Agent discovery
|
||||
std::vector<AgentInfo> DiscoverAgents(const AgentQuery& query);
|
||||
absl::StatusOr<AgentCapabilities> GetCapabilities(const std::string& agent_id);
|
||||
};
|
||||
|
||||
} // namespace agent
|
||||
} // namespace yaze
|
||||
```
|
||||
|
||||
### 4.2 Remote z3ed Access
|
||||
|
||||
```yaml
|
||||
# OpenAPI 3.0 Specification
|
||||
openapi: 3.0.0
|
||||
info:
|
||||
title: Z3ED Remote API
|
||||
version: 1.0.0
|
||||
|
||||
paths:
|
||||
/api/v1/command:
|
||||
post:
|
||||
summary: Execute z3ed command
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
command:
|
||||
type: string
|
||||
example: "rom read --address 0x1000 --length 16"
|
||||
session_id:
|
||||
type: string
|
||||
timeout_ms:
|
||||
type: integer
|
||||
default: 30000
|
||||
responses:
|
||||
200:
|
||||
description: Command executed successfully
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
result:
|
||||
type: object
|
||||
execution_time_ms:
|
||||
type: number
|
||||
|
||||
/api/v1/session:
|
||||
post:
|
||||
summary: Create new z3ed session
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
rom_path:
|
||||
type: string
|
||||
persist:
|
||||
type: boolean
|
||||
default: false
|
||||
responses:
|
||||
200:
|
||||
description: Session created
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
session_id:
|
||||
type: string
|
||||
expires_at:
|
||||
type: string
|
||||
format: date-time
|
||||
|
||||
/api/v1/websocket:
|
||||
get:
|
||||
summary: WebSocket endpoint for real-time updates
|
||||
responses:
|
||||
101:
|
||||
description: Switching Protocols
|
||||
```
|
||||
|
||||
### 4.3 WebSocket Protocol
|
||||
|
||||
```typescript
|
||||
// WebSocket message types
|
||||
interface Z3edWebSocketMessage {
|
||||
type: 'command' | 'event' | 'subscribe' | 'unsubscribe';
|
||||
id: string;
|
||||
payload: any;
|
||||
}
|
||||
|
||||
// Command execution
|
||||
interface CommandMessage {
|
||||
type: 'command';
|
||||
id: string;
|
||||
payload: {
|
||||
command: string;
|
||||
args: string[];
|
||||
stream: boolean; // Stream output as it's generated
|
||||
};
|
||||
}
|
||||
|
||||
// Event subscription
|
||||
interface SubscribeMessage {
|
||||
type: 'subscribe';
|
||||
id: string;
|
||||
payload: {
|
||||
events: Array<'editor.changed' | 'test.completed' | 'build.status'>;
|
||||
};
|
||||
}
|
||||
|
||||
// Server events
|
||||
interface EventMessage {
|
||||
type: 'event';
|
||||
id: string;
|
||||
payload: {
|
||||
event: string;
|
||||
data: any;
|
||||
timestamp: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## 5. Implementation Plan
|
||||
|
||||
### Phase 1: Foundation (Weeks 1-2)
|
||||
1. Implement core ROM operations commands
|
||||
2. Add REPL infrastructure
|
||||
3. Enhance Canvas Automation API with batch operations
|
||||
4. Create command discovery/introspection system
|
||||
|
||||
### Phase 2: Editor Integration (Weeks 3-4)
|
||||
1. Implement editor automation commands
|
||||
2. Add programmatic editor access API
|
||||
3. Create test recording infrastructure
|
||||
4. Build event subscription system
|
||||
|
||||
### Phase 3: Testing & CI (Weeks 5-6)
|
||||
1. Implement test generation API
|
||||
2. Add test execution commands
|
||||
3. Create CI/CD integration commands
|
||||
4. Build regression test framework
|
||||
|
||||
### Phase 4: Collaboration (Weeks 7-8)
|
||||
1. Implement multi-agent coordinator
|
||||
2. Add REST API endpoints
|
||||
3. Create WebSocket real-time protocol
|
||||
4. Build conflict resolution system
|
||||
|
||||
### Phase 5: UI & Polish (Weeks 9-10)
|
||||
1. Create Agent Status Dashboard
|
||||
2. Build Agent Control Panel
|
||||
3. Add comprehensive documentation
|
||||
4. Create example workflows
|
||||
|
||||
## 6. Example Workflows
|
||||
|
||||
### Workflow 1: Automated Dungeon Testing
|
||||
```bash
|
||||
# Start REPL session
|
||||
z3ed repl --rom zelda3.sfc
|
||||
|
||||
# Record baseline
|
||||
> rom snapshot --name baseline
|
||||
> editor dungeon --room 0
|
||||
|
||||
# Start test recording
|
||||
> test record --name dungeon_placement_test --start
|
||||
|
||||
# Perform operations
|
||||
> editor dungeon place-object --room 0 --type 0x22 --x 10 --y 15
|
||||
> editor dungeon place-object --room 0 --type 0x23 --x 20 --y 15
|
||||
> query stats --type dungeon
|
||||
|
||||
# Stop and generate test
|
||||
> test record --stop
|
||||
> test generate --from-recording dungeon_placement_test --output test_dungeon.cc
|
||||
```
|
||||
|
||||
### Workflow 2: Multi-Agent ROM Editing
|
||||
```python
|
||||
import z3ed_client
|
||||
|
||||
# Agent 1: Overworld specialist
|
||||
agent1 = z3ed_client.Agent("overworld_agent")
|
||||
agent1.connect("localhost:8080")
|
||||
|
||||
# Agent 2: Dungeon specialist
|
||||
agent2 = z3ed_client.Agent("dungeon_agent")
|
||||
agent2.connect("localhost:8080")
|
||||
|
||||
# Coordinator assigns tasks
|
||||
coordinator = z3ed_client.Coordinator()
|
||||
coordinator.queue_task({
|
||||
"type": "overworld",
|
||||
"action": "optimize_tilemap",
|
||||
"map_id": 0x00
|
||||
})
|
||||
coordinator.queue_task({
|
||||
"type": "dungeon",
|
||||
"action": "validate_rooms",
|
||||
"rooms": range(0, 296)
|
||||
})
|
||||
|
||||
# Agents work in parallel
|
||||
results = coordinator.wait_for_completion()
|
||||
```
|
||||
|
||||
### Workflow 3: AI-Powered Test Generation
|
||||
```bash
|
||||
# Analyze class for test generation
|
||||
z3ed test analyze --class OverworldEditor
|
||||
|
||||
# Generate comprehensive tests
|
||||
z3ed test generate \
|
||||
--target OverworldEditor \
|
||||
--include-edge-cases \
|
||||
--include-mocks \
|
||||
--framework gtest \
|
||||
--output overworld_editor_test.cc
|
||||
|
||||
# Run generated tests
|
||||
z3ed test run --file overworld_editor_test.cc --verbose
|
||||
|
||||
# Create regression test from bug
|
||||
z3ed test regression \
|
||||
--bug "Tiles corrupt when placing entrance at map boundary" \
|
||||
--repro-steps "1. Open map 0x00" "2. Place entrance at x=511,y=511" \
|
||||
--output regression_boundary_test.cc
|
||||
```
|
||||
|
||||
## 7. Security Considerations
|
||||
|
||||
### Authentication & Authorization
|
||||
- API key authentication for remote access
|
||||
- Role-based permissions (read-only, editor, admin)
|
||||
- Session management with expiration
|
||||
- Rate limiting per API key
|
||||
|
||||
### Input Validation
|
||||
- Command injection prevention
|
||||
- Path traversal protection
|
||||
- Memory address validation
|
||||
- File size limits for imports
|
||||
|
||||
### Audit Logging
|
||||
- All commands logged with timestamp and user
|
||||
- ROM modifications tracked
|
||||
- Rollback capability for destructive operations
|
||||
- Export audit trail for compliance
|
||||
|
||||
## 8. Performance Optimizations
|
||||
|
||||
### Caching
|
||||
- Command result caching for repeated queries
|
||||
- ROM state caching for snapshots
|
||||
- Compiled test cache
|
||||
- WebSocket connection pooling
|
||||
|
||||
### Batch Processing
|
||||
- Aggregate multiple operations into transactions
|
||||
- Parallel execution for independent commands
|
||||
- Lazy loading for large data sets
|
||||
- Progressive streaming for long operations
|
||||
|
||||
### Resource Management
|
||||
- Connection limits per client
|
||||
- Memory quotas for sessions
|
||||
- CPU throttling for intensive operations
|
||||
- Graceful degradation under load
|
||||
|
||||
## 9. Documentation Requirements
|
||||
|
||||
### API Reference
|
||||
- Complete command reference with examples
|
||||
- REST API OpenAPI specification
|
||||
- WebSocket protocol documentation
|
||||
- Error code reference
|
||||
|
||||
### Tutorials
|
||||
- "Getting Started with z3ed REPL"
|
||||
- "Automating ROM Testing"
|
||||
- "Multi-Agent Collaboration"
|
||||
- "Building Custom Commands"
|
||||
|
||||
### Integration Guides
|
||||
- Python client library
|
||||
- JavaScript/TypeScript SDK
|
||||
- CI/CD integration examples
|
||||
- VS Code extension
|
||||
|
||||
### Best Practices
|
||||
- Command naming conventions
|
||||
- Error handling patterns
|
||||
- Performance optimization tips
|
||||
- Security guidelines
|
||||
|
||||
## 10. Success Metrics
|
||||
|
||||
### Functionality
|
||||
- 100% coverage of editor operations via CLI
|
||||
- < 100ms command execution for simple operations
|
||||
- < 1s for complex batch operations
|
||||
- 99.9% API availability
|
||||
|
||||
### Developer Experience
|
||||
- Tab completion for all commands
|
||||
- Comprehensive error messages
|
||||
- Interactive help system
|
||||
- Example for every command
|
||||
|
||||
### Testing
|
||||
- 90% code coverage for new components
|
||||
- Automated regression tests for all commands
|
||||
- Performance benchmarks for critical paths
|
||||
- Integration tests for multi-agent scenarios
|
||||
|
||||
## Conclusion
|
||||
|
||||
These enhancements will transform z3ed from a basic CLI tool into a comprehensive automation platform for YAZE. The design prioritizes developer experience, AI agent capabilities, and robust testing infrastructure while maintaining backwards compatibility and performance.
|
||||
|
||||
The modular implementation plan allows for incremental delivery of value, with each phase providing immediately useful functionality. The foundation laid here will enable future innovations in ROM hacking automation and collaborative editing.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Agent Coordination & Documentation Improvement Plan
|
||||
|
||||
## Findings
|
||||
1. **Persona Inconsistency**: `coordination-board.md` uses a mix of legacy IDs (`CLAUDE_AIINF`) and new canonical IDs (`ai-infra-architect`).
|
||||
2. **Tool Underutilization**: The protocol in `AGENTS.md` relies entirely on manual Markdown edits, ignoring the built-in `z3ed agent` CLI tools (todo, handoff) described in `agent-architecture.md`.
|
||||
3. **Fragmented Docs**: There is no central entry point (`README.md`) for agents entering the directory.
|
||||
4. **Undefined Systems**: `claude-gemini-collaboration.md` references a "challenge system" and "leaderboard" that do not exist.
|
||||
|
||||
## Proposed Actions
|
||||
|
||||
### 1. Update `AGENTS.md` (The Protocol)
|
||||
* **Mandate CLI Tools**: Update the "Quick tasks" and "Substantial work" sections to recommend using `z3ed agent todo` for personal task tracking.
|
||||
* **Clarify Handoffs**: Explicitly mention using `z3ed agent handoff` for transferring context, with the Markdown board used for *public signaling*.
|
||||
* **Strict Persona Usage**: Remove "Legacy aliases" mapping and simply link to `personas.md` as the source of truth.
|
||||
|
||||
### 2. Cleanup `coordination-board.md` (The Board)
|
||||
* **Header Update**: Add a bold warning to use only IDs from `personas.md`.
|
||||
* **Retroactive Fix**: Update recent active entries to use the correct new IDs (e.g., convert `CLAUDE_AIINF` -> `ai-infra-architect` where appropriate).
|
||||
|
||||
### 3. Create `docs/internal/agents/README.md` (The Hub)
|
||||
* Create a simple index file that links to:
|
||||
* **Protocol**: `AGENTS.md`
|
||||
* **Roles**: `personas.md`
|
||||
* **Status**: `coordination-board.md`
|
||||
* **Tools**: `agent-architecture.md`
|
||||
* Provide a 3-step "Start Here" guide for new agents.
|
||||
|
||||
### 4. Deprecate `claude-gemini-collaboration.md`
|
||||
* Rename to `docs/internal/agents/archive/collaboration-concept-legacy.md` or remove the "Challenge System" sections if the file is still valuable for its architectural definitions.
|
||||
* *Recommendation*: If the "Architecture vs. Automation" split is still relevant, update the file to use `backend-infra-engineer` (Architecture) vs `GEMINI_AUTOM` (Automation) instead of "Claude vs Gemini".
|
||||
|
||||
## Execution Order
|
||||
1. Create `docs/internal/agents/README.md`.
|
||||
2. Update `AGENTS.md`.
|
||||
3. Clean up `coordination-board.md`.
|
||||
4. Refactor `claude-gemini-collaboration.md`.
|
||||
132
docs/internal/agents/archive/plans-2025-11/CLEANUP_SUMMARY.md
Normal file
132
docs/internal/agents/archive/plans-2025-11/CLEANUP_SUMMARY.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Documentation Cleanup Summary - November 2025
|
||||
|
||||
**Date:** 2025-11-24
|
||||
**Action:** Cleanup of speculative planning documents and AI-generated bloat from `/docs/internal/plans/`
|
||||
**Rationale:** Planning documents should live in GitHub issues and coordination board, not as static markdown. Only actionable, actively-tracked plans belong in the codebase.
|
||||
|
||||
## Files Deleted (Pure AI-Generated Bloat)
|
||||
|
||||
These files were entirely speculative with no corresponding implementation:
|
||||
|
||||
1. **asm-debug-prompt-engineering.md** (45KB)
|
||||
- Extensive prompt templates for 65816 debugging
|
||||
- No evidence of integration or use
|
||||
- Classified as AI-generated reference material
|
||||
|
||||
2. **ai-assisted-development-plan.md** (17KB)
|
||||
- Workflow proposal for AI-assisted development
|
||||
- All features marked "Active" with "Next Review" dates but never implemented
|
||||
- Generic architecture diagrams with no corresponding code
|
||||
|
||||
3. **app-dev-agent-tools.md** (21KB)
|
||||
- 824 lines specifying 16 agent tools (build, code analysis, debug, editor integration)
|
||||
- All tools in Phase 1-3 (theoretical)
|
||||
- No implementation in codebase or recent commits
|
||||
|
||||
4. **EDITOR_ROADMAPS_2025-11.md** (25KB)
|
||||
- Multi-agent analysis document referencing "imgui-frontend-engineer" agent analysis
|
||||
- Generic roadmap format with estimated effort hours
|
||||
- Duplicate content with dungeon_editor_ui_refactor.md
|
||||
|
||||
5. **message_system_improvement_plan.md** (2KB)
|
||||
- Duplicate of sections in message_editor_implementation_roadmap.md
|
||||
- Generic feature wishlist (JSON export, translation workspace, search & replace)
|
||||
- No distinct value
|
||||
|
||||
6. **graphics_system_improvement_plan.md** (2.8KB)
|
||||
- Feature wishlist (unified editor, palette management, sprite assembly)
|
||||
- No concrete deliverables or implementation plan
|
||||
- Superseded by architecture documentation
|
||||
|
||||
7. **ui_modernization.md** (3.3KB)
|
||||
- Describes patterns already documented in CLAUDE.md
|
||||
- Marked "Active" but content is obsolete (already implemented)
|
||||
- Redundant with existing guidelines
|
||||
|
||||
## Files Archived (Partially Implemented / Historical Reference)
|
||||
|
||||
These files have some value as reference but are not actively tracked work:
|
||||
|
||||
1. **emulator-debug-api-design.md** → `archive/plans-2025-11/`
|
||||
- Design document for emulator debugging API
|
||||
- Some features implemented (breakpoints, memory inspection)
|
||||
- Watchpoints and symbol loading still planned but deprioritized
|
||||
- Value: Technical reference for future work
|
||||
|
||||
2. **message_editor_implementation_roadmap.md** → `archive/plans-2025-11/`
|
||||
- References actual code (MessageData, MessagePreview classes)
|
||||
- Documents what's completed vs. what's missing (JSON import/export)
|
||||
- Some ongoing development but should be tracked in coordination board
|
||||
- Value: Implementation reference
|
||||
|
||||
3. **hex_editor_enhancements.md** → `archive/plans-2025-11/`
|
||||
- Phase 1 (Data Inspector) concept defined
|
||||
- Phases 2-4 unimplemented
|
||||
- Better tracked as GitHub issue than static plan
|
||||
- Value: Technical spike reference
|
||||
|
||||
4. **dungeon_editor_ui_refactor.md** → `archive/plans-2025-11/`
|
||||
- Actually referenced in git commit acab491a1f (component was removed)
|
||||
- Concrete refactoring steps with clear deliverables
|
||||
- Now completed/obsolete
|
||||
- Value: Historical record of refactoring
|
||||
|
||||
## Files Retained (Actively Tracked)
|
||||
|
||||
1. **web_port_strategy.md**
|
||||
- Strategic milestone document for WASM port
|
||||
- Multiple recent commits show active ongoing work (52b1a99, 56e05bf, 206e926, etc.)
|
||||
- Clear milestones with deliverables
|
||||
- Actively referenced in CI/build processes
|
||||
- Status: KEEP - actively developed feature
|
||||
|
||||
2. **ai-infra-improvements.md**
|
||||
- Structured phase-based plan (gRPC server, emulator RPCs, breakpoints, symbols)
|
||||
- More specific than other plans with concrete files to modify
|
||||
- Tracks infrastructure gaps with file references
|
||||
- Status: KEEP - tracks ongoing infrastructure work
|
||||
|
||||
3. **README.md**
|
||||
- Directory governance and guidelines
|
||||
- Actively enforces plan organization standards
|
||||
- Status: KEEP - essential for directory management
|
||||
|
||||
## Rationale for This Cleanup
|
||||
|
||||
### Problem
|
||||
- 14 planning files (400KB) in `/docs/internal/plans/`
|
||||
- Most marked "Active" with forward-looking dates (Nov 25 → Dec 2)
|
||||
- Little correlation with actual work in recent commits
|
||||
- Directory becoming a repository of speculative AI-generated content
|
||||
|
||||
### Solution
|
||||
- Keep only plans with active ongoing work (2 files)
|
||||
- Archive reference documents with partial implementation (4 files)
|
||||
- Delete pure speculation and AI-generated bloat (7 files)
|
||||
- Directory size reduced from 400KB to 13KB at root level
|
||||
|
||||
### Principle
|
||||
**Planning belongs in GitHub issues and coordination board, not in markdown files.**
|
||||
|
||||
Static plan documents should only exist for:
|
||||
1. Strategic initiatives (like WASM web port) with active commits
|
||||
2. Infrastructure work with concrete phases and file references
|
||||
3. Historical reference after completion
|
||||
|
||||
Speculative planning should use:
|
||||
- GitHub Discussions for RFCs
|
||||
- Issues with labels for feature requests
|
||||
- Coordination board for multi-agent work tracking
|
||||
|
||||
## Files in Archive
|
||||
|
||||
```
|
||||
docs/internal/agents/archive/plans-2025-11/
|
||||
├── CLEANUP_SUMMARY.md
|
||||
├── dungeon_editor_ui_refactor.md
|
||||
├── emulator-debug-api-design.md
|
||||
├── hex_editor_enhancements.md
|
||||
└── message_editor_implementation_roadmap.md
|
||||
```
|
||||
|
||||
These are available if needed as historical context but should not be referenced for active development.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Dungeon Editor UI Refactor Plan
|
||||
|
||||
## 1. Overview
|
||||
The Dungeon Editor currently uses a primitive "colored square" representation for objects in the object selector, despite having a full-fidelity rendering component (`DungeonObjectSelector`) available. This plan outlines the refactoring steps to integrate the game-accurate object browser into the main `ObjectEditorCard`, improving UX and eliminating code duplication.
|
||||
|
||||
## 2. Current State Analysis
|
||||
- **`ObjectEditorCard` (Active UI):** Reimplements object selection logic in `DrawObjectSelector()`. Renders objects as simple colored rectangles (`DrawObjectPreviewIcon`).
|
||||
- **`DungeonObjectSelector` (Component):** Contains `DrawObjectAssetBrowser()`, which uses `ObjectDrawer` to render actual tile graphics. This component is instantiated as a member `object_selector_` in `ObjectEditorCard` but is effectively unused.
|
||||
- **`DungeonEditorV2`:** Instantiates a separate, unused `DungeonObjectSelector` (`object_selector_`), adding to the confusion.
|
||||
|
||||
## 3. Implementation Plan
|
||||
|
||||
### Phase 1: Component Preparation
|
||||
1. **Expose UI Method:** Ensure `DungeonObjectSelector::DrawObjectAssetBrowser()` is public or accessible to `ObjectEditorCard`.
|
||||
2. **State Synchronization:** Ensure `DungeonObjectSelector` has access to the same `Rom` and `PaletteGroup` data as `ObjectEditorCard` so it can render correctly.
|
||||
|
||||
### Phase 2: Refactor ObjectEditorCard
|
||||
1. **Delegate Rendering:** Replace the body of `ObjectEditorCard::DrawObjectSelector()` with a call to `object_selector_.DrawObjectAssetBrowser()`.
|
||||
2. **Callback Wiring:**
|
||||
* In `ObjectEditorCard::Initialize` (or constructor), set up the callback for `object_selector_`.
|
||||
* When an object is selected in `object_selector_`, it should update `ObjectEditorCard::preview_object_` and `canvas_viewer_`.
|
||||
* Current logic:
|
||||
```cpp
|
||||
object_selector_.SetObjectSelectedCallback([this](const zelda3::RoomObject& obj) {
|
||||
this->preview_object_ = obj;
|
||||
this->has_preview_object_ = true;
|
||||
this->canvas_viewer_->SetPreviewObject(obj);
|
||||
this->interaction_mode_ = InteractionMode::Place;
|
||||
});
|
||||
```
|
||||
3. **Cleanup:** Remove private helper `DrawObjectPreviewIcon` and the old loop logic in `ObjectEditorCard`.
|
||||
|
||||
### Phase 3: Cleanup DungeonEditorV2
|
||||
1. **Remove Redundancy:** Remove the top-level `DungeonObjectSelector object_selector_` from `DungeonEditorV2`. The one inside `ObjectEditorCard` is sufficient.
|
||||
2. **Verify Initialization:** Ensure `DungeonEditorV2` correctly initializes `ObjectEditorCard` with the necessary dependencies.
|
||||
|
||||
## 4. Verification
|
||||
1. **Build:** Compile `yaze`.
|
||||
2. **Test:** Open Dungeon Editor -> Object Editor tab.
|
||||
3. **Expectation:** The object list should now show actual graphics (walls, chests, pots) instead of colored squares.
|
||||
4. **Interaction:** Clicking an object should correctly load it into the cursor for placement.
|
||||
|
||||
## 5. Dependencies
|
||||
- `src/app/editor/dungeon/object_editor_card.cc`
|
||||
- `src/app/editor/dungeon/object_editor_card.h`
|
||||
- `src/app/editor/dungeon/dungeon_object_selector.cc`
|
||||
- `src/app/editor/dungeon/dungeon_object_selector.h`
|
||||
- `src/app/editor/dungeon/dungeon_editor_v2.h`
|
||||
@@ -0,0 +1,516 @@
|
||||
# Emulator Debug API Design for AI Agent Integration
|
||||
|
||||
**Status:** Active
|
||||
**Owner (Agent ID):** snes-emulator-expert
|
||||
**Last Updated:** 2025-11-25
|
||||
**Next Review:** 2025-12-02
|
||||
**Coordination Board Entry:** link when claimed
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines the design for a comprehensive debugging API that enables AI agents to debug Zelda ROM hacks through the yaze emulator. The API provides execution control, memory inspection, disassembly, and analysis capabilities specifically tailored for 65816 and SPC700 debugging.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌──────────────────┐ ┌─────────────────┐ ┌──────────────────┐
|
||||
│ AI Agent │◄────►│ Tool Dispatcher │◄────►│ gRPC Service │
|
||||
│ (Claude/GPT) │ │ │ │ │
|
||||
└──────────────────┘ └─────────────────┘ └──────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌──────────────────┐
|
||||
│ Tool Handlers │ │ Emulator Service │
|
||||
│ │ │ Implementation │
|
||||
└─────────────────┘ └──────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Debug Infrastructure │
|
||||
├─────────────────┬───────────────────────┤
|
||||
│ Disassembler │ Step Controller │
|
||||
│ Symbol Provider │ Breakpoint Manager │
|
||||
│ Memory Tracker │ State Snapshots │
|
||||
└─────────────────┴───────────────────────┘
|
||||
```
|
||||
|
||||
## Phase 1 (MVP) Features
|
||||
|
||||
### 1. Execution Control API
|
||||
|
||||
```cpp
|
||||
// Tool Dispatcher Commands
|
||||
enum class EmulatorDebugTool {
|
||||
// Basic execution
|
||||
kDebugRun, // Run until breakpoint or pause
|
||||
kDebugStep, // Single instruction step
|
||||
kDebugStepOver, // Step over JSR/JSL calls
|
||||
kDebugStepOut, // Step out of current subroutine
|
||||
kDebugPause, // Pause execution
|
||||
kDebugReset, // Reset to power-on state
|
||||
|
||||
// Breakpoint management
|
||||
kDebugSetBreak, // Set execution breakpoint
|
||||
kDebugClearBreak, // Clear breakpoint by ID
|
||||
kDebugListBreaks, // List all breakpoints
|
||||
kDebugToggleBreak, // Enable/disable breakpoint
|
||||
};
|
||||
```
|
||||
|
||||
#### Example Tool Call Structure:
|
||||
```cpp
|
||||
struct DebugStepRequest {
|
||||
enum StepType {
|
||||
SINGLE, // One instruction
|
||||
OVER, // Step over calls
|
||||
OUT, // Step out of routine
|
||||
TO_ADDRESS // Run to specific address
|
||||
};
|
||||
|
||||
StepType type;
|
||||
uint32_t target_address; // For TO_ADDRESS
|
||||
uint32_t max_steps; // Timeout protection
|
||||
};
|
||||
|
||||
struct DebugStepResponse {
|
||||
bool success;
|
||||
uint32_t pc; // New program counter
|
||||
uint32_t instruction_count;
|
||||
DisassembledInstruction current_instruction;
|
||||
std::vector<std::string> call_stack;
|
||||
std::string message;
|
||||
};
|
||||
```
|
||||
|
||||
### 2. Memory Inspection API
|
||||
|
||||
```cpp
|
||||
enum class MemoryDebugTool {
|
||||
kMemoryRead, // Read memory region
|
||||
kMemoryWrite, // Write memory (for patching)
|
||||
kMemoryWatch, // Set memory watchpoint
|
||||
kMemoryCompare, // Compare two memory regions
|
||||
kMemorySearch, // Search for byte pattern
|
||||
kMemorySnapshot, // Save memory state
|
||||
kMemoryDiff, // Diff against snapshot
|
||||
};
|
||||
```
|
||||
|
||||
#### Memory Region Types:
|
||||
```cpp
|
||||
enum class MemoryRegion {
|
||||
WRAM, // Work RAM ($7E0000-$7FFFFF)
|
||||
SRAM, // Save RAM ($700000-$77FFFF)
|
||||
ROM, // ROM banks ($008000-$FFFFFF)
|
||||
VRAM, // Video RAM (PPU)
|
||||
OAM, // Sprite data
|
||||
CGRAM, // Palette data
|
||||
APU_RAM, // Audio RAM ($0000-$FFFF in SPC700 space)
|
||||
};
|
||||
|
||||
struct MemoryReadRequest {
|
||||
MemoryRegion region;
|
||||
uint32_t address; // 24-bit SNES address
|
||||
uint32_t size; // Bytes to read
|
||||
bool as_hex; // Format as hex dump
|
||||
bool with_symbols; // Include symbol annotations
|
||||
};
|
||||
|
||||
struct MemoryReadResponse {
|
||||
std::vector<uint8_t> data;
|
||||
std::string hex_dump;
|
||||
std::map<uint32_t, std::string> symbols; // Address -> symbol name
|
||||
std::string interpretation; // AI-friendly interpretation
|
||||
};
|
||||
```
|
||||
|
||||
### 3. Disassembly API
|
||||
|
||||
```cpp
|
||||
enum class DisassemblyTool {
|
||||
kDisassemble, // Disassemble at address
|
||||
kDisassembleRange, // Disassemble N instructions
|
||||
kDisassembleContext, // Show surrounding code
|
||||
kFindInstruction, // Search for instruction pattern
|
||||
kGetCallStack, // Get current call stack
|
||||
kTraceExecution, // Get execution history
|
||||
};
|
||||
```
|
||||
|
||||
#### Disassembly Request/Response:
|
||||
```cpp
|
||||
struct DisassemblyRequest {
|
||||
uint32_t address;
|
||||
uint32_t instruction_count;
|
||||
uint32_t context_before; // Instructions before target
|
||||
uint32_t context_after; // Instructions after target
|
||||
bool include_symbols;
|
||||
bool include_execution_counts;
|
||||
bool track_branches; // Show branch targets
|
||||
};
|
||||
|
||||
struct DisassemblyResponse {
|
||||
struct Line {
|
||||
uint32_t address;
|
||||
std::string hex_bytes; // "20 34 80"
|
||||
std::string mnemonic; // "JSR"
|
||||
std::string operands; // "$8034"
|
||||
std::string symbol; // "MainGameLoop"
|
||||
std::string comment; // "; Initialize game state"
|
||||
bool is_breakpoint;
|
||||
bool is_current_pc;
|
||||
uint32_t execution_count;
|
||||
uint32_t branch_target; // For jumps/branches
|
||||
};
|
||||
|
||||
std::vector<Line> lines;
|
||||
std::string formatted_text; // Human-readable disassembly
|
||||
std::vector<std::string> referenced_symbols;
|
||||
};
|
||||
```
|
||||
|
||||
### 4. Analysis API
|
||||
|
||||
```cpp
|
||||
enum class AnalysisTool {
|
||||
kAnalyzeRoutine, // Analyze subroutine behavior
|
||||
kFindReferences, // Find references to address
|
||||
kDetectPattern, // Detect common bug patterns
|
||||
kCompareRom, // Compare with original ROM
|
||||
kProfileExecution, // Performance profiling
|
||||
kTrackDataFlow, // Track value propagation
|
||||
};
|
||||
```
|
||||
|
||||
## Tool Dispatcher Integration
|
||||
|
||||
### New Tool Definitions
|
||||
|
||||
```cpp
|
||||
// In tool_dispatcher.h
|
||||
enum class ToolCallType {
|
||||
// ... existing tools ...
|
||||
|
||||
// Debugger - Execution Control
|
||||
kDebugRun,
|
||||
kDebugStep,
|
||||
kDebugStepOver,
|
||||
kDebugStepOut,
|
||||
kDebugRunToAddress,
|
||||
|
||||
// Debugger - Breakpoints
|
||||
kDebugSetBreakpoint,
|
||||
kDebugSetWatchpoint,
|
||||
kDebugClearBreakpoint,
|
||||
kDebugListBreakpoints,
|
||||
kDebugEnableBreakpoint,
|
||||
|
||||
// Debugger - Memory
|
||||
kDebugReadMemory,
|
||||
kDebugWriteMemory,
|
||||
kDebugSearchMemory,
|
||||
kDebugCompareMemory,
|
||||
kDebugSnapshotMemory,
|
||||
|
||||
// Debugger - Disassembly
|
||||
kDebugDisassemble,
|
||||
kDebugGetCallStack,
|
||||
kDebugGetExecutionTrace,
|
||||
kDebugFindInstruction,
|
||||
|
||||
// Debugger - Analysis
|
||||
kDebugAnalyzeRoutine,
|
||||
kDebugFindReferences,
|
||||
kDebugDetectBugs,
|
||||
kDebugProfileCode,
|
||||
};
|
||||
```
|
||||
|
||||
### Tool Handler Implementation
|
||||
|
||||
```cpp
|
||||
class DebuggerToolHandler {
|
||||
public:
|
||||
explicit DebuggerToolHandler(EmulatorServiceImpl* service);
|
||||
|
||||
absl::StatusOr<ToolResponse> HandleDebugStep(const nlohmann::json& params);
|
||||
absl::StatusOr<ToolResponse> HandleDebugSetBreakpoint(const nlohmann::json& params);
|
||||
absl::StatusOr<ToolResponse> HandleDebugDisassemble(const nlohmann::json& params);
|
||||
absl::StatusOr<ToolResponse> HandleDebugReadMemory(const nlohmann::json& params);
|
||||
absl::StatusOr<ToolResponse> HandleDebugAnalyze(const nlohmann::json& params);
|
||||
|
||||
private:
|
||||
EmulatorServiceImpl* emulator_service_;
|
||||
Disassembler65816 disassembler_;
|
||||
SymbolProvider symbol_provider_;
|
||||
MemoryAnalyzer analyzer_;
|
||||
};
|
||||
```
|
||||
|
||||
## Example AI Agent Interactions
|
||||
|
||||
### Example 1: Debugging a Crash
|
||||
|
||||
**Agent Query:**
|
||||
```json
|
||||
{
|
||||
"tool": "debug_analyze",
|
||||
"params": {
|
||||
"problem": "Game crashes when entering dungeon room 15",
|
||||
"context": {
|
||||
"room_id": 15,
|
||||
"entrance_id": 3
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**System Response:**
|
||||
```json
|
||||
{
|
||||
"analysis": {
|
||||
"crash_location": {
|
||||
"address": "0x008A45",
|
||||
"instruction": "LDA [$00],Y",
|
||||
"symbol": "LoadRoomHeader+0x15"
|
||||
},
|
||||
"cause": "Null pointer dereference",
|
||||
"explanation": "Direct page location $00 contains $0000, causing invalid long pointer access",
|
||||
"registers_at_crash": {
|
||||
"Y": "0x0010",
|
||||
"DP": "0x1E00",
|
||||
"$1E00": "0x0000"
|
||||
},
|
||||
"suggested_fix": "Check room header pointer before dereferencing",
|
||||
"call_stack": [
|
||||
"Module_Dungeon",
|
||||
"Dungeon_LoadRoom",
|
||||
"LoadRoomHeader"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Step-by-Step Debugging
|
||||
|
||||
**Agent Query:**
|
||||
```json
|
||||
{
|
||||
"tool": "debug_step",
|
||||
"params": {
|
||||
"type": "over",
|
||||
"with_context": true,
|
||||
"show_memory": ["$7E0010", "$7E0012"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"pc_before": "0x008034",
|
||||
"pc_after": "0x008037",
|
||||
"instruction_executed": {
|
||||
"address": "0x008034",
|
||||
"mnemonic": "JSR",
|
||||
"operand": "$8520",
|
||||
"symbol": "UpdateSprites",
|
||||
"cycles": 6
|
||||
},
|
||||
"context": {
|
||||
"before": [
|
||||
{"address": "0x008031", "instruction": "LDA $10", "value": "0x07"}
|
||||
],
|
||||
"after": [
|
||||
{"address": "0x008037", "instruction": "BEQ $8045"}
|
||||
]
|
||||
},
|
||||
"memory_values": {
|
||||
"$7E0010": "0x07",
|
||||
"$7E0012": "0x00"
|
||||
},
|
||||
"call_depth": 2
|
||||
}
|
||||
```
|
||||
|
||||
### Example 3: Finding Bug Patterns
|
||||
|
||||
**Agent Query:**
|
||||
```json
|
||||
{
|
||||
"tool": "debug_detect_bugs",
|
||||
"params": {
|
||||
"patterns": ["stack_overflow", "invalid_bank", "dma_collision"],
|
||||
"range": {
|
||||
"start": "0x008000",
|
||||
"end": "0x00FFFF"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"bugs_found": [
|
||||
{
|
||||
"type": "potential_stack_overflow",
|
||||
"location": "0x009A23",
|
||||
"description": "Recursive JSR without stack check",
|
||||
"severity": "high",
|
||||
"suggestion": "Add stack pointer validation before recursive call"
|
||||
},
|
||||
{
|
||||
"type": "invalid_bank_switch",
|
||||
"location": "0x00B456",
|
||||
"description": "PHB without corresponding PLB",
|
||||
"severity": "medium",
|
||||
"suggestion": "Ensure data bank is restored after operation"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Example 4: Memory Watchpoint
|
||||
|
||||
**Agent Query:**
|
||||
```json
|
||||
{
|
||||
"tool": "debug_set_watchpoint",
|
||||
"params": {
|
||||
"address": "0x7E0020",
|
||||
"size": 2,
|
||||
"type": "write",
|
||||
"condition": "value > 0x00FF",
|
||||
"description": "Monitor game state overflow"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"watchpoint_id": 5,
|
||||
"status": "active",
|
||||
"message": "Watchpoint set on $7E0020-$7E0021 for writes > 0x00FF"
|
||||
}
|
||||
```
|
||||
|
||||
## Phase 2 (Full) Features
|
||||
|
||||
### Advanced Analysis
|
||||
- **Control Flow Graphs**: Generate CFG for routines
|
||||
- **Data Flow Analysis**: Track value propagation through code
|
||||
- **Symbolic Execution**: Analyze possible execution paths
|
||||
- **Pattern Matching**: Detect specific code patterns (e.g., DMA setup, HDMA tables)
|
||||
- **Performance Profiling**: Cycle-accurate performance analysis
|
||||
|
||||
### Enhanced Debugging
|
||||
- **Conditional Breakpoints**: Complex expressions (e.g., "A > 0x10 && X == 0")
|
||||
- **Trace Recording**: Record full execution traces to file
|
||||
- **Reverse Debugging**: Step backwards through recorded execution
|
||||
- **Memory Diffing**: Visual diff between memory states
|
||||
- **SPC700 Debugging**: Full audio processor debugging support
|
||||
|
||||
### AI-Specific Features
|
||||
- **Semantic Analysis**: Understanding game logic from assembly
|
||||
- **Bug Pattern Database**: ML-trained bug detection
|
||||
- **Automated Fix Suggestions**: Propose assembly patches for bugs
|
||||
- **Test Case Generation**: Generate test scenarios for ROM hacks
|
||||
- **Documentation Generation**: Auto-document assembly routines
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
### Phase 1A (Immediate - Week 1-2)
|
||||
1. Basic step control (single, over, out)
|
||||
2. Simple breakpoints (address-based)
|
||||
3. Memory read/write operations
|
||||
4. Basic disassembly at address
|
||||
|
||||
### Phase 1B (Short-term - Week 3-4)
|
||||
1. Call stack tracking
|
||||
2. Symbol resolution
|
||||
3. Memory watchpoints
|
||||
4. Execution trace (last N instructions)
|
||||
|
||||
### Phase 1C (Medium-term - Week 5-6)
|
||||
1. Pattern-based bug detection
|
||||
2. Memory snapshots and comparison
|
||||
3. Advanced breakpoint conditions
|
||||
4. Performance metrics
|
||||
|
||||
### Phase 2 (Long-term - Month 2+)
|
||||
1. Full analysis suite
|
||||
2. SPC700 debugging
|
||||
3. Reverse debugging
|
||||
4. AI-specific enhancements
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Technical Metrics
|
||||
- Response time < 100ms for step operations
|
||||
- Support for 100+ simultaneous breakpoints without performance impact
|
||||
- Accurate disassembly for 100% of valid 65816 opcodes
|
||||
- Symbol resolution for all loaded ASM files
|
||||
|
||||
### User Experience Metrics
|
||||
- AI agents can identify crash causes in < 5 interactions
|
||||
- Step debugging provides sufficient context without overwhelming
|
||||
- Memory inspection clearly shows relevant game state
|
||||
- Bug detection has < 10% false positive rate
|
||||
|
||||
## Integration Points
|
||||
|
||||
### With Existing yaze Components
|
||||
- **Rom Class**: Read-only access to ROM data
|
||||
- **Emulator Core**: Direct CPU/PPU/APU state access
|
||||
- **Symbol Files**: Integration with usdasm output
|
||||
- **Canvas System**: Visual debugging overlays (Phase 2)
|
||||
|
||||
### With AI Infrastructure
|
||||
- **Tool Dispatcher**: Seamless tool call routing
|
||||
- **Prompt Builder**: Context-aware debugging prompts
|
||||
- **Agent Memory**: Persistent debugging session state
|
||||
- **Response Formatter**: Human-readable debug output
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Read-Only by Default**: Prevent accidental ROM corruption
|
||||
2. **Sandboxed Execution**: Limit memory access to emulated space
|
||||
3. **Rate Limiting**: Prevent runaway debugging loops
|
||||
4. **Audit Logging**: Track all debugging operations
|
||||
5. **Session Isolation**: Separate debug sessions per agent
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
- Disassembler accuracy for all opcodes
|
||||
- Step controller call stack tracking
|
||||
- Breakpoint manager hit detection
|
||||
- Symbol provider resolution
|
||||
|
||||
### Integration Tests
|
||||
- Full debugging session workflows
|
||||
- gRPC service communication
|
||||
- Tool dispatcher routing
|
||||
- Memory state consistency
|
||||
|
||||
### End-to-End Tests
|
||||
- AI agent debugging scenarios
|
||||
- Bug detection accuracy
|
||||
- Performance under load
|
||||
- Error recovery paths
|
||||
|
||||
## Documentation Requirements
|
||||
|
||||
1. **API Reference**: Complete gRPC service documentation
|
||||
2. **Tool Guide**: How to use each debugging tool
|
||||
3. **Assembly Primer**: 65816 basics for AI agents
|
||||
4. **Common Patterns**: Debugging patterns for Zelda3
|
||||
5. **Troubleshooting**: Common issues and solutions
|
||||
|
||||
## Conclusion
|
||||
|
||||
This debugging API design provides a comprehensive foundation for AI agents to effectively debug SNES ROM hacks. The phased approach ensures quick delivery of core features while building toward advanced analysis capabilities. The integration with existing yaze infrastructure and focus on 65816-specific debugging makes this a powerful tool for ROM hacking assistance.
|
||||
|
||||
The API balances technical depth with usability, providing both low-level control for precise debugging and high-level analysis for pattern recognition. This enables AI agents to assist with everything from simple crash debugging to complex performance optimization.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Plan: Hex Editor Enhancements (Inspired by ImHex)
|
||||
|
||||
**Status:** Active
|
||||
**Owner (Agent ID):** imgui-frontend-engineer
|
||||
**Last Updated:** 2025-11-25
|
||||
**Next Review:** 2025-12-02
|
||||
**Coordination Board Entry:** link when claimed
|
||||
|
||||
This document outlines the roadmap for enhancing the `yaze` Memory/Hex Editor to provide robust analysis tools similar to ImHex.
|
||||
|
||||
## Phase 1: Data Inspector (High Priority)
|
||||
|
||||
**Goal:** Provide immediate context for the selected byte(s) in the Hex Editor without mental math.
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **Create `DataInspector` Component:**
|
||||
* A standalone ImGui widget (`src/app/editor/code/data_inspector.h/cc`).
|
||||
* Accepts a `const uint8_t* data_ptr` and `size_t max_len`.
|
||||
2. **Standard Types:**
|
||||
* Decode and display Little Endian values:
|
||||
* `int8_t` / `uint8_t`
|
||||
* `int16_t` / `uint16_t`
|
||||
* `int24_t` / `uint24_t` (Common SNES pointers)
|
||||
* `int32_t` / `uint32_t`
|
||||
* Display Binary representation (`00001111`).
|
||||
3. **SNES-Specific Types (The "Yaze" Value):**
|
||||
* **SNES LoROM Address:** Convert the physical offset to `$BB:AAAA` format.
|
||||
* **RGB555 Color:** Interpret 2 bytes as `0bbbbbgggggrrrrr`. Show a colored rectangle preview.
|
||||
* **Tile Attribute:** Interpret byte as `vhopppcc` (Vertical/Horizontal flip, Priority, Palette, Tile High bit).
|
||||
4. **Integration:**
|
||||
* Modify `MemoryEditorWithDiffChecker` to instantiate and render `DataInspector` in a sidebar or child window next to the main hex grid.
|
||||
* Hook into the hex editor's "Selection Changed" event (or poll selection state) to update the Inspector.
|
||||
|
||||
## Phase 2: Entropy Navigation (Navigation)
|
||||
|
||||
**Goal:** Visualize the ROM's structure to quickly find free space, graphics, or code.
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **Entropy Calculator:**
|
||||
* Create a utility to calculate Shannon entropy for blocks of data (e.g., 256-byte chunks).
|
||||
* Run this calculation in a background thread when a ROM is loaded to generate an `EntropyMap`.
|
||||
2. **Minimap Widget:**
|
||||
* Render a thin vertical bar next to the hex scrollbar.
|
||||
* Map the file offset to vertical pixels.
|
||||
* **Color Coding:**
|
||||
* **Black:** Zeroes (`0x00` fill).
|
||||
* **Dark Grey:** `0xFF` fill (common flash erase value).
|
||||
* **Blue:** Low entropy (Text, Tables).
|
||||
* **Red:** High entropy (Compressed Graphics, Code).
|
||||
3. **Interaction:**
|
||||
* Clicking the minimap jumps the Hex Editor to that offset.
|
||||
|
||||
## Phase 3: Structure Templates (Advanced Analysis)
|
||||
|
||||
**Goal:** Define and visualize complex data structures on top of the raw hex.
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **Template Definition System:**
|
||||
* Define a C++-based schema builder (e.g., `StructBuilder("Header").AddString("Title", 21).AddByte("MapMode")...`).
|
||||
2. **Visualizer:**
|
||||
* Render these structures as a tree view (ImGui TreeNodes).
|
||||
* When a tree node is hovered, highlight the corresponding bytes in the Hex Editor grid.
|
||||
3. **Standard Templates:**
|
||||
* Implement templates for known ALTTP structures: `SNES Header`, `Dungeon Header`, `Sprite Properties`.
|
||||
|
||||
## Phase 4: Disassembly Integration
|
||||
|
||||
**Goal:** Seamless transition between data viewing and code analysis.
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **Context Menu:** Add "Disassemble Here" to the Hex Editor right-click menu.
|
||||
2. **Disassembly View:**
|
||||
* Invoke the `disassembler` (already present in `app/emu/debug`) on the selected range.
|
||||
* Display the output in a popup or switch to the Assembly Editor.
|
||||
|
||||
---
|
||||
|
||||
## Initial Work Item: Data Inspector
|
||||
|
||||
We will begin with Phase 1. This requires creating the `DataInspector` class and hooking it into `MemoryEditorWithDiffChecker`.
|
||||
@@ -0,0 +1,774 @@
|
||||
# Message Editor Implementation Roadmap
|
||||
|
||||
**Status**: Active Development
|
||||
**Owner (Agent ID)**: imgui-frontend-engineer
|
||||
**Last Updated**: 2025-11-25
|
||||
**Next Review**: 2025-12-02
|
||||
**Coordination Board Entry**: link when claimed
|
||||
**Related Docs**:
|
||||
- `docs/internal/architecture/message_system.md` (Gemini's architecture vision)
|
||||
- `docs/internal/plans/message_system_improvement_plan.md` (Gemini's feature proposals)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This roadmap bridges Gemini's architectural vision with practical implementation steps for completing the Message Editor. The current implementation has the **core foundation** in place (message parsing, dictionary system, preview rendering) but lacks several key features proposed in Gemini's plan, particularly around **JSON import/export**, **translation workflows**, and **theme integration**.
|
||||
|
||||
---
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### What's Working (Completed Features)
|
||||
|
||||
#### Core Data Layer ✅
|
||||
- **MessageData**: Full implementation with raw/parsed representations
|
||||
- **DictionaryEntry**: Compression system with dictionary optimization
|
||||
- **TextElement**: Command and special character parsing
|
||||
- **Character Encoding**: Complete CharEncoder table (0x00-0x66)
|
||||
- **ROM Reading**: `ReadAllTextData()` successfully loads all 396 messages
|
||||
- **ROM Writing**: `Save()` handles two-bank text data with overflow detection
|
||||
|
||||
#### Message Preview System ✅
|
||||
- **MessagePreview**: Live rendering of messages as they appear in-game
|
||||
- **Font Graphics**: 2BPP font tiles loaded and displayed at 0x70000
|
||||
- **Character Widths**: Proportional font support via width table at 0x74ADF
|
||||
- **Preview Bitmap**: Real-time message rendering with proper palette support
|
||||
|
||||
#### Editor UI ✅
|
||||
- **Card System**: Four dockable cards (Message List, Editor, Font Atlas, Dictionary)
|
||||
- **Message List**: Table view with ID, contents, and address columns
|
||||
- **Text Editor**: Multiline input with live preview updates
|
||||
- **Command Insertion**: Buttons to insert text commands and special characters
|
||||
- **Dictionary Display**: Read-only view of all 97 dictionary entries
|
||||
- **Expanded Messages**: Basic support for loading external message bins
|
||||
|
||||
#### Testing Coverage ✅
|
||||
- **Unit Tests**: 20+ tests covering parsing, encoding, dictionary optimization
|
||||
- **Integration Tests**: ROM-dependent tests verify actual game data
|
||||
- **Command Parsing**: Regression tests for argument handling bugs
|
||||
|
||||
#### CLI Integration ✅
|
||||
- **Message List**: `z3ed message list --format json --range 0-100`
|
||||
- **Message Read**: `z3ed message read --id 5 --format json`
|
||||
- **Message Search**: `z3ed message search --query "Link"`
|
||||
- **Message Stats**: `z3ed message stats --format json`
|
||||
|
||||
### What's Missing (Gaps vs. Gemini's Vision)
|
||||
|
||||
#### 1. JSON Import/Export ❌ (HIGH PRIORITY)
|
||||
**Status**: Not implemented
|
||||
**Gemini's Vision**:
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": 0,
|
||||
"address": 917504,
|
||||
"text": "[W:00][SPD:00]Welcome to [D:05]...",
|
||||
"context": "Uncle dying in sewers"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
**Current Gap**:
|
||||
- No `SerializeMessages()` or `DeserializeMessages()` in `MessageData`
|
||||
- No UI for export/import operations
|
||||
- No context field for translator notes
|
||||
- CLI has JSON output but not JSON input
|
||||
|
||||
**Impact**: Cannot version control text, cannot use external editors, cannot collaborate with translators
|
||||
|
||||
---
|
||||
|
||||
#### 2. Translation Workspace ❌ (MEDIUM PRIORITY)
|
||||
**Status**: Not implemented
|
||||
**Gemini's Vision**: Side-by-side view with reference ROM/JSON and editable translation
|
||||
|
||||
**Current Gap**:
|
||||
- No reference text display
|
||||
- No side-by-side layout
|
||||
- No translation progress tracking
|
||||
- No language-specific dictionary optimization
|
||||
|
||||
**Impact**: Manual translation workflows are tedious and error-prone
|
||||
|
||||
---
|
||||
|
||||
#### 3. Search & Replace ⚠️ (PARTIAL)
|
||||
**Status**: Stub implementation exists
|
||||
**Gemini's Vision**: Regex support, batch replace across all messages
|
||||
|
||||
**Current Implementation**:
|
||||
- `Find()` method exists in `MessageEditor` (lines 574-600)
|
||||
- Basic UI skeleton present (search input, case sensitivity toggle)
|
||||
- **Missing**: Replace functionality, regex support, "Find All", multi-message operations
|
||||
|
||||
**Impact**: Global text edits require manual per-message changes
|
||||
|
||||
---
|
||||
|
||||
#### 4. Theme Integration ❌ (LOW PRIORITY - UI POLISH)
|
||||
**Status**: Not implemented
|
||||
**Current Issues**:
|
||||
- No hardcoded `ImVec4` colors found (GOOD!)
|
||||
- Not using `AgentUITheme` system for consistency
|
||||
- Missing semantic color names for message editor components
|
||||
|
||||
**Impact**: Message Editor UI may not match rest of application theme
|
||||
|
||||
---
|
||||
|
||||
#### 5. Expanded ROM Support ⚠️ (PARTIAL)
|
||||
**Status**: Basic implementation exists
|
||||
**Gemini's Vision**: Repointing text blocks to expanded ROM space (Banks 10+), automatic bank switching
|
||||
|
||||
**Current Implementation**:
|
||||
- Can load expanded message bins (lines 322-334)
|
||||
- Can save expanded messages (lines 497-508)
|
||||
- **Missing**: Repointing logic, bank management, automatic overflow handling
|
||||
|
||||
**Impact**: Cannot support large translation projects that exceed vanilla space
|
||||
|
||||
---
|
||||
|
||||
#### 6. Scripting Integration ❌ (FUTURE)
|
||||
**Status**: Not planned
|
||||
**Gemini's Vision**: Lua/Python API for procedural text generation
|
||||
|
||||
**Current Gap**: No scripting hooks in message system
|
||||
|
||||
**Impact**: Low - nice-to-have for advanced users
|
||||
|
||||
---
|
||||
|
||||
## Architectural Decisions Required
|
||||
|
||||
### Decision 1: JSON Schema Design
|
||||
**Question**: What fields should the JSON export include?
|
||||
|
||||
**Proposal**:
|
||||
```json
|
||||
{
|
||||
"version": "1.0",
|
||||
"rom_name": "zelda3.sfc",
|
||||
"messages": [
|
||||
{
|
||||
"id": 0,
|
||||
"address": 917504,
|
||||
"address_hex": "0xE0000",
|
||||
"text": "[W:00][SPD:00]Welcome...",
|
||||
"context": "Optional translator note",
|
||||
"dictionary_optimized": true,
|
||||
"expanded": false
|
||||
}
|
||||
],
|
||||
"dictionary": [
|
||||
{
|
||||
"id": 0,
|
||||
"token": "[D:00]",
|
||||
"contents": "the"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Trade-offs**:
|
||||
- Verbose but human-readable
|
||||
- Includes metadata for validation
|
||||
- Context field for translator workflow
|
||||
|
||||
**Status**: ✅ RECOMMENDED
|
||||
|
||||
---
|
||||
|
||||
### Decision 2: Translation Workspace Layout
|
||||
**Question**: How should reference vs. translation be displayed?
|
||||
|
||||
**Option A**: Side-by-side split pane
|
||||
```
|
||||
┌────────────────┬────────────────┐
|
||||
│ Reference │ Translation │
|
||||
│ (English) │ (Spanish) │
|
||||
│ [Read-only] │ [Editable] │
|
||||
│ │ │
|
||||
│ Message 0: │ Message 0: │
|
||||
│ "Welcome to │ "Bienvenido a │
|
||||
│ Hyrule" │ Hyrule" │
|
||||
└────────────────┴────────────────┘
|
||||
```
|
||||
|
||||
**Option B**: Top-bottom with context panel
|
||||
```
|
||||
┌────────────────────────────────┐
|
||||
│ Reference: "Welcome to Hyrule" │
|
||||
│ Context: Uncle's dying words │
|
||||
├────────────────────────────────┤
|
||||
│ Translation: │
|
||||
│ [Editable text box] │
|
||||
└────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Recommendation**: Option A for large screens, Option B for narrow windows
|
||||
|
||||
**Status**: ⚠️ NEEDS USER FEEDBACK
|
||||
|
||||
---
|
||||
|
||||
### Decision 3: Dictionary Auto-Optimization
|
||||
**Question**: Should we auto-generate optimal dictionary for new languages?
|
||||
|
||||
**Challenges**:
|
||||
- Dictionary optimization is NP-hard (longest common substring problem)
|
||||
- Need to preserve ROM space constraints (97 entries max)
|
||||
- Different languages have different common phrases
|
||||
|
||||
**Proposal**:
|
||||
1. Provide "Analyze Translation" button that suggests optimal dictionary
|
||||
2. Let user accept/reject suggestions
|
||||
3. Preserve manual dictionary entries
|
||||
|
||||
**Status**: ⚠️ NEEDS RESEARCH
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority Matrix
|
||||
|
||||
### Phase 1: Foundation (Sprint 1-2 weeks)
|
||||
**Goal**: JSON import/export with UI integration
|
||||
|
||||
#### Task 1.1: Implement JSON Serialization
|
||||
**Location**: `src/app/editor/message/message_data.h`, `message_data.cc`
|
||||
**Priority**: P0 (Blocker for translation workflow)
|
||||
**Estimated Effort**: 3 days
|
||||
|
||||
**Implementation**:
|
||||
```cpp
|
||||
// In MessageData
|
||||
nlohmann::json SerializeToJson() const;
|
||||
static absl::StatusOr<MessageData> DeserializeFromJson(const nlohmann::json& j);
|
||||
|
||||
// Free functions
|
||||
absl::Status ExportMessagesToJson(
|
||||
const std::vector<MessageData>& messages,
|
||||
const std::vector<DictionaryEntry>& dictionary,
|
||||
const std::string& output_path);
|
||||
|
||||
absl::StatusOr<std::vector<MessageData>> ImportMessagesFromJson(
|
||||
const std::string& input_path);
|
||||
```
|
||||
|
||||
**Dependencies**: nlohmann/json (already in project via CPM)
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Export all 396 messages to valid JSON
|
||||
- [ ] Import JSON and verify byte-for-byte ROM equivalence
|
||||
- [ ] Handle malformed JSON with clear error messages
|
||||
- [ ] Preserve dictionary optimization
|
||||
- [ ] Include context field in schema
|
||||
|
||||
---
|
||||
|
||||
#### Task 1.2: Add Export/Import UI
|
||||
**Location**: `src/app/editor/message/message_editor.cc`
|
||||
**Priority**: P0
|
||||
**Estimated Effort**: 2 days
|
||||
|
||||
**UI Additions**:
|
||||
```cpp
|
||||
void MessageEditor::DrawExportImportPanel() {
|
||||
if (ImGui::Button("Export to JSON")) {
|
||||
std::string path = util::FileDialogWrapper::ShowSaveFileDialog("json");
|
||||
PRINT_IF_ERROR(ExportMessagesToJson(list_of_texts_,
|
||||
message_preview_.all_dictionaries_,
|
||||
path));
|
||||
}
|
||||
|
||||
if (ImGui::Button("Import from JSON")) {
|
||||
std::string path = util::FileDialogWrapper::ShowOpenFileDialog();
|
||||
auto result = ImportMessagesFromJson(path);
|
||||
if (result.ok()) {
|
||||
list_of_texts_ = result.value();
|
||||
RefreshMessageList();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] File dialogs open with correct filters
|
||||
- [ ] Progress indicator for large exports
|
||||
- [ ] Confirmation dialog on import (warns about overwriting)
|
||||
- [ ] Error popup on import failure with details
|
||||
|
||||
---
|
||||
|
||||
#### Task 1.3: CLI JSON Import Support
|
||||
**Location**: `src/cli/handlers/game/message.cc`
|
||||
**Priority**: P1
|
||||
**Estimated Effort**: 1 day
|
||||
|
||||
**Implementation**:
|
||||
```bash
|
||||
z3ed message import --json messages.json --rom zelda3.sfc --output zelda3_translated.sfc
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Import JSON and write to ROM
|
||||
- [ ] Validate JSON schema before import
|
||||
- [ ] Verify ROM size constraints
|
||||
- [ ] Dry-run mode (validate without writing)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Translation Workflow (Sprint 2-3 weeks)
|
||||
**Goal**: Side-by-side translation UI
|
||||
|
||||
#### Task 2.1: Add Translation Mode Card
|
||||
**Location**: `src/app/editor/message/message_editor.h`, `message_editor.cc`
|
||||
**Priority**: P1
|
||||
**Estimated Effort**: 5 days
|
||||
|
||||
**New Components**:
|
||||
```cpp
|
||||
class TranslationWorkspace {
|
||||
public:
|
||||
void Initialize(Rom* reference_rom, Rom* translation_rom);
|
||||
void DrawUI();
|
||||
void LoadReferenceFromJson(const std::string& path);
|
||||
|
||||
private:
|
||||
void DrawSideBySideView();
|
||||
void DrawProgressTracker();
|
||||
void UpdateTranslationProgress();
|
||||
|
||||
std::vector<MessageData> reference_messages_;
|
||||
std::vector<MessageData> translation_messages_;
|
||||
std::map<int, bool> translation_complete_flags_;
|
||||
Rom* reference_rom_ = nullptr;
|
||||
Rom* translation_rom_ = nullptr;
|
||||
};
|
||||
```
|
||||
|
||||
**UI Mockup**:
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Translation Progress: 42/396 (10.6%) │
|
||||
├──────────────────────┬──────────────────────────┤
|
||||
│ Reference (EN) │ Translation (ES) │
|
||||
├──────────────────────┼──────────────────────────┤
|
||||
│ Message 0: │ Message 0: │
|
||||
│ "Welcome to Hyrule" │ [Editable input box] │
|
||||
│ │ │
|
||||
│ Dictionary: [D:05] │ Dictionary: [D:05] │
|
||||
├──────────────────────┴──────────────────────────┤
|
||||
│ [Previous] [Next] [Mark Complete] [Skip] │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Load reference ROM or JSON
|
||||
- [ ] Display messages side-by-side
|
||||
- [ ] Track translation progress (per-message completion)
|
||||
- [ ] Keyboard shortcuts for navigation (Ctrl+N, Ctrl+P)
|
||||
- [ ] Auto-save translated ROM on completion
|
||||
|
||||
---
|
||||
|
||||
#### Task 2.2: Context/Notes System
|
||||
**Location**: `src/app/editor/message/message_data.h`
|
||||
**Priority**: P2
|
||||
**Estimated Effort**: 2 days
|
||||
|
||||
**Schema Addition**:
|
||||
```cpp
|
||||
struct MessageData {
|
||||
// ... existing fields ...
|
||||
std::string context; // Translator notes, scene context
|
||||
std::string screenshot_path; // Optional screenshot reference
|
||||
|
||||
nlohmann::json SerializeToJson() const {
|
||||
return {
|
||||
{"id", ID},
|
||||
{"address", Address},
|
||||
{"text", RawString},
|
||||
{"context", context},
|
||||
{"screenshot", screenshot_path}
|
||||
};
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**UI Addition**:
|
||||
```cpp
|
||||
void MessageEditor::DrawContextPanel() {
|
||||
ImGui::InputTextMultiline("Context Notes", ¤t_message_.context);
|
||||
if (!current_message_.screenshot_path.empty()) {
|
||||
ImGui::Image(LoadScreenshot(current_message_.screenshot_path));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Context field persists in JSON export/import
|
||||
- [ ] Context displayed in translation workspace
|
||||
- [ ] Optional screenshot attachment (stored as relative path)
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Search & Replace (Sprint 3-1 week)
|
||||
**Goal**: Complete Find/Replace implementation
|
||||
|
||||
#### Task 3.1: Implement Replace Functionality
|
||||
**Location**: `src/app/editor/message/message_editor.cc`
|
||||
**Priority**: P2
|
||||
**Estimated Effort**: 2 days
|
||||
|
||||
**Implementation**:
|
||||
```cpp
|
||||
absl::Status MessageEditor::Replace(const std::string& find_text,
|
||||
const std::string& replace_text,
|
||||
bool case_sensitive,
|
||||
bool whole_word,
|
||||
bool all_messages) {
|
||||
int replaced_count = 0;
|
||||
|
||||
if (all_messages) {
|
||||
for (auto& message : list_of_texts_) {
|
||||
replaced_count += ReplaceInMessage(message, find_text, replace_text,
|
||||
case_sensitive, whole_word);
|
||||
}
|
||||
} else {
|
||||
replaced_count += ReplaceInMessage(current_message_, find_text,
|
||||
replace_text, case_sensitive, whole_word);
|
||||
}
|
||||
|
||||
return absl::OkStatus();
|
||||
}
|
||||
```
|
||||
|
||||
**UI Updates**:
|
||||
```cpp
|
||||
void MessageEditor::DrawFindReplacePanel() {
|
||||
static char find_text[256] = "";
|
||||
static char replace_text[256] = "";
|
||||
|
||||
ImGui::InputText("Find", find_text, IM_ARRAYSIZE(find_text));
|
||||
ImGui::InputText("Replace", replace_text, IM_ARRAYSIZE(replace_text));
|
||||
|
||||
ImGui::Checkbox("Case Sensitive", &case_sensitive_);
|
||||
ImGui::Checkbox("Whole Word", &match_whole_word_);
|
||||
ImGui::Checkbox("All Messages", &replace_all_messages_);
|
||||
|
||||
if (ImGui::Button("Replace")) {
|
||||
PRINT_IF_ERROR(Replace(find_text, replace_text, case_sensitive_,
|
||||
match_whole_word_, false));
|
||||
}
|
||||
|
||||
if (ImGui::Button("Replace All")) {
|
||||
PRINT_IF_ERROR(Replace(find_text, replace_text, case_sensitive_,
|
||||
match_whole_word_, true));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Replace in current message
|
||||
- [ ] Replace in all messages
|
||||
- [ ] Case-sensitive/insensitive options
|
||||
- [ ] Whole word matching
|
||||
- [ ] Undo support (requires history stack)
|
||||
|
||||
---
|
||||
|
||||
#### Task 3.2: Add Regex Support
|
||||
**Location**: `src/app/editor/message/message_editor.cc`
|
||||
**Priority**: P3 (Nice-to-have)
|
||||
**Estimated Effort**: 2 days
|
||||
|
||||
**Implementation**:
|
||||
```cpp
|
||||
absl::Status MessageEditor::ReplaceRegex(const std::string& pattern,
|
||||
const std::string& replacement,
|
||||
bool all_messages) {
|
||||
std::regex regex_pattern;
|
||||
try {
|
||||
regex_pattern = std::regex(pattern);
|
||||
} catch (const std::regex_error& e) {
|
||||
return absl::InvalidArgumentError(
|
||||
absl::StrFormat("Invalid regex: %s", e.what()));
|
||||
}
|
||||
|
||||
// Perform replacement...
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Regex validation with error messages
|
||||
- [ ] Capture group support ($1, $2, etc.)
|
||||
- [ ] Preview matches before replacement
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: UI Polish (Sprint 4-1 week)
|
||||
**Goal**: Theme integration and UX improvements
|
||||
|
||||
#### Task 4.1: Integrate AgentUITheme
|
||||
**Location**: `src/app/editor/message/message_editor.cc`
|
||||
**Priority**: P3
|
||||
**Estimated Effort**: 1 day
|
||||
|
||||
**Implementation**:
|
||||
```cpp
|
||||
void MessageEditor::DrawMessageList() {
|
||||
const auto& theme = AgentUI::GetTheme();
|
||||
|
||||
AgentUI::PushPanelStyle();
|
||||
ImGui::PushStyleColor(ImGuiCol_Header, theme.panel_bg_darker);
|
||||
ImGui::PushStyleColor(ImGuiCol_HeaderHovered, theme.accent_color);
|
||||
|
||||
// ... table rendering ...
|
||||
|
||||
ImGui::PopStyleColor(2);
|
||||
AgentUI::PopPanelStyle();
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] All panels use `AgentUI::PushPanelStyle()`
|
||||
- [ ] Section headers use `AgentUI::RenderSectionHeader()`
|
||||
- [ ] Buttons use `AgentUI::StyledButton()` where appropriate
|
||||
- [ ] Color scheme matches rest of editor
|
||||
|
||||
---
|
||||
|
||||
#### Task 4.2: Add Keyboard Shortcuts
|
||||
**Location**: `src/app/editor/message/message_editor.cc`
|
||||
**Priority**: P3
|
||||
**Estimated Effort**: 1 day
|
||||
|
||||
**Shortcuts**:
|
||||
- `Ctrl+F`: Open Find/Replace
|
||||
- `Ctrl+E`: Export to JSON
|
||||
- `Ctrl+I`: Import from JSON
|
||||
- `Ctrl+S`: Save ROM
|
||||
- `Ctrl+N`: Next message (in translation mode)
|
||||
- `Ctrl+P`: Previous message (in translation mode)
|
||||
|
||||
**Implementation**:
|
||||
```cpp
|
||||
void MessageEditor::HandleKeyboardShortcuts() {
|
||||
if (ImGui::IsKeyPressed(ImGuiKey_F) &&
|
||||
ImGui::GetIO().KeyCtrl) {
|
||||
show_find_replace_ = true;
|
||||
}
|
||||
|
||||
// ... other shortcuts ...
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance Criteria**:
|
||||
- [ ] Shortcuts don't conflict with global shortcuts
|
||||
- [ ] Shortcuts displayed in tooltips
|
||||
- [ ] Configurable shortcuts (future enhancement)
|
||||
|
||||
---
|
||||
|
||||
## Test Strategy
|
||||
|
||||
### Unit Tests
|
||||
**Location**: `test/unit/message_data_test.cc` (new file)
|
||||
```cpp
|
||||
TEST(MessageDataTest, SerializeToJson_BasicMessage) {
|
||||
MessageData msg;
|
||||
msg.ID = 0;
|
||||
msg.Address = 0xE0000;
|
||||
msg.RawString = "Hello World";
|
||||
msg.context = "Test message";
|
||||
|
||||
auto json = msg.SerializeToJson();
|
||||
|
||||
EXPECT_EQ(json["id"], 0);
|
||||
EXPECT_EQ(json["text"], "Hello World");
|
||||
EXPECT_EQ(json["context"], "Test message");
|
||||
}
|
||||
|
||||
TEST(MessageDataTest, DeserializeFromJson_RoundTrip) {
|
||||
MessageData original;
|
||||
original.ID = 5;
|
||||
original.RawString = "[W:00][K]Test";
|
||||
|
||||
auto json = original.SerializeToJson();
|
||||
auto result = MessageData::DeserializeFromJson(json);
|
||||
|
||||
ASSERT_TRUE(result.ok());
|
||||
EXPECT_EQ(result.value().ID, 5);
|
||||
EXPECT_EQ(result.value().RawString, "[W:00][K]Test");
|
||||
}
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
**Location**: `test/integration/message_export_test.cc` (new file)
|
||||
```cpp
|
||||
TEST_F(MessageRomTest, ExportImport_RoundTrip) {
|
||||
// Export all messages to JSON
|
||||
std::string json_path = "/tmp/messages.json";
|
||||
EXPECT_OK(ExportMessagesToJson(list_of_texts_, dictionary_, json_path));
|
||||
|
||||
// Import back
|
||||
auto imported = ImportMessagesFromJson(json_path);
|
||||
ASSERT_TRUE(imported.ok());
|
||||
|
||||
// Verify identical
|
||||
EXPECT_EQ(imported.value().size(), list_of_texts_.size());
|
||||
for (size_t i = 0; i < list_of_texts_.size(); ++i) {
|
||||
EXPECT_EQ(imported.value()[i].RawString, list_of_texts_[i].RawString);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### E2E Tests
|
||||
**Location**: `test/e2e/message_editor_workflow_test.cc` (new file)
|
||||
```cpp
|
||||
TEST_F(MessageEditorE2ETest, TranslationWorkflow) {
|
||||
// Open translation workspace
|
||||
EXPECT_OK(ClickButton("Translation Mode"));
|
||||
|
||||
// Load reference ROM
|
||||
EXPECT_OK(OpenFileDialog("reference_rom.sfc"));
|
||||
|
||||
// Navigate to message 0
|
||||
EXPECT_EQ(GetCurrentMessageID(), 0);
|
||||
|
||||
// Edit translation
|
||||
EXPECT_OK(SetTextBoxValue("Bienvenido a Hyrule"));
|
||||
|
||||
// Mark complete
|
||||
EXPECT_OK(ClickButton("Mark Complete"));
|
||||
|
||||
// Verify progress
|
||||
EXPECT_EQ(GetTranslationProgress(), "1/396");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Risks
|
||||
|
||||
### External Dependencies
|
||||
1. **nlohmann/json**: Already integrated via CPM ✅
|
||||
2. **ImGui Test Engine**: Available for E2E tests ✅
|
||||
3. **File Dialog**: `util::FileDialogWrapper` already exists ✅
|
||||
|
||||
### Technical Risks
|
||||
|
||||
#### Risk 1: JSON Schema Evolution
|
||||
**Impact**: Breaking changes to JSON format
|
||||
**Mitigation**:
|
||||
- Include version number in schema
|
||||
- Implement forward/backward compatibility
|
||||
- Provide migration tool for old exports
|
||||
|
||||
#### Risk 2: Dictionary Auto-Optimization Complexity
|
||||
**Impact**: Algorithm may be too slow for real-time use
|
||||
**Mitigation**:
|
||||
- Run optimization in background thread
|
||||
- Provide progress indicator
|
||||
- Allow cancellation
|
||||
|
||||
#### Risk 3: Large ROM Size with Expanded Messages
|
||||
**Impact**: May exceed bank boundaries
|
||||
**Mitigation**:
|
||||
- Implement repointing logic early (Phase 5)
|
||||
- Warn user when approaching limits
|
||||
- Suggest dictionary optimization
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Quantitative Metrics
|
||||
- [ ] 100% message export/import success rate (no data loss)
|
||||
- [ ] JSON schema supports all 396 vanilla messages
|
||||
- [ ] Translation workspace reduces edit time by 50% vs. current workflow
|
||||
- [ ] Search/Replace handles batch operations in <1 second
|
||||
- [ ] 90%+ test coverage for new code
|
||||
|
||||
### Qualitative Metrics
|
||||
- [ ] Translator feedback: "Translation workflow is intuitive"
|
||||
- [ ] No hardcoded colors in Message Editor
|
||||
- [ ] UI matches yaze style guide
|
||||
- [ ] Documentation complete for all new features
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
| Phase | Duration | Effort (Dev Days) |
|
||||
|-------|----------|-------------------|
|
||||
| Phase 1: JSON Export/Import | 2 weeks | 6 days |
|
||||
| Phase 2: Translation Workspace | 3 weeks | 9 days |
|
||||
| Phase 3: Search & Replace | 1 week | 4 days |
|
||||
| Phase 4: UI Polish | 1 week | 2 days |
|
||||
| **Total** | **7 weeks** | **21 dev days** |
|
||||
|
||||
**Note**: Timeline assumes single developer working full-time. Adjust for part-time work or team collaboration.
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements (Post-MVP)
|
||||
|
||||
1. **Scripting API** (Gemini's vision)
|
||||
- Expose MessageData to Lua/Python
|
||||
- Allow procedural text generation
|
||||
- Useful for randomizers
|
||||
|
||||
2. **Cloud Translation Integration**
|
||||
- Google Translate API for quick drafts
|
||||
- DeepL API for quality translations
|
||||
- Requires API key management
|
||||
|
||||
3. **Collaborative Editing**
|
||||
- Real-time multi-user translation
|
||||
- Conflict resolution for concurrent edits
|
||||
- Requires network infrastructure
|
||||
|
||||
4. **ROM Patch Generation**
|
||||
- Export as `.ips` or `.bps` patch files
|
||||
- Useful for distribution without ROM sharing
|
||||
- Requires patch generation library
|
||||
|
||||
5. **Message Validation**
|
||||
- Check for overlong messages (exceeds textbox width)
|
||||
- Verify all messages have terminators
|
||||
- Flag unused dictionary entries
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Q**: Should we support multiple translation languages simultaneously?
|
||||
**A**: TBD - May require multi-ROM workspace UI
|
||||
|
||||
2. **Q**: How should we handle custom dictionary entries for expanded ROMs?
|
||||
**A**: TBD - Need research into ROM space allocation
|
||||
|
||||
3. **Q**: Should translation progress be persisted?
|
||||
**A**: TBD - Could store in `.yaze` project file
|
||||
|
||||
4. **Q**: Do we need undo/redo for message editing?
|
||||
**A**: TBD - ImGui InputTextMultiline has built-in undo, may be sufficient
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The Message Editor has a **solid foundation** with core parsing, preview, and UI systems in place. The main gaps are **JSON export/import** (P0), **translation workspace** (P1), and **search/replace** (P2).
|
||||
|
||||
**Recommended Next Steps**:
|
||||
1. Start with Phase 1 (JSON export/import) - this unblocks all translation workflows
|
||||
2. Get user feedback on translation workspace mockups before Phase 2
|
||||
3. Defer theme integration to Phase 4 - not blocking functionality
|
||||
|
||||
**Estimated Effort**: ~7 weeks to MVP, ~21 dev days total.
|
||||
|
||||
**Success Criteria**: Translator can export messages to JSON, edit in external tool, and re-import without data loss. Side-by-side translation workspace reduces manual comparison time by 50%.
|
||||
164
docs/internal/agents/archive/release-checklist-2025-11-20.md
Normal file
164
docs/internal/agents/archive/release-checklist-2025-11-20.md
Normal file
@@ -0,0 +1,164 @@
|
||||
# Release Checklist - feat/http-api-phase2 → master
|
||||
|
||||
**Release Coordinator**: CLAUDE_RELEASE_COORD
|
||||
**Target Commit**: 43118254e6 - "fix: apply /std:c++latest unconditionally on Windows for std::filesystem"
|
||||
**CI Run**: #485 - https://github.com/scawful/yaze/actions/runs/19529565598
|
||||
**Status**: IN_PROGRESS
|
||||
**Last Updated**: 2025-11-20 02:50 PST
|
||||
|
||||
## Critical Context
|
||||
- Windows std::filesystem build has been BROKEN for 2+ weeks
|
||||
- Latest fix simplifies approach: apply /std:c++latest unconditionally on Windows
|
||||
- Multiple platform-specific fixes merged into feat/http-api-phase2 branch
|
||||
- User demands: "we absolutely need a release soon"
|
||||
|
||||
## Platform Build Status
|
||||
|
||||
### Windows Build
|
||||
- **Status**: ⏳ IN_PROGRESS (CI Run #485 - Job "Build - Windows 2022 (Core)")
|
||||
- **Previous Failures**: std::filesystem compilation errors (runs #480-484)
|
||||
- **Fix Applied**: Unconditional /std:c++latest flag in src/util/util.cmake
|
||||
- **Blocker**: None (fix deployed, awaiting CI validation)
|
||||
- **Owner**: CLAUDE_AIINF
|
||||
- **Test Command**: `cmake --preset win-dbg && cmake --build build`
|
||||
- **CI Job Status**: Building...
|
||||
|
||||
### Linux Build
|
||||
- **Status**: ⏳ IN_PROGRESS (CI Run #485 - Job "Build - Ubuntu 22.04 (GCC-12)")
|
||||
- **Previous Failures**:
|
||||
- Circular dependency resolved (commit 0812a84a22) ✅
|
||||
- FLAGS symbol conflicts in run #19528789779 ❌ (NEW BLOCKER)
|
||||
- **Known Issues**: FLAGS symbol redefinition (FLAGS_rom, FLAGS_norom, FLAGS_quiet)
|
||||
- **Blocker**: CRITICAL - Previous run showed FLAGS conflicts in yaze_emu_test linking
|
||||
- **Owner**: CLAUDE_LIN_BUILD (specialist agent monitoring)
|
||||
- **Test Command**: `cmake --preset lin-dbg && cmake --build build`
|
||||
- **CI Job Status**: Building...
|
||||
|
||||
### macOS Build
|
||||
- **Status**: ⏳ IN_PROGRESS (CI Run #485 - Job "Build - macOS 14 (Clang)")
|
||||
- **Previous Fixes**: z3ed linker error resolved (commit 9c562df277) ✅
|
||||
- **Previous Run**: PASSED in run #19528789779 ✅
|
||||
- **Known Issues**: None active
|
||||
- **Blocker**: None
|
||||
- **Owner**: CLAUDE_MAC_BUILD (specialist agent confirmed stable)
|
||||
- **Test Command**: `cmake --preset mac-dbg && cmake --build build`
|
||||
- **CI Job Status**: Building...
|
||||
|
||||
## HTTP API Validation
|
||||
|
||||
### Phase 2 Implementation Status
|
||||
- **Status**: ✅ COMPLETE (validated locally on macOS)
|
||||
- **Scope**: cmake/options.cmake, src/cli/cli_main.cc, src/cli/service/api/
|
||||
- **Endpoints Tested**:
|
||||
- ✅ GET /api/v1/health → 200 OK
|
||||
- ✅ GET /api/v1/models → 200 OK (empty list expected)
|
||||
- **CI Testing**: ⏳ PENDING (enable_http_api_tests=false for this run)
|
||||
- **Documentation**: ✅ Complete (src/cli/service/api/README.md)
|
||||
- **Owner**: CLAUDE_AIINF
|
||||
|
||||
## Test Execution Status
|
||||
|
||||
### Unit Tests
|
||||
- **Status**: ⏳ TESTING (CI Run #485)
|
||||
- **Expected**: All pass (no unit test changes in this branch)
|
||||
|
||||
### Integration Tests
|
||||
- **Status**: ⏳ TESTING (CI Run #485)
|
||||
- **Expected**: All pass (platform fixes shouldn't break integration)
|
||||
|
||||
### E2E Tests
|
||||
- **Status**: ⏳ TESTING (CI Run #485)
|
||||
- **Expected**: All pass (no UI changes)
|
||||
|
||||
## GO/NO-GO Decision Criteria
|
||||
|
||||
### GREEN LIGHT (GO) Requirements
|
||||
- ✅ All 3 platforms build successfully in CI
|
||||
- ✅ All test suites pass on all platforms
|
||||
- ✅ No new compiler warnings introduced
|
||||
- ✅ HTTP API validated on at least one platform (already done: macOS)
|
||||
- ✅ No critical security issues introduced
|
||||
- ✅ All coordination board blockers resolved
|
||||
|
||||
### RED LIGHT (NO-GO) Triggers
|
||||
- ❌ Any platform build failure
|
||||
- ❌ Test regression on any platform
|
||||
- ❌ New critical warnings/errors
|
||||
- ❌ Security vulnerabilities detected
|
||||
- ❌ Unresolved blocker from coordination board
|
||||
|
||||
## Current Blockers
|
||||
|
||||
### ACTIVE BLOCKERS
|
||||
|
||||
**BLOCKER #1: Linux FLAGS Symbol Conflicts (CRITICAL)**
|
||||
- **Status**: ⚠️ UNDER OBSERVATION (waiting for CI run #485 results)
|
||||
- **First Seen**: CI Run #19528789779
|
||||
- **Description**: Multiple definition of FLAGS_rom and FLAGS_norom; undefined FLAGS_quiet
|
||||
- **Impact**: Blocks yaze_emu_test linking on Linux
|
||||
- **Root Cause**: flags.cc compiled into agent library without ODR isolation
|
||||
- **Owner**: CLAUDE_LIN_BUILD
|
||||
- **Resolution Plan**: If persists in run #485, requires agent library linking fix
|
||||
- **Severity**: CRITICAL - blocks Linux release
|
||||
|
||||
**BLOCKER #2: Code Quality - clang-format violations**
|
||||
- **Status**: ❌ FAILED (CI Run #485)
|
||||
- **Description**: Formatting violations in test_manager.h, editor_manager.h, menu_orchestrator.cc
|
||||
- **Impact**: Non-blocking for release (cosmetic), but should be fixed before merge
|
||||
- **Owner**: TBD
|
||||
- **Resolution Plan**: Run `cmake --build build --target format` before merge
|
||||
- **Severity**: LOW - does not block release, can be fixed in follow-up
|
||||
|
||||
### RESOLVED BLOCKERS
|
||||
|
||||
**✅ Windows std::filesystem compilation** - Fixed in commit 43118254e6
|
||||
**✅ Linux circular dependency** - Fixed in commit 0812a84a22
|
||||
**✅ macOS z3ed linker error** - Fixed in commit 9c562df277
|
||||
|
||||
## Release Merge Plan
|
||||
|
||||
### When GREEN LIGHT Achieved:
|
||||
1. **Verify CI run #485 passes all jobs**
|
||||
2. **Run smoke build verification**: `scripts/agents/smoke-build.sh {preset} {target}` on all platforms
|
||||
3. **Update coordination board** with final status
|
||||
4. **Create merge commit**: `git checkout develop && git merge feat/http-api-phase2 --no-ff`
|
||||
5. **Run final test suite**: `scripts/agents/run-tests.sh {preset}`
|
||||
6. **Merge to master**: `git checkout master && git merge develop --no-ff`
|
||||
7. **Tag release**: `git tag -a v0.x.x -m "Release v0.x.x - Windows std::filesystem fix + HTTP API Phase 2"`
|
||||
8. **Push with tags**: `git push origin master develop --tags`
|
||||
9. **Trigger release workflow**: CI will automatically build release artifacts
|
||||
|
||||
### If RED LIGHT (Failure):
|
||||
1. **Identify failing job** in CI run #485
|
||||
2. **Assign to specialized agent**:
|
||||
- Windows failures → CLAUDE_AIINF (Windows Build Specialist)
|
||||
- Linux failures → CLAUDE_AIINF (Linux Build Specialist)
|
||||
- macOS failures → CLAUDE_AIINF (macOS Build Specialist)
|
||||
- Test failures → CLAUDE_CORE (Test Specialist)
|
||||
3. **Create emergency fix** on feat/http-api-phase2 branch
|
||||
4. **Trigger new CI run** and update this checklist
|
||||
5. **Repeat until GREEN LIGHT**
|
||||
|
||||
## Monitoring Protocol
|
||||
|
||||
**CLAUDE_RELEASE_COORD will check CI status every 5 minutes and update coordination board with:**
|
||||
- Platform build progress (queued/in_progress/success/failure)
|
||||
- Test execution status
|
||||
- Any new blockers discovered
|
||||
- ETA to GREEN LIGHT decision
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ⏳ Monitor CI run #485 - https://github.com/scawful/yaze/actions/runs/19529565598
|
||||
2. ⏳ Wait for Windows build job to complete (critical validation)
|
||||
3. ⏳ Wait for Linux build job to complete
|
||||
4. ⏳ Wait for macOS build job to complete
|
||||
5. ⏳ Wait for test jobs to complete on all platforms
|
||||
6. ⏳ Make GO/NO-GO decision
|
||||
7. ⏳ Execute merge plan if GREEN LIGHT
|
||||
|
||||
---
|
||||
|
||||
**Coordination Board**: `docs/internal/agents/coordination-board.md`
|
||||
**Build Reference**: `docs/public/build/quick-reference.md`
|
||||
**HTTP API Docs**: `src/cli/service/api/README.md`
|
||||
@@ -0,0 +1,436 @@
|
||||
# Architecture Documentation Review & Improvements Report
|
||||
|
||||
**Review Date**: November 21, 2025
|
||||
**Reviewed By**: Claude Documentation Janitor
|
||||
**Status**: Complete
|
||||
**Documents Reviewed**: 8 architecture files
|
||||
|
||||
## Executive Summary
|
||||
|
||||
A comprehensive review of Gemini's architecture documentation has been completed. The documentation is well-structured and generally accurate, with clear explanations of complex systems. Several improvements have been made to enhance accuracy, completeness, and usability:
|
||||
|
||||
**Key Accomplishments**:
|
||||
- Consolidated duplicate graphics system documentation
|
||||
- Enhanced accuracy with specific file paths and method signatures
|
||||
- Added best practices and contributing guidelines
|
||||
- Created comprehensive navigation hub (README.md)
|
||||
- Added cross-references to CLAUDE.md
|
||||
- Improved technical depth and implementation details
|
||||
|
||||
**Quality Assessment**: **Good Work Overall** - Documentation demonstrates solid understanding of the codebase with clear explanations and good organization. Improvements focused on completeness and precision.
|
||||
|
||||
---
|
||||
|
||||
## Document-by-Document Review
|
||||
|
||||
### 1. graphics_system_architecture.md & graphics_system.md
|
||||
|
||||
**Status**: CONSOLIDATED & ENHANCED
|
||||
|
||||
**Original Issues**:
|
||||
- Two separate files covering similar content (69 lines vs 74 lines each)
|
||||
- Partial overlap in coverage (both covered Arena, Bitmap, compression)
|
||||
- graphics_system.md lacked detailed rendering pipeline explanation
|
||||
- Missing specific method signatures
|
||||
- No best practices section
|
||||
|
||||
**Improvements Made**:
|
||||
✓ Merged both documents into enhanced graphics_system_architecture.md
|
||||
✓ Added comprehensive rendering pipeline with 4 distinct phases
|
||||
✓ Added detailed component descriptions with key methods
|
||||
✓ Added structured table for compression formats with sheet indices
|
||||
✓ Enhanced Canvas Interactions section with coordinate system details
|
||||
✓ Added Best Practices section (6 key practices)
|
||||
✓ Expanded Future Improvements section
|
||||
✓ Added specific decompression function names: `DecompressV2()`, `CompressV3()`
|
||||
✓ Deleted duplicate graphics_system.md
|
||||
|
||||
**Code Accuracy Verified**:
|
||||
- Arena holds 223 Bitmap objects (confirmed in arena.h line 82)
|
||||
- Compression functions exist as documented (compression.h lines 226, 241)
|
||||
- IRenderer pattern documented correctly (irenderer.h exists)
|
||||
- BackgroundBuffer management confirmed (arena.h lines 127-128)
|
||||
|
||||
**Lines of Documentation**: 178 (consolidated from 143, increased for comprehensiveness)
|
||||
|
||||
---
|
||||
|
||||
### 2. dungeon_editor_system.md
|
||||
|
||||
**Status**: SIGNIFICANTLY ENHANCED
|
||||
|
||||
**Original Issues**:
|
||||
- Component descriptions lacked full context
|
||||
- Missing file paths in location column
|
||||
- No best practices for contributors
|
||||
- Future Improvements section was minimal
|
||||
- No discussion of callback-based communication pattern
|
||||
- Missing discussion of coordinate system details
|
||||
|
||||
**Improvements Made**:
|
||||
✓ Added architectural pattern explanation (coordinator pattern)
|
||||
✓ Expanded Key Components table with full file paths (8 → 8 entries, more detail)
|
||||
✓ Added DungeonRoomSelector and DungeonObjectValidator to component table
|
||||
✓ Added "Best Practices for Contributors" section with:
|
||||
- Guidelines for adding new editor modes (6 steps)
|
||||
- Object editing best practices with code examples
|
||||
- Callback communication patterns
|
||||
- Coordinate system management guidance
|
||||
- Batch operation efficiency tips
|
||||
✓ Expanded Future Improvements (3 → 6 improvements)
|
||||
✓ Enhanced Interaction Flow with more detail
|
||||
✓ Added coordinate system explanation (0-63 grid)
|
||||
|
||||
**Code Accuracy Verified**:
|
||||
- DungeonEditorV2 inherits from Editor (dungeon_editor_v2.h line 42)
|
||||
- DungeonEditorSystem has EditorMode enum with kObjects mode (dungeon_editor_system.h lines 40-49)
|
||||
- UndoPoint structure confirmed (dungeon_object_editor.h line 93)
|
||||
- Component file paths verified in actual directory structure
|
||||
|
||||
**Added Practical Guidance**: Code examples and step-by-step procedures for common tasks
|
||||
|
||||
---
|
||||
|
||||
### 3. room_data_persistence.md
|
||||
|
||||
**Status**: ENHANCED WITH DETAILS
|
||||
|
||||
**Original Issues**:
|
||||
- Lacked specific ROM address information
|
||||
- Missing details about pointer tables
|
||||
- No discussion of thread safety
|
||||
- Method signatures incomplete
|
||||
- Bank boundary considerations mentioned but not explained
|
||||
|
||||
**Improvements Made**:
|
||||
✓ Added specific method signatures and parameters
|
||||
✓ Enhanced description of room ID range (0x000-0x127)
|
||||
✓ Clarified pointer table lookup process
|
||||
✓ Emphasized critical importance of repointing logic
|
||||
✓ Expanded ROM address references with constants from dungeon_rom_addresses.h
|
||||
✓ Better explanation of room size calculation for safety
|
||||
✓ Clarified thread safety aspects of bulk loading
|
||||
|
||||
**Code Accuracy Verified**:
|
||||
- LoadRoom method signature confirmed (dungeon_room_loader.h line 26)
|
||||
- LoadAllRooms uses std::array<zelda3::Room, 0x128> (dungeon_room_loader.h line 27)
|
||||
- Room ID range 0x000-0x127 confirmed (296 rooms = 0x128)
|
||||
|
||||
**Still Accurate**: Saving strategy marked as "Planned/In-Progress" - correctly reflects implementation status
|
||||
|
||||
---
|
||||
|
||||
### 4. undo_redo_system.md
|
||||
|
||||
**Status**: VERIFIED ACCURATE
|
||||
|
||||
**Verification Results**:
|
||||
✓ UndoPoint structure matches implementation exactly (dungeon_object_editor.h lines 93-98)
|
||||
- objects: std::vector<RoomObject>
|
||||
- selection: SelectionState
|
||||
- editing: EditingState
|
||||
- timestamp: std::chrono::steady_clock::time_point
|
||||
✓ Undo/Redo workflow documented correctly
|
||||
✓ Best practices align with implementation patterns
|
||||
✓ Batch operation guidance is sound
|
||||
|
||||
**No Changes Required**: Documentation is accurate and complete as written
|
||||
|
||||
**Quality Notes**: Clear explanation of state snapshot pattern, good guidance on batch operations
|
||||
|
||||
---
|
||||
|
||||
### 5. overworld_editor_system.md
|
||||
|
||||
**Status**: VERIFIED ACCURATE
|
||||
|
||||
**Verification Results**:
|
||||
✓ All component descriptions match actual classes
|
||||
✓ Interaction flow accurately describes actual workflow
|
||||
✓ Coordinate systems explanation is correct
|
||||
✓ Large maps configuration documented correctly
|
||||
✓ Deferred loading section accurately describes implementation
|
||||
|
||||
**Code Cross-Check Completed**:
|
||||
- OverworldEditor class confirmed (overworld_editor.h line 64)
|
||||
- Overworld system coordinator pattern verified
|
||||
- OverworldMap data model description accurate
|
||||
- Entity renderer pattern confirmed
|
||||
|
||||
**No Changes Required**: Documentation is accurate and comprehensive
|
||||
|
||||
---
|
||||
|
||||
### 6. overworld_map_data.md
|
||||
|
||||
**Status**: VERIFIED ACCURATE WITH ENHANCEMENTS
|
||||
|
||||
**Original Issues**:
|
||||
- Good documentation but could be more precise with ROM address constants
|
||||
- ZSCustomOverworld section mentioned need to verify exact implementation
|
||||
|
||||
**Improvements Made**:
|
||||
✓ Verified all ROM address constants against overworld_map.h
|
||||
✓ Confirmed ZSCustomOverworld property names and storage locations
|
||||
✓ Storage locations now explicitly listed (OverworldCustomAreaSpecificBGPalette, etc.)
|
||||
✓ Cross-referenced with overworld.h for implementation accuracy
|
||||
|
||||
**Code Cross-Check Completed**:
|
||||
- OverworldMap ROM addresses verified (overworld_map.h lines 21-73)
|
||||
- Custom property constants confirmed:
|
||||
- OverworldCustomAreaSpecificBGPalette = 0x140000 (line 21)
|
||||
- OverworldCustomMosaicArray = 0x140200 (line 39)
|
||||
- OverworldCustomSubscreenOverlayArray = 0x140340 (line 28)
|
||||
- OverworldCustomAnimatedGFXArray = 0x1402A0 (line 31)
|
||||
- ZSCustomOverworld v3 constants verified
|
||||
|
||||
**No Changes Required**: Documentation is accurate as written
|
||||
|
||||
---
|
||||
|
||||
### 7. zscustomoverworld_integration.md
|
||||
|
||||
**Status**: SIGNIFICANTLY ENHANCED
|
||||
|
||||
**Original Issues**:
|
||||
- Version detection section marked as "need to verify"
|
||||
- Missing ROM storage locations table
|
||||
- No implementation code examples
|
||||
- ConfigureMultiAreaMap method not fully explained
|
||||
- Missing details on feature enables
|
||||
|
||||
**Improvements Made**:
|
||||
✓ Added complete ROM storage locations table with:
|
||||
- Feature names and constants
|
||||
- ROM addresses (0x140000 range)
|
||||
- Data sizes (1-8 bytes per map)
|
||||
- Usage notes for each feature
|
||||
✓ Clarified version detection using overworld_version_helper.h
|
||||
✓ Added asm_version check details
|
||||
✓ Provided code example for custom properties access
|
||||
✓ Emphasized never setting area_size directly
|
||||
✓ Explained ConfigureMultiAreaMap 5-step process
|
||||
✓ Added table of feature enable flag addresses
|
||||
|
||||
**Code Examples Added**:
|
||||
```cpp
|
||||
// Proper multi-area configuration
|
||||
absl::Status Overworld::ConfigureMultiAreaMap(int parent_index, AreaSizeEnum size);
|
||||
|
||||
// Custom properties access pattern
|
||||
if (rom->asm_version >= 1) {
|
||||
map.SetupCustomTileset(rom->asm_version);
|
||||
uint16_t custom_bg_color = map.area_specific_bg_color_;
|
||||
}
|
||||
```
|
||||
|
||||
**Code Accuracy Verified**:
|
||||
- ConfigureMultiAreaMap method signature confirmed (overworld.h line 204)
|
||||
- SetupCustomTileset method confirmed (overworld_map.h line 257)
|
||||
- All ROM address constants verified against source (overworld_map.h lines 21-73)
|
||||
|
||||
---
|
||||
|
||||
### 8. TEST_INFRASTRUCTURE_IMPROVEMENTS.md
|
||||
|
||||
**Status**: NOTED BUT NOT REVIEWED
|
||||
|
||||
**Reasoning**: This file focuses on test infrastructure and was not part of the core architecture review scope. It exists as a separate documentation artifact.
|
||||
|
||||
---
|
||||
|
||||
## New Documentation Created
|
||||
|
||||
### docs/internal/architecture/README.md
|
||||
|
||||
**Status**: CREATED
|
||||
|
||||
**Purpose**: Comprehensive navigation hub for all architecture documentation
|
||||
|
||||
**Contents**:
|
||||
- Overview of architecture documentation purpose
|
||||
- Quick reference guide organized by component
|
||||
- Design patterns used in the project (5 patterns documented)
|
||||
- Contributing guidelines
|
||||
- Architecture evolution notes
|
||||
- Status and maintenance information
|
||||
|
||||
**Key Sections**:
|
||||
1. Core Architecture Guides (5 major systems)
|
||||
2. Quick Reference by Component (organized by source directory)
|
||||
3. Design Patterns Used (Modular/Component-Based, Callbacks, Singleton, Progressive Loading, Snapshot-Based Undo/Redo)
|
||||
4. Contributing Guidelines (7 key principles)
|
||||
5. Related Documents (links to CLAUDE.md, README.md)
|
||||
6. Architecture Evolution (historical context)
|
||||
|
||||
**File Size**: 400+ lines, comprehensive navigation and guidance
|
||||
|
||||
---
|
||||
|
||||
## CLAUDE.md Enhancements
|
||||
|
||||
**Changes Made**:
|
||||
✓ Added new "Architecture Documentation" section
|
||||
✓ Provided links to all 8 architecture documents with brief descriptions
|
||||
✓ Reorganized "Important File Locations" section
|
||||
✓ Updated file path references to match actual locations
|
||||
|
||||
**New Section**:
|
||||
```markdown
|
||||
## Architecture Documentation
|
||||
|
||||
Detailed architectural guides are available in `docs/internal/architecture/`:
|
||||
- Graphics System
|
||||
- Dungeon Editor System
|
||||
- Room Data Persistence
|
||||
- Overworld Editor System
|
||||
- Overworld Map Data
|
||||
- Undo/Redo System
|
||||
- ZSCustomOverworld Integration
|
||||
- Architecture Index
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary of Inaccuracies Found
|
||||
|
||||
**Critical Issues**: None found
|
||||
|
||||
**Minor Issues Addressed**:
|
||||
1. Duplicate graphics system documentation (fixed by consolidation)
|
||||
2. Incomplete method signatures (enhanced with full details)
|
||||
3. Missing ROM address constants (added from source)
|
||||
4. Vague component descriptions (expanded with file paths and roles)
|
||||
5. Missing implementation examples (added where helpful)
|
||||
|
||||
---
|
||||
|
||||
## Recommendations for Future Documentation
|
||||
|
||||
### Short-Term (For Next Review)
|
||||
|
||||
1. **Test Architecture Documentation**: Create document for test structure and patterns
|
||||
2. **ROM Structure Guide**: Detailed reference of ALttP ROM layout and bank addressing
|
||||
3. **Asar Integration Details**: More comprehensive guide to assembly patching
|
||||
4. **CLI Tool Architecture**: Document z3ed CLI and TUI component design
|
||||
|
||||
### Medium-Term (Next Quarter)
|
||||
|
||||
1. **Performance Optimization Guide**: Document optimization patterns and bottlenecks
|
||||
2. **Thread Safety Guidelines**: Comprehensive guide to concurrent operations
|
||||
3. **Graphics Format Reference**: Detailed 2BPP/3BPP/Indexed format guide
|
||||
4. **ROM Hacking Patterns**: Common patterns and anti-patterns in the codebase
|
||||
|
||||
### Long-Term (Strategic)
|
||||
|
||||
1. **API Reference Documentation**: Auto-generated API docs from inline comments
|
||||
2. **Architecture Decision Records (ADRs)**: Document why certain patterns were chosen
|
||||
3. **Migration Guides**: Documentation for code refactoring and API changes
|
||||
4. **Video Tutorials**: Visual architecture walkthroughs
|
||||
|
||||
---
|
||||
|
||||
## Gemini's Documentation Quality Assessment
|
||||
|
||||
### Strengths
|
||||
|
||||
✓ **Clear Structure**: Documents are well-organized with logical sections
|
||||
✓ **Good Explanations**: Complex systems explained in accessible language
|
||||
✓ **Accurate Understanding**: Demonstrates solid grasp of codebase
|
||||
✓ **Component Relationships**: Clear description of how pieces interact
|
||||
✓ **Practical Focus**: Includes real examples and workflows
|
||||
✓ **ROM Knowledge**: Correct handling of SNES-specific details
|
||||
|
||||
### Areas for Improvement
|
||||
|
||||
⚠ **Precision**: Could include more specific file paths and method signatures
|
||||
⚠ **Completeness**: Some sections could benefit from code examples
|
||||
⚠ **Verification**: Some implementation details marked as "need to verify"
|
||||
⚠ **Best Practices**: Could include more contributor guidance
|
||||
⚠ **Cross-References**: Could link between related documents more
|
||||
|
||||
### Growth Opportunities
|
||||
|
||||
1. **Deepen ROM Knowledge**: Learn more about pointer tables and memory banking
|
||||
2. **Study Design Patterns**: Research the specific patterns used (coordinator, callback, singleton)
|
||||
3. **Add Examples**: Include real code snippets from the project
|
||||
4. **Test Verification**: Verify documentation against actual test cases
|
||||
5. **Performance Details**: Document performance implications of design choices
|
||||
|
||||
---
|
||||
|
||||
## Checklist of Deliverables
|
||||
|
||||
✓ **Updated/Corrected Versions of All Documents**:
|
||||
- graphics_system_architecture.md (merged and enhanced)
|
||||
- dungeon_editor_system.md (enhanced)
|
||||
- room_data_persistence.md (enhanced)
|
||||
- overworld_editor_system.md (verified accurate)
|
||||
- overworld_map_data.md (verified accurate)
|
||||
- undo_redo_system.md (verified accurate)
|
||||
- zscustomoverworld_integration.md (enhanced)
|
||||
- graphics_system.md (deleted - consolidated)
|
||||
|
||||
✓ **New Documentation Created**:
|
||||
- docs/internal/architecture/README.md (navigation hub)
|
||||
|
||||
✓ **Integration Updates**:
|
||||
- CLAUDE.md (added Architecture Documentation section with links)
|
||||
|
||||
✓ **Summary Report**:
|
||||
- This document (comprehensive findings and recommendations)
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
### Updated Files
|
||||
1. `/Users/scawful/Code/yaze/docs/internal/architecture/graphics_system_architecture.md`
|
||||
- Status: Enhanced (178 lines, was 74)
|
||||
- Changes: Consolidated duplicate, added rendering pipeline, best practices, code examples
|
||||
|
||||
2. `/Users/scawful/Code/yaze/docs/internal/architecture/dungeon_editor_system.md`
|
||||
- Status: Enhanced
|
||||
- Changes: Added best practices section, contributor guidelines, expanded components, examples
|
||||
|
||||
3. `/Users/scawful/Code/yaze/docs/internal/architecture/room_data_persistence.md`
|
||||
- Status: Enhanced
|
||||
- Changes: Improved method signatures, ROM address details, thread safety notes
|
||||
|
||||
4. `/Users/scawful/Code/yaze/docs/internal/architecture/zscustomoverworld_integration.md`
|
||||
- Status: Enhanced
|
||||
- Changes: Added ROM storage table, implementation details, code examples
|
||||
|
||||
5. `/Users/scawful/Code/yaze/CLAUDE.md`
|
||||
- Status: Enhanced
|
||||
- Changes: Added Architecture Documentation section with links and descriptions
|
||||
|
||||
### New Files
|
||||
1. `/Users/scawful/Code/yaze/docs/internal/architecture/README.md` (400+ lines)
|
||||
- Comprehensive navigation hub with quick references, design patterns, and guidelines
|
||||
|
||||
### Deleted Files
|
||||
1. `/Users/scawful/Code/yaze/docs/internal/architecture/graphics_system.md`
|
||||
- Consolidated into graphics_system_architecture.md
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
Gemini's architecture documentation demonstrates a solid understanding of the YAZE codebase. The documentation is clear, well-organized, and largely accurate. The improvements made focus on:
|
||||
|
||||
1. **Consolidating Duplicates**: Merged two graphics system documents into one comprehensive guide
|
||||
2. **Enhancing Accuracy**: Added specific file paths, method signatures, and ROM addresses
|
||||
3. **Improving Usability**: Created navigation hub and added cross-references
|
||||
4. **Adding Guidance**: Included best practices and contributor guidelines
|
||||
5. **Ensuring Completeness**: Expanded sections that were marked incomplete
|
||||
|
||||
The architecture documentation now serves as an excellent resource for developers working on or understanding the YAZE codebase. The navigation hub makes it easy to find relevant information, and the added examples and best practices provide practical guidance for contributors.
|
||||
|
||||
**Overall Quality Rating**: **8/10** - Good work with solid understanding, some areas for even greater depth and precision.
|
||||
|
||||
---
|
||||
|
||||
**Report Prepared By**: Documentation Janitor
|
||||
**Date**: November 21, 2025
|
||||
**Architecture Documentation Status**: **Ready for Use**
|
||||
@@ -0,0 +1,303 @@
|
||||
# Dungeon Graphics Rendering Bug Report
|
||||
|
||||
**Status**: CRITICAL - Objects not rendering
|
||||
**Affected System**: Dungeon Object Editor
|
||||
**Root Causes**: 4 critical bugs identified
|
||||
**Research By**: zelda3-hacking-expert + backend-infra-engineer agents
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Dungeon objects are not rendering correctly due to **incorrect ROM addresses** and **missing palette application**. Four critical bugs have been identified in the rendering pipeline.
|
||||
|
||||
---
|
||||
|
||||
## CRITICAL BUG #1: Wrong ROM Addresses in ObjectParser ⚠️
|
||||
|
||||
**Priority**: P0 - BLOCKER
|
||||
**File**: `src/zelda3/dungeon/object_parser.cc` (Lines 10-14)
|
||||
**Impact**: Objects read garbage data from ROM
|
||||
|
||||
### Current Code (WRONG)
|
||||
```cpp
|
||||
static constexpr int kRoomObjectSubtype1 = 0x0A8000; // ❌ PLACEHOLDER
|
||||
static constexpr int kRoomObjectSubtype2 = 0x0A9000; // ❌ PLACEHOLDER
|
||||
static constexpr int kRoomObjectSubtype3 = 0x0AA000; // ❌ PLACEHOLDER
|
||||
static constexpr int kRoomObjectTileAddress = 0x0AB000; // ❌ PLACEHOLDER
|
||||
```
|
||||
|
||||
**These addresses don't exist in ALTTP's ROM!** They are placeholders from early development.
|
||||
|
||||
### Fix (CORRECT)
|
||||
```cpp
|
||||
// ALTTP US 1.0 ROM addresses (PC format)
|
||||
static constexpr int kRoomObjectSubtype1 = 0x0F8000; // SNES: $08:8000
|
||||
static constexpr int kRoomObjectSubtype2 = 0x0F83F0; // SNES: $08:83F0
|
||||
static constexpr int kRoomObjectSubtype3 = 0x0F84F0; // SNES: $08:84F0
|
||||
static constexpr int kRoomObjectTileAddress = 0x091B52; // SNES: $09:1B52
|
||||
```
|
||||
|
||||
### Explanation
|
||||
|
||||
**How ALTTP Object Graphics Work**:
|
||||
```
|
||||
1. Object ID (e.g., $10 = wall) → Subtype Table Lookup
|
||||
├─ Read pointer from: kRoomObjectSubtype1 + (ID * 2)
|
||||
└─ Pointer is 16-bit offset from kRoomObjectTileAddress
|
||||
|
||||
2. Calculate Tile Data Address
|
||||
├─ tile_data_addr = kRoomObjectTileAddress + offset
|
||||
└─ Each tile = 2 bytes (TileInfo word)
|
||||
|
||||
3. TileInfo Word Format (16-bit: vhopppcccccccccc)
|
||||
├─ v (bit 15): Vertical flip
|
||||
├─ h (bit 14): Horizontal flip
|
||||
├─ o (bit 13): Priority/Over flag
|
||||
├─ ppp (bits 10-12): Palette index (0-7)
|
||||
└─ cccccccccc (bits 0-9): CHR tile ID (0-1023)
|
||||
```
|
||||
|
||||
**Example for Object $10 (Wall)**:
|
||||
```
|
||||
1. Subtype 1 table: 0x0F8000 + ($10 * 2) = 0x0F8020
|
||||
2. Read offset: [Low, High] = $0234
|
||||
3. Tile data: 0x091B52 + $0234 = 0x091D86
|
||||
4. Read TileInfo words (8 tiles = 16 bytes)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CRITICAL BUG #2: Missing Palette Application ⚠️
|
||||
|
||||
**Priority**: P0 - BLOCKER
|
||||
**File**: `src/zelda3/dungeon/object_drawer.cc` (Lines 76-104)
|
||||
**Impact**: Black screen or wrong colors
|
||||
|
||||
### The Problem
|
||||
|
||||
`ObjectDrawer` writes palette index values (0-255) to the bitmap, but **never applies the dungeon palette** to the SDL surface. The bitmap has no color information!
|
||||
|
||||
**Current Flow**:
|
||||
```
|
||||
ObjectDrawer writes index values → memcpy to SDL surface → Display ❌
|
||||
↑
|
||||
No palette applied!
|
||||
```
|
||||
|
||||
**Should Be**:
|
||||
```
|
||||
ObjectDrawer writes index values → Apply palette → memcpy to SDL → Display ✅
|
||||
```
|
||||
|
||||
### Fix
|
||||
|
||||
**Add to `ObjectDrawer::DrawObjectList()` after line 77**:
|
||||
|
||||
```cpp
|
||||
absl::Status ObjectDrawer::DrawObjectList(
|
||||
const std::vector<RoomObject>& objects,
|
||||
gfx::BackgroundBuffer& bg1,
|
||||
gfx::BackgroundBuffer& bg2,
|
||||
const gfx::PaletteGroup& palette_group) {
|
||||
|
||||
// Draw all objects
|
||||
for (const auto& object : objects) {
|
||||
RETURN_IF_ERROR(DrawObject(object, bg1, bg2, palette_group));
|
||||
}
|
||||
|
||||
// ✅ FIX: Apply dungeon palette to background buffers
|
||||
auto& bg1_bmp = bg1.bitmap();
|
||||
auto& bg2_bmp = bg2.bitmap();
|
||||
|
||||
if (!palette_group.empty()) {
|
||||
const auto& dungeon_palette = palette_group[0]; // Main dungeon palette (90 colors)
|
||||
bg1_bmp.SetPalette(dungeon_palette);
|
||||
bg2_bmp.SetPalette(dungeon_palette);
|
||||
}
|
||||
|
||||
// Sync bitmap data to SDL surfaces AFTER palette is applied
|
||||
if (bg1_bmp.modified() && bg1_bmp.surface() && !bg1_bmp.data().empty()) {
|
||||
SDL_LockSurface(bg1_bmp.surface());
|
||||
memcpy(bg1_bmp.surface()->pixels, bg1_bmp.data().data(), bg1_bmp.data().size());
|
||||
SDL_UnlockSurface(bg1_bmp.surface());
|
||||
}
|
||||
|
||||
if (bg2_bmp.modified() && bg2_bmp.surface() && !bg2_bmp.data().empty()) {
|
||||
SDL_LockSurface(bg2_bmp.surface());
|
||||
memcpy(bg2_bmp.surface()->pixels, bg2_bmp.data().data(), bg2_bmp.data().size());
|
||||
SDL_UnlockSurface(bg2_bmp.surface());
|
||||
}
|
||||
|
||||
return absl::OkStatus();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## BUG #3: Incorrect Palette Offset Calculation
|
||||
|
||||
**Priority**: P1 - HIGH
|
||||
**File**: `src/zelda3/dungeon/object_drawer.cc` (Line 900)
|
||||
**Impact**: Wrong colors for objects
|
||||
|
||||
### Current Code (WRONG)
|
||||
```cpp
|
||||
// Line 899-900
|
||||
uint8_t palette_offset = (tile_info.palette_ & 0x0F) * 8;
|
||||
```
|
||||
|
||||
**Problem**: Uses 4 bits (`& 0x0F`) but dungeon graphics are 3BPP with only 3-bit palette indices!
|
||||
|
||||
### Fix
|
||||
```cpp
|
||||
// Dungeon graphics are 3BPP (8 colors per palette)
|
||||
// Only use 3 bits for palette index (0-7)
|
||||
uint8_t palette_offset = (tile_info.palette_ & 0x07) * 8;
|
||||
```
|
||||
|
||||
### Dungeon Palette Structure
|
||||
|
||||
From `snes_palette.cc` line 198:
|
||||
- Total: **90 colors** per dungeon palette
|
||||
- Colors 0-29: Main graphics (palettes 0-3)
|
||||
- Colors 30-59: Secondary graphics (palettes 4-7)
|
||||
- Colors 60-89: Sprite graphics (palettes 8-11)
|
||||
|
||||
Each sub-palette has 8 colors (3BPP), arranged:
|
||||
- Palette 0: Colors 0-7
|
||||
- Palette 1: Colors 8-15
|
||||
- Palette 2: Colors 16-23
|
||||
- Palette 3: Colors 24-29 (NOT 24-31!)
|
||||
|
||||
---
|
||||
|
||||
## BUG #4: Palette Metadata Not Initialized
|
||||
|
||||
**Priority**: P2 - MEDIUM
|
||||
**File**: `src/app/gfx/render/background_buffer.cc` (constructor)
|
||||
**Impact**: `ApplyPaletteByMetadata()` may not work correctly
|
||||
|
||||
### Fix
|
||||
|
||||
Ensure BackgroundBuffer initializes bitmap metadata:
|
||||
|
||||
```cpp
|
||||
BackgroundBuffer::BackgroundBuffer(int width, int height)
|
||||
: width_(width), height_(height) {
|
||||
buffer_.resize((width / 8) * (height / 8));
|
||||
std::vector<uint8_t> data(width * height, 0);
|
||||
|
||||
// Create 8-bit indexed color bitmap
|
||||
bitmap_.Create(width, height, 8, data);
|
||||
|
||||
// Set metadata for dungeon rendering
|
||||
auto& metadata = bitmap_.metadata();
|
||||
metadata.source_bpp = 3; // 3BPP dungeon graphics
|
||||
metadata.palette_format = 0; // Full palette (90 colors)
|
||||
metadata.source_type = "dungeon_background";
|
||||
metadata.palette_colors = 90; // Dungeon main palette size
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Complete Rendering Pipeline
|
||||
|
||||
### Correct Flow
|
||||
```
|
||||
1. ROM Data (0x0F8000+) → ObjectParser
|
||||
├─ Read subtype table
|
||||
├─ Calculate tile data offset
|
||||
└─ Parse TileInfo words
|
||||
|
||||
2. TileInfo[] → ObjectDrawer
|
||||
├─ For each tile:
|
||||
│ ├─ Calculate position in graphics sheet
|
||||
│ ├─ Read 8x8 indexed pixels (0-7)
|
||||
│ ├─ Apply palette offset: pixel + (palette * 8)
|
||||
│ └─ Write to BackgroundBuffer bitmap
|
||||
└─ Apply dungeon palette to bitmap (SetPalette)
|
||||
|
||||
3. BackgroundBuffer → SDL Surface
|
||||
├─ memcpy indexed pixel data
|
||||
└─ SDL uses surface palette for display
|
||||
|
||||
4. SDL Surface → ImGui Texture → Screen
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ROM Address Reference
|
||||
|
||||
| Structure | SNES Address | PC Address | Purpose |
|
||||
|-----------|-------------|-----------|---------|
|
||||
| Subtype 1 Table | `$08:8000` | `0x0F8000` | Objects $00-$FF pointers (512 bytes) |
|
||||
| Subtype 2 Table | `$08:83F0` | `0x0F83F0` | Objects $100-$1FF pointers (256 bytes) |
|
||||
| Subtype 3 Table | `$08:84F0` | `0x0F84F0` | Objects $F00-$FFF pointers (256 bytes) |
|
||||
| Tile Data Base | `$09:1B52` | `0x091B52` | TileInfo word arrays (~8KB) |
|
||||
| Graphics Sheets | `$0C:8000+` | `0x0C8000+` | 4BPP compressed CHR data |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. ✅ **Fix Bug #1** (ROM addresses) - 5 minutes
|
||||
2. ✅ **Fix Bug #2** (palette application) - 15 minutes
|
||||
3. ✅ **Fix Bug #3** (palette offset) - 5 minutes
|
||||
4. ⚠️ **Fix Bug #4** (metadata) - 10 minutes (verify needed)
|
||||
|
||||
**Total Time**: ~35 minutes to fix all critical bugs
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
After fixes:
|
||||
- [ ] Load dungeon room 0x01 (Eastern Palace entrance)
|
||||
- [ ] Verify gray stone walls render correctly
|
||||
- [ ] Check that objects have distinct colors
|
||||
- [ ] Verify no black/transparent artifacts
|
||||
- [ ] Test multiple rooms with different palettes
|
||||
- [ ] Verify BG1 and BG2 layers are distinct
|
||||
|
||||
---
|
||||
|
||||
## Debugging Commands
|
||||
|
||||
Add these logs to verify the fix:
|
||||
|
||||
```cpp
|
||||
// In ObjectParser::ReadTileData() after reading first tile:
|
||||
if (i == 0) {
|
||||
printf("[ObjectParser] Object 0x%03X: tile_addr=0x%06X word=0x%04X → id=%03X pal=%d\n",
|
||||
object_id, tile_offset, tile_word, tile_info.id_, tile_info.palette_);
|
||||
}
|
||||
|
||||
// In ObjectDrawer::DrawObjectList() after applying palette:
|
||||
if (!palette_group.empty()) {
|
||||
const auto& pal = palette_group[0];
|
||||
printf("[ObjectDrawer] Applied palette: %zu colors, first=RGB(%d,%d,%d)\n",
|
||||
pal.size(), pal[0].rom_color().red, pal[0].rom_color().green, pal[0].rom_color().blue);
|
||||
}
|
||||
|
||||
// In DrawTileToBitmap() after palette calculation:
|
||||
printf("[Tile] ID=0x%03X pal_idx=%d offset=%d pixel[0]=%d\n",
|
||||
tile_info.id_, tile_info.palette_, palette_offset, tiledata[0]);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- **ALTTP Disassembly**: https://github.com/
|
||||
|
||||
spannerisms/ALTTPR-estrela
|
||||
- **ZScream Source**: DungeonObjectData.cs (C# implementation)
|
||||
- **yaze Graphics System**: CLAUDE.md Pattern 4 (Bitmap sync requirements)
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-21
|
||||
**Research By**: CLAUDE_CORE (zelda3-hacking-expert + backend-infra-engineer)
|
||||
**Status**: Ready for implementation
|
||||
@@ -0,0 +1,258 @@
|
||||
# Agent Documentation Audit Report
|
||||
|
||||
**Audit Date**: 2025-11-23
|
||||
**Auditor**: CLAUDE_DOCS (Documentation Janitor)
|
||||
**Total Files Reviewed**: 30 markdown files
|
||||
**Total Size**: 9,149 lines, ~175KB
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The `/docs/internal/agents/` directory contains valuable agent collaboration infrastructure but has accumulated task-specific documentation that should be **archived** (5 files), **consolidated** (3 file groups), and one **template** file that should remain. The coordination-board.md is oversized (83KB) and needs archival strategy.
|
||||
|
||||
**Key Findings**:
|
||||
- **3 Gemini-specific task prompts** (gemini-master, gemini3-overworld-fix, gemini-task-checklist) are completed or superseded
|
||||
- **2 Onboarding documents** (COLLABORATION_KICKOFF, CODEX_ONBOARDING) are one-time setup docs for past kickoffs
|
||||
- **1 Handoff document** (CLAUDE_AIINF_HANDOFF) documents a completed session handoff
|
||||
- **Core infrastructure documents** (coordination-board, personas, agent-architecture) should remain
|
||||
- **System reference documents** (gemini-overworld-system-reference, gemini-dungeon-system-reference) are valuable for context
|
||||
- **Initiative documents** (initiative-v040, initiative-test-slimdown) are active/in-progress
|
||||
|
||||
---
|
||||
|
||||
## File-by-File Audit Table
|
||||
|
||||
| File | Size | Relevance (1-5) | Status | Recommended Action | Justification |
|
||||
|------|------|-----------------|--------|-------------------|---------------|
|
||||
| **ACTIVE CORE** |
|
||||
| coordination-board.md | 83KB | 5 | ACTIVE | ARCHIVE OLD ENTRIES | Live coordination hub; archive entries >2 weeks old to separate file |
|
||||
| personas.md | 1.8KB | 5 | ACTIVE | KEEP | Defines CLAUDE_CORE, CLAUDE_AIINF, CLAUDE_DOCS, GEMINI_AUTOM personas |
|
||||
| agent-architecture.md | 14KB | 5 | ACTIVE | KEEP | Foundational reference for agent roles, capabilities, interaction patterns |
|
||||
| **ACTIVE INITIATIVES** |
|
||||
| initiative-v040.md | 8.4KB | 5 | ACTIVE | KEEP | Ongoing v0.4.0 development (SDL3, emulator accuracy); linked from coordination-board |
|
||||
| initiative-test-slimdown.md | 2.3KB | 4 | IN_PROGRESS | KEEP | Scoped test infrastructure work; referenced in coordination-board |
|
||||
| initiative-template.md | 1.3KB | 4 | ACTIVE | KEEP | Reusable template for future initiatives |
|
||||
| **ACTIVE COLLABORATION FRAMEWORK** |
|
||||
| claude-gemini-collaboration.md | 12KB | 4 | ACTIVE | KEEP | Documents Claude-Gemini teamwork structure; still relevant |
|
||||
| agent-leaderboard.md | 10KB | 3 | SEMI-ACTIVE | CONSIDER ARCHIVING | Gamification artifact from 2025-11-20; update score tracking to coordination-board |
|
||||
| **GEMINI TASK-SPECIFIC (COMPLETED/SUPERSEDED)** |
|
||||
| gemini-build-setup.md | 2.2KB | 3 | COMPLETED | ARCHIVE | Build guide for Gemini; superseded by docs/public/build/quick-reference.md |
|
||||
| gemini-master-prompt.md | 7.1KB | 2 | COMPLETED | ARCHIVE | Session context doc for Gemini session; work is complete (fixed ASM version checks) |
|
||||
| gemini3-overworld-fix-prompt.md | 5.4KB | 2 | COMPLETED | ARCHIVE | Specific bug fix prompt for overworld regression; issue resolved (commit aed7967e29) |
|
||||
| gemini-task-checklist.md | 6.5KB | 2 | COMPLETED | ARCHIVE | Gemini task checklist from 2025-11-20 session; all items completed or handed off |
|
||||
| gemini-overworld-reference.md | 7.0KB | 3 | REFERENCE | CONSOLIDATE | Duplicate info from gemini-overworld-system-reference.md; merge into system-reference |
|
||||
| **DUNGEON/OVERWORLD SYSTEM REFERENCES** |
|
||||
| gemini-overworld-system-reference.md | 11KB | 4 | REFERENCE | KEEP | Technical deep-dive for overworld system; valuable ongoing reference |
|
||||
| gemini-dungeon-system-reference.md | 14KB | 4 | REFERENCE | KEEP | Technical deep-dive for dungeon system; valuable ongoing reference |
|
||||
| **HANDOFF DOCUMENTS** |
|
||||
| CLAUDE_AIINF_HANDOFF.md | 7.3KB | 2 | COMPLETED | ARCHIVE | Session handoff from 2025-11-20; work documented in coordination-board |
|
||||
| **ONE-TIME SETUP/KICKOFF** |
|
||||
| COLLABORATION_KICKOFF.md | 5.3KB | 2 | COMPLETED | ARCHIVE | Kickoff for Claude-Gemini collaboration; framework now in place |
|
||||
| CODEX_ONBOARDING.md | 5.9KB | 2 | COMPLETED | ARCHIVE | Onboarding guide for Codex agent; role is now established |
|
||||
| **DEVELOPMENT GUIDES** |
|
||||
| overworld-agent-guide.md | 14KB | 3 | REFERENCE | CONSOLIDATE | Overlaps with gemini-overworld-system-reference.md; merge content |
|
||||
| ai-agent-debugging-guide.md | 22KB | 4 | ACTIVE | KEEP | Comprehensive debug reference for AI agents working on yaze |
|
||||
| ai-development-tools.md | 17KB | 4 | ACTIVE | KEEP | Development tools reference for AI agents |
|
||||
| ai-infrastructure-initiative.md | 11KB | 3 | REFERENCE | CONSIDER ARCHIVING | Infrastructure planning doc; mostly superseded by active initiatives |
|
||||
| **Z3ED DOCUMENTATION** |
|
||||
| z3ed-command-abstraction.md | 15KB | 3 | REFERENCE | CONSOLIDATE | CLI refactoring reference; merge with z3ed-refactoring.md |
|
||||
| z3ed-refactoring.md | 9.7KB | 3 | REFERENCE | CONSOLIDATE | CLI refactoring summary; merge with command-abstraction.md |
|
||||
| **INFRASTRUCTURE** |
|
||||
| CI-TEST-AUDIT-REPORT.md | 5.6KB | 2 | COMPLETED | ARCHIVE | Test audit from 2025-11-20; findings incorporated into CI/test docs |
|
||||
| filesystem-tool.md | 6.0KB | 2 | REFERENCE | ARCHIVE | Tool documentation; likely obsolete or incorporated elsewhere |
|
||||
| dev-assist-agent.md | 8.4KB | 3 | REFERENCE | REVIEW | Development assistant design doc; check if still relevant |
|
||||
| ai-modularity.md | 6.6KB | 3 | REFERENCE | REVIEW | Modularity initiative doc; check completion status |
|
||||
| gh-actions-remote.md | 1.6KB | 1 | REFERENCE | DELETE | GitHub Actions remote reference; likely outdated tool documentation |
|
||||
|
||||
---
|
||||
|
||||
## Consolidation Recommendations
|
||||
|
||||
### Group A: Gemini Overworld References (CONSOLIDATE)
|
||||
**Files to Merge**:
|
||||
- `gemini-overworld-reference.md` (7.0KB)
|
||||
- `gemini-overworld-system-reference.md` (11KB)
|
||||
- `overworld-agent-guide.md` (14KB)
|
||||
|
||||
**Action**: Keep `gemini-overworld-system-reference.md` as the authoritative reference. Archive the other two files.
|
||||
|
||||
**Reasoning**: All three files cover similar ground (overworld architecture, file structure, data models). System-reference is most comprehensive and is actively used by agents.
|
||||
|
||||
---
|
||||
|
||||
### Group B: z3ed Refactoring References (CONSOLIDATE)
|
||||
**Files to Merge**:
|
||||
- `z3ed-command-abstraction.md` (15KB)
|
||||
- `z3ed-refactoring.md` (9.7KB)
|
||||
|
||||
**Action**: Keep `z3ed-refactoring.md` as the summary. Move detailed command abstraction specifics to a new `z3ed-implementation-details.md` in `docs/internal/` (not agents/).
|
||||
|
||||
**Reasoning**: Refactoring is complete, but CLI architecture docs are valuable for future CLI work. Separate implementation details from agent coordination.
|
||||
|
||||
---
|
||||
|
||||
### Group C: Gemini Task-Specific Prompts (ARCHIVE)
|
||||
**Files to Archive**:
|
||||
- `gemini-master-prompt.md` (7.1KB)
|
||||
- `gemini3-overworld-fix-prompt.md` (5.4KB)
|
||||
- `gemini-task-checklist.md` (6.5KB)
|
||||
- `gemini-build-setup.md` (2.2KB)
|
||||
|
||||
**Action**: Move to `docs/internal/agents/archive/gemini-session-2025-11-20/` with a README explaining the session context.
|
||||
|
||||
**Reasoning**: These are session-specific task documents. The work they document is complete. They're valuable for understanding past sessions but shouldn't clutter the active agent docs directory.
|
||||
|
||||
---
|
||||
|
||||
### Group D: Session Handoffs & Kickoffs (ARCHIVE)
|
||||
**Files to Archive**:
|
||||
- `CLAUDE_AIINF_HANDOFF.md` (7.3KB)
|
||||
- `COLLABORATION_KICKOFF.md` (5.3KB)
|
||||
- `CODEX_ONBOARDING.md` (5.9KB)
|
||||
|
||||
**Action**: Move to `docs/internal/agents/archive/session-handoffs/` with dates in filenames.
|
||||
|
||||
**Reasoning**: One-time setup documents. The collaboration framework is now established and documented in `claude-gemini-collaboration.md`. Handoff information has been integrated into coordination-board.
|
||||
|
||||
---
|
||||
|
||||
### Group E: Completed Audits & Reports (ARCHIVE)
|
||||
**Files to Archive**:
|
||||
- `CI-TEST-AUDIT-REPORT.md` (5.6KB)
|
||||
|
||||
**Action**: Move to `docs/internal/agents/archive/reports/` with date.
|
||||
|
||||
**Reasoning**: Audit findings have been incorporated into active test documentation. The report itself is a historical artifact.
|
||||
|
||||
---
|
||||
|
||||
## Immediate Actions
|
||||
|
||||
### Priority 1: Resolve Coordination Board Size (CRITICAL)
|
||||
**Current**: 83KB file makes it unwieldy
|
||||
**Action**:
|
||||
1. Create `coordination-board-archive.md` in same directory
|
||||
2. Move entries older than 2 weeks to archive (keep last 60-80 entries, ~40KB max)
|
||||
3. Update coordination-board.md header with note about archival strategy
|
||||
4. Create a script to automate monthly archival
|
||||
|
||||
**Expected Result**: Faster file loads, easier to find current work
|
||||
|
||||
---
|
||||
|
||||
### Priority 2: Create Archive Structure
|
||||
```
|
||||
docs/internal/agents/
|
||||
├── archive/
|
||||
│ ├── gemini-session-2025-11-20/
|
||||
│ │ ├── README.md (context)
|
||||
│ │ ├── gemini-master-prompt.md
|
||||
│ │ ├── gemini3-overworld-fix-prompt.md
|
||||
│ │ ├── gemini-task-checklist.md
|
||||
│ │ └── gemini-build-setup.md
|
||||
│ ├── session-handoffs/
|
||||
│ │ ├── 2025-11-20-CLAUDE_AIINF_HANDOFF.md
|
||||
│ │ ├── 2025-11-20-COLLABORATION_KICKOFF.md
|
||||
│ │ └── 2025-11-20-CODEX_ONBOARDING.md
|
||||
│ └── reports/
|
||||
│ └── 2025-11-20-CI-TEST-AUDIT-REPORT.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Priority 3: Consolidate Overlapping Documents
|
||||
1. **Merge overworld guides**: Keep `gemini-overworld-system-reference.md`, archive others
|
||||
2. **Merge z3ed docs**: Keep `z3ed-refactoring.md`, consolidate implementation details
|
||||
3. **Review low-relevance files**: Check `dev-assist-agent.md`, `ai-modularity.md`, `ai-infrastructure-initiative.md`
|
||||
|
||||
---
|
||||
|
||||
## Files to Keep (Active Core)
|
||||
|
||||
These files should remain in `/docs/internal/agents/` with no changes:
|
||||
|
||||
| File | Reason |
|
||||
|------|--------|
|
||||
| `coordination-board.md` | Live coordination hub (after archival cleanup) |
|
||||
| `personas.md` | Defines agent roles and responsibilities |
|
||||
| `agent-architecture.md` | Foundational reference for agent systems |
|
||||
| `initiative-v040.md` | Active development initiative |
|
||||
| `initiative-test-slimdown.md` | Active development initiative |
|
||||
| `initiative-template.md` | Reusable template for future work |
|
||||
| `claude-gemini-collaboration.md` | Active team collaboration framework |
|
||||
| `gemini-overworld-system-reference.md` | Technical deep-dive (widely used) |
|
||||
| `gemini-dungeon-system-reference.md` | Technical deep-dive (widely used) |
|
||||
| `ai-agent-debugging-guide.md` | Debugging reference for agents |
|
||||
| `ai-development-tools.md` | Development tools reference |
|
||||
|
||||
---
|
||||
|
||||
## Files Requiring Further Review
|
||||
|
||||
These files need owner confirmation before archival:
|
||||
|
||||
| File | Status | Recommendation |
|
||||
|------|--------|-----------------|
|
||||
| `dev-assist-agent.md` | UNCLEAR | Contact owner to confirm if still active |
|
||||
| `ai-modularity.md` | UNCLEAR | Check if modularity initiative is complete |
|
||||
| `ai-infrastructure-initiative.md` | SEMI-ACTIVE | May be superseded by v0.4.0 initiative |
|
||||
| `agent-leaderboard.md` | SEMI-ACTIVE | Consider moving gamification tracking to coordination-board |
|
||||
| `gh-actions-remote.md` | LIKELY-OBSOLETE | Very small file; verify it's not actively referenced |
|
||||
|
||||
---
|
||||
|
||||
## Summary of Recommendations
|
||||
|
||||
### Files to Archive (7-8 files, ~45KB)
|
||||
- Gemini task-specific prompts (4 files)
|
||||
- Session handoffs and kickoffs (3 files)
|
||||
- Completed audit reports (1 file)
|
||||
|
||||
### Files to Consolidate (3 groups)
|
||||
- Overworld references: Keep system-reference.md, archive others
|
||||
- Z3ed references: Merge into single document
|
||||
- Review low-relevance infrastructure initiatives
|
||||
|
||||
### Files to Keep (11 files, ~80KB)
|
||||
- Core coordination and architecture files
|
||||
- Active initiatives
|
||||
- Technical deep-dives used by agents
|
||||
|
||||
### Structure Improvement
|
||||
- Create `archive/` subdirectory with documented substructure
|
||||
- Establish coordination-board.md archival strategy
|
||||
- Aim for <100KB total in active agents directory
|
||||
|
||||
---
|
||||
|
||||
## Implementation Timeline
|
||||
|
||||
**Week 1 (Nov 23-29)**:
|
||||
- [ ] Create archive directory structure
|
||||
- [ ] Move files to archive (priority: task-specific prompts)
|
||||
- [ ] Update cross-references in remaining docs
|
||||
|
||||
**Week 2 (Nov 30-Dec 6)**:
|
||||
- [ ] Archive coordination-board entries (manual or scripted)
|
||||
- [ ] Consolidate overlapping system references
|
||||
- [ ] Review unclear files with owners
|
||||
|
||||
**Ongoing**:
|
||||
- [ ] Implement monthly coordination-board.md archival
|
||||
- [ ] Keep active initiatives up-to-date
|
||||
|
||||
---
|
||||
|
||||
## Notes for Future Archival
|
||||
|
||||
When adding new agent documentation:
|
||||
|
||||
1. **Session-specific docs** (task prompts, handoffs, prompts) → Archive after completion
|
||||
2. **One-time setup docs** (kickoffs, onboarding) → Archive after 2 weeks
|
||||
3. **Active infrastructure** (coordination, personas, initiatives) → Keep in root
|
||||
4. **System references** (architecture, debugging) → Keep if actively used by agents
|
||||
5. **Completed work reports** → Archive with date in filename
|
||||
|
||||
**Target**: Keep `/docs/internal/agents/` under 100KB with ~15-20 active files. Move everything else to `archive/`.
|
||||
|
||||
@@ -48,7 +48,7 @@ Established three-agent team:
|
||||
- **Codex (CODEX)**: Documentation, coordination, QA, organization
|
||||
|
||||
Files created:
|
||||
- `docs/internal/agents/agent-leaderboard.md` - Competitive tracking
|
||||
- `docs/internal/agents/archive/legacy-2025-11/agent-leaderboard-archived-2025-11-25.md` - Competitive tracking
|
||||
- `docs/internal/agents/claude-gemini-collaboration.md` - Collaboration framework
|
||||
- `docs/internal/agents/CODEX_ONBOARDING.md` - Codex welcome guide
|
||||
- `docs/internal/agents/coordination-board.md` - Updated with team assignments
|
||||
@@ -156,7 +156,7 @@ Files created:
|
||||
|
||||
### Key Documents
|
||||
- **Coordination Board**: `docs/internal/agents/coordination-board.md`
|
||||
- **Leaderboard**: `docs/internal/agents/agent-leaderboard.md`
|
||||
- **Leaderboard**: `docs/internal/agents/archive/legacy-2025-11/agent-leaderboard-archived-2025-11-25.md`
|
||||
- **Collaboration Guide**: `docs/internal/agents/claude-gemini-collaboration.md`
|
||||
- **Testing Docs**: `docs/internal/testing/README.md`
|
||||
|
||||
@@ -149,7 +149,7 @@ Welcome aboard! Claude and Gemini have been duking it out fixing critical build
|
||||
## Getting Started
|
||||
|
||||
1. **Read the coordination board**: `docs/internal/agents/coordination-board.md`
|
||||
2. **Check the leaderboard**: `docs/internal/agents/agent-leaderboard.md`
|
||||
2. **Check the leaderboard**: `docs/internal/agents/archive/legacy-2025-11/agent-leaderboard-archived-2025-11-25.md`
|
||||
3. **Pick a task** from the list above (start with Documentation Cleanup)
|
||||
4. **Post on coordination board** when you start/finish tasks
|
||||
5. **Join the friendly rivalry** - may the best AI win! 🏆
|
||||
@@ -12,7 +12,7 @@ Accelerate yaze release by combining Claude's architectural expertise with Gemin
|
||||
|
||||
### Documents Created
|
||||
|
||||
1. **Agent Leaderboard** (`docs/internal/agents/agent-leaderboard.md`)
|
||||
1. **Agent Leaderboard** (`docs/internal/agents/archive/legacy-2025-11/agent-leaderboard-archived-2025-11-25.md`)
|
||||
- Objective scoring system (points based on impact)
|
||||
- Current scores: Claude 725 pts, Gemini 90 pts
|
||||
- Friendly trash talk section
|
||||
@@ -144,7 +144,7 @@ Accelerate yaze release by combining Claude's architectural expertise with Gemin
|
||||
|
||||
## Resources
|
||||
|
||||
- **Leaderboard**: `docs/internal/agents/agent-leaderboard.md`
|
||||
- **Leaderboard**: `docs/internal/agents/archive/legacy-2025-11/agent-leaderboard-archived-2025-11-25.md`
|
||||
- **Framework**: `docs/internal/agents/claude-gemini-collaboration.md`
|
||||
- **Coordination**: `docs/internal/agents/coordination-board.md`
|
||||
- **CI Status Script**: `scripts/agents/get-gh-workflow-status.sh`
|
||||
142
docs/internal/agents/archive/testing-docs-2025/archive-index.md
Normal file
142
docs/internal/agents/archive/testing-docs-2025/archive-index.md
Normal file
@@ -0,0 +1,142 @@
|
||||
# Testing Documentation Archive (November 2025)
|
||||
|
||||
This directory contains testing-related documentation that was archived during a comprehensive cleanup of `/docs/internal/testing/` to reduce duplication and improve maintainability.
|
||||
|
||||
## Archive Rationale
|
||||
|
||||
The testing directory contained 25 markdown files with significant duplication of content from:
|
||||
- `test/README.md` - The canonical test suite documentation
|
||||
- `docs/public/build/quick-reference.md` - The canonical build reference
|
||||
- `docs/internal/ci-and-testing.md` - CI/CD pipeline documentation
|
||||
|
||||
## Archived Files (6 total)
|
||||
|
||||
### Bloated/Redundant Documentation
|
||||
|
||||
1. **testing-strategy.md** (843 lines)
|
||||
- Duplicates the tiered testing strategy from `test/README.md`
|
||||
- Reason: Content moved to canonical test/README.md
|
||||
- Reference: See test/README.md for current strategy
|
||||
|
||||
2. **TEST_INFRASTRUCTURE_IMPROVEMENT_PLAN.md** (2257 lines)
|
||||
- Massive improvement proposal document
|
||||
- Duplicates much of test/README.md and docs/internal/ci-and-testing.md
|
||||
- Reason: Content integrated into existing canonical docs
|
||||
- Reference: Implementation recommendations are in docs/internal/ci-and-testing.md
|
||||
|
||||
3. **ci-improvements-proposal.md** (690 lines)
|
||||
- Detailed CI/CD improvement proposals
|
||||
- Overlaps significantly with docs/internal/ci-and-testing.md
|
||||
- Reason: Improvements documented in canonical CI/testing doc
|
||||
- Reference: See docs/internal/ci-and-testing.md
|
||||
|
||||
4. **cmake-validation.md** (672 lines)
|
||||
- CMake validation guide
|
||||
- Duplicates content from docs/public/build/quick-reference.md
|
||||
- Reason: Build validation covered in quick-reference.md
|
||||
- Reference: See docs/public/build/quick-reference.md
|
||||
|
||||
5. **integration-plan.md** (505 lines)
|
||||
- Testing infrastructure integration planning document
|
||||
- Much of content duplicated in test/README.md
|
||||
- Reason: Integration approach implemented and documented elsewhere
|
||||
- Reference: See test/README.md for current integration approach
|
||||
|
||||
6. **matrix-testing-strategy.md** (499 lines)
|
||||
- Platform/configuration matrix testing strategy
|
||||
- Some unique content but much is duplicated in other docs
|
||||
- Reason: Matrix testing implementation is in scripts/
|
||||
- Reference: Check scripts/test-config-matrix.sh and related scripts
|
||||
|
||||
## Deleted Files (14 total - Already in git staging)
|
||||
|
||||
These files were completely duplicative and offered no unique value:
|
||||
|
||||
1. **QUICKSTART.md** - Exact duplicate of QUICK_START_GUIDE.md
|
||||
2. **QUICK_START_GUIDE.md** - Duplicates test/README.md Quick Start section
|
||||
3. **QUICK_REFERENCE.md** - Redundant quick reference for symbol detection
|
||||
4. **README_TESTING.md** - Duplicate hub documentation
|
||||
5. **TESTING_INDEX.md** - Navigation index (redundant)
|
||||
6. **ARCHITECTURE_HANDOFF.md** - AI-generated project status document
|
||||
7. **INITIATIVE.md** - AI-generated project initiative document
|
||||
8. **EXECUTIVE_SUMMARY.md** - AI-generated executive summary
|
||||
9. **IMPLEMENTATION_GUIDE.md** - Symbol detection implementation guide (superseded)
|
||||
10. **MATRIX_TESTING_README.md** - Matrix testing system documentation
|
||||
11. **MATRIX_TESTING_IMPLEMENTATION.md** - Matrix testing implementation guide
|
||||
12. **MATRIX_TESTING_CHECKLIST.md** - Matrix testing checklist
|
||||
13. **SYMBOL_DETECTION_README.md** - Duplicate of symbol-conflict-detection.md
|
||||
14. **TEST_INFRASTRUCTURE_IMPROVEMENT_PLAN.md** - (see archived files above)
|
||||
|
||||
## Files Retained (5 total in docs/internal/testing/)
|
||||
|
||||
1. **dungeon-gui-test-design.md** (1007 lines)
|
||||
- Unique architectural test design for dungeon editor
|
||||
- Specific to DungeonEditorV2 testing with ImGuiTestEngine
|
||||
- Rationale: Contains unique architectural and testing patterns not found elsewhere
|
||||
|
||||
2. **pre-push-checklist.md** (335 lines)
|
||||
- Practical developer checklist for pre-commit validation
|
||||
- Links to scripts and CI verification
|
||||
- Rationale: Useful operational checklist referenced by developers
|
||||
|
||||
3. **README.md** (414 lines)
|
||||
- Hub documentation for testing infrastructure
|
||||
- Links to canonical testing documents and resources
|
||||
- Rationale: Serves as navigation hub to various testing documents
|
||||
|
||||
4. **symbol-conflict-detection.md** (440 lines)
|
||||
- Complete documentation for symbol conflict detection system
|
||||
- Details on symbol extraction, detection, and pre-commit hooks
|
||||
- Rationale: Complete reference for symbol conflict system
|
||||
|
||||
5. **sample-symbol-database.json** (1133 bytes)
|
||||
- Example JSON database for symbol conflict detection
|
||||
- Supporting documentation for symbol system
|
||||
- Rationale: Example data for understanding symbol database format
|
||||
|
||||
## Canonical Documentation References
|
||||
|
||||
When working with testing, refer to these canonical sources:
|
||||
|
||||
- **Test Suite Overview**: `test/README.md` (407 lines)
|
||||
- Tiered testing strategy, test structure, running tests
|
||||
- How to write new tests, CI configuration
|
||||
|
||||
- **Build & Test Quick Reference**: `docs/public/build/quick-reference.md`
|
||||
- CMake presets, common build commands
|
||||
- Test execution quick reference
|
||||
|
||||
- **CI/CD Pipeline**: `docs/internal/ci-and-testing.md`
|
||||
- CI workflow configuration, test infrastructure
|
||||
- GitHub Actions integration
|
||||
|
||||
- **CLAUDE.md**: Project root CLAUDE.md
|
||||
- References canonical test documentation
|
||||
- Links to quick-reference.md and test/README.md
|
||||
|
||||
## How to Restore
|
||||
|
||||
If you need to reference archived content:
|
||||
|
||||
```bash
|
||||
# View specific archived document
|
||||
cat docs/internal/agents/archive/testing-docs-2025/testing-strategy.md
|
||||
|
||||
# Restore if needed
|
||||
mv docs/internal/agents/archive/testing-docs-2025/<filename>.md docs/internal/testing/
|
||||
```
|
||||
|
||||
## Cleanup Results
|
||||
|
||||
- **Before**: 25 markdown files (12,170 total lines)
|
||||
- **After**: 5 markdown files (2,943 total lines)
|
||||
- **Reduction**: 75.8% fewer files, 75.8% fewer lines
|
||||
- **Result**: Cleaner documentation structure, easier to maintain, reduced duplication
|
||||
|
||||
## Related Cleanup
|
||||
|
||||
This cleanup was performed as part of documentation janitor work to:
|
||||
- Remove AI-generated spam and duplicate documentation
|
||||
- Enforce single source of truth for each documentation topic
|
||||
- Keep root documentation directory clean
|
||||
- Maintain clear, authoritative documentation structure
|
||||
@@ -0,0 +1,690 @@
|
||||
# CI/CD Improvements Proposal
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document proposes specific improvements to the YAZE CI/CD pipeline to catch build failures earlier, reduce wasted CI time, and provide faster feedback to developers.
|
||||
|
||||
**Goals**:
|
||||
- Reduce time-to-first-failure from ~15 minutes to <5 minutes
|
||||
- Catch 90% of failures in fast jobs (<5 min)
|
||||
- Reduce PR iteration time from hours to minutes
|
||||
- Prevent platform-specific issues from reaching CI
|
||||
|
||||
**ROI**:
|
||||
- **Time Saved**: ~10 minutes per failed build × ~30 failures/month = **5 hours/month**
|
||||
- **Developer Experience**: Faster feedback → less context switching
|
||||
- **CI Cost**: Minimal (fast jobs use fewer resources)
|
||||
|
||||
---
|
||||
|
||||
## Current CI Pipeline Analysis
|
||||
|
||||
### Current Jobs
|
||||
|
||||
| Job | Platform | Duration | Cost | Catches |
|
||||
|-----|----------|----------|------|---------|
|
||||
| build | Ubuntu/macOS/Windows | 15-20 min | High | Compilation errors |
|
||||
| test | Ubuntu/macOS/Windows | 5 min | Medium | Test failures |
|
||||
| windows-agent | Windows | 30 min | High | AI stack issues |
|
||||
| code-quality | Ubuntu | 2 min | Low | Format/lint issues |
|
||||
| memory-sanitizer | Ubuntu | 20 min | High | Memory bugs |
|
||||
| z3ed-agent-test | macOS | 15 min | High | Agent integration |
|
||||
|
||||
**Total PR Time**: ~40 minutes (parallel), ~90 minutes (worst case)
|
||||
|
||||
### Issues with Current Pipeline
|
||||
|
||||
1. **Long feedback loop**: 15-20 minutes to find out if headers are missing
|
||||
2. **Wasted resources**: Full 20-minute builds that fail in first 2 minutes
|
||||
3. **No early validation**: CMake configuration succeeds, but compilation fails later
|
||||
4. **Symbol conflicts detected late**: Link errors only appear after full compile
|
||||
5. **Platform-specific issues**: Discovered after 15+ minutes per platform
|
||||
|
||||
---
|
||||
|
||||
## Proposed Improvements
|
||||
|
||||
### Improvement 1: Configuration Validation Job
|
||||
|
||||
**Goal**: Catch CMake errors in <2 minutes
|
||||
|
||||
**Implementation**:
|
||||
```yaml
|
||||
config-validation:
|
||||
name: "Config Validation - ${{ matrix.preset }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: true # Stop immediately if any fails
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-22.04
|
||||
preset: ci-linux
|
||||
- os: macos-14
|
||||
preset: ci-macos
|
||||
- os: windows-2022
|
||||
preset: ci-windows
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup build environment
|
||||
uses: ./.github/actions/setup-build
|
||||
with:
|
||||
platform: ${{ matrix.platform }}
|
||||
preset: ${{ matrix.preset }}
|
||||
|
||||
- name: Validate CMake configuration
|
||||
run: |
|
||||
cmake --preset ${{ matrix.preset }} \
|
||||
-DCMAKE_VERBOSE_MAKEFILE=OFF
|
||||
|
||||
- name: Check include paths
|
||||
run: |
|
||||
grep "INCLUDE_DIRECTORIES" build/CMakeCache.txt || \
|
||||
(echo "Include paths not configured" && exit 1)
|
||||
|
||||
- name: Validate presets
|
||||
run: cmake --preset ${{ matrix.preset }} --list-presets
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- ✅ Fails in <2 minutes for CMake errors
|
||||
- ✅ Catches missing dependencies immediately
|
||||
- ✅ Validates include path propagation
|
||||
- ✅ Low resource usage (no compilation)
|
||||
|
||||
**What it catches**:
|
||||
- CMake syntax errors
|
||||
- Missing dependencies (immediate)
|
||||
- Invalid preset definitions
|
||||
- Include path misconfiguration
|
||||
|
||||
---
|
||||
|
||||
### Improvement 2: Compile-Only Job
|
||||
|
||||
**Goal**: Catch compilation errors in <5 minutes
|
||||
|
||||
**Implementation**:
|
||||
```yaml
|
||||
compile-check:
|
||||
name: "Compile Check - ${{ matrix.preset }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
needs: [config-validation] # Run after config validation passes
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-22.04
|
||||
preset: ci-linux
|
||||
platform: linux
|
||||
- os: macos-14
|
||||
preset: ci-macos
|
||||
platform: macos
|
||||
- os: windows-2022
|
||||
preset: ci-windows
|
||||
platform: windows
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup build environment
|
||||
uses: ./.github/actions/setup-build
|
||||
with:
|
||||
platform: ${{ matrix.platform }}
|
||||
preset: ${{ matrix.preset }}
|
||||
|
||||
- name: Configure project
|
||||
run: cmake --preset ${{ matrix.preset }}
|
||||
|
||||
- name: Compile representative files
|
||||
run: |
|
||||
# Compile 10-20 key files to catch most header issues
|
||||
cmake --build build --target rom.cc.o bitmap.cc.o \
|
||||
overworld.cc.o resource_catalog.cc.o \
|
||||
dungeon.cc.o sprite.cc.o palette.cc.o \
|
||||
asar_wrapper.cc.o controller.cc.o canvas.cc.o \
|
||||
--parallel 4
|
||||
|
||||
- name: Check for common issues
|
||||
run: |
|
||||
# Platform-specific checks
|
||||
if [ "${{ matrix.platform }}" = "windows" ]; then
|
||||
echo "Checking for /std:c++latest flag..."
|
||||
grep "std:c++latest" build/compile_commands.json || \
|
||||
echo "Warning: C++20 flag may be missing"
|
||||
fi
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- ✅ Catches header issues in ~5 minutes
|
||||
- ✅ Tests actual compilation without full build
|
||||
- ✅ Platform-specific early detection
|
||||
- ✅ ~70% faster than full build
|
||||
|
||||
**What it catches**:
|
||||
- Missing headers
|
||||
- Include path problems
|
||||
- Preprocessor errors
|
||||
- Template instantiation issues
|
||||
- Platform-specific compilation errors
|
||||
|
||||
---
|
||||
|
||||
### Improvement 3: Symbol Conflict Job
|
||||
|
||||
**Goal**: Detect ODR violations before linking
|
||||
|
||||
**Implementation**:
|
||||
```yaml
|
||||
symbol-check:
|
||||
name: "Symbol Check - ${{ matrix.platform }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
needs: [build] # Run after full build completes
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-22.04
|
||||
platform: linux
|
||||
- os: macos-14
|
||||
platform: macos
|
||||
- os: windows-2022
|
||||
platform: windows
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Download build artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: build-${{ matrix.platform }}
|
||||
path: build
|
||||
|
||||
- name: Check for symbol conflicts (Unix)
|
||||
if: matrix.platform != 'windows'
|
||||
run: ./scripts/verify-symbols.sh --build-dir build
|
||||
|
||||
- name: Check for symbol conflicts (Windows)
|
||||
if: matrix.platform == 'windows'
|
||||
shell: pwsh
|
||||
run: .\scripts\verify-symbols.ps1 -BuildDir build
|
||||
|
||||
- name: Upload conflict report
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: symbol-conflicts-${{ matrix.platform }}
|
||||
path: build/symbol-report.txt
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- ✅ Catches ODR violations before linking
|
||||
- ✅ Detects FLAGS conflicts (Linux-specific)
|
||||
- ✅ Platform-specific symbol issues
|
||||
- ✅ Runs in parallel with tests (~3 minutes)
|
||||
|
||||
**What it catches**:
|
||||
- Duplicate symbol definitions
|
||||
- FLAGS_* conflicts (gflags)
|
||||
- ODR violations
|
||||
- Link-time errors (predicted)
|
||||
|
||||
---
|
||||
|
||||
### Improvement 4: Fail-Fast Strategy
|
||||
|
||||
**Goal**: Stop wasting resources on doomed builds
|
||||
|
||||
**Current Behavior**: All jobs run even if one fails
|
||||
**Proposed Behavior**: Stop non-essential jobs if critical jobs fail
|
||||
|
||||
**Implementation**:
|
||||
```yaml
|
||||
jobs:
|
||||
# Critical path: These must pass
|
||||
config-validation:
|
||||
# ... (as above)
|
||||
|
||||
compile-check:
|
||||
needs: [config-validation]
|
||||
strategy:
|
||||
fail-fast: true # Stop all platforms if one fails
|
||||
|
||||
build:
|
||||
needs: [compile-check]
|
||||
strategy:
|
||||
fail-fast: false # Allow other platforms to continue
|
||||
|
||||
# Non-critical: These can be skipped if builds fail
|
||||
integration-tests:
|
||||
needs: [build]
|
||||
if: success() # Only run if build succeeded
|
||||
|
||||
windows-agent:
|
||||
needs: [build, test]
|
||||
if: success() && github.event_name != 'pull_request'
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- ✅ Saves ~60 minutes of CI time per failed build
|
||||
- ✅ Faster feedback (no waiting for doomed jobs)
|
||||
- ✅ Reduced resource usage
|
||||
|
||||
---
|
||||
|
||||
### Improvement 5: Preset Matrix Testing
|
||||
|
||||
**Goal**: Validate all presets can configure
|
||||
|
||||
**Implementation**:
|
||||
```yaml
|
||||
preset-validation:
|
||||
name: "Preset Validation"
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-22.04, macos-14, windows-2022]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Test all presets for platform
|
||||
run: |
|
||||
for preset in $(cmake --list-presets | grep ${{ matrix.os }} | awk '{print $1}'); do
|
||||
echo "Testing preset: $preset"
|
||||
cmake --preset "$preset" --list-presets || exit 1
|
||||
done
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- ✅ Catches invalid preset definitions
|
||||
- ✅ Validates CMake configuration across all presets
|
||||
- ✅ Fast (<2 minutes)
|
||||
|
||||
---
|
||||
|
||||
## Proposed CI Pipeline (New)
|
||||
|
||||
### Job Dependencies
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ config-validation │ (2 min, fail-fast)
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ compile-check │ (5 min, fail-fast)
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ build │ (15 min, parallel)
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
├──────────┬──────────┬──────────┐
|
||||
▼ ▼ ▼ ▼
|
||||
┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐
|
||||
│ test │ │ symbol │ │quality │ │sanitize│
|
||||
│ (5 min)│ │(3 min) │ │(2 min) │ │(20 min)│
|
||||
└────────┘ └────────┘ └────────┘ └────────┘
|
||||
```
|
||||
|
||||
### Time Comparison
|
||||
|
||||
**Current Pipeline**:
|
||||
- First failure: ~15 minutes (compilation error)
|
||||
- Total time: ~40 minutes (if all succeed)
|
||||
|
||||
**Proposed Pipeline**:
|
||||
- First failure: ~2 minutes (CMake error) or ~5 minutes (compilation error)
|
||||
- Total time: ~40 minutes (if all succeed)
|
||||
|
||||
**Time Saved**:
|
||||
- CMake errors: **13 minutes saved** (15 min → 2 min)
|
||||
- Compilation errors: **10 minutes saved** (15 min → 5 min)
|
||||
- Symbol conflicts: **Caught earlier** (no failed PRs)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Quick Wins (Week 1)
|
||||
|
||||
1. **Add config-validation job**
|
||||
- Copy composite actions
|
||||
- Add new job to `ci.yml`
|
||||
- Test on feature branch
|
||||
|
||||
2. **Add symbol-check script**
|
||||
- Already created: `scripts/verify-symbols.sh`
|
||||
- Add Windows version: `scripts/verify-symbols.ps1`
|
||||
- Test locally
|
||||
|
||||
3. **Update job dependencies**
|
||||
- Make `build` depend on `config-validation`
|
||||
- Add fail-fast to compile-check
|
||||
|
||||
**Deliverables**:
|
||||
- ✅ Config validation catches CMake errors in <2 min
|
||||
- ✅ Symbol checker available for CI
|
||||
- ✅ Fail-fast prevents wasted CI time
|
||||
|
||||
### Phase 2: Compilation Checks (Week 2)
|
||||
|
||||
1. **Add compile-check job**
|
||||
- Identify representative files
|
||||
- Create compilation target list
|
||||
- Add to CI workflow
|
||||
|
||||
2. **Platform-specific smoke tests**
|
||||
- Windows: Check `/std:c++latest`
|
||||
- Linux: Check `-std=c++20`
|
||||
- macOS: Check framework links
|
||||
|
||||
**Deliverables**:
|
||||
- ✅ Compilation errors caught in <5 min
|
||||
- ✅ Platform-specific issues detected early
|
||||
|
||||
### Phase 3: Symbol Validation (Week 3)
|
||||
|
||||
1. **Add symbol-check job**
|
||||
- Integrate `verify-symbols.sh`
|
||||
- Upload conflict reports
|
||||
- Add to required checks
|
||||
|
||||
2. **Create symbol conflict guide**
|
||||
- Document common issues
|
||||
- Provide fix examples
|
||||
- Link from CI failures
|
||||
|
||||
**Deliverables**:
|
||||
- ✅ ODR violations caught before merge
|
||||
- ✅ FLAGS conflicts detected automatically
|
||||
|
||||
### Phase 4: Optimization (Week 4)
|
||||
|
||||
1. **Fine-tune fail-fast**
|
||||
- Identify critical vs optional jobs
|
||||
- Set up conditional execution
|
||||
- Test resource savings
|
||||
|
||||
2. **Add caching improvements**
|
||||
- Cache compiled objects
|
||||
- Share artifacts between jobs
|
||||
- Optimize dependency downloads
|
||||
|
||||
**Deliverables**:
|
||||
- ✅ ~60 minutes CI time saved per failed build
|
||||
- ✅ Faster PR iteration
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Before Improvements
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Time to first failure | 15-20 min |
|
||||
| CI failures per month | ~30 |
|
||||
| Wasted CI time/month | ~8 hours |
|
||||
| PR iteration time | 2-4 hours |
|
||||
| Symbol conflicts caught | 0% (manual) |
|
||||
|
||||
### After Improvements (Target)
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Time to first failure | **2-5 min** |
|
||||
| CI failures per month | **<10** |
|
||||
| Wasted CI time/month | **<2 hours** |
|
||||
| PR iteration time | **30-60 min** |
|
||||
| Symbol conflicts caught | **100%** |
|
||||
|
||||
### ROI Calculation
|
||||
|
||||
**Time Savings**:
|
||||
- 20 failures/month × 10 min saved = **200 minutes/month**
|
||||
- 10 failed PRs avoided = **~4 hours/month**
|
||||
- **Total: ~5-6 hours/month saved**
|
||||
|
||||
**Developer Experience**:
|
||||
- Faster feedback → less context switching
|
||||
- Earlier error detection → easier debugging
|
||||
- Fewer CI failures → less frustration
|
||||
|
||||
---
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
### Risk 1: False Positives
|
||||
**Risk**: New checks catch issues that aren't real problems
|
||||
**Mitigation**:
|
||||
- Test thoroughly before enabling as required
|
||||
- Allow overrides for known false positives
|
||||
- Iterate on filtering logic
|
||||
|
||||
### Risk 2: Increased Complexity
|
||||
**Risk**: More jobs = harder to understand CI failures
|
||||
**Mitigation**:
|
||||
- Clear job names and descriptions
|
||||
- Good error messages with links to docs
|
||||
- Dependency graph visualization
|
||||
|
||||
### Risk 3: Slower PR Merges
|
||||
**Risk**: More required checks = slower to merge
|
||||
**Mitigation**:
|
||||
- Make only critical checks required
|
||||
- Run expensive checks post-merge
|
||||
- Provide override mechanism for emergencies
|
||||
|
||||
---
|
||||
|
||||
## Alternative Approaches Considered
|
||||
|
||||
### Approach 1: Pre-commit Hooks
|
||||
**Pros**: Catch issues before pushing
|
||||
**Cons**: Developers can skip, not enforced
|
||||
**Decision**: Provide optional hooks, but rely on CI
|
||||
|
||||
### Approach 2: GitHub Actions Matrix Expansion
|
||||
**Pros**: Test more combinations
|
||||
**Cons**: Significantly more CI time
|
||||
**Decision**: Focus on critical paths, expand later if needed
|
||||
|
||||
### Approach 3: Self-Hosted Runners
|
||||
**Pros**: Faster builds, more control
|
||||
**Cons**: Maintenance overhead, security concerns
|
||||
**Decision**: Stick with GitHub runners for now
|
||||
|
||||
---
|
||||
|
||||
## Related Work
|
||||
|
||||
### Similar Implementations
|
||||
- **LLVM Project**: Uses compile-only jobs for fast feedback
|
||||
- **Chromium**: Extensive smoke testing before full builds
|
||||
- **Abseil**: Symbol conflict detection in CI
|
||||
|
||||
### Best Practices
|
||||
1. **Fail Fast**: Stop early if critical checks fail
|
||||
2. **Layered Testing**: Quick checks first, expensive checks later
|
||||
3. **Clear Feedback**: Good error messages with actionable advice
|
||||
4. **Caching**: Reuse work across jobs when possible
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: New CI Jobs (YAML)
|
||||
|
||||
### Config Validation Job
|
||||
```yaml
|
||||
config-validation:
|
||||
name: "Config Validation - ${{ matrix.name }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: true
|
||||
matrix:
|
||||
include:
|
||||
- name: "Ubuntu 22.04"
|
||||
os: ubuntu-22.04
|
||||
preset: ci-linux
|
||||
platform: linux
|
||||
- name: "macOS 14"
|
||||
os: macos-14
|
||||
preset: ci-macos
|
||||
platform: macos
|
||||
- name: "Windows 2022"
|
||||
os: windows-2022
|
||||
preset: ci-windows
|
||||
platform: windows
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup build environment
|
||||
uses: ./.github/actions/setup-build
|
||||
with:
|
||||
platform: ${{ matrix.platform }}
|
||||
preset: ${{ matrix.preset }}
|
||||
|
||||
- name: Validate CMake configuration
|
||||
run: cmake --preset ${{ matrix.preset }}
|
||||
|
||||
- name: Check configuration
|
||||
shell: bash
|
||||
run: |
|
||||
# Check include paths
|
||||
grep "INCLUDE_DIRECTORIES" build/CMakeCache.txt
|
||||
|
||||
# Check preset is valid
|
||||
cmake --preset ${{ matrix.preset }} --list-presets
|
||||
```
|
||||
|
||||
### Compile Check Job
|
||||
```yaml
|
||||
compile-check:
|
||||
name: "Compile Check - ${{ matrix.name }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
needs: [config-validation]
|
||||
strategy:
|
||||
fail-fast: true
|
||||
matrix:
|
||||
include:
|
||||
- name: "Ubuntu 22.04"
|
||||
os: ubuntu-22.04
|
||||
preset: ci-linux
|
||||
platform: linux
|
||||
- name: "macOS 14"
|
||||
os: macos-14
|
||||
preset: ci-macos
|
||||
platform: macos
|
||||
- name: "Windows 2022"
|
||||
os: windows-2022
|
||||
preset: ci-windows
|
||||
platform: windows
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Setup build environment
|
||||
uses: ./.github/actions/setup-build
|
||||
with:
|
||||
platform: ${{ matrix.platform }}
|
||||
preset: ${{ matrix.preset }}
|
||||
|
||||
- name: Configure project
|
||||
run: cmake --preset ${{ matrix.preset }}
|
||||
|
||||
- name: Smoke compilation test
|
||||
shell: bash
|
||||
run: ./scripts/pre-push-test.sh --smoke-only --preset ${{ matrix.preset }}
|
||||
```
|
||||
|
||||
### Symbol Check Job
|
||||
```yaml
|
||||
symbol-check:
|
||||
name: "Symbol Check - ${{ matrix.name }}"
|
||||
runs-on: ${{ matrix.os }}
|
||||
needs: [build]
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- name: "Ubuntu 22.04"
|
||||
os: ubuntu-22.04
|
||||
platform: linux
|
||||
- name: "macOS 14"
|
||||
os: macos-14
|
||||
platform: macos
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Download build artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: build-${{ matrix.platform }}
|
||||
path: build
|
||||
|
||||
- name: Check for symbol conflicts
|
||||
shell: bash
|
||||
run: ./scripts/verify-symbols.sh --build-dir build
|
||||
|
||||
- name: Upload conflict report
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: symbol-conflicts-${{ matrix.platform }}
|
||||
path: build/symbol-report.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: Cost Analysis
|
||||
|
||||
### Current Monthly CI Usage (Estimated)
|
||||
|
||||
| Job | Duration | Runs/Month | Total Time |
|
||||
|-----|----------|------------|------------|
|
||||
| build (3 platforms) | 15 min × 3 | 100 PRs | **75 hours** |
|
||||
| test (3 platforms) | 5 min × 3 | 100 PRs | **25 hours** |
|
||||
| windows-agent | 30 min | 30 | **15 hours** |
|
||||
| code-quality | 2 min | 100 PRs | **3.3 hours** |
|
||||
| memory-sanitizer | 20 min | 50 PRs | **16.7 hours** |
|
||||
| z3ed-agent-test | 15 min | 30 | **7.5 hours** |
|
||||
| **Total** | | | **142.5 hours** |
|
||||
|
||||
### Proposed Monthly CI Usage
|
||||
|
||||
| Job | Duration | Runs/Month | Total Time |
|
||||
|-----|----------|------------|------------|
|
||||
| config-validation (3) | 2 min × 3 | 100 PRs | **10 hours** |
|
||||
| compile-check (3) | 5 min × 3 | 100 PRs | **25 hours** |
|
||||
| build (3 platforms) | 15 min × 3 | 80 PRs | **60 hours** (↓20%) |
|
||||
| test (3 platforms) | 5 min × 3 | 80 PRs | **20 hours** (↓20%) |
|
||||
| symbol-check (2) | 3 min × 2 | 80 PRs | **8 hours** |
|
||||
| windows-agent | 30 min | 25 | **12.5 hours** (↓17%) |
|
||||
| code-quality | 2 min | 100 PRs | **3.3 hours** |
|
||||
| memory-sanitizer | 20 min | 40 PRs | **13.3 hours** (↓20%) |
|
||||
| z3ed-agent-test | 15 min | 25 | **6.25 hours** (↓17%) |
|
||||
| **Total** | | | **158.4 hours** (+11%) |
|
||||
|
||||
**Net Change**: +16 hours/month (11% increase)
|
||||
|
||||
**BUT**:
|
||||
- Fewer failed builds (20% reduction)
|
||||
- Faster feedback (10-15 min saved per failure)
|
||||
- Better developer experience (invaluable)
|
||||
|
||||
**Conclusion**: Slight increase in total CI time, but significant improvement in efficiency and developer experience
|
||||
@@ -0,0 +1,672 @@
|
||||
# CMake Configuration Validation
|
||||
|
||||
Comprehensive guide to validating CMake configuration and catching dependency issues early.
|
||||
|
||||
## Overview
|
||||
|
||||
The CMake validation toolkit provides four powerful tools to catch configuration issues before they cause build failures:
|
||||
|
||||
1. **validate-cmake-config.cmake** - Validates CMake cache and configuration
|
||||
2. **check-include-paths.sh** - Verifies include paths in compile commands
|
||||
3. **visualize-deps.py** - Generates dependency graphs
|
||||
4. **test-cmake-presets.sh** - Tests all CMake presets
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Validate configuration after running cmake
|
||||
cmake --preset mac-dbg
|
||||
cmake -P scripts/validate-cmake-config.cmake build
|
||||
|
||||
# 2. Check include paths
|
||||
./scripts/check-include-paths.sh build
|
||||
|
||||
# 3. Visualize dependencies
|
||||
python3 scripts/visualize-deps.py build --format graphviz --stats
|
||||
|
||||
# 4. Test all presets for your platform
|
||||
./scripts/test-cmake-presets.sh --platform mac
|
||||
```
|
||||
|
||||
## Tool 1: validate-cmake-config.cmake
|
||||
|
||||
### Purpose
|
||||
Validates CMake configuration by checking:
|
||||
- Required targets exist
|
||||
- Feature flags are consistent
|
||||
- Compiler settings are correct
|
||||
- Platform-specific configuration (especially Windows/Abseil)
|
||||
- Output directories are created
|
||||
- Common configuration issues
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Validate default build directory
|
||||
cmake -P scripts/validate-cmake-config.cmake
|
||||
|
||||
# Validate specific build directory
|
||||
cmake -P scripts/validate-cmake-config.cmake build_ai
|
||||
|
||||
# Validate after configuration
|
||||
cmake --preset win-ai
|
||||
cmake -P scripts/validate-cmake-config.cmake build
|
||||
```
|
||||
|
||||
### Exit Codes
|
||||
- **0** - All checks passed
|
||||
- **1** - Validation failed (errors detected)
|
||||
|
||||
### What It Checks
|
||||
|
||||
#### 1. Required Targets
|
||||
Ensures core targets exist:
|
||||
- `yaze_common` - Common interface library
|
||||
|
||||
#### 2. Feature Flag Consistency
|
||||
- When `YAZE_ENABLE_AI` is ON, `YAZE_ENABLE_GRPC` must also be ON
|
||||
- When `YAZE_ENABLE_GRPC` is ON, validates gRPC version is set
|
||||
|
||||
#### 3. Compiler Configuration
|
||||
- C++ standard is set to 23
|
||||
- MSVC runtime library is configured correctly on Windows
|
||||
- Compiler flags are propagated correctly
|
||||
|
||||
#### 4. Abseil Configuration (Windows)
|
||||
**CRITICAL for Windows builds with gRPC:**
|
||||
- Checks `CMAKE_MSVC_RUNTIME_LIBRARY` is set to `MultiThreaded`
|
||||
- Validates `ABSL_PROPAGATE_CXX_STD` is enabled
|
||||
- Verifies Abseil include directories exist
|
||||
|
||||
This prevents the "Abseil missing include paths" issue.
|
||||
|
||||
#### 5. Output Directories
|
||||
- `build/bin` exists
|
||||
- `build/lib` exists
|
||||
|
||||
#### 6. Common Issues
|
||||
- LTO enabled in Debug builds (warning)
|
||||
- Missing compile_commands.json
|
||||
- Generator expressions not expanded
|
||||
|
||||
### Example Output
|
||||
|
||||
```
|
||||
=== CMake Configuration Validator ===
|
||||
✓ Build directory: build
|
||||
✓ Loaded 342 cache variables
|
||||
|
||||
=== Validating required targets ===
|
||||
✓ Required target exists: yaze_common
|
||||
|
||||
=== Validating feature flags ===
|
||||
✓ gRPC enabled: ON
|
||||
✓ gRPC version: 1.67.1
|
||||
✓ Tests enabled
|
||||
✓ AI features enabled
|
||||
|
||||
=== Validating compiler flags ===
|
||||
✓ C++ standard: 23
|
||||
✓ CXX flags set: /EHsc /W4 /bigobj
|
||||
|
||||
=== Validating Windows/Abseil configuration ===
|
||||
✓ MSVC runtime: MultiThreaded$<$<CONFIG:Debug>:Debug>
|
||||
✓ Abseil CXX standard propagation enabled
|
||||
|
||||
=== Validation Summary ===
|
||||
✓ All validation checks passed!
|
||||
Configuration is ready for build
|
||||
```
|
||||
|
||||
## Tool 2: check-include-paths.sh
|
||||
|
||||
### Purpose
|
||||
Validates include paths in compile_commands.json to catch missing includes before compilation.
|
||||
|
||||
**Key Problem Solved:** On Windows, Abseil includes from gRPC were sometimes not propagated, causing build failures. This tool catches that early.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Check default build directory
|
||||
./scripts/check-include-paths.sh
|
||||
|
||||
# Check specific build directory
|
||||
./scripts/check-include-paths.sh build_ai
|
||||
|
||||
# Verbose mode (shows all include directories)
|
||||
VERBOSE=1 ./scripts/check-include-paths.sh build
|
||||
```
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **jq** (optional but recommended): `brew install jq` / `apt install jq`
|
||||
- Without jq, uses basic grep parsing
|
||||
|
||||
### What It Checks
|
||||
|
||||
#### 1. Common Dependencies
|
||||
- SDL2 includes
|
||||
- ImGui includes
|
||||
- yaml-cpp includes
|
||||
|
||||
#### 2. Platform-Specific Includes
|
||||
Validates platform-specific headers based on detected OS
|
||||
|
||||
#### 3. Abseil Includes (Windows Critical)
|
||||
When gRPC is enabled:
|
||||
- Checks `build/_deps/grpc-build/third_party/abseil-cpp` exists
|
||||
- Validates Abseil paths are in compile commands
|
||||
- Warns about unexpanded generator expressions
|
||||
|
||||
#### 4. Suspicious Configurations
|
||||
- No `-I` flags at all (error)
|
||||
- Relative paths with `../` (warning)
|
||||
- Duplicate include paths (warning)
|
||||
|
||||
### Exit Codes
|
||||
- **0** - All checks passed or warnings only
|
||||
- **1** - Critical errors detected
|
||||
|
||||
### Example Output
|
||||
|
||||
```
|
||||
=== Include Path Validation ===
|
||||
Build directory: build
|
||||
✓ Using jq for JSON parsing
|
||||
|
||||
=== Common Dependencies ===
|
||||
✓ SDL2 includes found
|
||||
✓ ImGui includes found
|
||||
⚠ yaml-cpp includes not found (may be optional)
|
||||
|
||||
=== Platform-Specific Includes ===
|
||||
Platform: macOS
|
||||
✓ SDL2 framework/library
|
||||
|
||||
=== Checking Abseil Includes (Windows Issue) ===
|
||||
gRPC build detected - checking Abseil paths...
|
||||
✓ Abseil from gRPC build: build/_deps/grpc-build/third_party/abseil-cpp
|
||||
|
||||
=== Suspicious Configurations ===
|
||||
✓ Include flags present (234/245 commands)
|
||||
✓ No duplicate include paths
|
||||
|
||||
=== Summary ===
|
||||
Checks performed: 5
|
||||
Warnings: 1
|
||||
✓ All include path checks passed!
|
||||
```
|
||||
|
||||
## Tool 3: visualize-deps.py
|
||||
|
||||
### Purpose
|
||||
Generates visual dependency graphs and detects circular dependencies.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Generate GraphViz diagram (default)
|
||||
python3 scripts/visualize-deps.py build
|
||||
|
||||
# Generate Mermaid diagram
|
||||
python3 scripts/visualize-deps.py build --format mermaid -o deps.mmd
|
||||
|
||||
# Generate text tree
|
||||
python3 scripts/visualize-deps.py build --format text
|
||||
|
||||
# Show statistics
|
||||
python3 scripts/visualize-deps.py build --stats
|
||||
```
|
||||
|
||||
### Output Formats
|
||||
|
||||
#### 1. GraphViz (DOT)
|
||||
```bash
|
||||
python3 scripts/visualize-deps.py build --format graphviz -o dependencies.dot
|
||||
|
||||
# Render to PNG
|
||||
dot -Tpng dependencies.dot -o dependencies.png
|
||||
|
||||
# Render to SVG (better for large graphs)
|
||||
dot -Tsvg dependencies.dot -o dependencies.svg
|
||||
```
|
||||
|
||||
**Color Coding:**
|
||||
- Blue boxes: Executables
|
||||
- Green boxes: Libraries
|
||||
- Gray boxes: Unknown type
|
||||
- Red arrows: Circular dependencies
|
||||
|
||||
#### 2. Mermaid
|
||||
```bash
|
||||
python3 scripts/visualize-deps.py build --format mermaid -o dependencies.mmd
|
||||
```
|
||||
|
||||
View at https://mermaid.live/edit or include in Markdown:
|
||||
|
||||
````markdown
|
||||
```mermaid
|
||||
graph LR
|
||||
yaze_app-->yaze_lib
|
||||
yaze_lib-->SDL2
|
||||
```
|
||||
````
|
||||
|
||||
#### 3. Text Tree
|
||||
```bash
|
||||
python3 scripts/visualize-deps.py build --format text
|
||||
```
|
||||
|
||||
Simple text representation for quick overview.
|
||||
|
||||
### Circular Dependency Detection
|
||||
|
||||
The tool automatically detects and highlights circular dependencies:
|
||||
|
||||
```
|
||||
✗ Found 1 circular dependencies
|
||||
libA -> libB -> libC -> libA
|
||||
```
|
||||
|
||||
Circular dependencies in graphs are shown with red arrows.
|
||||
|
||||
### Statistics Output
|
||||
|
||||
With `--stats` flag:
|
||||
```
|
||||
=== Dependency Statistics ===
|
||||
Total targets: 47
|
||||
Total dependencies: 156
|
||||
Average dependencies per target: 3.32
|
||||
|
||||
Most connected targets:
|
||||
yaze_lib: 23 dependencies
|
||||
yaze_app: 18 dependencies
|
||||
yaze_cli: 15 dependencies
|
||||
...
|
||||
```
|
||||
|
||||
## Tool 4: test-cmake-presets.sh
|
||||
|
||||
### Purpose
|
||||
Tests that all CMake presets can configure successfully, ensuring no configuration regressions.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Test all presets for current platform
|
||||
./scripts/test-cmake-presets.sh
|
||||
|
||||
# Test specific preset
|
||||
./scripts/test-cmake-presets.sh --preset mac-ai
|
||||
|
||||
# Test only Mac presets
|
||||
./scripts/test-cmake-presets.sh --platform mac
|
||||
|
||||
# Test in parallel (4 jobs)
|
||||
./scripts/test-cmake-presets.sh --parallel 4
|
||||
|
||||
# Quick mode (don't clean between tests)
|
||||
./scripts/test-cmake-presets.sh --quick
|
||||
|
||||
# Verbose output
|
||||
./scripts/test-cmake-presets.sh --verbose
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--parallel N` | Test N presets in parallel (default: 4) |
|
||||
| `--preset PRESET` | Test only specific preset |
|
||||
| `--platform PLATFORM` | Test only presets for platform (mac/win/lin) |
|
||||
| `--quick` | Skip cleaning between tests (faster) |
|
||||
| `--verbose` | Show full CMake output |
|
||||
|
||||
### Platform Detection
|
||||
|
||||
Automatically skips presets for other platforms:
|
||||
- On macOS: Only tests `mac-*` and generic presets
|
||||
- On Linux: Only tests `lin-*` and generic presets
|
||||
- On Windows: Only tests `win-*` and generic presets
|
||||
|
||||
### Example Output
|
||||
|
||||
```
|
||||
=== CMake Preset Configuration Tester ===
|
||||
Platform: mac
|
||||
Parallel jobs: 4
|
||||
|
||||
Presets to test:
|
||||
- mac-dbg
|
||||
- mac-rel
|
||||
- mac-ai
|
||||
- dev
|
||||
- ci
|
||||
|
||||
Running tests in parallel (jobs: 4)...
|
||||
|
||||
✓ mac-dbg configured successfully (12s)
|
||||
✓ dev configured successfully (15s)
|
||||
✓ mac-rel configured successfully (11s)
|
||||
✓ mac-ai configured successfully (45s)
|
||||
✓ ci configured successfully (18s)
|
||||
|
||||
=== Test Summary ===
|
||||
Total presets tested: 5
|
||||
Passed: 5
|
||||
Failed: 0
|
||||
✓ All presets configured successfully!
|
||||
```
|
||||
|
||||
### Failure Handling
|
||||
|
||||
When a preset fails:
|
||||
```
|
||||
✗ win-ai failed (34s)
|
||||
Log saved to: preset_test_win-ai.log
|
||||
|
||||
=== Test Summary ===
|
||||
Total presets tested: 3
|
||||
Passed: 2
|
||||
Failed: 1
|
||||
Failed presets:
|
||||
- win-ai
|
||||
|
||||
Check log files for details: preset_test_*.log
|
||||
```
|
||||
|
||||
## Integration with CI
|
||||
|
||||
### Add to GitHub Actions Workflow
|
||||
|
||||
```yaml
|
||||
name: CMake Validation
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
validate-cmake:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Configure CMake
|
||||
run: cmake --preset ci-linux
|
||||
|
||||
- name: Validate Configuration
|
||||
run: cmake -P scripts/validate-cmake-config.cmake build
|
||||
|
||||
- name: Check Include Paths
|
||||
run: ./scripts/check-include-paths.sh build
|
||||
|
||||
- name: Detect Circular Dependencies
|
||||
run: python3 scripts/visualize-deps.py build --stats
|
||||
```
|
||||
|
||||
### Pre-Configuration Check
|
||||
|
||||
Run validation as first CI step to fail fast:
|
||||
|
||||
```yaml
|
||||
- name: Fast Configuration Check
|
||||
run: |
|
||||
cmake --preset minimal
|
||||
cmake -P scripts/validate-cmake-config.cmake build
|
||||
```
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
### Issue 1: Missing Abseil Includes on Windows
|
||||
|
||||
**Symptom:**
|
||||
```
|
||||
✗ Missing required include: Abseil from gRPC build
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
1. Ensure `ABSL_PROPAGATE_CXX_STD` is ON in cmake/dependencies/grpc.cmake
|
||||
2. Reconfigure with `--fresh`: `cmake --preset win-ai --fresh`
|
||||
3. Check that gRPC was built successfully
|
||||
|
||||
**Prevention:**
|
||||
Run `cmake -P scripts/validate-cmake-config.cmake` after every configuration.
|
||||
|
||||
### Issue 2: Circular Dependencies
|
||||
|
||||
**Symptom:**
|
||||
```
|
||||
✗ Found 2 circular dependencies
|
||||
libA -> libB -> libA
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
1. Visualize full graph: `python3 scripts/visualize-deps.py build --format graphviz -o deps.dot`
|
||||
2. Render: `dot -Tpng deps.dot -o deps.png`
|
||||
3. Identify and break cycles by:
|
||||
- Moving shared code to a new library
|
||||
- Using forward declarations instead of includes
|
||||
- Restructuring dependencies
|
||||
|
||||
### Issue 3: Preset Configuration Fails
|
||||
|
||||
**Symptom:**
|
||||
```
|
||||
✗ mac-ai failed (34s)
|
||||
Log saved to: preset_test_mac-ai.log
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
1. Check log file: `cat preset_test_mac-ai.log`
|
||||
2. Common causes:
|
||||
- Missing dependencies (gRPC build failure)
|
||||
- Incompatible compiler flags
|
||||
- Platform condition mismatch
|
||||
3. Test preset manually: `cmake --preset mac-ai -B test_build -v`
|
||||
|
||||
### Issue 4: Generator Expressions Not Expanded
|
||||
|
||||
**Symptom:**
|
||||
```
|
||||
⚠ Generator expressions found in compile commands (may not be expanded)
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
This is usually harmless. Generator expressions like `$<BUILD_INTERFACE:...>` are CMake-internal and won't appear in final compile commands. If build fails, the issue is elsewhere.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Run Validation After Every Configuration
|
||||
|
||||
```bash
|
||||
# Configure
|
||||
cmake --preset mac-ai
|
||||
|
||||
# Validate immediately
|
||||
cmake -P scripts/validate-cmake-config.cmake build
|
||||
./scripts/check-include-paths.sh build
|
||||
```
|
||||
|
||||
### 2. Test All Presets Before Committing
|
||||
|
||||
```bash
|
||||
# Quick test of all platform presets
|
||||
./scripts/test-cmake-presets.sh --platform mac --parallel 4
|
||||
```
|
||||
|
||||
### 3. Check Dependencies When Adding New Targets
|
||||
|
||||
```bash
|
||||
# After adding new target to CMakeLists.txt
|
||||
cmake --preset dev
|
||||
python3 scripts/visualize-deps.py build --stats
|
||||
```
|
||||
|
||||
Look for:
|
||||
- Unexpected high dependency counts
|
||||
- New circular dependencies
|
||||
|
||||
### 4. Use in Git Hooks
|
||||
|
||||
Create `.git/hooks/pre-commit`:
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Validate CMake configuration before commit
|
||||
|
||||
if [ -f "build/CMakeCache.txt" ]; then
|
||||
echo "Validating CMake configuration..."
|
||||
cmake -P scripts/validate-cmake-config.cmake build || exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### 5. Periodic Full Validation
|
||||
|
||||
Weekly or before releases:
|
||||
```bash
|
||||
# Full validation suite
|
||||
./scripts/test-cmake-presets.sh --parallel 4
|
||||
cmake --preset dev
|
||||
cmake -P scripts/validate-cmake-config.cmake build
|
||||
./scripts/check-include-paths.sh build
|
||||
python3 scripts/visualize-deps.py build --format graphviz --stats -o deps.dot
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Tool doesn't run on Windows
|
||||
|
||||
**Bash scripts:**
|
||||
Use Git Bash, WSL, or MSYS2 to run `.sh` scripts.
|
||||
|
||||
**CMake scripts:**
|
||||
Should work natively on Windows:
|
||||
```powershell
|
||||
cmake -P scripts\validate-cmake-config.cmake build
|
||||
```
|
||||
|
||||
### jq not found
|
||||
|
||||
Install jq for better JSON parsing:
|
||||
```bash
|
||||
# macOS
|
||||
brew install jq
|
||||
|
||||
# Ubuntu/Debian
|
||||
sudo apt install jq
|
||||
|
||||
# Windows (via Chocolatey)
|
||||
choco install jq
|
||||
```
|
||||
|
||||
Scripts will work without jq but with reduced functionality.
|
||||
|
||||
### Python script fails
|
||||
|
||||
Ensure Python 3.7+ is installed:
|
||||
```bash
|
||||
python3 --version
|
||||
```
|
||||
|
||||
No external dependencies required - uses only standard library.
|
||||
|
||||
### GraphViz rendering fails
|
||||
|
||||
Install GraphViz:
|
||||
```bash
|
||||
# macOS
|
||||
brew install graphviz
|
||||
|
||||
# Ubuntu/Debian
|
||||
sudo apt install graphviz
|
||||
|
||||
# Windows (via Chocolatey)
|
||||
choco install graphviz
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Custom Validation Rules
|
||||
|
||||
Edit `scripts/validate-cmake-config.cmake` to add project-specific checks:
|
||||
|
||||
```cmake
|
||||
# Add after existing checks
|
||||
log_header "Custom Project Checks"
|
||||
|
||||
if(DEFINED CACHE_MY_CUSTOM_FLAG)
|
||||
if(CACHE_MY_CUSTOM_FLAG)
|
||||
log_success "Custom flag enabled"
|
||||
else()
|
||||
log_error "Custom flag must be enabled for this build"
|
||||
endif()
|
||||
endif()
|
||||
```
|
||||
|
||||
### Automated Dependency Reports
|
||||
|
||||
Generate weekly dependency reports:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# weekly-deps-report.sh
|
||||
|
||||
DATE=$(date +%Y-%m-%d)
|
||||
REPORT_DIR="reports/$DATE"
|
||||
mkdir -p "$REPORT_DIR"
|
||||
|
||||
# Configure
|
||||
cmake --preset ci
|
||||
|
||||
# Generate all formats
|
||||
python3 scripts/visualize-deps.py build \
|
||||
--format graphviz --stats -o "$REPORT_DIR/deps.dot"
|
||||
|
||||
python3 scripts/visualize-deps.py build \
|
||||
--format mermaid -o "$REPORT_DIR/deps.mmd"
|
||||
|
||||
python3 scripts/visualize-deps.py build \
|
||||
--format text -o "$REPORT_DIR/deps.txt"
|
||||
|
||||
# Render GraphViz
|
||||
dot -Tsvg "$REPORT_DIR/deps.dot" -o "$REPORT_DIR/deps.svg"
|
||||
|
||||
echo "Report generated in $REPORT_DIR"
|
||||
```
|
||||
|
||||
### CI Matrix Testing
|
||||
|
||||
Test all presets across platforms:
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
test-presets:
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
runs-on: ${{ matrix.os }}
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Test Presets
|
||||
run: ./scripts/test-cmake-presets.sh --parallel 2
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| Validate config | `cmake -P scripts/validate-cmake-config.cmake build` |
|
||||
| Check includes | `./scripts/check-include-paths.sh build` |
|
||||
| Visualize deps | `python3 scripts/visualize-deps.py build` |
|
||||
| Test all presets | `./scripts/test-cmake-presets.sh` |
|
||||
| Test one preset | `./scripts/test-cmake-presets.sh --preset mac-ai` |
|
||||
| Generate PNG graph | `python3 scripts/visualize-deps.py build -o d.dot && dot -Tpng d.dot -o d.png` |
|
||||
| Check for cycles | `python3 scripts/visualize-deps.py build --stats` |
|
||||
| Verbose include check | `VERBOSE=1 ./scripts/check-include-paths.sh build` |
|
||||
|
||||
## See Also
|
||||
|
||||
- [Build Quick Reference](../../public/build/quick-reference.md) - Build commands
|
||||
- [Build Troubleshooting](../../BUILD-TROUBLESHOOTING.md) - Common build issues
|
||||
- [CMakePresets.json](../../../CMakePresets.json) - All available presets
|
||||
- [GitHub Actions Workflows](../../../.github/workflows/) - CI configuration
|
||||
390
docs/internal/agents/archive/testing-docs-2025/gap-analysis.md
Normal file
390
docs/internal/agents/archive/testing-docs-2025/gap-analysis.md
Normal file
@@ -0,0 +1,390 @@
|
||||
# Testing Infrastructure Gap Analysis
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Recent CI failures revealed critical gaps in our testing infrastructure that allowed platform-specific build failures to reach CI. This document analyzes what we currently test, what we missed, and what infrastructure is needed to catch issues earlier.
|
||||
|
||||
**Date**: 2025-11-20
|
||||
**Triggered By**: Multiple CI failures in commits 43a0e5e314, c2bb90a3f1, and related fixes
|
||||
|
||||
---
|
||||
|
||||
## 1. Issues We Didn't Catch Locally
|
||||
|
||||
### 1.1 Windows Abseil Include Path Issues (c2bb90a3f1)
|
||||
**Problem**: Abseil headers not found during Windows/clang-cl compilation
|
||||
**Why it wasn't caught**:
|
||||
- No local pre-push compilation check
|
||||
- CMake configuration validates successfully, but compilation fails later
|
||||
- Include path propagation from gRPC/Abseil not validated until full compile
|
||||
|
||||
**What would have caught it**:
|
||||
- ✅ Smoke compilation test (compile subset of files to catch header issues)
|
||||
- ✅ CMake configuration validator (check include path propagation)
|
||||
- ✅ Header dependency checker
|
||||
|
||||
### 1.2 Linux FLAGS Symbol Conflicts (43a0e5e314, eb77bbeaff)
|
||||
**Problem**: ODR (One Definition Rule) violation - multiple `FLAGS` symbols across libraries
|
||||
**Why it wasn't caught**:
|
||||
- Symbol conflicts only appear at link time
|
||||
- No cross-library symbol conflict detection
|
||||
- Static analysis doesn't catch ODR violations
|
||||
- Unit tests don't link full dependency graph
|
||||
|
||||
**What would have caught it**:
|
||||
- ✅ Symbol conflict scanner (nm/objdump analysis)
|
||||
- ✅ ODR violation detector
|
||||
- ✅ Full integration build test (link all libraries together)
|
||||
|
||||
### 1.3 Platform-Specific Configuration Issues
|
||||
**Problem**: Preprocessor flags, compiler detection, and platform-specific code paths
|
||||
**Why it wasn't caught**:
|
||||
- No local cross-platform validation
|
||||
- CMake configuration differences between platforms not tested
|
||||
- Compiler detection logic (clang-cl vs MSVC) not validated
|
||||
|
||||
**What would have caught it**:
|
||||
- ✅ CMake configuration dry-run on multiple platforms
|
||||
- ✅ Preprocessor flag validation
|
||||
- ✅ Compiler detection smoke test
|
||||
|
||||
---
|
||||
|
||||
## 2. Current Testing Coverage
|
||||
|
||||
### 2.1 What We Test Well
|
||||
|
||||
#### Unit Tests (test/unit/)
|
||||
- **Coverage**: Core algorithms, data structures, parsers
|
||||
- **Speed**: Fast (<1s for most tests)
|
||||
- **Isolation**: Mocked dependencies, no ROM required
|
||||
- **CI**: ✅ Runs on every PR
|
||||
- **Example**: `hex_test.cc`, `asar_wrapper_test.cc`, `snes_palette_test.cc`
|
||||
|
||||
**Strengths**:
|
||||
- Catches logic errors quickly
|
||||
- Good for TDD
|
||||
- Platform-independent
|
||||
|
||||
**Gaps**:
|
||||
- Doesn't catch build system issues
|
||||
- Doesn't catch linking problems
|
||||
- Doesn't validate dependencies
|
||||
|
||||
#### Integration Tests (test/integration/)
|
||||
- **Coverage**: Multi-component interactions, ROM operations
|
||||
- **Speed**: Slower (1-10s per test)
|
||||
- **Dependencies**: May require ROM files
|
||||
- **CI**: ✅ Runs on develop/master
|
||||
- **Example**: `asar_integration_test.cc`, `dungeon_editor_v2_test.cc`
|
||||
|
||||
**Strengths**:
|
||||
- Tests component interactions
|
||||
- Validates ROM operations
|
||||
|
||||
**Gaps**:
|
||||
- Still doesn't catch platform-specific issues
|
||||
- Doesn't validate symbol conflicts
|
||||
- Doesn't test cross-library linking
|
||||
|
||||
#### E2E Tests (test/e2e/)
|
||||
- **Coverage**: Full UI workflows, user interactions
|
||||
- **Speed**: Very slow (10-60s per test)
|
||||
- **Dependencies**: GUI, ImGuiTestEngine
|
||||
- **CI**: ⚠️ Limited (only on macOS z3ed-agent-test)
|
||||
- **Example**: `dungeon_editor_smoke_test.cc`, `canvas_selection_test.cc`
|
||||
|
||||
**Strengths**:
|
||||
- Validates real user workflows
|
||||
- Tests UI responsiveness
|
||||
|
||||
**Gaps**:
|
||||
- Not run consistently across platforms
|
||||
- Slow feedback loop
|
||||
- Requires display/window system
|
||||
|
||||
### 2.2 What We DON'T Test
|
||||
|
||||
#### Build System Validation
|
||||
- ❌ CMake configuration correctness per preset
|
||||
- ❌ Include path propagation from dependencies
|
||||
- ❌ Compiler flag compatibility
|
||||
- ❌ Linker flag validation
|
||||
- ❌ Cross-preset compatibility
|
||||
|
||||
#### Symbol-Level Issues
|
||||
- ❌ ODR (One Definition Rule) violations
|
||||
- ❌ Duplicate symbol detection across libraries
|
||||
- ❌ Symbol visibility (public/private)
|
||||
- ❌ ABI compatibility between libraries
|
||||
|
||||
#### Platform-Specific Compilation
|
||||
- ❌ Header-only compilation checks
|
||||
- ❌ Preprocessor branch coverage
|
||||
- ❌ Platform macro validation
|
||||
- ❌ Compiler-specific feature detection
|
||||
|
||||
#### Dependency Health
|
||||
- ❌ Include path conflicts
|
||||
- ❌ Library version mismatches
|
||||
- ❌ Transitive dependency validation
|
||||
- ❌ Static vs shared library conflicts
|
||||
|
||||
---
|
||||
|
||||
## 3. CI/CD Coverage Analysis
|
||||
|
||||
### 3.1 Current CI Matrix (.github/workflows/ci.yml)
|
||||
|
||||
| Platform | Build | Test (stable) | Test (unit) | Test (integration) | Test (AI) |
|
||||
|----------|-------|---------------|-------------|-------------------|-----------|
|
||||
| Ubuntu 22.04 (GCC-12) | ✅ | ✅ | ✅ | ❌ | ❌ |
|
||||
| macOS 14 (Clang) | ✅ | ✅ | ✅ | ❌ | ✅ |
|
||||
| Windows 2022 (Core) | ✅ | ✅ | ✅ | ❌ | ❌ |
|
||||
| Windows 2022 (AI) | ✅ | ✅ | ✅ | ❌ | ❌ |
|
||||
|
||||
**CI Job Flow**:
|
||||
1. **build**: Configure + compile full project
|
||||
2. **test**: Run stable + unit tests
|
||||
3. **windows-agent**: Full AI stack (gRPC + AI runtime)
|
||||
4. **code-quality**: clang-format, cppcheck, clang-tidy
|
||||
5. **memory-sanitizer**: AddressSanitizer (Linux only)
|
||||
6. **z3ed-agent-test**: Full agent test suite (macOS only)
|
||||
|
||||
### 3.2 CI Gaps
|
||||
|
||||
#### Missing Early Feedback
|
||||
- ❌ No compilation-only job (fails after 15-20 min build)
|
||||
- ❌ No CMake configuration validation job (would catch in <1 min)
|
||||
- ❌ No symbol conflict checking job
|
||||
|
||||
#### Limited Platform Coverage
|
||||
- ⚠️ Only Linux gets AddressSanitizer
|
||||
- ⚠️ Only macOS gets full z3ed agent tests
|
||||
- ⚠️ Windows AI stack not tested on PRs (only post-merge)
|
||||
|
||||
#### Incomplete Testing
|
||||
- ❌ Integration tests not run in CI
|
||||
- ❌ E2E tests not run on Linux/Windows
|
||||
- ❌ No ROM-dependent testing
|
||||
- ❌ No performance regression detection
|
||||
|
||||
---
|
||||
|
||||
## 4. Developer Workflow Gaps
|
||||
|
||||
### 4.1 Pre-Commit Hooks
|
||||
**Current State**: None
|
||||
**Gap**: No automatic checks before local commits
|
||||
|
||||
**Should Include**:
|
||||
- clang-format check
|
||||
- Build system sanity check
|
||||
- Copyright header validation
|
||||
|
||||
### 4.2 Pre-Push Validation
|
||||
**Current State**: Manual testing only
|
||||
**Gap**: Easy to push broken code to CI
|
||||
|
||||
**Should Include**:
|
||||
- Smoke build test (quick compilation check)
|
||||
- Unit test run
|
||||
- Symbol conflict detection
|
||||
|
||||
### 4.3 Local Cross-Platform Testing
|
||||
**Current State**: Developer-dependent
|
||||
**Gap**: No easy way to test across platforms locally
|
||||
|
||||
**Should Include**:
|
||||
- Docker-based Linux testing
|
||||
- VM-based Windows testing (for macOS/Linux devs)
|
||||
- Preset validation tool
|
||||
|
||||
---
|
||||
|
||||
## 5. Root Cause Analysis by Issue Type
|
||||
|
||||
### 5.1 Windows Abseil Include Paths
|
||||
|
||||
**Timeline**:
|
||||
- ✅ Local macOS build succeeds
|
||||
- ✅ CMake configuration succeeds on all platforms
|
||||
- ❌ Windows compilation fails 15 minutes into CI
|
||||
- ❌ Fix attempt 1 fails (14d1f5de4c)
|
||||
- ❌ Fix attempt 2 fails (c2bb90a3f1)
|
||||
- ✅ Final fix succeeds
|
||||
|
||||
**Why Multiple Attempts**:
|
||||
1. No local Windows testing environment
|
||||
2. CMake configuration doesn't validate actual compilation
|
||||
3. No header-only compilation check
|
||||
4. 15-20 minute feedback cycle from CI
|
||||
|
||||
**Prevention**:
|
||||
- Header compilation smoke test
|
||||
- CMake include path validator
|
||||
- Local Windows testing (Docker/VM)
|
||||
|
||||
### 5.2 Linux FLAGS Symbol Conflicts
|
||||
|
||||
**Timeline**:
|
||||
- ✅ Local macOS build succeeds
|
||||
- ✅ Unit tests pass
|
||||
- ❌ Linux full build fails at link time
|
||||
- ❌ ODR violation: multiple `FLAGS` definitions
|
||||
- ✅ Fix: move FLAGS definition, rename conflicts
|
||||
|
||||
**Why It Happened**:
|
||||
1. gflags creates `FLAGS_*` symbols in headers
|
||||
2. Multiple translation units define same symbols
|
||||
3. macOS linker more permissive than Linux ld
|
||||
4. No symbol conflict detection
|
||||
|
||||
**Prevention**:
|
||||
- Symbol conflict scanner
|
||||
- ODR violation checker
|
||||
- Cross-platform link test
|
||||
|
||||
---
|
||||
|
||||
## 6. Recommended Testing Levels
|
||||
|
||||
We propose a **5-level testing pyramid**:
|
||||
|
||||
### Level 0: Static Analysis (< 1s)
|
||||
- clang-format
|
||||
- clang-tidy on changed files
|
||||
- Copyright headers
|
||||
- CMakeLists.txt syntax
|
||||
|
||||
### Level 1: Configuration Validation (< 10s)
|
||||
- CMake configure dry-run
|
||||
- Include path validation
|
||||
- Compiler detection check
|
||||
- Preprocessor flag validation
|
||||
|
||||
### Level 2: Smoke Compilation (< 2 min)
|
||||
- Compile subset of files (1 file per library)
|
||||
- Header-only compilation
|
||||
- Template instantiation check
|
||||
- Platform-specific branch validation
|
||||
|
||||
### Level 3: Symbol Validation (< 5 min)
|
||||
- Full project compilation
|
||||
- Symbol conflict detection (nm/dumpbin)
|
||||
- ODR violation check
|
||||
- Library dependency graph
|
||||
|
||||
### Level 4: Test Execution (5-30 min)
|
||||
- Unit tests (fast)
|
||||
- Integration tests (medium)
|
||||
- E2E tests (slow)
|
||||
- ROM-dependent tests (optional)
|
||||
|
||||
---
|
||||
|
||||
## 7. Actionable Recommendations
|
||||
|
||||
### 7.1 Immediate Actions (This Initiative)
|
||||
|
||||
1. **Create pre-push scripts** (`scripts/pre-push-test.sh`, `scripts/pre-push-test.ps1`)
|
||||
- Run Level 0-2 checks locally
|
||||
- Estimated time: <2 minutes
|
||||
- Blocks 90% of CI failures
|
||||
|
||||
2. **Create symbol conflict detector** (`scripts/verify-symbols.sh`)
|
||||
- Scan built libraries for duplicate symbols
|
||||
- Run as part of pre-push
|
||||
- Catches ODR violations
|
||||
|
||||
3. **Document testing strategy** (`docs/internal/testing/testing-strategy.md`)
|
||||
- Clear explanation of each test level
|
||||
- When to run which tests
|
||||
- CI vs local testing
|
||||
|
||||
4. **Create pre-push checklist** (`docs/internal/testing/pre-push-checklist.md`)
|
||||
- Interactive checklist for developers
|
||||
- Links to tools and scripts
|
||||
|
||||
### 7.2 Short-Term Improvements (Next Sprint)
|
||||
|
||||
1. **Add CI compile-only job**
|
||||
- Runs in <5 minutes
|
||||
- Catches compilation issues before full build
|
||||
- Fails fast
|
||||
|
||||
2. **Add CI symbol checking job**
|
||||
- Runs after compile-only
|
||||
- Detects ODR violations
|
||||
- Platform-specific
|
||||
|
||||
3. **Add CMake configuration validation job**
|
||||
- Tests all presets
|
||||
- Validates include paths
|
||||
- <2 minutes
|
||||
|
||||
4. **Enable integration tests in CI**
|
||||
- Run on develop/master only (not PRs)
|
||||
- Requires ROM file handling
|
||||
|
||||
### 7.3 Long-Term Improvements (Future)
|
||||
|
||||
1. **Docker-based local testing**
|
||||
- Linux environment for macOS/Windows devs
|
||||
- Matches CI exactly
|
||||
- Fast feedback
|
||||
|
||||
2. **Cross-platform test matrix locally**
|
||||
- Run tests across multiple platforms
|
||||
- Automated VM/container management
|
||||
|
||||
3. **Performance regression detection**
|
||||
- Benchmark suite
|
||||
- Historical tracking
|
||||
- Automatic alerts
|
||||
|
||||
4. **Coverage tracking**
|
||||
- Line coverage per PR
|
||||
- Coverage trends over time
|
||||
- Uncovered code reports
|
||||
|
||||
---
|
||||
|
||||
## 8. Success Metrics
|
||||
|
||||
### 8.1 Developer Experience
|
||||
- **Target**: <2 minutes pre-push validation time
|
||||
- **Target**: 90% reduction in CI build failures
|
||||
- **Target**: <3 attempts to fix CI issues (down from 5-10)
|
||||
|
||||
### 8.2 CI Efficiency
|
||||
- **Target**: <5 minutes to first failure signal
|
||||
- **Target**: 50% reduction in wasted CI time
|
||||
- **Target**: 95% PR pass rate (up from ~70%)
|
||||
|
||||
### 8.3 Code Quality
|
||||
- **Target**: Zero ODR violations
|
||||
- **Target**: Zero platform-specific include issues
|
||||
- **Target**: 100% symbol conflict detection
|
||||
|
||||
---
|
||||
|
||||
## 9. Reference
|
||||
|
||||
### Similar Issues in Recent History
|
||||
- Windows std::filesystem support (19196ca87c, b556b155a5)
|
||||
- Linux circular dependency (0812a84a22, e36d81f357)
|
||||
- macOS z3ed linker error (9c562df277)
|
||||
- Windows clang-cl detection (84cdb09a5b, cbdc6670a1)
|
||||
|
||||
### Related Documentation
|
||||
- `docs/public/build/quick-reference.md` - Build commands
|
||||
- `docs/public/build/troubleshooting.md` - Platform-specific fixes
|
||||
- `CLAUDE.md` - Build system guidelines
|
||||
- `.github/workflows/ci.yml` - CI configuration
|
||||
|
||||
### Tools Used
|
||||
- `nm` (Unix) / `dumpbin` (Windows) - Symbol inspection
|
||||
- `clang-tidy` - Static analysis
|
||||
- `cppcheck` - Code quality
|
||||
- `cmake --preset <name> --list-presets` - Preset validation
|
||||
@@ -0,0 +1,505 @@
|
||||
# Testing Infrastructure Integration Plan
|
||||
|
||||
**Owner**: CLAUDE_TEST_COORD
|
||||
**Status**: Draft
|
||||
**Created**: 2025-11-20
|
||||
**Target Completion**: 2025-12-15
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines the rollout plan for comprehensive testing infrastructure improvements across the yaze project. The goal is to reduce CI failures, catch issues earlier, and provide developers with fast, reliable testing tools.
|
||||
|
||||
## Current State Assessment
|
||||
|
||||
### What's Working Well
|
||||
|
||||
✅ **Test Organization**:
|
||||
- Clear directory structure (unit/integration/e2e/benchmarks)
|
||||
- Good test coverage for core systems
|
||||
- ImGui Test Engine integration for GUI testing
|
||||
|
||||
✅ **CI/CD**:
|
||||
- Multi-platform matrix (Linux, macOS, Windows)
|
||||
- Automated test execution on every commit
|
||||
- Test result artifacts on failure
|
||||
|
||||
✅ **Helper Scripts**:
|
||||
- `run-tests.sh` for preset-based testing
|
||||
- `smoke-build.sh` for quick build verification
|
||||
- `run-gh-workflow.sh` for remote CI triggers
|
||||
|
||||
### Current Gaps
|
||||
|
||||
❌ **Developer Experience**:
|
||||
- No pre-push validation hooks
|
||||
- Long CI feedback loop (10-15 minutes)
|
||||
- Unclear what tests to run locally
|
||||
- Format checking often forgotten
|
||||
|
||||
❌ **Test Infrastructure**:
|
||||
- No symbol conflict detection tools
|
||||
- No CMake configuration validators
|
||||
- Platform-specific test failures hard to reproduce locally
|
||||
- Flaky test tracking is manual
|
||||
|
||||
❌ **Documentation**:
|
||||
- Testing docs scattered across multiple files
|
||||
- No clear "before you push" checklist
|
||||
- Platform-specific troubleshooting incomplete
|
||||
- Release testing process not documented
|
||||
|
||||
## Goals and Success Criteria
|
||||
|
||||
### Primary Goals
|
||||
|
||||
1. **Fast Local Feedback** (<5 minutes for pre-push checks)
|
||||
2. **Early Issue Detection** (catch 90% of CI failures locally)
|
||||
3. **Clear Documentation** (developers know exactly what to run)
|
||||
4. **Automated Validation** (pre-push hooks, format checking)
|
||||
5. **Platform Parity** (reproducible CI failures locally)
|
||||
|
||||
### Success Metrics
|
||||
|
||||
- **CI Failure Rate**: Reduce from ~20% to <5%
|
||||
- **Time to Fix**: Average time from failure to fix <30 minutes
|
||||
- **Developer Satisfaction**: Positive feedback on testing workflow
|
||||
- **Test Runtime**: Unit tests complete in <10s, full suite in <5min
|
||||
- **Coverage**: Maintain >80% test coverage for critical paths
|
||||
|
||||
## Rollout Phases
|
||||
|
||||
### Phase 1: Documentation and Tools (Week 1-2) ✅ COMPLETE
|
||||
|
||||
**Status**: COMPLETE
|
||||
**Completion Date**: 2025-11-20
|
||||
|
||||
#### Deliverables
|
||||
|
||||
- ✅ Master testing documentation (`docs/internal/testing/README.md`)
|
||||
- ✅ Developer quick-start guide (`docs/public/developer/testing-quick-start.md`)
|
||||
- ✅ Integration plan (this document)
|
||||
- ✅ Updated release checklist with testing requirements
|
||||
|
||||
#### Validation
|
||||
|
||||
- ✅ All documents reviewed and approved
|
||||
- ✅ Links between documents verified
|
||||
- ✅ Content accuracy checked against actual implementation
|
||||
|
||||
### Phase 2: Pre-Push Validation (Week 3)
|
||||
|
||||
**Status**: PLANNED
|
||||
**Target Date**: 2025-11-27
|
||||
|
||||
#### Deliverables
|
||||
|
||||
1. **Pre-Push Script** (`scripts/pre-push.sh`)
|
||||
- Run unit tests automatically
|
||||
- Check code formatting
|
||||
- Verify build compiles
|
||||
- Exit with error if any check fails
|
||||
- Run in <2 minutes
|
||||
|
||||
2. **Git Hook Integration** (`.git/hooks/pre-push`)
|
||||
- Optional installation script
|
||||
- Easy enable/disable mechanism
|
||||
- Clear output showing progress
|
||||
- Skip with `--no-verify` flag
|
||||
|
||||
3. **Developer Documentation**
|
||||
- How to install pre-push hook
|
||||
- How to customize checks
|
||||
- How to skip when needed
|
||||
|
||||
#### Implementation Steps
|
||||
|
||||
```bash
|
||||
# 1. Create pre-push script
|
||||
scripts/pre-push.sh
|
||||
|
||||
# 2. Create hook installer
|
||||
scripts/install-git-hooks.sh
|
||||
|
||||
# 3. Update documentation
|
||||
docs/public/developer/git-workflow.md
|
||||
docs/public/developer/testing-quick-start.md
|
||||
|
||||
# 4. Test on all platforms
|
||||
- macOS: Verify script runs correctly
|
||||
- Linux: Verify script runs correctly
|
||||
- Windows: Create PowerShell equivalent
|
||||
```
|
||||
|
||||
#### Validation
|
||||
|
||||
- [ ] Script runs in <2 minutes on all platforms
|
||||
- [ ] All checks are meaningful (catch real issues)
|
||||
- [ ] False positive rate <5%
|
||||
- [ ] Developers report positive feedback
|
||||
|
||||
### Phase 3: Symbol Conflict Detection (Week 4)
|
||||
|
||||
**Status**: PLANNED
|
||||
**Target Date**: 2025-12-04
|
||||
|
||||
#### Background
|
||||
|
||||
Recent Linux build failures were caused by symbol conflicts (FLAGS_rom, FLAGS_norom redefinition). We need automated detection to prevent this.
|
||||
|
||||
#### Deliverables
|
||||
|
||||
1. **Symbol Conflict Checker** (`scripts/check-symbols.sh`)
|
||||
- Parse CMake target link graphs
|
||||
- Detect duplicate symbol definitions
|
||||
- Report conflicts with file locations
|
||||
- Run in <30 seconds
|
||||
|
||||
2. **CI Integration**
|
||||
- Add symbol check job to `.github/workflows/ci.yml`
|
||||
- Run on every PR
|
||||
- Fail build if conflicts detected
|
||||
|
||||
3. **Documentation**
|
||||
- Troubleshooting guide for symbol conflicts
|
||||
- Best practices for avoiding conflicts
|
||||
|
||||
#### Implementation Steps
|
||||
|
||||
```bash
|
||||
# 1. Create symbol checker
|
||||
scripts/check-symbols.sh
|
||||
# - Use nm/objdump to list symbols
|
||||
# - Compare across linked targets
|
||||
# - Detect duplicates
|
||||
|
||||
# 2. Add to CI
|
||||
.github/workflows/ci.yml
|
||||
# - New job: symbol-check
|
||||
# - Runs after build
|
||||
|
||||
# 3. Document usage
|
||||
docs/internal/testing/symbol-conflict-detection.md
|
||||
```
|
||||
|
||||
#### Validation
|
||||
|
||||
- [ ] Detects known symbol conflicts (FLAGS_rom case)
|
||||
- [ ] Zero false positives on current codebase
|
||||
- [ ] Runs in <30 seconds
|
||||
- [ ] Clear, actionable error messages
|
||||
|
||||
### Phase 4: CMake Configuration Validation (Week 5)
|
||||
|
||||
**Status**: PLANNED
|
||||
**Target Date**: 2025-12-11
|
||||
|
||||
#### Deliverables
|
||||
|
||||
1. **CMake Preset Validator** (`scripts/validate-cmake-presets.sh`)
|
||||
- Verify all presets configure successfully
|
||||
- Check for missing variables
|
||||
- Validate preset inheritance
|
||||
- Test preset combinations
|
||||
|
||||
2. **Build Matrix Tester** (`scripts/test-build-matrix.sh`)
|
||||
- Test common preset/platform combinations
|
||||
- Verify all targets build
|
||||
- Check for missing dependencies
|
||||
|
||||
3. **Documentation**
|
||||
- CMake troubleshooting guide
|
||||
- Preset creation guidelines
|
||||
|
||||
#### Implementation Steps
|
||||
|
||||
```bash
|
||||
# 1. Create validators
|
||||
scripts/validate-cmake-presets.sh
|
||||
scripts/test-build-matrix.sh
|
||||
|
||||
# 2. Add to CI (optional job)
|
||||
.github/workflows/cmake-validation.yml
|
||||
|
||||
# 3. Document
|
||||
docs/internal/testing/cmake-validation.md
|
||||
```
|
||||
|
||||
#### Validation
|
||||
|
||||
- [ ] All current presets validate successfully
|
||||
- [ ] Catches common configuration errors
|
||||
- [ ] Runs in <5 minutes for full matrix
|
||||
- [ ] Provides clear error messages
|
||||
|
||||
### Phase 5: Platform Matrix Testing (Week 6)
|
||||
|
||||
**Status**: PLANNED
|
||||
**Target Date**: 2025-12-18
|
||||
|
||||
#### Deliverables
|
||||
|
||||
1. **Local Platform Testing** (`scripts/test-all-platforms.sh`)
|
||||
- Run tests on all configured platforms
|
||||
- Parallel execution for speed
|
||||
- Aggregate results
|
||||
- Report differences across platforms
|
||||
|
||||
2. **CI Enhancement**
|
||||
- Add platform-specific test suites
|
||||
- Better artifact collection
|
||||
- Test result comparison across platforms
|
||||
|
||||
3. **Documentation**
|
||||
- Platform-specific testing guide
|
||||
- Troubleshooting platform differences
|
||||
|
||||
#### Implementation Steps
|
||||
|
||||
```bash
|
||||
# 1. Create platform tester
|
||||
scripts/test-all-platforms.sh
|
||||
|
||||
# 2. Enhance CI
|
||||
.github/workflows/ci.yml
|
||||
# - Better artifact collection
|
||||
# - Result comparison
|
||||
|
||||
# 3. Document
|
||||
docs/internal/testing/platform-testing.md
|
||||
```
|
||||
|
||||
#### Validation
|
||||
|
||||
- [ ] Detects platform-specific failures
|
||||
- [ ] Clear reporting of differences
|
||||
- [ ] Runs in <10 minutes (parallel)
|
||||
- [ ] Useful for debugging platform issues
|
||||
|
||||
## Training and Communication
|
||||
|
||||
### Developer Training
|
||||
|
||||
**Target Audience**: All contributors
|
||||
|
||||
**Format**: Written documentation + optional video walkthrough
|
||||
|
||||
**Topics**:
|
||||
1. How to run tests locally (5 minutes)
|
||||
2. Understanding test categories (5 minutes)
|
||||
3. Using pre-push hooks (5 minutes)
|
||||
4. Debugging test failures (10 minutes)
|
||||
5. CI workflow overview (5 minutes)
|
||||
|
||||
**Materials**:
|
||||
- ✅ Quick start guide (already created)
|
||||
- ✅ Testing guide (already exists)
|
||||
- [ ] Video walkthrough (optional, Phase 6)
|
||||
|
||||
### Communication Plan
|
||||
|
||||
**Announcements**:
|
||||
1. **Phase 1 Complete**: Email/Slack announcement with links to new docs
|
||||
2. **Phase 2 Ready**: Announce pre-push hooks, encourage adoption
|
||||
3. **Phase 3-5**: Update as each phase completes
|
||||
4. **Final Rollout**: Comprehensive announcement when all phases done
|
||||
|
||||
**Channels**:
|
||||
- GitHub Discussions
|
||||
- Project README updates
|
||||
- CONTRIBUTING.md updates
|
||||
- Coordination board updates
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
### Risk 1: Developer Resistance to Pre-Push Hooks
|
||||
|
||||
**Mitigation**:
|
||||
- Make hooks optional (install script)
|
||||
- Keep checks fast (<2 minutes)
|
||||
- Allow easy skip with `--no-verify`
|
||||
- Provide clear value proposition
|
||||
|
||||
### Risk 2: False Positives Causing Frustration
|
||||
|
||||
**Mitigation**:
|
||||
- Test extensively before rollout
|
||||
- Monitor false positive rate
|
||||
- Provide clear bypass mechanisms
|
||||
- Iterate based on feedback
|
||||
|
||||
### Risk 3: Tools Break on Platform Updates
|
||||
|
||||
**Mitigation**:
|
||||
- Test on all platforms before rollout
|
||||
- Document platform-specific requirements
|
||||
- Version-pin critical dependencies
|
||||
- Maintain fallback paths
|
||||
|
||||
### Risk 4: CI Becomes Too Slow
|
||||
|
||||
**Mitigation**:
|
||||
- Use parallel execution
|
||||
- Cache aggressively
|
||||
- Make expensive checks optional
|
||||
- Profile and optimize bottlenecks
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If any phase causes significant issues:
|
||||
|
||||
1. **Immediate**: Disable problematic feature (remove hook, comment out CI job)
|
||||
2. **Investigate**: Gather feedback and logs
|
||||
3. **Fix**: Address root cause
|
||||
4. **Re-enable**: Gradual rollout with fixes
|
||||
5. **Document**: Update docs with lessons learned
|
||||
|
||||
## Success Indicators
|
||||
|
||||
### Week-by-Week Targets
|
||||
|
||||
- **Week 2**: Documentation complete and published ✅
|
||||
- **Week 3**: Pre-push hooks adopted by 50% of active developers
|
||||
- **Week 4**: Symbol conflicts detected before reaching CI
|
||||
- **Week 5**: CMake preset validation catches configuration errors
|
||||
- **Week 6**: Platform-specific failures reproducible locally
|
||||
|
||||
### Final Success Criteria (End of Phase 5)
|
||||
|
||||
- ✅ All documentation complete and reviewed
|
||||
- [ ] CI failure rate <5% (down from ~20%)
|
||||
- [ ] Average time to fix CI failure <30 minutes
|
||||
- [ ] 80%+ developers using pre-push hooks
|
||||
- [ ] Zero symbol conflict issues reaching production
|
||||
- [ ] Platform parity: local tests match CI results
|
||||
|
||||
## Maintenance and Long-Term Support
|
||||
|
||||
### Ongoing Responsibilities
|
||||
|
||||
**Testing Infrastructure Lead** (CLAUDE_TEST_COORD):
|
||||
- Monitor CI failure rates
|
||||
- Respond to testing infrastructure issues
|
||||
- Update documentation as needed
|
||||
- Coordinate with platform specialists
|
||||
|
||||
**Platform Specialists**:
|
||||
- Maintain platform-specific test helpers
|
||||
- Troubleshoot platform-specific failures
|
||||
- Keep documentation current
|
||||
|
||||
**All Developers**:
|
||||
- Report testing infrastructure issues
|
||||
- Suggest improvements
|
||||
- Keep tests passing locally before pushing
|
||||
|
||||
### Quarterly Reviews
|
||||
|
||||
**Schedule**: Every 3 months
|
||||
|
||||
**Review**:
|
||||
1. CI failure rate trends
|
||||
2. Test runtime trends
|
||||
3. Developer feedback
|
||||
4. New platform/tool needs
|
||||
5. Documentation updates
|
||||
|
||||
**Adjustments**:
|
||||
- Update scripts for new platforms
|
||||
- Optimize slow tests
|
||||
- Add new helpers as needed
|
||||
- Archive obsolete tools/docs
|
||||
|
||||
## Budget and Resources
|
||||
|
||||
### Time Investment
|
||||
|
||||
**Initial Rollout** (Phases 1-5): ~6 weeks
|
||||
- Documentation: 1 week ✅
|
||||
- Pre-push validation: 1 week
|
||||
- Symbol detection: 1 week
|
||||
- CMake validation: 1 week
|
||||
- Platform testing: 1 week
|
||||
- Buffer/testing: 1 week
|
||||
|
||||
**Ongoing Maintenance**: ~4 hours/month
|
||||
- Monitoring CI
|
||||
- Updating docs
|
||||
- Fixing issues
|
||||
- Quarterly reviews
|
||||
|
||||
### Infrastructure Costs
|
||||
|
||||
**Current**: $0 (using GitHub Actions free tier)
|
||||
|
||||
**Projected**: $0 (within free tier limits)
|
||||
|
||||
**Potential Future Costs**:
|
||||
- GitHub Actions minutes (if exceed free tier)
|
||||
- External CI service (if needed)
|
||||
- Test infrastructure hosting (if needed)
|
||||
|
||||
## Appendix: Related Work
|
||||
|
||||
### Completed by Other Agents
|
||||
|
||||
**GEMINI_AUTOM**:
|
||||
- ✅ Remote workflow trigger support
|
||||
- ✅ HTTP API testing infrastructure
|
||||
- ✅ Helper scripts for agents
|
||||
|
||||
**CLAUDE_AIINF**:
|
||||
- ✅ Platform-specific build fixes
|
||||
- ✅ CMake preset expansion
|
||||
- ✅ gRPC integration improvements
|
||||
|
||||
**CODEX**:
|
||||
- ✅ Documentation audit and consolidation
|
||||
- ✅ Build verification scripts
|
||||
- ✅ Coordination board setup
|
||||
|
||||
### Planned by Other Agents
|
||||
|
||||
**CLAUDE_TEST_ARCH**:
|
||||
- Pre-push testing automation
|
||||
- Gap analysis of test coverage
|
||||
|
||||
**CLAUDE_CMAKE_VALIDATOR**:
|
||||
- CMake configuration validation tools
|
||||
- Preset verification
|
||||
|
||||
**CLAUDE_SYMBOL_CHECK**:
|
||||
- Symbol conflict detection
|
||||
- Link graph analysis
|
||||
|
||||
**CLAUDE_MATRIX_TEST**:
|
||||
- Platform matrix testing
|
||||
- Cross-platform validation
|
||||
|
||||
## Questions and Clarifications
|
||||
|
||||
**Q: Are pre-push hooks mandatory?**
|
||||
A: No, they're optional but strongly recommended. Developers can install with `scripts/install-git-hooks.sh` and remove anytime.
|
||||
|
||||
**Q: How long will pre-push checks take?**
|
||||
A: Target is <2 minutes. Unit tests (<10s) + format check (<5s) + build verification (~1min).
|
||||
|
||||
**Q: What if I need to push despite failing checks?**
|
||||
A: Use `git push --no-verify` to bypass hooks. This should be rare and only for emergencies.
|
||||
|
||||
**Q: Will this slow down CI?**
|
||||
A: No. Most tools run locally to catch issues before CI. Some new CI jobs are optional/parallel.
|
||||
|
||||
**Q: What if tools break on my platform?**
|
||||
A: Report in GitHub issues with platform details. We'll fix or provide platform-specific workaround.
|
||||
|
||||
## References
|
||||
|
||||
- [Testing Documentation](README.md)
|
||||
- [Quick Start Guide](../../public/developer/testing-quick-start.md)
|
||||
- [Coordination Board](../agents/coordination-board.md)
|
||||
- [Release Checklist](../release-checklist.md)
|
||||
- [CI Workflow](../../../.github/workflows/ci.yml)
|
||||
|
||||
---
|
||||
|
||||
**Next Actions**: Proceed to Phase 2 (Pre-Push Validation) once Phase 1 is approved and published.
|
||||
@@ -0,0 +1,499 @@
|
||||
# Matrix Testing Strategy
|
||||
|
||||
**Owner**: CLAUDE_MATRIX_TEST (Platform Matrix Testing Specialist)
|
||||
**Last Updated**: 2025-11-20
|
||||
**Status**: ACTIVE
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document defines the strategy for comprehensive platform/configuration matrix testing to catch issues across CMake flag combinations, platforms, and build configurations.
|
||||
|
||||
**Key Goals**:
|
||||
- Catch cross-configuration issues before they reach production
|
||||
- Prevent "works on my machine" problems
|
||||
- Document problematic flag combinations
|
||||
- Make matrix testing accessible to developers locally
|
||||
- Minimize CI time while maximizing coverage
|
||||
|
||||
**Quick Links**:
|
||||
- Configuration reference: `/docs/internal/configuration-matrix.md`
|
||||
- GitHub Actions workflow: `/.github/workflows/matrix-test.yml`
|
||||
- Local test script: `/scripts/test-config-matrix.sh`
|
||||
|
||||
## 1. Problem Statement
|
||||
|
||||
### Current Gaps
|
||||
|
||||
Before this initiative, yaze only tested:
|
||||
1. **Default configurations**: `ci-linux`, `ci-macos`, `ci-windows` presets
|
||||
2. **Single feature toggles**: One dimension at a time
|
||||
3. **No interaction testing**: Missing edge cases like "GRPC=ON but REMOTE_AUTOMATION=OFF"
|
||||
|
||||
### Real Bugs Caught by Matrix Testing
|
||||
|
||||
Examples of issues a configuration matrix would catch:
|
||||
|
||||
**Example 1: GRPC Without Automation**
|
||||
```cmake
|
||||
# Broken: User enables gRPC but disables remote automation
|
||||
cmake -B build -DYAZE_ENABLE_GRPC=ON -DYAZE_ENABLE_REMOTE_AUTOMATION=OFF
|
||||
# Result: gRPC headers included but server code never compiled → link errors
|
||||
```
|
||||
|
||||
**Example 2: HTTP API Without CLI Stack**
|
||||
```cmake
|
||||
# Broken: User wants HTTP API but disables agent CLI
|
||||
cmake -B build -DYAZE_ENABLE_HTTP_API=ON -DYAZE_ENABLE_AGENT_CLI=OFF
|
||||
# Result: REST endpoints defined but no command dispatcher → runtime errors
|
||||
```
|
||||
|
||||
**Example 3: AI Runtime Without JSON**
|
||||
```cmake
|
||||
# Broken: User enables AI with Gemini but disables JSON
|
||||
cmake -B build -DYAZE_ENABLE_AI_RUNTIME=ON -DYAZE_ENABLE_JSON=OFF
|
||||
# Result: Gemini parser requires JSON but it's not available → compile errors
|
||||
```
|
||||
|
||||
**Example 4: Windows GRPC Version Mismatch**
|
||||
```cmake
|
||||
# Broken on Windows: gRPC version incompatible with MSVC ABI
|
||||
cmake -B build (with gRPC <1.67.1)
|
||||
# Result: Symbol errors, linker failures on Visual Studio
|
||||
```
|
||||
|
||||
## 2. Matrix Testing Approach
|
||||
|
||||
### Strategy: Smart, Not Exhaustive
|
||||
|
||||
Instead of testing all 2^18 = 262,144 combinations:
|
||||
|
||||
1. **Baseline**: Default configuration (most common user scenario)
|
||||
2. **Extremes**: All ON, All OFF (catch hidden assumptions)
|
||||
3. **Interactions**: Known problematic combinations
|
||||
4. **Tiers**: Progressive validation by feature complexity
|
||||
5. **Platforms**: Run critical tests on each OS
|
||||
|
||||
### Testing Tiers
|
||||
|
||||
#### Tier 1: Core Platforms (Every Commit)
|
||||
|
||||
**When**: On push to `master` or `develop`, every PR
|
||||
**What**: The three critical presets that users will actually use
|
||||
**Time**: ~15 minutes total
|
||||
|
||||
```
|
||||
ci-linux (gRPC + Agent, Linux)
|
||||
ci-macos (gRPC + Agent UI + Agent, macOS)
|
||||
ci-windows (gRPC, Windows)
|
||||
```
|
||||
|
||||
**Why**: These reflect real user workflows. If they break, users are impacted immediately.
|
||||
|
||||
#### Tier 2: Feature Combinations (Nightly / On-Demand)
|
||||
|
||||
**When**: Nightly at 2 AM UTC, manual dispatch, or `[matrix]` in commit message
|
||||
**What**: 6-8 specific flag combinations per platform
|
||||
**Time**: ~45 minutes total (parallel across 3 platforms × 7 configs)
|
||||
|
||||
```
|
||||
Linux: minimal, grpc-only, full-ai, cli-no-grpc, http-api, no-json
|
||||
macOS: minimal, full-ai, agent-ui, universal
|
||||
Windows: minimal, full-ai, grpc-remote, z3ed-cli
|
||||
```
|
||||
|
||||
**Why**: Tests dangerous interactions without exponential explosion. Each config tests a realistic user workflow.
|
||||
|
||||
#### Tier 3: Platform-Specific (As Needed)
|
||||
|
||||
**When**: When platform-specific issues arise
|
||||
**What**: Architecture-specific builds (ARM64, universal binary, etc.)
|
||||
**Time**: ~20 minutes
|
||||
|
||||
```
|
||||
Windows ARM64: Debug + Release
|
||||
macOS Universal: arm64 + x86_64
|
||||
Linux ARM: Cross-compile tests
|
||||
```
|
||||
|
||||
**Why**: Catches architecture-specific issues that only appear on target platforms.
|
||||
|
||||
### Configuration Selection Rationale
|
||||
|
||||
#### Why "Minimal"?
|
||||
|
||||
Tests the smallest viable configuration:
|
||||
- Validates core ROM reading/writing works without extras
|
||||
- Ensures build system doesn't have "feature X requires feature Y" errors
|
||||
- Catches over-linked libraries
|
||||
|
||||
#### Why "gRPC Only"?
|
||||
|
||||
Tests server-side automation without AI:
|
||||
- Validates gRPC infrastructure
|
||||
- Tests GUI automation system
|
||||
- Ensures protocol buffer compilation
|
||||
- Minimal dependencies for headless servers
|
||||
|
||||
#### Why "Full AI Stack"?
|
||||
|
||||
Tests maximum feature complexity:
|
||||
- All AI features enabled
|
||||
- Both Gemini and Ollama paths
|
||||
- Remote automation + Agent UI
|
||||
- Catches subtle linking issues with yaml-cpp, OpenSSL, etc.
|
||||
|
||||
#### Why "No JSON"?
|
||||
|
||||
Tests optional JSON dependency:
|
||||
- Ensures Ollama works without JSON
|
||||
- Validates graceful degradation
|
||||
- Catches hardcoded JSON assumptions
|
||||
|
||||
#### Why Platform-Specific?
|
||||
|
||||
Each platform has unique constraints:
|
||||
- **Windows**: MSVC ABI compatibility, gRPC version pinning
|
||||
- **macOS**: Universal binary (arm64 + x86_64), Homebrew dependencies
|
||||
- **Linux**: GCC version, glibc compatibility, system library versions
|
||||
|
||||
## 3. Problematic Flag Combinations
|
||||
|
||||
### Pattern 1: Hidden Dependencies (Fixed)
|
||||
|
||||
**Configuration**:
|
||||
```cmake
|
||||
YAZE_ENABLE_GRPC=ON
|
||||
YAZE_ENABLE_REMOTE_AUTOMATION=OFF # ← Inconsistent!
|
||||
```
|
||||
|
||||
**Problem**: gRPC headers included, but no automation server compiled → link errors
|
||||
|
||||
**Fix**: CMake now forces:
|
||||
```cmake
|
||||
if(YAZE_ENABLE_REMOTE_AUTOMATION AND NOT YAZE_ENABLE_GRPC)
|
||||
set(YAZE_ENABLE_GRPC ON ... FORCE)
|
||||
endif()
|
||||
```
|
||||
|
||||
**Matrix Test**: `grpc-only` configuration validates this constraint.
|
||||
|
||||
### Pattern 2: Orphaned Features (Fixed)
|
||||
|
||||
**Configuration**:
|
||||
```cmake
|
||||
YAZE_ENABLE_HTTP_API=ON
|
||||
YAZE_ENABLE_AGENT_CLI=OFF # ← HTTP API needs a CLI context!
|
||||
```
|
||||
|
||||
**Problem**: REST endpoints defined but no command dispatcher
|
||||
|
||||
**Fix**: CMake forces:
|
||||
```cmake
|
||||
if(YAZE_ENABLE_HTTP_API AND NOT YAZE_ENABLE_AGENT_CLI)
|
||||
set(YAZE_ENABLE_AGENT_CLI ON ... FORCE)
|
||||
endif()
|
||||
```
|
||||
|
||||
**Matrix Test**: `http-api` configuration validates this.
|
||||
|
||||
### Pattern 3: Optional Dependency Breakage
|
||||
|
||||
**Configuration**:
|
||||
```cmake
|
||||
YAZE_ENABLE_AI_RUNTIME=ON
|
||||
YAZE_ENABLE_JSON=OFF # ← Gemini requires JSON!
|
||||
```
|
||||
|
||||
**Problem**: Gemini service can't parse responses
|
||||
|
||||
**Status**: Currently relies on developer discipline
|
||||
**Matrix Test**: `no-json` + `full-ai` would catch this
|
||||
|
||||
### Pattern 4: Platform-Specific ABI Mismatch
|
||||
|
||||
**Configuration**: Windows with gRPC <1.67.1
|
||||
|
||||
**Problem**: MSVC ABI differences, symbol mismatch
|
||||
|
||||
**Status**: Documented in `ci-windows` preset
|
||||
**Matrix Test**: `grpc-remote` on Windows validates gRPC version
|
||||
|
||||
### Pattern 5: Architecture-Specific Issues
|
||||
|
||||
**Configuration**: macOS universal binary with platform-specific dependencies
|
||||
|
||||
**Problem**: Homebrew packages may not have arm64 support
|
||||
|
||||
**Status**: Requires dependency audit
|
||||
**Matrix Test**: `universal` on macOS tests both arm64 and x86_64
|
||||
|
||||
## 4. Matrix Testing Tools
|
||||
|
||||
### Local Testing: `scripts/test-config-matrix.sh`
|
||||
|
||||
Developers run this before pushing to validate all critical configurations locally.
|
||||
|
||||
#### Quick Start
|
||||
```bash
|
||||
# Test all configurations on current platform
|
||||
./scripts/test-config-matrix.sh
|
||||
|
||||
# Test specific configuration
|
||||
./scripts/test-config-matrix.sh --config minimal
|
||||
|
||||
# Smoke test (configure only, no build)
|
||||
./scripts/test-config-matrix.sh --smoke
|
||||
|
||||
# Verbose with timing
|
||||
./scripts/test-config-matrix.sh --verbose
|
||||
```
|
||||
|
||||
#### Features
|
||||
- **Fast feedback**: ~2-3 minutes for all configurations
|
||||
- **Smoke mode**: Configure without building (30 seconds)
|
||||
- **Platform detection**: Automatically runs platform-appropriate presets
|
||||
- **Result tracking**: Clear pass/fail summary
|
||||
- **Debug logging**: Full CMake/build output in `build_matrix/<config>/`
|
||||
|
||||
#### Output Example
|
||||
```
|
||||
Config: minimal
|
||||
Status: PASSED
|
||||
Description: No AI, no gRPC
|
||||
Build time: 2.3s
|
||||
|
||||
Config: full-ai
|
||||
Status: PASSED
|
||||
Description: All features enabled
|
||||
Build time: 45.2s
|
||||
|
||||
============
|
||||
2/2 configs passed
|
||||
============
|
||||
```
|
||||
|
||||
### CI Testing: `.github/workflows/matrix-test.yml`
|
||||
|
||||
Automated nightly testing across all three platforms.
|
||||
|
||||
#### Execution
|
||||
- **Trigger**: Nightly (2 AM UTC) + manual dispatch + `[matrix]` in commit message
|
||||
- **Platforms**: Linux (ubuntu-22.04), macOS (14), Windows (2022)
|
||||
- **Configurations per platform**: 6-7 distinct flag combinations
|
||||
- **Total runtime**: ~45 minutes (all jobs in parallel)
|
||||
- **Report**: Pass/fail summary + artifact upload on failure
|
||||
|
||||
#### What It Tests
|
||||
|
||||
**Linux (6 configs)**:
|
||||
1. `minimal` - No AI, no gRPC
|
||||
2. `grpc-only` - gRPC without automation
|
||||
3. `full-ai` - All features
|
||||
4. `cli-no-grpc` - CLI only
|
||||
5. `http-api` - REST endpoints
|
||||
6. `no-json` - Ollama mode
|
||||
|
||||
**macOS (4 configs)**:
|
||||
1. `minimal` - GUI, no AI
|
||||
2. `full-ai` - All features
|
||||
3. `agent-ui` - Agent UI panels only
|
||||
4. `universal` - arm64 + x86_64 binary
|
||||
|
||||
**Windows (4 configs)**:
|
||||
1. `minimal` - No AI
|
||||
2. `full-ai` - All features
|
||||
3. `grpc-remote` - gRPC + automation
|
||||
4. `z3ed-cli` - CLI executable
|
||||
|
||||
## 5. Integration with Development Workflow
|
||||
|
||||
### For Developers
|
||||
|
||||
Before pushing code to `develop` or `master`:
|
||||
|
||||
```bash
|
||||
# 1. Make changes
|
||||
git add src/...
|
||||
|
||||
# 2. Test locally
|
||||
./scripts/test-config-matrix.sh
|
||||
|
||||
# 3. If all pass, commit
|
||||
git commit -m "feature: add new thing"
|
||||
|
||||
# 4. Push
|
||||
git push
|
||||
```
|
||||
|
||||
### For CI/CD
|
||||
|
||||
**On every push to develop/master**:
|
||||
1. Standard CI runs (Tier 1 tests)
|
||||
2. Code quality checks
|
||||
3. If green, wait for nightly matrix test
|
||||
|
||||
**Nightly**:
|
||||
1. All Tier 2 combinations run in parallel
|
||||
2. Failures trigger alerts
|
||||
3. Success confirms no new cross-configuration issues
|
||||
|
||||
### For Pull Requests
|
||||
|
||||
Option A: **Include `[matrix]` in commit message**
|
||||
```bash
|
||||
git commit -m "fix: handle edge case [matrix]"
|
||||
git push # Triggers matrix test immediately
|
||||
```
|
||||
|
||||
Option B: **Manual dispatch**
|
||||
- Go to `.github/workflows/matrix-test.yml`
|
||||
- Click "Run workflow"
|
||||
- Select desired tier
|
||||
|
||||
## 6. Monitoring & Maintenance
|
||||
|
||||
### What to Watch
|
||||
|
||||
**Daily**: Check nightly matrix test results
|
||||
- Link: GitHub Actions > `Configuration Matrix Testing`
|
||||
- Alert if any configuration fails
|
||||
|
||||
**Weekly**: Review failure patterns
|
||||
- Are certain flag combinations always failing?
|
||||
- Is a platform having consistent issues?
|
||||
- Do dependencies need version updates?
|
||||
|
||||
**Monthly**: Audit the matrix configuration
|
||||
- Do new flags need testing?
|
||||
- Are deprecated flags still tested?
|
||||
- Can any Tier 2 configs be combined?
|
||||
|
||||
### Adding New Configurations
|
||||
|
||||
When adding a new feature flag:
|
||||
|
||||
1. **Update `cmake/options.cmake`**
|
||||
- Define the option
|
||||
- Document dependencies
|
||||
- Add constraint enforcement
|
||||
|
||||
2. **Update `/docs/internal/configuration-matrix.md`**
|
||||
- Add to Section 1 (flags)
|
||||
- Update Section 2 (constraints)
|
||||
- Add to relevant Tier in Section 3
|
||||
|
||||
3. **Update `/scripts/test-config-matrix.sh`**
|
||||
- Add to `CONFIGS` array
|
||||
- Test locally: `./scripts/test-config-matrix.sh --config new-config`
|
||||
|
||||
4. **Update `/.github/workflows/matrix-test.yml`**
|
||||
- Add matrix job entries for each platform
|
||||
- Estimate runtime impact
|
||||
|
||||
## 7. Troubleshooting Common Issues
|
||||
|
||||
### Issue: "Configuration failed" locally
|
||||
|
||||
```bash
|
||||
# Check the cmake log
|
||||
tail -50 build_matrix/<config>/config.log
|
||||
|
||||
# Check if presets exist
|
||||
cmake --list-presets
|
||||
```
|
||||
|
||||
### Issue: "Build failed" locally
|
||||
|
||||
```bash
|
||||
# Get full build output
|
||||
./scripts/test-config-matrix.sh --config <name> --verbose
|
||||
|
||||
# Check for missing dependencies
|
||||
# On macOS: brew list | grep <dep>
|
||||
# On Linux: apt list --installed | grep <dep>
|
||||
```
|
||||
|
||||
### Issue: Test passes locally but fails in CI
|
||||
|
||||
**Likely causes**:
|
||||
1. Different CMake version (CI uses latest)
|
||||
2. Different compiler (GCC vs Clang vs MSVC)
|
||||
3. Missing system library
|
||||
|
||||
**Solutions**:
|
||||
- Check `.github/actions/setup-build` for CI environment
|
||||
- Match local compiler: `cmake --preset ci-linux -DCMAKE_CXX_COMPILER=gcc-13`
|
||||
- Add dependency: Update `cmake/dependencies.cmake`
|
||||
|
||||
## 8. Future Improvements
|
||||
|
||||
### Short Term (Next Sprint)
|
||||
|
||||
- [ ] Add binary size tracking per configuration
|
||||
- [ ] Add compile time benchmarks
|
||||
- [ ] Auto-generate configuration compatibility matrix chart
|
||||
- [ ] Add `--ci-mode` flag to local script (simulates GH Actions)
|
||||
|
||||
### Medium Term (Next Quarter)
|
||||
|
||||
- [ ] Integrate with release pipeline (validate all Tier 2 before release)
|
||||
- [ ] Add performance regression tests per configuration
|
||||
- [ ] Create configuration validator tool (warns on suspicious combinations)
|
||||
- [ ] Document platform-specific dependency versions
|
||||
|
||||
### Long Term (Next Year)
|
||||
|
||||
- [ ] Separate `YAZE_ENABLE_AI` and `YAZE_ENABLE_AI_RUNTIME` (currently coupled)
|
||||
- [ ] Add Tier 0 (smoke tests) that run on every commit
|
||||
- [ ] Create web dashboard of matrix test results
|
||||
- [ ] Add "configuration suggestion" tool (infer optimal flags for user's hardware)
|
||||
|
||||
## 9. Reference: Configuration Categories
|
||||
|
||||
### GUI User (Desktop)
|
||||
```cmake
|
||||
YAZE_BUILD_GUI=ON
|
||||
YAZE_BUILD_AGENT_UI=ON
|
||||
YAZE_ENABLE_GRPC=OFF # No network overhead
|
||||
YAZE_ENABLE_AI=OFF # Unnecessary for GUI-only
|
||||
```
|
||||
|
||||
### Server/Headless (Automation)
|
||||
```cmake
|
||||
YAZE_BUILD_GUI=OFF
|
||||
YAZE_ENABLE_GRPC=ON
|
||||
YAZE_ENABLE_REMOTE_AUTOMATION=ON
|
||||
YAZE_ENABLE_AI=OFF # Optional
|
||||
```
|
||||
|
||||
### Full-Featured Developer
|
||||
```cmake
|
||||
YAZE_BUILD_GUI=ON
|
||||
YAZE_BUILD_AGENT_UI=ON
|
||||
YAZE_ENABLE_GRPC=ON
|
||||
YAZE_ENABLE_REMOTE_AUTOMATION=ON
|
||||
YAZE_ENABLE_AI_RUNTIME=ON
|
||||
YAZE_ENABLE_HTTP_API=ON
|
||||
```
|
||||
|
||||
### CLI-Only (z3ed Agent)
|
||||
```cmake
|
||||
YAZE_BUILD_GUI=OFF
|
||||
YAZE_BUILD_Z3ED=ON
|
||||
YAZE_ENABLE_GRPC=ON
|
||||
YAZE_ENABLE_AI_RUNTIME=ON
|
||||
YAZE_ENABLE_HTTP_API=ON
|
||||
```
|
||||
|
||||
### Minimum (Embedded/Library)
|
||||
```cmake
|
||||
YAZE_BUILD_GUI=OFF
|
||||
YAZE_BUILD_CLI=OFF
|
||||
YAZE_BUILD_TESTS=OFF
|
||||
YAZE_ENABLE_GRPC=OFF
|
||||
YAZE_ENABLE_AI=OFF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Questions?** Check `/docs/internal/configuration-matrix.md` or ask in coordination-board.md.
|
||||
@@ -0,0 +1,843 @@
|
||||
# YAZE Testing Strategy
|
||||
|
||||
## Purpose
|
||||
|
||||
This document defines the comprehensive testing strategy for YAZE, explaining what each test level catches, when to run tests, and how to debug failures. It serves as the authoritative guide for developers and AI agents.
|
||||
|
||||
**Last Updated**: 2025-11-20
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Testing Philosophy](#1-testing-philosophy)
|
||||
2. [Test Pyramid](#2-test-pyramid)
|
||||
3. [Test Categories](#3-test-categories)
|
||||
4. [When to Run Tests](#4-when-to-run-tests)
|
||||
5. [Test Organization](#5-test-organization)
|
||||
6. [Platform-Specific Testing](#6-platform-specific-testing)
|
||||
7. [CI/CD Testing](#7-cicd-testing)
|
||||
8. [Debugging Test Failures](#8-debugging-test-failures)
|
||||
|
||||
---
|
||||
|
||||
## 1. Testing Philosophy
|
||||
|
||||
### Core Principles
|
||||
|
||||
1. **Fast Feedback**: Developers should get test results in <2 minutes locally
|
||||
2. **Fail Early**: Catch issues at the lowest/fastest test level possible
|
||||
3. **Confidence**: Tests should give confidence that code works across platforms
|
||||
4. **Automation**: All tests should be automatable in CI
|
||||
5. **Clarity**: Test failures should clearly indicate what broke and where
|
||||
|
||||
### Testing Goals
|
||||
|
||||
- **Prevent Regressions**: Ensure new changes don't break existing functionality
|
||||
- **Catch Build Issues**: Detect compilation/linking problems before CI
|
||||
- **Validate Logic**: Verify algorithms and data structures work correctly
|
||||
- **Test Integration**: Ensure components work together
|
||||
- **Validate UX**: Confirm UI workflows function as expected
|
||||
|
||||
---
|
||||
|
||||
## 2. Test Pyramid
|
||||
|
||||
YAZE uses a **5-level testing pyramid**, from fastest (bottom) to slowest (top):
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ E2E Tests (E2E) │ Minutes │ Few tests
|
||||
│ Full UI workflows │ │ High value
|
||||
├─────────────────────┤ │
|
||||
┌─ │ Integration (INT) │ Seconds │
|
||||
│ │ Multi-component │ │
|
||||
│ ├─────────────────────┤ │
|
||||
Tests │ │ Unit Tests (UT) │ <1 second │
|
||||
│ │ Isolated logic │ │
|
||||
└─ ├─────────────────────┤ │
|
||||
│ Symbol Validation │ Minutes │
|
||||
│ ODR, conflicts │ ▼
|
||||
├─────────────────────┤
|
||||
│ Smoke Compilation │ ~2 min
|
||||
│ Header checks │
|
||||
Build ├─────────────────────┤
|
||||
Checks │ Config Validation │ ~10 sec
|
||||
│ CMake, includes │
|
||||
├─────────────────────┤
|
||||
│ Static Analysis │ <1 sec │ Many checks
|
||||
│ Format, lint │ │ Fast feedback
|
||||
└─────────────────────┘ ▼
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Test Categories
|
||||
|
||||
### Level 0: Static Analysis (< 1 second)
|
||||
|
||||
**Purpose**: Catch trivial issues before compilation
|
||||
|
||||
**Tools**:
|
||||
- `clang-format` - Code formatting
|
||||
- `clang-tidy` - Static analysis (subset of files)
|
||||
- `cppcheck` - Additional static checks
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ Formatting violations
|
||||
- ✅ Common code smells
|
||||
- ✅ Potential null pointer dereferences
|
||||
- ✅ Unused variables
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Build system issues
|
||||
- ❌ Linking problems
|
||||
- ❌ Runtime logic errors
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
# Format check (don't modify)
|
||||
cmake --build build --target yaze-format-check
|
||||
|
||||
# Static analysis on changed files
|
||||
git diff --name-only HEAD | grep -E '\.(cc|h)$' | \
|
||||
xargs clang-tidy-14 --header-filter='src/.*'
|
||||
```
|
||||
|
||||
**Run in CI**: ✅ Every PR (code-quality job)
|
||||
|
||||
---
|
||||
|
||||
### Level 1: Configuration Validation (< 10 seconds)
|
||||
|
||||
**Purpose**: Validate CMake configuration without full compilation
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ CMake syntax errors
|
||||
- ✅ Missing dependencies (immediate)
|
||||
- ✅ Invalid preset combinations
|
||||
- ✅ Include path misconfigurations
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Actual compilation errors
|
||||
- ❌ Header availability issues
|
||||
- ❌ Linking problems
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
# Validate a preset
|
||||
./scripts/pre-push-test.sh --config-only
|
||||
|
||||
# Test multiple presets
|
||||
for preset in mac-dbg mac-rel mac-ai; do
|
||||
cmake --preset "$preset" --list-presets > /dev/null
|
||||
done
|
||||
```
|
||||
|
||||
**Run in CI**: 🔄 Proposed (new job)
|
||||
|
||||
---
|
||||
|
||||
### Level 2: Smoke Compilation (< 2 minutes)
|
||||
|
||||
**Purpose**: Quick compilation check to catch header/include issues
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ Missing headers
|
||||
- ✅ Include path problems
|
||||
- ✅ Preprocessor errors
|
||||
- ✅ Template instantiation issues
|
||||
- ✅ Platform-specific compilation
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Linking errors
|
||||
- ❌ Symbol conflicts
|
||||
- ❌ Runtime behavior
|
||||
|
||||
**Strategy**:
|
||||
- Compile 1-2 representative files per library
|
||||
- Focus on files with many includes
|
||||
- Test platform-specific code paths
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
./scripts/pre-push-test.sh --smoke-only
|
||||
```
|
||||
|
||||
**Run in CI**: 🔄 Proposed (compile-only job, <5 min)
|
||||
|
||||
---
|
||||
|
||||
### Level 3: Symbol Validation (< 5 minutes)
|
||||
|
||||
**Purpose**: Detect symbol conflicts and ODR violations
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ Duplicate symbol definitions
|
||||
- ✅ ODR (One Definition Rule) violations
|
||||
- ✅ Missing symbols (link errors)
|
||||
- ✅ Symbol visibility issues
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Runtime logic errors
|
||||
- ❌ Performance issues
|
||||
- ❌ Memory leaks
|
||||
|
||||
**Tools**:
|
||||
- `nm` (Unix/macOS) - Symbol inspection
|
||||
- `dumpbin /symbols` (Windows) - Symbol inspection
|
||||
- `c++filt` - Symbol demangling
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
./scripts/verify-symbols.sh
|
||||
```
|
||||
|
||||
**Run in CI**: 🔄 Proposed (symbol-check job)
|
||||
|
||||
---
|
||||
|
||||
### Level 4: Unit Tests (< 1 second each)
|
||||
|
||||
**Purpose**: Fast, isolated testing of individual components
|
||||
|
||||
**Location**: `test/unit/`
|
||||
|
||||
**Characteristics**:
|
||||
- No external dependencies (ROM, network, filesystem)
|
||||
- Mocked dependencies via test doubles
|
||||
- Single-component focus
|
||||
- Deterministic (no flaky tests)
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ Algorithm correctness
|
||||
- ✅ Data structure behavior
|
||||
- ✅ Edge cases and error handling
|
||||
- ✅ Isolated component logic
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Component interactions
|
||||
- ❌ ROM data handling
|
||||
- ❌ UI workflows
|
||||
- ❌ Platform-specific issues
|
||||
|
||||
**Examples**:
|
||||
- `test/unit/core/hex_test.cc` - Hex conversion logic
|
||||
- `test/unit/gfx/snes_palette_test.cc` - Palette operations
|
||||
- `test/unit/zelda3/object_parser_test.cc` - Object parsing
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
./build/bin/yaze_test --unit
|
||||
```
|
||||
|
||||
**Run in CI**: ✅ Every PR (test job)
|
||||
|
||||
**Writing Guidelines**:
|
||||
```cpp
|
||||
// GOOD: Fast, isolated, no dependencies
|
||||
TEST(UnitTest, SnesPaletteConversion) {
|
||||
gfx::SnesColor color(0x7C00); // Red in SNES format
|
||||
EXPECT_EQ(color.red(), 31);
|
||||
EXPECT_EQ(color.rgb(), 0xFF0000);
|
||||
}
|
||||
|
||||
// BAD: Depends on ROM file
|
||||
TEST(UnitTest, LoadOverworldMapColors) {
|
||||
Rom rom;
|
||||
rom.LoadFromFile("zelda3.sfc"); // ❌ External dependency
|
||||
auto colors = rom.ReadPalette(0x1BD308);
|
||||
EXPECT_EQ(colors.size(), 128);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Level 5: Integration Tests (1-10 seconds each)
|
||||
|
||||
**Purpose**: Test interactions between components
|
||||
|
||||
**Location**: `test/integration/`
|
||||
|
||||
**Characteristics**:
|
||||
- Multi-component interactions
|
||||
- May require ROM files (optional)
|
||||
- Real implementations (minimal mocking)
|
||||
- Slower but more realistic
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ Component interaction bugs
|
||||
- ✅ Data flow between systems
|
||||
- ✅ ROM operations
|
||||
- ✅ Resource management
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Full UI workflows
|
||||
- ❌ User interactions
|
||||
- ❌ Visual rendering
|
||||
|
||||
**Examples**:
|
||||
- `test/integration/asar_integration_test.cc` - Asar patching + ROM
|
||||
- `test/integration/dungeon_editor_v2_test.cc` - Dungeon editor logic
|
||||
- `test/integration/zelda3/overworld_integration_test.cc` - Overworld loading
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
./build/bin/yaze_test --integration
|
||||
```
|
||||
|
||||
**Run in CI**: ⚠️ Limited (develop/master only, not PRs)
|
||||
|
||||
**Writing Guidelines**:
|
||||
```cpp
|
||||
// GOOD: Tests component interaction
|
||||
TEST(IntegrationTest, AsarPatchRom) {
|
||||
Rom rom;
|
||||
ASSERT_TRUE(rom.LoadFromFile("zelda3.sfc"));
|
||||
|
||||
AsarWrapper asar;
|
||||
auto result = asar.ApplyPatch("test.asm", rom);
|
||||
ASSERT_TRUE(result.ok());
|
||||
|
||||
// Verify ROM was patched correctly
|
||||
EXPECT_EQ(rom.ReadByte(0x12345), 0xAB);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Level 6: End-to-End (E2E) Tests (10-60 seconds each)
|
||||
|
||||
**Purpose**: Validate full user workflows through the UI
|
||||
|
||||
**Location**: `test/e2e/`
|
||||
|
||||
**Characteristics**:
|
||||
- Full application stack
|
||||
- Real UI (ImGui + SDL)
|
||||
- User interaction simulation
|
||||
- Requires display/window system
|
||||
|
||||
**What It Catches**:
|
||||
- ✅ Complete user workflows
|
||||
- ✅ UI responsiveness
|
||||
- ✅ Visual rendering (screenshots)
|
||||
- ✅ Cross-editor interactions
|
||||
|
||||
**What It Misses**:
|
||||
- ❌ Performance issues
|
||||
- ❌ Memory leaks (unless with sanitizers)
|
||||
- ❌ Platform-specific edge cases
|
||||
|
||||
**Tools**:
|
||||
- `ImGuiTestEngine` - UI automation
|
||||
- `ImGui_TestEngineHook_*` - Test engine integration
|
||||
|
||||
**Examples**:
|
||||
- `test/e2e/dungeon_editor_smoke_test.cc` - Open dungeon editor, load ROM
|
||||
- `test/e2e/canvas_selection_test.cc` - Select tiles on canvas
|
||||
- `test/e2e/overworld/overworld_e2e_test.cc` - Overworld editing workflow
|
||||
|
||||
**Run Locally**:
|
||||
```bash
|
||||
# Headless (fast)
|
||||
./build/bin/yaze_test --e2e
|
||||
|
||||
# With GUI visible (slow, for debugging)
|
||||
./build/bin/yaze_test --e2e --show-gui --normal
|
||||
```
|
||||
|
||||
**Run in CI**: ⚠️ macOS only (z3ed-agent-test job)
|
||||
|
||||
**Writing Guidelines**:
|
||||
```cpp
|
||||
void E2ETest_DungeonEditorSmokeTest(ImGuiTestContext* ctx) {
|
||||
ctx->SetRef("DockSpaceViewport");
|
||||
|
||||
// Open File menu
|
||||
ctx->MenuCheck("File/Load ROM", true);
|
||||
|
||||
// Enter ROM path
|
||||
ctx->ItemInput("##rom_path");
|
||||
ctx->KeyCharsAppend("zelda3.sfc");
|
||||
|
||||
// Click Load button
|
||||
ctx->ItemClick("Load");
|
||||
|
||||
// Verify editor opened
|
||||
ctx->WindowFocus("Dungeon Editor");
|
||||
IM_CHECK(ctx->WindowIsOpen("Dungeon Editor"));
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. When to Run Tests
|
||||
|
||||
### 4.1 During Development (Continuous)
|
||||
|
||||
**Frequency**: After every significant change
|
||||
|
||||
**Run**:
|
||||
- Level 0: Static analysis (IDE integration)
|
||||
- Level 4: Unit tests for changed components
|
||||
|
||||
**Tools**:
|
||||
- VSCode C++ extension (clang-tidy)
|
||||
- File watchers (`entr`, `watchexec`)
|
||||
|
||||
```bash
|
||||
# Watch mode for unit tests
|
||||
find src test -name "*.cc" | entr -c ./build/bin/yaze_test --unit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.2 Before Committing (Pre-Commit)
|
||||
|
||||
**Frequency**: Before `git commit`
|
||||
|
||||
**Run**:
|
||||
- Level 0: Format check
|
||||
- Level 4: Unit tests for changed files
|
||||
|
||||
**Setup** (optional):
|
||||
```bash
|
||||
# Install pre-commit hook
|
||||
cat > .git/hooks/pre-commit << 'EOF'
|
||||
#!/bin/bash
|
||||
# Format check
|
||||
if ! cmake --build build --target yaze-format-check; then
|
||||
echo "❌ Format check failed. Run: cmake --build build --target yaze-format"
|
||||
exit 1
|
||||
fi
|
||||
EOF
|
||||
chmod +x .git/hooks/pre-commit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4.3 Before Pushing (Pre-Push)
|
||||
|
||||
**Frequency**: Before `git push` to remote
|
||||
|
||||
**Run**:
|
||||
- Level 0: Static analysis
|
||||
- Level 1: Configuration validation
|
||||
- Level 2: Smoke compilation
|
||||
- Level 3: Symbol validation
|
||||
- Level 4: All unit tests
|
||||
|
||||
**Time Budget**: < 2 minutes
|
||||
|
||||
**Command**:
|
||||
```bash
|
||||
# Unix/macOS
|
||||
./scripts/pre-push-test.sh
|
||||
|
||||
# Windows
|
||||
.\scripts\pre-push-test.ps1
|
||||
```
|
||||
|
||||
**What It Prevents**:
|
||||
- 90% of CI build failures
|
||||
- ODR violations
|
||||
- Include path issues
|
||||
- Symbol conflicts
|
||||
|
||||
---
|
||||
|
||||
### 4.4 After Pull Request Creation
|
||||
|
||||
**Frequency**: Automatically on every PR
|
||||
|
||||
**Run** (CI):
|
||||
- Level 0: Static analysis (code-quality job)
|
||||
- Level 2: Full compilation (build job)
|
||||
- Level 4: Unit tests (test job)
|
||||
- Level 4: Stable tests (test job)
|
||||
|
||||
**Time**: 15-20 minutes
|
||||
|
||||
**Outcome**: ✅ Required for merge
|
||||
|
||||
---
|
||||
|
||||
### 4.5 After Merge to Develop/Master
|
||||
|
||||
**Frequency**: Post-merge (develop/master only)
|
||||
|
||||
**Run** (CI):
|
||||
- All PR checks
|
||||
- Level 5: Integration tests
|
||||
- Level 6: E2E tests (macOS)
|
||||
- Memory sanitizers (Linux)
|
||||
- Full AI stack tests (Windows/macOS)
|
||||
|
||||
**Time**: 30-45 minutes
|
||||
|
||||
**Outcome**: ⚠️ Optional (but monitored)
|
||||
|
||||
---
|
||||
|
||||
### 4.6 Before Release
|
||||
|
||||
**Frequency**: Release candidates
|
||||
|
||||
**Run**:
|
||||
- All CI tests
|
||||
- Manual exploratory testing
|
||||
- Performance benchmarks
|
||||
- Cross-platform smoke testing
|
||||
|
||||
**Checklist**: See `docs/internal/release-checklist.md`
|
||||
|
||||
---
|
||||
|
||||
## 5. Test Organization
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
test/
|
||||
├── unit/ # Level 4: Fast, isolated tests
|
||||
│ ├── core/ # Core utilities
|
||||
│ ├── gfx/ # Graphics system
|
||||
│ ├── zelda3/ # Game logic
|
||||
│ ├── cli/ # CLI components
|
||||
│ ├── gui/ # GUI widgets
|
||||
│ └── emu/ # Emulator
|
||||
│
|
||||
├── integration/ # Level 5: Multi-component tests
|
||||
│ ├── ai/ # AI integration
|
||||
│ ├── editor/ # Editor systems
|
||||
│ └── zelda3/ # Game system integration
|
||||
│
|
||||
├── e2e/ # Level 6: Full workflow tests
|
||||
│ ├── overworld/ # Overworld editor E2E
|
||||
│ ├── zscustomoverworld/ # ZSCustomOverworld E2E
|
||||
│ └── rom_dependent/ # ROM-required E2E
|
||||
│
|
||||
├── benchmarks/ # Performance tests
|
||||
├── mocks/ # Test doubles
|
||||
└── test_utils.cc # Test utilities
|
||||
```
|
||||
|
||||
### Naming Conventions
|
||||
|
||||
**Files**:
|
||||
- Unit: `<component>_test.cc`
|
||||
- Integration: `<feature>_integration_test.cc`
|
||||
- E2E: `<workflow>_e2e_test.cc`
|
||||
|
||||
**Test Names**:
|
||||
```cpp
|
||||
// Unit
|
||||
TEST(UnitTest, ComponentName_Behavior_ExpectedOutcome) { }
|
||||
|
||||
// Integration
|
||||
TEST(IntegrationTest, SystemName_Interaction_ExpectedOutcome) { }
|
||||
|
||||
// E2E
|
||||
void E2ETest_WorkflowName_StepDescription(ImGuiTestContext* ctx) { }
|
||||
```
|
||||
|
||||
### Test Labels (CTest)
|
||||
|
||||
Tests are labeled for selective execution:
|
||||
|
||||
- `stable` - No ROM required, fast
|
||||
- `unit` - Unit tests only
|
||||
- `integration` - Integration tests
|
||||
- `e2e` - End-to-end tests
|
||||
- `rom_dependent` - Requires ROM file
|
||||
|
||||
```bash
|
||||
# Run only stable tests
|
||||
ctest --preset stable
|
||||
|
||||
# Run unit tests
|
||||
./build/bin/yaze_test --unit
|
||||
|
||||
# Run ROM-dependent tests
|
||||
./build/bin/yaze_test --rom-dependent --rom-path zelda3.sfc
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Platform-Specific Testing
|
||||
|
||||
### 6.1 Cross-Platform Considerations
|
||||
|
||||
**Different Linker Behavior**:
|
||||
- macOS: More permissive (weak symbols)
|
||||
- Linux: Strict ODR enforcement
|
||||
- Windows: MSVC vs clang-cl differences
|
||||
|
||||
**Strategy**: Test on Linux for strictest validation
|
||||
|
||||
**Different Compilers**:
|
||||
- GCC (Linux): `-Werror=odr`
|
||||
- Clang (macOS/Linux): More warnings
|
||||
- clang-cl (Windows): MSVC compatibility mode
|
||||
|
||||
**Strategy**: Use verbose presets (`*-dbg-v`) to see all warnings
|
||||
|
||||
### 6.2 Local Cross-Platform Testing
|
||||
|
||||
**For macOS Developers**:
|
||||
```bash
|
||||
# Test Linux build locally (future: Docker)
|
||||
docker run --rm -v $(pwd):/workspace yaze-linux-builder \
|
||||
cmake --preset lin-dbg && cmake --build build --target yaze
|
||||
```
|
||||
|
||||
**For Linux Developers**:
|
||||
```bash
|
||||
# Test macOS build locally (requires macOS VM)
|
||||
# Future: GitHub Actions remote testing
|
||||
```
|
||||
|
||||
**For Windows Developers**:
|
||||
```powershell
|
||||
# Test via WSL (Linux build)
|
||||
wsl bash -c "cmake --preset lin-dbg && cmake --build build"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. CI/CD Testing
|
||||
|
||||
### 7.1 Current CI Matrix
|
||||
|
||||
| Job | Platform | Preset | Duration | Runs On |
|
||||
|-----|----------|--------|----------|---------|
|
||||
| build | Ubuntu 22.04 | ci-linux | ~15 min | All PRs |
|
||||
| build | macOS 14 | ci-macos | ~20 min | All PRs |
|
||||
| build | Windows 2022 | ci-windows | ~25 min | All PRs |
|
||||
| test | Ubuntu 22.04 | ci-linux | ~5 min | All PRs |
|
||||
| test | macOS 14 | ci-macos | ~5 min | All PRs |
|
||||
| test | Windows 2022 | ci-windows | ~5 min | All PRs |
|
||||
| windows-agent | Windows 2022 | ci-windows-ai | ~30 min | Post-merge |
|
||||
| code-quality | Ubuntu 22.04 | - | ~2 min | All PRs |
|
||||
| memory-sanitizer | Ubuntu 22.04 | sanitizer | ~20 min | PRs |
|
||||
| z3ed-agent-test | macOS 14 | mac-ai | ~15 min | Develop/master |
|
||||
|
||||
### 7.2 Proposed CI Improvements
|
||||
|
||||
**New Jobs**:
|
||||
|
||||
1. **compile-only** (< 5 min)
|
||||
- Run BEFORE full build
|
||||
- Compile 10-20 representative files
|
||||
- Fast feedback on include issues
|
||||
|
||||
2. **symbol-check** (< 3 min)
|
||||
- Run AFTER build
|
||||
- Detect ODR violations
|
||||
- Platform-specific (Linux most strict)
|
||||
|
||||
3. **config-validation** (< 2 min)
|
||||
- Test all presets can configure
|
||||
- Validate include paths
|
||||
- Catch CMake errors early
|
||||
|
||||
**Benefits**:
|
||||
- 90% of issues caught in <5 minutes
|
||||
- Reduced wasted CI time
|
||||
- Faster developer feedback
|
||||
|
||||
---
|
||||
|
||||
## 8. Debugging Test Failures
|
||||
|
||||
### 8.1 Local Test Failures
|
||||
|
||||
**Unit Test Failure**:
|
||||
```bash
|
||||
# Run specific test
|
||||
./build/bin/yaze_test "TestSuiteName.TestName"
|
||||
|
||||
# Run with verbose output
|
||||
./build/bin/yaze_test --verbose "TestSuiteName.*"
|
||||
|
||||
# Run with debugger
|
||||
lldb -- ./build/bin/yaze_test "TestSuiteName.TestName"
|
||||
```
|
||||
|
||||
**Integration Test Failure**:
|
||||
```bash
|
||||
# Ensure ROM is available
|
||||
export YAZE_TEST_ROM_PATH=/path/to/zelda3.sfc
|
||||
./build/bin/yaze_test --integration --verbose
|
||||
```
|
||||
|
||||
**E2E Test Failure**:
|
||||
```bash
|
||||
# Run with GUI visible (slow motion)
|
||||
./build/bin/yaze_test --e2e --show-gui --cinematic
|
||||
|
||||
# Take screenshots on failure
|
||||
YAZE_E2E_SCREENSHOT_DIR=/tmp/screenshots \
|
||||
./build/bin/yaze_test --e2e
|
||||
```
|
||||
|
||||
### 8.2 CI Test Failures
|
||||
|
||||
**Step 1: Identify Job**
|
||||
- Which platform failed? (Linux/macOS/Windows)
|
||||
- Which job failed? (build/test/code-quality)
|
||||
- Which test failed? (check CI logs)
|
||||
|
||||
**Step 2: Reproduce Locally**
|
||||
```bash
|
||||
# Use matching CI preset
|
||||
cmake --preset ci-linux # or ci-macos, ci-windows
|
||||
cmake --build build
|
||||
|
||||
# Run same test
|
||||
./build/bin/yaze_test --unit
|
||||
```
|
||||
|
||||
**Step 3: Platform-Specific Issues**
|
||||
|
||||
**If Windows-only failure**:
|
||||
- Check for MSVC/clang-cl differences
|
||||
- Validate include paths (Abseil, gRPC)
|
||||
- Check preprocessor macros (`_WIN32`, etc.)
|
||||
|
||||
**If Linux-only failure**:
|
||||
- Check for ODR violations (duplicate symbols)
|
||||
- Validate linker flags
|
||||
- Check for gflags `FLAGS` conflicts
|
||||
|
||||
**If macOS-only failure**:
|
||||
- Check for framework dependencies
|
||||
- Validate Objective-C++ code
|
||||
- Check for Apple SDK issues
|
||||
|
||||
### 8.3 Build Failures
|
||||
|
||||
**CMake Configuration Failure**:
|
||||
```bash
|
||||
# Verbose CMake output
|
||||
cmake --preset ci-linux -DCMAKE_VERBOSE_MAKEFILE=ON
|
||||
|
||||
# Check CMake cache
|
||||
cat build/CMakeCache.txt | grep ERROR
|
||||
|
||||
# Check include paths
|
||||
cmake --build build --target help | grep INCLUDE
|
||||
```
|
||||
|
||||
**Compilation Failure**:
|
||||
```bash
|
||||
# Verbose compilation
|
||||
cmake --build build --preset ci-linux -v
|
||||
|
||||
# Single file compilation
|
||||
cd build
|
||||
ninja -v path/to/file.cc.o
|
||||
```
|
||||
|
||||
**Linking Failure**:
|
||||
```bash
|
||||
# Check symbols in library
|
||||
nm -gU build/lib/libyaze_core.a | grep FLAGS
|
||||
|
||||
# Check duplicate symbols
|
||||
./scripts/verify-symbols.sh --verbose
|
||||
|
||||
# Check ODR violations
|
||||
nm build/lib/*.a | c++filt | grep " [TDR] " | sort | uniq -d
|
||||
```
|
||||
|
||||
### 8.4 Common Failure Patterns
|
||||
|
||||
**Pattern 1: "FLAGS redefined"**
|
||||
- **Cause**: gflags creates `FLAGS_*` symbols in multiple TUs
|
||||
- **Solution**: Define FLAGS in exactly one .cc file
|
||||
- **Prevention**: Run `./scripts/verify-symbols.sh`
|
||||
|
||||
**Pattern 2: "Abseil headers not found"**
|
||||
- **Cause**: Include paths not propagated from gRPC
|
||||
- **Solution**: Add explicit Abseil include directory
|
||||
- **Prevention**: Run smoke compilation test
|
||||
|
||||
**Pattern 3: "std::filesystem not available"**
|
||||
- **Cause**: Missing C++17/20 standard flag
|
||||
- **Solution**: Add `/std:c++latest` (Windows) or `-std=c++20`
|
||||
- **Prevention**: Validate compiler flags in CMake
|
||||
|
||||
**Pattern 4: "Multiple definition of X"**
|
||||
- **Cause**: Header-only library included in multiple TUs
|
||||
- **Solution**: Use `inline` or move to single TU
|
||||
- **Prevention**: Symbol conflict checker
|
||||
|
||||
---
|
||||
|
||||
## 9. Best Practices
|
||||
|
||||
### 9.1 Writing Tests
|
||||
|
||||
1. **Fast**: Unit tests should complete in <100ms
|
||||
2. **Isolated**: No external dependencies (files, network, ROM)
|
||||
3. **Deterministic**: Same input → same output, always
|
||||
4. **Clear**: Test name describes what is tested
|
||||
5. **Focused**: One assertion per test (ideally)
|
||||
|
||||
### 9.2 Test Data
|
||||
|
||||
**Good**:
|
||||
```cpp
|
||||
// Inline test data
|
||||
const uint8_t palette_data[] = {0x00, 0x7C, 0xFF, 0x03};
|
||||
auto palette = gfx::SnesPalette(palette_data, 4);
|
||||
```
|
||||
|
||||
**Bad**:
|
||||
```cpp
|
||||
// External file dependency
|
||||
auto palette = gfx::SnesPalette::LoadFromFile("test_palette.bin"); // ❌
|
||||
```
|
||||
|
||||
### 9.3 Assertions
|
||||
|
||||
**Prefer `EXPECT_*` over `ASSERT_*`**:
|
||||
- `EXPECT_*` continues on failure (more info)
|
||||
- `ASSERT_*` stops immediately (for fatal errors)
|
||||
|
||||
```cpp
|
||||
// Good: Continue testing after failure
|
||||
EXPECT_EQ(color.red(), 31);
|
||||
EXPECT_EQ(color.green(), 0);
|
||||
EXPECT_EQ(color.blue(), 0);
|
||||
|
||||
// Bad: Only see first failure
|
||||
ASSERT_EQ(color.red(), 31);
|
||||
ASSERT_EQ(color.green(), 0); // Never executed if red fails
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Resources
|
||||
|
||||
### Documentation
|
||||
- **Gap Analysis**: `docs/internal/testing/gap-analysis.md`
|
||||
- **Pre-Push Checklist**: `docs/internal/testing/pre-push-checklist.md`
|
||||
- **Quick Reference**: `docs/public/build/quick-reference.md`
|
||||
|
||||
### Scripts
|
||||
- **Pre-Push Test**: `scripts/pre-push-test.sh` (Unix/macOS)
|
||||
- **Pre-Push Test**: `scripts/pre-push-test.ps1` (Windows)
|
||||
- **Symbol Checker**: `scripts/verify-symbols.sh`
|
||||
|
||||
### CI Configuration
|
||||
- **Workflow**: `.github/workflows/ci.yml`
|
||||
- **Composite Actions**: `.github/actions/`
|
||||
|
||||
### Tools
|
||||
- **Test Runner**: `test/yaze_test.cc`
|
||||
- **Test Utilities**: `test/test_utils.h`
|
||||
- **Google Test**: https://google.github.io/googletest/
|
||||
- **ImGui Test Engine**: https://github.com/ocornut/imgui_test_engine
|
||||
@@ -709,6 +709,6 @@ LOG_WARNING("Emulator",
|
||||
## Related Documentation
|
||||
|
||||
- **FileSystemTool**: `filesystem-tool.md`
|
||||
- **AI Infrastructure**: `ai-infrastructure-initiative.md`
|
||||
- **AI Infrastructure (archived)**: `archive/legacy-2025-11/ai-infrastructure-initiative-archived-2025-11-25.md`
|
||||
- **Agent Architecture**: `agent-architecture.md`
|
||||
- **Development Plan**: `../plans/ai-assisted-development-plan.md`
|
||||
@@ -254,5 +254,5 @@ agent.SetAIEnabled(true);
|
||||
## Related Documentation
|
||||
|
||||
- [Build Tool Documentation](filesystem-tool.md)
|
||||
- [AI Infrastructure Initiative](ai-infrastructure-initiative.md)
|
||||
- [Test Suite Configuration](../../test-suite-configuration.md)
|
||||
- [AI Infrastructure Initiative (archived)](archive/legacy-2025-11/ai-infrastructure-initiative-archived-2025-11-25.md)
|
||||
- [Test Suite Configuration](../../test-suite-configuration.md)
|
||||
@@ -0,0 +1,424 @@
|
||||
# WASM Debug Infrastructure for AI Integration
|
||||
|
||||
**Date:** November 25, 2025 (Updated)
|
||||
**Status:** Current - Active debugging API
|
||||
**Version:** 2.3.0
|
||||
**Purpose:** Comprehensive debug API for Gemini/Antigravity AI integration to analyze rendering issues and game state in the yaze web application.
|
||||
|
||||
**Note:** This document is the high-level overview for WASM debugging. For detailed API reference, see `wasm-yazeDebug-api-reference.md`. For general WASM status and control APIs, see `wasm_dev_status.md`.
|
||||
|
||||
## Overview
|
||||
|
||||
The WASM debug infrastructure provides JavaScript access to internal yaze data structures for AI-powered debugging of dungeon palette rendering issues and other visual artifacts.
|
||||
|
||||
## Memory Configuration
|
||||
|
||||
The WASM build uses optimized memory settings configured in `src/app/app.cmake`:
|
||||
|
||||
| Setting | Value | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `INITIAL_MEMORY` | 256MB | Reduces heap resizing during ROM load (~200MB needed for overworld) |
|
||||
| `MAXIMUM_MEMORY` | 1GB | Prevents runaway allocations |
|
||||
| `STACK_SIZE` | 8MB | Handles recursive operations during asset decompression |
|
||||
| `ALLOW_MEMORY_GROWTH` | 1 | Enables dynamic heap expansion |
|
||||
|
||||
**Emscripten Flags:**
|
||||
```
|
||||
-s INITIAL_MEMORY=268435456 -s ALLOW_MEMORY_GROWTH=1 -s MAXIMUM_MEMORY=1073741824 -s STACK_SIZE=8388608
|
||||
```
|
||||
|
||||
## ROM Loading Progress
|
||||
|
||||
The WASM build reports loading progress through C++ `WasmLoadingManager`. Progress can be monitored in the browser UI or console.
|
||||
|
||||
### Loading Stages
|
||||
|
||||
| Progress | Stage |
|
||||
|----------|-------|
|
||||
| 0% | Reading ROM file... |
|
||||
| 5% | Loading ROM data... |
|
||||
| 10% | Initializing editors... |
|
||||
| 18% | Loading graphics sheets... |
|
||||
| 26% | Loading overworld... |
|
||||
| 34% | Loading dungeons... |
|
||||
| 42% | Loading screen editor... |
|
||||
| 50%+ | Loading remaining editors... |
|
||||
| 100% | Complete |
|
||||
|
||||
### Monitoring Loading in Console
|
||||
|
||||
```javascript
|
||||
// Check ROM loading status
|
||||
window.yaze.control.getRomStatus()
|
||||
|
||||
// Check graphics loading status
|
||||
window.yazeDebug.arena.getStatus()
|
||||
|
||||
// Get editor state after loading
|
||||
window.yaze.editor.getSnapshot()
|
||||
```
|
||||
|
||||
### Loading Indicator JavaScript API
|
||||
|
||||
```javascript
|
||||
// Called by C++ via WasmLoadingManager
|
||||
window.createLoadingIndicator(id, taskName) // Creates loading overlay
|
||||
window.updateLoadingProgress(id, progress, msg) // Updates progress (0.0-1.0)
|
||||
window.removeLoadingIndicator(id) // Removes loading overlay
|
||||
window.isLoadingCancelled(id) // Check if user cancelled
|
||||
```
|
||||
|
||||
## Files Modified/Created
|
||||
|
||||
### Core Debug Inspector
|
||||
- **`src/web/yaze_debug_inspector.cc`** (renamed from `palette_inspector.cpp`)
|
||||
- Main WASM debug inspector providing JavaScript bindings via Emscripten
|
||||
- Exports functions for palette, ROM, overworld, arena, and emulator debugging
|
||||
|
||||
### JavaScript API
|
||||
- **`src/web/shell.html`**
|
||||
- Added `window.yazeDebug` API object for browser console access
|
||||
- Enhanced `paletteInspector` with better error handling
|
||||
- Added pixel inspector overlay for visual debugging
|
||||
|
||||
### File System Fixes
|
||||
- **`src/web/app.js`**
|
||||
- Fixed race condition in `initPersistentFS()`
|
||||
- Added detection for C++ runtime-initialized IDBFS
|
||||
- Improved user feedback when file system is initializing
|
||||
|
||||
### Build System
|
||||
- **`src/app/app.cmake`**
|
||||
- Updated to include `web/yaze_debug_inspector.cc` for WASM builds
|
||||
|
||||
## Debug API Reference
|
||||
|
||||
### JavaScript API (`window.yazeDebug`)
|
||||
|
||||
```javascript
|
||||
// Check if API is ready
|
||||
window.yazeDebug.isReady() // Returns: boolean
|
||||
|
||||
// Capabilities
|
||||
window.yazeDebug.capabilities // ['palette', 'arena', 'timeline', 'pixel-inspector', 'rom', 'overworld']
|
||||
|
||||
// Palette Debugging
|
||||
window.yazeDebug.palette.getEvents() // Get palette debug events
|
||||
window.yazeDebug.palette.getFullState() // Full palette state with metadata
|
||||
window.yazeDebug.palette.getData() // Raw palette data
|
||||
window.yazeDebug.palette.getComparisons() // Color comparison data
|
||||
window.yazeDebug.palette.samplePixel(x,y) // Sample pixel at coordinates
|
||||
window.yazeDebug.palette.clear() // Clear debug events
|
||||
|
||||
// ROM Debugging
|
||||
window.yazeDebug.rom.getStatus() // ROM load state, size, title
|
||||
window.yazeDebug.rom.readBytes(address, count) // Read up to 256 bytes from ROM
|
||||
window.yazeDebug.rom.getPaletteGroup(groupName, idx) // Get palette group by name
|
||||
|
||||
// Overworld Debugging
|
||||
window.yazeDebug.overworld.getMapInfo(mapId) // Map properties (0-159)
|
||||
window.yazeDebug.overworld.getTileInfo(mapId, x, y) // Tile data at coordinates
|
||||
|
||||
// Arena (Graphics) Debugging
|
||||
window.yazeDebug.arena.getStatus() // Texture queue size, active sheets
|
||||
window.yazeDebug.arena.getSheetInfo(idx) // Details for specific graphics sheet
|
||||
|
||||
// Timeline Analysis
|
||||
window.yazeDebug.timeline.get() // Ordered event timeline
|
||||
|
||||
// AI Analysis Helpers
|
||||
window.yazeDebug.analysis.getSummary() // Diagnostic summary
|
||||
window.yazeDebug.analysis.getHypothesis() // AI hypothesis analysis
|
||||
window.yazeDebug.analysis.getFullState() // Combined full state
|
||||
|
||||
// Utility Functions
|
||||
window.yazeDebug.dumpAll() // Complete state dump (JSON)
|
||||
window.yazeDebug.formatForAI() // Human-readable format for AI
|
||||
```
|
||||
|
||||
### C++ EMSCRIPTEN_BINDINGS
|
||||
|
||||
```cpp
|
||||
EMSCRIPTEN_BINDINGS(yaze_debug_inspector) {
|
||||
// Palette debug functions
|
||||
function("getDungeonPaletteEvents", &getDungeonPaletteEvents);
|
||||
function("getColorComparisons", &getColorComparisons);
|
||||
function("samplePixelAt", &samplePixelAt);
|
||||
function("clearPaletteDebugEvents", &clearPaletteDebugEvents);
|
||||
function("getFullPaletteState", &getFullPaletteState);
|
||||
function("getPaletteData", &getPaletteData);
|
||||
function("getEventTimeline", &getEventTimeline);
|
||||
function("getDiagnosticSummary", &getDiagnosticSummary);
|
||||
function("getHypothesisAnalysis", &getHypothesisAnalysis);
|
||||
|
||||
// Arena debug functions
|
||||
function("getArenaStatus", &getArenaStatus);
|
||||
function("getGfxSheetInfo", &getGfxSheetInfo);
|
||||
|
||||
// ROM debug functions
|
||||
function("getRomStatus", &getRomStatus);
|
||||
function("readRomBytes", &readRomBytes);
|
||||
function("getRomPaletteGroup", &getRomPaletteGroup);
|
||||
|
||||
// Overworld debug functions
|
||||
function("getOverworldMapInfo", &getOverworldMapInfo);
|
||||
function("getOverworldTileInfo", &getOverworldTileInfo);
|
||||
|
||||
// Emulator debug functions
|
||||
function("getEmulatorStatus", &getEmulatorStatus);
|
||||
function("readEmulatorMemory", &readEmulatorMemory);
|
||||
function("getEmulatorVideoState", &getEmulatorVideoState);
|
||||
|
||||
// Combined state
|
||||
function("getFullDebugState", &getFullDebugState);
|
||||
}
|
||||
```
|
||||
|
||||
## File System Issues & Fixes
|
||||
|
||||
### Problem: Silent File Opening Failure
|
||||
|
||||
**Symptoms:**
|
||||
- Clicking "Open ROM" did nothing
|
||||
- No error messages shown to user
|
||||
- Console showed `FS not ready yet; still initializing` even after `IDBFS synced successfully`
|
||||
|
||||
**Root Cause:**
|
||||
The application had two separate IDBFS initialization paths:
|
||||
1. **C++ runtime** (`main.cc` → `MountFilesystems()`) - Initializes IDBFS and logs `IDBFS synced successfully`
|
||||
2. **JavaScript** (`app.js` → `initPersistentFS()`) - Tried to mount IDBFS again, failed silently
|
||||
|
||||
The JavaScript code never set `fsReady = true` because:
|
||||
- It waited for IDBFS to be available
|
||||
- C++ already mounted IDBFS
|
||||
- The JS `FS.syncfs()` callback never fired (already synced)
|
||||
- `fsReady` stayed `false`, blocking all file operations
|
||||
|
||||
**Fix Applied:**
|
||||
|
||||
```javascript
|
||||
function initPersistentFS() {
|
||||
// ... existing code ...
|
||||
|
||||
// Check if /roms already exists (C++ may have already set up the FS)
|
||||
var romsExists = false;
|
||||
try {
|
||||
FS.stat('/roms');
|
||||
romsExists = true;
|
||||
} catch (e) {
|
||||
// Directory doesn't exist
|
||||
}
|
||||
|
||||
if (romsExists) {
|
||||
// C++ already mounted IDBFS, just mark as ready
|
||||
console.log('[WASM] FS already initialized by C++ runtime');
|
||||
fsReady = true;
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
|
||||
// ... continue with JS mounting if needed ...
|
||||
}
|
||||
```
|
||||
|
||||
### Problem: No User Feedback During FS Init
|
||||
|
||||
**Symptoms:**
|
||||
- User clicked "Open ROM" during initialization
|
||||
- Nothing happened, no error shown
|
||||
- Only console warning logged
|
||||
|
||||
**Fix Applied:**
|
||||
|
||||
```javascript
|
||||
function ensureFSReady(showAlert = true) {
|
||||
if (fsReady && typeof FS !== 'undefined') return true;
|
||||
if (fsInitPromise) {
|
||||
console.warn('FS not ready yet; still initializing.');
|
||||
if (showAlert) {
|
||||
// Show status message in header
|
||||
var status = document.getElementById('header-status');
|
||||
if (status) {
|
||||
status.textContent = 'File system initializing... please wait';
|
||||
status.style.color = '#ffaa00';
|
||||
setTimeout(function() {
|
||||
status.textContent = 'Ready';
|
||||
status.style.color = '';
|
||||
}, 3000);
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
// ... rest of function
|
||||
}
|
||||
```
|
||||
|
||||
### Problem: JSON Parse Errors in Problems Panel
|
||||
|
||||
**Symptoms:**
|
||||
- Console error: `Failed to parse palette events JSON: unexpected character at line 1 column 2`
|
||||
- Problems panel showed errors when no palette events existed
|
||||
|
||||
**Root Cause:**
|
||||
- `Module.getDungeonPaletteEvents()` returned empty string `""` instead of `"[]"`
|
||||
- JSON.parse failed on empty string
|
||||
|
||||
**Fix Applied:**
|
||||
|
||||
```javascript
|
||||
// Handle empty or invalid responses
|
||||
if (!eventsStr || eventsStr.length === 0 || eventsStr === '[]') {
|
||||
var list = document.getElementById('problems-list');
|
||||
if (list) {
|
||||
list.innerHTML = '<div class="problems-empty">No palette events yet.</div>';
|
||||
}
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
## Valid Palette Group Names
|
||||
|
||||
For `getRomPaletteGroup(groupName, paletteIndex)`:
|
||||
|
||||
| Group Name | Description |
|
||||
|------------|-------------|
|
||||
| `ow_main` | Overworld main palettes |
|
||||
| `ow_aux` | Overworld auxiliary palettes |
|
||||
| `ow_animated` | Overworld animated palettes |
|
||||
| `hud` | HUD/UI palettes |
|
||||
| `global_sprites` | Global sprite palettes |
|
||||
| `armors` | Link armor palettes |
|
||||
| `swords` | Sword palettes |
|
||||
| `shields` | Shield palettes |
|
||||
| `sprites_aux1` | Sprite auxiliary 1 |
|
||||
| `sprites_aux2` | Sprite auxiliary 2 |
|
||||
| `sprites_aux3` | Sprite auxiliary 3 |
|
||||
| `dungeon_main` | Dungeon main palettes |
|
||||
| `grass` | Grass palettes |
|
||||
| `3d_object` | 3D object palettes |
|
||||
| `ow_mini_map` | Overworld minimap palettes |
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
# Build WASM with debug infrastructure
|
||||
./scripts/build-wasm.sh debug
|
||||
|
||||
# Serve locally (sets COOP/COEP headers for SharedArrayBuffer)
|
||||
./scripts/serve-wasm.sh 8080
|
||||
```
|
||||
|
||||
**Important:** The dev server must set COOP/COEP headers for SharedArrayBuffer support. Use `./scripts/serve-wasm.sh` which handles this automatically.
|
||||
|
||||
## Testing the API
|
||||
|
||||
Open browser console after loading the application:
|
||||
|
||||
```javascript
|
||||
// Verify API is loaded
|
||||
window.yazeDebug.isReady()
|
||||
|
||||
// Get ROM status (after loading a ROM)
|
||||
window.yazeDebug.rom.getStatus()
|
||||
|
||||
// Read bytes from ROM address 0x10000
|
||||
window.yazeDebug.rom.readBytes(0x10000, 32)
|
||||
|
||||
// Get dungeon palette group
|
||||
window.yazeDebug.rom.getPaletteGroup('dungeon_main', 0)
|
||||
|
||||
// Get overworld map info for Light World map 0
|
||||
window.yazeDebug.overworld.getMapInfo(0)
|
||||
|
||||
// Full debug dump for AI analysis
|
||||
window.yazeDebug.dumpAll()
|
||||
```
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **Emulator debug functions** require emulator to be initialized and running
|
||||
2. **Overworld tile info** requires overworld to be loaded in editor
|
||||
3. **Palette sampling** works on the visible canvas area only
|
||||
4. **ROM byte reading** limited to 256 bytes per call to prevent large responses
|
||||
5. **Memory reading** from emulator limited to 256 bytes per call
|
||||
6. **Loading indicator** managed by C++ `WasmLoadingManager` - don't create separate JS indicators
|
||||
|
||||
## Quick Start for AI Agents
|
||||
|
||||
1. **Load a ROM**: Use the file picker or drag-and-drop a `.sfc` file
|
||||
2. **Wait for loading**: Monitor progress via loading overlay or `window.yaze.control.getRomStatus()`
|
||||
3. **Verify ready state**: `window.yaze.control.isReady()` should return `true`
|
||||
4. **Start debugging**: Use `window.yazeDebug.dumpAll()` for full state or specific APIs
|
||||
|
||||
```javascript
|
||||
// Complete verification sequence
|
||||
if (window.yaze.control.isReady()) {
|
||||
const status = window.yaze.control.getRomStatus();
|
||||
if (status.loaded) {
|
||||
console.log('ROM loaded:', status.filename);
|
||||
console.log('AI-ready dump:', window.yazeDebug.formatForAI());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Gemini Antigravity AI Integration
|
||||
|
||||
The web interface includes dedicated tools for AI assistants that struggle to discover ImGui elements.
|
||||
|
||||
### window.aiTools API
|
||||
|
||||
High-level helper functions with console output for AI readability:
|
||||
|
||||
```javascript
|
||||
// Get full application state (ROM, editor, cards, layouts)
|
||||
window.aiTools.getAppState()
|
||||
|
||||
// Get current editor snapshot
|
||||
window.aiTools.getEditorState()
|
||||
|
||||
// Card management
|
||||
window.aiTools.getVisibleCards()
|
||||
window.aiTools.getAvailableCards()
|
||||
window.aiTools.showCard('Room Selector')
|
||||
window.aiTools.hideCard('Object Editor')
|
||||
|
||||
// Navigation
|
||||
window.aiTools.navigateTo('room:0') // Go to dungeon room
|
||||
window.aiTools.navigateTo('map:5') // Go to overworld map
|
||||
window.aiTools.navigateTo('Dungeon') // Switch editor
|
||||
|
||||
// Data access
|
||||
window.aiTools.getRoomData(0) // Dungeon room data
|
||||
window.aiTools.getMapData(0) // Overworld map data
|
||||
|
||||
// Documentation
|
||||
window.aiTools.dumpAPIReference() // Complete API reference
|
||||
```
|
||||
|
||||
### Nav Bar Dropdowns
|
||||
|
||||
The web UI includes four dedicated dropdown menus:
|
||||
|
||||
| Dropdown | Purpose |
|
||||
|----------|---------|
|
||||
| **Editor** | Quick switch between all 13 editors |
|
||||
| **Emulator** | Show/Run/Pause/Step/Reset + Memory Viewer |
|
||||
| **Layouts** | Preset card configurations |
|
||||
| **AI Tools** | All `window.aiTools` functions via UI |
|
||||
|
||||
### Command Palette (Ctrl+K)
|
||||
|
||||
All AI tools accessible via palette:
|
||||
- `Editor: <name>` - Switch editors
|
||||
- `Emulator: <action>` - Control emulator
|
||||
- `AI: Get App State` - Application state
|
||||
- `AI: API Reference` - Full API documentation
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] Add dungeon room state debugging
|
||||
- [ ] Add sprite debugging
|
||||
- [ ] Add memory watch points
|
||||
- [ ] Add breakpoint support for emulator
|
||||
- [ ] Add texture atlas visualization
|
||||
- [ ] Add palette history tracking
|
||||
286
docs/internal/agents/archive/wasm-docs-2025/wasm_dev_status.md
Normal file
286
docs/internal/agents/archive/wasm-docs-2025/wasm_dev_status.md
Normal file
@@ -0,0 +1,286 @@
|
||||
# WASM / Web Agent Integration Status
|
||||
|
||||
**Last Updated:** November 25, 2025
|
||||
**Status:** Functional MVP with Agent APIs (ROM loading fixed, loading progress added, control APIs implemented, performance optimizations applied)
|
||||
|
||||
## Overview
|
||||
This document tracks the development state of the `yaze` WASM web application, specifically focusing on the AI Agent integration (`z3ed` console) and the modern UI overhaul.
|
||||
|
||||
## 1. Completed Features
|
||||
|
||||
### ROM Loading & Initialization (November 2025 Fixes)
|
||||
* **ROM File Validation (`rom_file_manager.cc`):**
|
||||
* Fixed minimum ROM size check from 1MB to 512KB (was rejecting valid 1MB Zelda 3 ROMs)
|
||||
* **CMake WASM Configuration (`app.cmake`):**
|
||||
* Added `MODULARIZE=1` and `EXPORT_NAME='createYazeModule'` to match `app.js` expectations
|
||||
* Added missing exports: `_yazeHandleDroppedFile`, `_yazeHandleDropError`, `_yazeHandleDragEnter`, `_yazeHandleDragLeave`, `_malloc`, `_free`
|
||||
* Added missing runtime methods: `lengthBytesUTF8`, `IDBFS`, `allocateUTF8`
|
||||
* **JavaScript Fixes (`filesystem_manager.js`):**
|
||||
* Fixed `Module.ccall` return type from `'null'` (string) to `null`
|
||||
* Fixed direct function fallback to properly allocate/free memory for string parameters
|
||||
* **Drop Zone (`drop_zone.js`):**
|
||||
* Disabled duplicate auto-initialization (conflicted with C++ handler)
|
||||
* Now delegates to `FilesystemManager.handleRomUpload` instead of calling non-existent function
|
||||
* **Loading Progress (`editor_manager.cc`):**
|
||||
* Added `WasmLoadingManager` integration to `LoadAssets()`
|
||||
* Shows progress for each editor: "Loading overworld...", "Loading dungeons...", etc.
|
||||
* **UI Streamlining (`shell.html`, `app.js`):**
|
||||
* Removed HTML welcome screen - canvas is always visible
|
||||
* Loading overlay shows during initialization with status messages
|
||||
|
||||
### AI Agent Integration
|
||||
* **Core Bridge (`wasm_terminal_bridge.cc`):**
|
||||
* Exposes `Z3edProcessCommand` to JavaScript.
|
||||
* Exposes `GetGlobalBrowserAIService()` and `GetGlobalRom()` to C++ handlers.
|
||||
* **Browser Agent (`browser_agent.cc`):**
|
||||
* **`agent chat`**: Fully functional with conversation history. Uses `std::thread` for non-blocking AI calls.
|
||||
* **`agent plan`**: Generates text-based implementation plans (asynchronous).
|
||||
* **`agent diff`**: Shows the "pending plan" (conceptual diff).
|
||||
* **`agent list/describe`**: Introspects ROM resources via `ResourceCatalog`.
|
||||
* **`agent todo`**: Fully implemented with persistent storage.
|
||||
* **Browser AI Service** (`src/cli/service/ai/browser_ai_service.cc`):
|
||||
* Implements `AIService` interface for browser-based AI calls
|
||||
* Uses `IHttpClient` from network abstraction layer (CORS-compatible)
|
||||
* Supports Gemini API (text and vision models)
|
||||
* Secures API keys via sessionStorage (cleared on tab close)
|
||||
* Comprehensive error handling with `absl::Status`
|
||||
* **Browser Storage** (`src/app/platform/wasm/wasm_browser_storage.cc`):
|
||||
* Non-hardcoded API key management via sessionStorage/localStorage
|
||||
* User-provided keys, never embedded in binary
|
||||
* Namespaced storage to avoid conflicts
|
||||
* **Persistence (`todo_manager.cc`):**
|
||||
* Updated to use `WasmStorage` (IndexedDB) when compiled for Emscripten. TODOs persist across reloads.
|
||||
|
||||
### UI & UX
|
||||
* **Drag & Drop (`wasm_drop_handler.cc`):**
|
||||
* Supports `.sfc`, `.smc`, `.zip`.
|
||||
* Automatically writes to `/roms/` in MEMFS and loads the ROM.
|
||||
* Stubbed support for `.pal` / `.tpl`.
|
||||
* **Modern Interface:**
|
||||
* **`main.css`**: Unified design system (VS Code dark theme variables).
|
||||
* **`app.js`**: Extracted logic from `shell.html`. Handles terminal resize, zoom, and PWA updates.
|
||||
* **Components**: `terminal.css`, `collab_console.css`, etc., updated to use CSS variables.
|
||||
|
||||
### WASM Control APIs (November 2025)
|
||||
|
||||
The WASM build now exposes comprehensive JavaScript APIs for programmatic control, enabling LLM agents with DOM access to interact with the editor.
|
||||
|
||||
#### Editor State APIs (`window.yaze.editor`)
|
||||
* **`getSnapshot()`**: Get current editor state (type, ROM status, active data)
|
||||
* **`getCurrentRoom()`**: Get dungeon room info (room_id, active_rooms, visible_cards)
|
||||
* **`getCurrentMap()`**: Get overworld map info (map_id, world, world_name)
|
||||
* **`getSelection()`**: Get current selection in active editor
|
||||
|
||||
#### Read-only Data APIs (`window.yaze.data`)
|
||||
* **Dungeon Data:**
|
||||
* `getRoomTiles(roomId)` - Get room tile data (layer1, layer2)
|
||||
* `getRoomObjects(roomId)` - Get objects in a room
|
||||
* `getRoomProperties(roomId)` - Get room properties (music, palette, tileset)
|
||||
* **Overworld Data:**
|
||||
* `getMapTiles(mapId)` - Get map tile data
|
||||
* `getMapEntities(mapId)` - Get entities (entrances, exits, items, sprites)
|
||||
* `getMapProperties(mapId)` - Get map properties (gfx_group, palette, area_size)
|
||||
* **Palette Data:**
|
||||
* `getPalette(group, id)` - Get palette colors
|
||||
* `getPaletteGroups()` - List available palette groups
|
||||
|
||||
#### GUI Automation APIs (`window.yaze.gui`)
|
||||
* **Element Discovery:**
|
||||
* `discover()` - List all interactive UI elements with metadata
|
||||
* `getElementBounds(id)` - Get element position and dimensions (backed by `WidgetIdRegistry`)
|
||||
* `waitForElement(id, timeout)` - Async wait for element to appear
|
||||
* **Interaction:**
|
||||
* `click(target)` - Click by element ID or {x, y} coordinates
|
||||
* `doubleClick(target)` - Double-click
|
||||
* `drag(from, to, steps)` - Drag operation
|
||||
* `pressKey(key, modifiers)` - Send keyboard input
|
||||
* `type(text, delay)` - Type text string
|
||||
* `scroll(dx, dy)` - Scroll canvas
|
||||
* **Utility:**
|
||||
* `takeScreenshot(format)` - Capture canvas as base64
|
||||
* `getCanvasInfo()` - Get canvas dimensions
|
||||
* `isReady()` - Check if GUI API is ready
|
||||
|
||||
**Widget Tracking Infrastructure** (November 2025):
|
||||
The `WidgetIdRegistry` system tracks all ImGui widget bounds in real-time:
|
||||
- **Real-time Bounds**: `GetUIElementTree()` and `GetUIElementBounds()` query live widget positions via `WidgetIdRegistry`
|
||||
- **Frame Lifecycle**: Integrated into `Controller::OnLoad()` with `BeginFrame()` and `EndFrame()` hooks
|
||||
- **Bounds Data**: Includes `min_x`, `min_y`, `max_x`, `max_y` for accurate GUI automation
|
||||
- **Metadata**: Returns `imgui_id`, `last_seen_frame`, widget type, visibility, enabled state
|
||||
- **Key Files**: `src/app/gui/automation/widget_id_registry.h`, `src/app/gui/automation/widget_measurement.h`
|
||||
|
||||
#### Control APIs (`window.yaze.control`)
|
||||
* **Editor Control:** `switchEditor()`, `getCurrentEditor()`, `getAvailableEditors()`
|
||||
* **Card Control:** `openCard()`, `closeCard()`, `toggleCard()`, `getVisibleCards()`
|
||||
* **Layout Control:** `setCardLayout()`, `getAvailableLayouts()`, `saveCurrentLayout()`
|
||||
* **Menu Actions:** `triggerMenuAction()`, `getAvailableMenuActions()`
|
||||
* **Session Control:** `getSessionInfo()`, `createSession()`, `switchSession()`
|
||||
* **ROM Control:** `getRomStatus()`, `readRomBytes()`, `writeRomBytes()`, `saveRom()`
|
||||
|
||||
#### Extended UI Control APIs (November 2025)
|
||||
|
||||
**Async Editor Switching (`yazeDebug.switchToEditorAsync`)**:
|
||||
Promise-based editor switching with operation tracking for reliable LLM automation.
|
||||
* Returns `Promise<{success, editor, session_id, error}>` after editor transition completes
|
||||
* Supports all 14 editor types: Assembly, Dungeon, Graphics, Music, Overworld, Palette, Screen, Sprite, Message, Hex, Agent, Settings, World, Map
|
||||
* 5-second timeout with proper error reporting
|
||||
|
||||
**Card Control API (`yazeDebug.cards`)**:
|
||||
* `show(cardId)` - Show a specific card by ID (e.g., "dungeon.room_selector")
|
||||
* `hide(cardId)` - Hide a specific card
|
||||
* `toggle(cardId)` - Toggle card visibility
|
||||
* `getState()` - Get visibility state of all cards
|
||||
* `getInCategory(category)` - List cards in a category (dungeon, overworld, etc.)
|
||||
* `showGroup(groupName)` - Show predefined card groups (dungeon_editing, overworld_editing, etc.)
|
||||
* `hideGroup(groupName)` - Hide predefined card groups
|
||||
* `getGroups()` - List available card groups
|
||||
|
||||
**Sidebar Control API (`yazeDebug.sidebar`)**:
|
||||
* `isTreeView()` - Check if tree view mode is active
|
||||
* `setTreeView(enabled)` - Switch between tree view (200px) and icon mode (48px)
|
||||
* `toggle()` - Toggle between view modes
|
||||
* `getState()` - Get sidebar state (mode, width, collapsed)
|
||||
|
||||
**Right Panel Control API (`yazeDebug.rightPanel`)**:
|
||||
* `open(panelName)` - Open specific panel: properties, agent, proposals, settings, help
|
||||
* `close()` - Close current panel
|
||||
* `toggle(panelName)` - Toggle panel visibility
|
||||
* `getState()` - Get panel state (active, expanded, width)
|
||||
* `openProperties()` - Convenience method for properties panel
|
||||
* `openAgent()` - Convenience method for agent chat panel
|
||||
|
||||
**Tree View Sidebar**:
|
||||
New hierarchical sidebar mode (200px wide) with:
|
||||
* Category icons and expandable tree nodes
|
||||
* Checkboxes for each card with visibility toggles
|
||||
* Visible count badges per category
|
||||
* "Show All" / "Hide All" buttons per category
|
||||
* Toggle button to switch to icon mode
|
||||
|
||||
**Selection Properties Panel**:
|
||||
New right-side panel for editing selected entities:
|
||||
* Context-aware property display based on selection type
|
||||
* Supports dungeon rooms, objects, sprites, entrances
|
||||
* Supports overworld maps, tiles, sprites, entrances, exits, items
|
||||
* Supports graphics sheets and palettes
|
||||
* Position/size editors with clamping
|
||||
* Byte/word property editors with hex display
|
||||
* Flag property editors with checkboxes
|
||||
* Advanced and raw data toggles
|
||||
|
||||
**Key Files:**
|
||||
* `src/app/platform/wasm/wasm_control_api.cc` - C++ implementation
|
||||
* `src/app/platform/wasm/wasm_control_api.h` - API declarations
|
||||
* `src/web/core/agent_automation.js` - GUI automation layer
|
||||
* `src/web/debug/yaze_debug_inspector.cc` - Extended WASM bindings
|
||||
* `src/app/editor/system/editor_card_registry.cc` - Tree view sidebar implementation
|
||||
* `src/app/editor/ui/right_panel_manager.cc` - Right panel management
|
||||
* `src/app/editor/ui/selection_properties_panel.cc` - Properties panel implementation
|
||||
|
||||
### Performance Optimizations & Bug Fixes (November 2025)
|
||||
|
||||
A comprehensive audit and fix of the WASM web layer was performed to address performance issues, memory leaks, and race conditions.
|
||||
|
||||
#### JavaScript Performance Fixes (`app.js`)
|
||||
* **Event Sanitization Optimization:**
|
||||
* Removed redundant document-level event listeners (canvas-only now)
|
||||
* Added WeakMap caching to avoid re-sanitizing the same event objects
|
||||
* Optimized to check only relevant properties per event type category
|
||||
* ~50% reduction in sanitization overhead
|
||||
* **Console Log Buffer:**
|
||||
* Replaced O(n) `Array.shift()` with O(1) circular buffer implementation
|
||||
* Uses modulo arithmetic for constant-time log rotation
|
||||
* **Polling Cleanup:**
|
||||
* Added timeout tracking and max retry limits for module initialization
|
||||
* Proper interval cleanup when components are destroyed
|
||||
* Added `window.YAZE_MODULE_READY` flag for reliable initialization detection
|
||||
|
||||
#### Memory Leak Fixes
|
||||
* **Service Worker Cache (`service-worker.js`):**
|
||||
* Added `MAX_RUNTIME_CACHE_SIZE` (50 entries) with LRU eviction
|
||||
* New `trimRuntimeCache()` function enforces size limits
|
||||
* `addToRuntimeCacheWithEviction()` wrapper for cache operations
|
||||
* **Confirmation Callbacks (`wasm_error_handler.cc`):**
|
||||
* Added `CallbackEntry` struct with timestamps for timeout tracking
|
||||
* Auto-cleanup of callbacks older than 5 minutes
|
||||
* Page unload handler via `js_register_cleanup_handler()`
|
||||
* **Loading Indicators (`loading_indicator.js`):**
|
||||
* Added try-catch error handling to ensure cleanup on errors
|
||||
* Stale indicator cleanup (5-minute timeout)
|
||||
* Periodic cleanup interval with proper lifecycle management
|
||||
|
||||
#### Race Condition Fixes
|
||||
* **Module Initialization (`app.js`):**
|
||||
* Added `window.YAZE_MODULE_READY` flag set AFTER promise resolves
|
||||
* Updated `waitForModule()` to check both Module existence AND ready flag
|
||||
* Prevents code from seeing incomplete Module state
|
||||
* **FS Ready State (`filesystem_manager.js`):**
|
||||
* Restructured `initPersistentFS()` with synchronous lock pattern
|
||||
* Promise created immediately before async operations
|
||||
* Eliminates race where two calls could create duplicate promises
|
||||
* **Redundant FS Exposure:**
|
||||
* Added `fsExposed` flag to prevent wasteful redundant calls
|
||||
* Reduced from 3 setTimeout calls to 1 conditional retry
|
||||
|
||||
#### C++ WASM Fixes
|
||||
* **Memory Safety (`wasm_storage.cc`):**
|
||||
* Added `free(data_ptr)` in error paths of `LoadRom()` to prevent memory leaks
|
||||
* Ensures allocated memory is freed even when operations fail
|
||||
* **Cleanup Handlers (`wasm_error_handler.cc`):**
|
||||
* Added `cleanupConfirmCallbacks()` function for page unload
|
||||
* Registered via `js_register_cleanup_handler()` in `Initialize()`
|
||||
|
||||
#### Drop Zone Optimization (`drop_zone.js`, `filesystem_manager.js`)
|
||||
* **Eliminated Double File Reading:**
|
||||
* Added new `FilesystemManager.handleRomData(filename, data)` method
|
||||
* Accepts pre-read `Uint8Array` instead of `File` object
|
||||
* Drop zone now passes already-read data instead of re-reading
|
||||
* Reduces CPU and memory usage for ROM uploads
|
||||
|
||||
**Key Files Modified:**
|
||||
* `src/web/app.js` - Event sanitization, console buffer, module init
|
||||
* `src/web/core/filesystem_manager.js` - FS init race fix, handleRomData
|
||||
* `src/web/core/loading_indicator.js` - Stale cleanup, error handling
|
||||
* `src/web/components/drop_zone.js` - Use handleRomData
|
||||
* `src/web/pwa/service-worker.js` - Cache eviction
|
||||
* `src/app/platform/wasm/wasm_storage.cc` - Memory free on error
|
||||
* `src/app/platform/wasm/wasm_error_handler.cc` - Callback cleanup
|
||||
|
||||
## 2. Technical Debt & Known Issues
|
||||
|
||||
* **`SimpleChatSession`**: This C++ class relies on `VimMode` and raw TTY input, which is incompatible with WASM. We bypassed this by implementing a custom `HandleChatCommand` in `browser_agent.cc`. The original `SimpleChatSession` remains unused in the browser build.
|
||||
* **Emscripten Fetch Blocking**: The `EmscriptenHttpClient` implementation contains a `cv.wait()` which blocks the main thread. We worked around this by spawning `std::thread` in the command handlers, but the HTTP client itself remains synchronous-blocking if called directly on the main thread.
|
||||
* **Single-Threaded Rendering**: Dungeon graphics loading happens on the main thread (`DungeonEditorV2::DrawRoomTab`), causing UI freezes on large ROMs.
|
||||
|
||||
## 3. Next Steps / Roadmap
|
||||
|
||||
### Short Term
|
||||
1. **Palette Import**: Implement the logic in `wasm_drop_handler.cc` (or `main.cc` callback) to parse `.pal` files and apply them to `PaletteManager`.
|
||||
2. **Deep Linking**: Add logic to `app.js` and `main.cc` to parse URL query parameters (e.g., `?rom=url`) for easy sharing.
|
||||
|
||||
### Medium Term
|
||||
1. **In-Memory Proposal Registry**:
|
||||
* Implement a `WasmProposalRegistry` that mimics the file-based `ProposalRegistry`.
|
||||
* Store "sandboxes" as `Rom` copies in memory (or IndexedDB blobs).
|
||||
* Enable `agent apply` to execute the plans generated by `agent plan`.
|
||||
2. **Multithreaded Graphics**:
|
||||
* Refactor `DungeonEditorV2` to use `WasmWorkerPool` for `LoadRoomGraphics`.
|
||||
* Requires decoupling `Room` data structures from the loading logic to pass data across threads safely.
|
||||
|
||||
## 4. Key Files
|
||||
|
||||
* **C++ Logic**:
|
||||
* `src/cli/handlers/agent/browser_agent.cc` (Agent commands)
|
||||
* `src/cli/wasm_terminal_bridge.cc` (JS <-> C++ Bridge)
|
||||
* `src/app/platform/wasm/wasm_drop_handler.cc` (File drag & drop)
|
||||
* `src/app/platform/wasm/wasm_control_api.cc` (Control API implementation)
|
||||
* `src/app/platform/wasm/wasm_control_api.h` (Control API declarations)
|
||||
* `src/cli/service/agent/todo_manager.cc` (Persistence logic)
|
||||
|
||||
* **Web Frontend**:
|
||||
* `src/web/shell.html` (Entry point)
|
||||
* `src/web/app.js` (Main UI logic)
|
||||
* `src/web/core/agent_automation.js` (GUI Automation layer)
|
||||
* `src/web/styles/main.css` (Theme definitions)
|
||||
* `src/web/components/terminal.js` (Console UI component)
|
||||
* `src/web/components/collaboration_ui.js` (Collaboration UI)
|
||||
@@ -0,0 +1,687 @@
|
||||
### WASM Debugging Guide: Dungeon Editor
|
||||
|
||||
**Status:** Current (November 2025)
|
||||
**Last Updated:** 2025-11-25
|
||||
**Version:** 2.3.0
|
||||
|
||||
The WASM build includes a powerful, hidden "Debug Inspector" that bypasses the need for GDB/LLDB by exposing C++ state directly to the browser console.
|
||||
|
||||
**Cross-Reference:** For comprehensive debug API reference, see `wasm-debug-infrastructure.md` and `wasm-yazeDebug-api-reference.md`.
|
||||
|
||||
#### 1. The "God Mode" Console Inspector
|
||||
The file `src/web/yaze_debug_inspector.cc` binds C++ functions to the global `Module` object. You can invoke these directly from Chrome/Firefox DevTools.
|
||||
|
||||
**Status & State:**
|
||||
* `Module.getEmulatorStatus()`: Returns JSON with CPU registers (A, X, Y, PC), Flags, and PPU state.
|
||||
* `Module.getFullDebugState()`: Returns a massive JSON dump suitable for pasting into an AI prompt for analysis.
|
||||
* `Module.getArenaStatus()`: Checks the memory arena used for dungeon rendering (vital for "out of memory" rendering glitches).
|
||||
|
||||
**Memory Inspection:**
|
||||
* `Module.readEmulatorMemory(addr, length)`: Reads WRAM/SRAM.
|
||||
* *Example:* Check Link's X-Coordinate ($20): `Module.readEmulatorMemory(0x7E0020, 2)`
|
||||
* `Module.readRom(addr, length)`: Verifies if your ROM patch actually applied in memory.
|
||||
|
||||
**Graphics & Palette:**
|
||||
* `Module.getDungeonPaletteEvents()`: Returns a log of recent palette uploads. Use this if colors look wrong or "flashy" in the editor.
|
||||
|
||||
#### 2. The Command Line Bridge
|
||||
The terminal you see in the web app isn't just a UI toy; it's a direct bridge to the C++ backend.
|
||||
|
||||
* **Architecture**: `src/web/terminal.js` captures your keystrokes and calls `Module.ccall('Z3edProcessCommand', ...)` which routes to `src/cli/wasm_terminal_bridge.cc`.
|
||||
* **Debug Tip**: If the editor UI freezes, the terminal often remains responsive (running on a separate event cadence). You can use it to:
|
||||
1. Save your work: `save`
|
||||
2. Dump state: (If a custom command exists)
|
||||
3. Reset the emulator.
|
||||
|
||||
#### 3. The Hidden "Debug Controls" Card
|
||||
The code in `src/app/editor/dungeon/dungeon_editor_v2.cc` contains a function `DrawDebugControlsCard()`, controlled by the boolean `show_debug_controls_`.
|
||||
|
||||
* **Current Status**: This is currently **hidden** by default and likely has no UI toggle in the public build.
|
||||
* **Recommended Task**: Create a `z3ed` CLI command to toggle this boolean.
|
||||
* *Implementation*: Add a command `editor debug toggle` in `wasm_terminal_bridge.cc` that finds the active `DungeonEditorV2` instance and flips `show_debug_controls_ = !show_debug_controls_`. This would give you on-screen access to render passes and layer toggles.
|
||||
|
||||
#### 4. Feature Parity
|
||||
There are **NO** `__EMSCRIPTEN__` checks inside the `src/app/editor/dungeon/` logic.
|
||||
* **Implication**: If a logic bug exists in WASM, it likely exists in the Native macOS/Linux build too. Reproduce bugs on Desktop first for easier debugging (breakpoints, etc.), then verify the fix on Web.
|
||||
* **Exception**: Rendering glitches are likely WASM-specific due to the single-threaded `TickFrame` loop vs. the multi-threaded desktop renderer.
|
||||
|
||||
#### 5. Thread Pool Configuration
|
||||
The WASM build uses a fixed thread pool (`PTHREAD_POOL_SIZE=8` in CMakePresets.json).
|
||||
* **Warning Signs**: If you see "Tried to spawn a new thread, but the thread pool is exhausted", heavy parallel operations are exceeding the pool size.
|
||||
* **Fix**: Increase `PTHREAD_POOL_SIZE` in CMakePresets.json and rebuild with `--clean`
|
||||
* **Root Cause**: Often happens during ROM loading when multiple graphics sheets are decompressed in parallel.
|
||||
|
||||
#### 6. Memory Configuration
|
||||
The WASM build uses optimized memory settings to reduce heap resize operations:
|
||||
|
||||
| Setting | Value | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `INITIAL_MEMORY` | 256MB | Reduces heap resizing during ROM load |
|
||||
| `MAXIMUM_MEMORY` | 1GB | Prevents runaway allocations |
|
||||
| `STACK_SIZE` | 8MB | Handles recursive asset decompression |
|
||||
|
||||
* **Warning Signs**: Console shows `_emscripten_resize_heap` calls during loading
|
||||
* **Common Causes**: Overworld map loading (~160MB for 160 maps), sprite preview buffers, dungeon object emulator
|
||||
* **Optimization Applied**: Lazy initialization for SNES emulator instances and sprite preview buffers
|
||||
|
||||
#### 7. ROM Loading Progress
|
||||
The C++ `WasmLoadingManager` controls loading progress display. Monitor loading status:
|
||||
|
||||
```javascript
|
||||
// Check if ROM is loaded
|
||||
window.yaze.control.getRomStatus()
|
||||
// Returns: { loaded: true/false, filename: "...", title: "...", size: ... }
|
||||
|
||||
// Check arena (graphics) status
|
||||
window.yazeDebug.arena.getStatus()
|
||||
|
||||
// Full editor state after loading
|
||||
window.yaze.editor.getSnapshot()
|
||||
```
|
||||
|
||||
**Loading Progress Stages:**
|
||||
| Progress | Stage |
|
||||
|----------|-------|
|
||||
| 10% | Initializing editors... |
|
||||
| 18% | Loading graphics sheets... |
|
||||
| 26% | Loading overworld... |
|
||||
| 34% | Loading dungeons... |
|
||||
| 42%+ | Loading remaining editors... |
|
||||
| 100% | Complete |
|
||||
|
||||
#### 8. Prompting AI Agents (Claude Code / Gemini Antigravity)
|
||||
|
||||
This section provides precise prompts and workflows for AI agents to interact with the YAZE WASM app.
|
||||
|
||||
##### Step 1: Verify the App is Ready
|
||||
|
||||
Before any operations, the AI must confirm the WASM module is initialized:
|
||||
|
||||
```javascript
|
||||
// Check module ready state
|
||||
window.YAZE_MODULE_READY // Should be true
|
||||
|
||||
// Check if APIs are available
|
||||
typeof window.yaze !== 'undefined' &&
|
||||
typeof window.yaze.control !== 'undefined' // Should be true
|
||||
```
|
||||
|
||||
**If not ready**, wait and retry:
|
||||
```javascript
|
||||
// Poll until ready (max 10 seconds)
|
||||
async function waitForYaze() {
|
||||
for (let i = 0; i < 100; i++) {
|
||||
if (window.YAZE_MODULE_READY && window.yaze?.control) return true;
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
await waitForYaze();
|
||||
```
|
||||
|
||||
##### Step 2: Load a ROM File
|
||||
|
||||
**Option A: User Drag-and-Drop (Recommended)**
|
||||
Prompt the user: *"Please drag and drop your Zelda 3 ROM (.sfc or .smc) onto the canvas."*
|
||||
|
||||
Then verify:
|
||||
```javascript
|
||||
// Check if ROM loaded successfully
|
||||
const status = window.yaze.control.getRomStatus();
|
||||
console.log(status);
|
||||
// Expected: { loaded: true, filename: "zelda3.sfc", title: "...", size: 1048576 }
|
||||
```
|
||||
|
||||
**Option B: Check if ROM Already Loaded**
|
||||
```javascript
|
||||
window.yaze.control.getRomStatus().loaded // true if ROM is present
|
||||
```
|
||||
|
||||
**Option C: Load from IndexedDB (if previously saved)**
|
||||
```javascript
|
||||
// List saved ROMs
|
||||
FilesystemManager.listSavedRoms && FilesystemManager.listSavedRoms();
|
||||
```
|
||||
|
||||
##### Step 3: Open the Dungeon Editor
|
||||
|
||||
```javascript
|
||||
// Switch to dungeon editor
|
||||
window.yaze.control.switchEditor('Dungeon');
|
||||
|
||||
// Verify switch was successful
|
||||
const snapshot = window.yaze.editor.getSnapshot();
|
||||
console.log(snapshot.editor_type); // Should be "Dungeon"
|
||||
```
|
||||
|
||||
##### Step 4: Navigate to a Specific Room
|
||||
|
||||
```javascript
|
||||
// Get current room info
|
||||
window.yaze.editor.getCurrentRoom();
|
||||
// Returns: { room_id: 0, active_rooms: [...], visible_cards: [...] }
|
||||
|
||||
// Navigate to room 42 (Hyrule Castle Entrance)
|
||||
window.aiTools.navigateTo('room:42');
|
||||
|
||||
// Or use control API
|
||||
window.yaze.control.openCard('Room 42');
|
||||
```
|
||||
|
||||
##### Step 5: Inspect Room Data
|
||||
|
||||
```javascript
|
||||
// Get room properties
|
||||
const props = window.yaze.data.getRoomProperties(42);
|
||||
console.log(props);
|
||||
// Returns: { music: 5, palette: 2, tileset: 7, ... }
|
||||
|
||||
// Get room objects (chests, torches, blocks, etc.)
|
||||
const objects = window.yaze.data.getRoomObjects(42);
|
||||
console.log(objects);
|
||||
|
||||
// Get room tile data
|
||||
const tiles = window.yaze.data.getRoomTiles(42);
|
||||
console.log(tiles.layer1, tiles.layer2);
|
||||
```
|
||||
|
||||
##### Example AI Prompt Workflow
|
||||
|
||||
**User asks:** "Show me the objects in room 42 of the dungeon editor"
|
||||
|
||||
**AI should execute:**
|
||||
```javascript
|
||||
// 1. Verify ready
|
||||
if (!window.YAZE_MODULE_READY) throw new Error("WASM not ready");
|
||||
|
||||
// 2. Check ROM
|
||||
const rom = window.yaze.control.getRomStatus();
|
||||
if (!rom.loaded) throw new Error("No ROM loaded - please drag a ROM file onto the canvas");
|
||||
|
||||
// 3. Switch to dungeon editor
|
||||
window.yaze.control.switchEditor('Dungeon');
|
||||
|
||||
// 4. Navigate to room
|
||||
window.aiTools.navigateTo('room:42');
|
||||
|
||||
// 5. Get and display data
|
||||
const objects = window.yaze.data.getRoomObjects(42);
|
||||
console.log("Room 42 Objects:", JSON.stringify(objects, null, 2));
|
||||
```
|
||||
|
||||
#### 9. JavaScript Tips for Browser Debugging
|
||||
|
||||
##### Console Shortcuts
|
||||
|
||||
```javascript
|
||||
// Alias for quick access
|
||||
const y = window.yaze;
|
||||
const yd = window.yazeDebug;
|
||||
|
||||
// Quick status check
|
||||
y.control.getRomStatus()
|
||||
y.editor.getSnapshot()
|
||||
yd.arena.getStatus()
|
||||
```
|
||||
|
||||
##### Error Handling Pattern
|
||||
|
||||
Always wrap API calls in try-catch when automating:
|
||||
```javascript
|
||||
function safeCall(fn, fallback = null) {
|
||||
try {
|
||||
return fn();
|
||||
} catch (e) {
|
||||
console.error('[YAZE API Error]', e.message);
|
||||
return fallback;
|
||||
}
|
||||
}
|
||||
|
||||
// Usage
|
||||
const status = safeCall(() => window.yaze.control.getRomStatus(), { loaded: false });
|
||||
```
|
||||
|
||||
##### Async Operations
|
||||
|
||||
Some operations are async. Use proper await patterns:
|
||||
```javascript
|
||||
// Wait for element to appear in GUI
|
||||
await window.yaze.gui.waitForElement('dungeon-room-canvas', 5000);
|
||||
|
||||
// Then interact
|
||||
window.yaze.gui.click('dungeon-room-canvas');
|
||||
```
|
||||
|
||||
##### Debugging State Issues
|
||||
|
||||
```javascript
|
||||
// Full state dump for debugging
|
||||
const debugState = {
|
||||
moduleReady: window.YAZE_MODULE_READY,
|
||||
romStatus: window.yaze?.control?.getRomStatus?.() || 'API unavailable',
|
||||
editorSnapshot: window.yaze?.editor?.getSnapshot?.() || 'API unavailable',
|
||||
arenaStatus: window.yazeDebug?.arena?.getStatus?.() || 'API unavailable',
|
||||
consoleErrors: window._yazeConsoleLogs?.filter(l => l.includes('[ERROR]')) || []
|
||||
};
|
||||
console.log(JSON.stringify(debugState, null, 2));
|
||||
```
|
||||
|
||||
##### Monitoring Loading Progress
|
||||
|
||||
```javascript
|
||||
// Set up a loading progress monitor
|
||||
let lastProgress = 0;
|
||||
const progressInterval = setInterval(() => {
|
||||
const status = window.yaze?.control?.getRomStatus?.();
|
||||
if (status?.loaded) {
|
||||
console.log('ROM loaded successfully!');
|
||||
clearInterval(progressInterval);
|
||||
}
|
||||
}, 500);
|
||||
|
||||
// Clear after 30 seconds timeout
|
||||
setTimeout(() => clearInterval(progressInterval), 30000);
|
||||
```
|
||||
|
||||
##### Inspecting Graphics Issues
|
||||
|
||||
```javascript
|
||||
// Check if graphics sheets are loaded
|
||||
const arenaStatus = window.yazeDebug.arena.getStatus();
|
||||
console.log('Loaded sheets:', arenaStatus.loaded_sheets);
|
||||
console.log('Pending textures:', arenaStatus.pending_textures);
|
||||
|
||||
// Check for palette issues
|
||||
const paletteEvents = Module.getDungeonPaletteEvents?.() || 'Not available';
|
||||
console.log('Recent palette changes:', paletteEvents);
|
||||
```
|
||||
|
||||
##### Memory Usage Check
|
||||
|
||||
```javascript
|
||||
// Check WASM memory usage
|
||||
const memInfo = {
|
||||
heapSize: Module.HEAPU8?.length || 0,
|
||||
heapSizeMB: ((Module.HEAPU8?.length || 0) / 1024 / 1024).toFixed(2) + ' MB'
|
||||
};
|
||||
console.log('Memory:', memInfo);
|
||||
```
|
||||
|
||||
#### 10. Gemini Antigravity AI Tools
|
||||
|
||||
For AI assistants using the Antigravity browser extension, use the high-level `window.aiTools` API:
|
||||
|
||||
##### Complete Workflow Example
|
||||
|
||||
```javascript
|
||||
// Step-by-step for Gemini/Antigravity
|
||||
async function inspectDungeonRoom(roomId) {
|
||||
// 1. Verify environment
|
||||
if (!window.YAZE_MODULE_READY) {
|
||||
return { error: "WASM module not ready. Please wait for initialization." };
|
||||
}
|
||||
|
||||
// 2. Check ROM
|
||||
const romStatus = window.yaze.control.getRomStatus();
|
||||
if (!romStatus.loaded) {
|
||||
return { error: "No ROM loaded. Please drag a Zelda 3 ROM onto the canvas." };
|
||||
}
|
||||
|
||||
// 3. Switch to dungeon editor
|
||||
window.yaze.control.switchEditor('Dungeon');
|
||||
|
||||
// 4. Navigate to room
|
||||
window.aiTools.navigateTo(`room:${roomId}`);
|
||||
|
||||
// 5. Gather all data
|
||||
return {
|
||||
room_id: roomId,
|
||||
properties: window.yaze.data.getRoomProperties(roomId),
|
||||
objects: window.yaze.data.getRoomObjects(roomId),
|
||||
tiles: window.yaze.data.getRoomTiles(roomId),
|
||||
editor_state: window.yaze.editor.getCurrentRoom()
|
||||
};
|
||||
}
|
||||
|
||||
// Usage
|
||||
const data = await inspectDungeonRoom(42);
|
||||
console.log(JSON.stringify(data, null, 2));
|
||||
```
|
||||
|
||||
##### Quick Reference Commands
|
||||
|
||||
```javascript
|
||||
// Get full application state with console output
|
||||
window.aiTools.getAppState()
|
||||
|
||||
// Get dungeon room data
|
||||
window.aiTools.getRoomData(0)
|
||||
|
||||
// Navigate directly to a room
|
||||
window.aiTools.navigateTo('room:42')
|
||||
|
||||
// Show/hide editor cards
|
||||
window.aiTools.showCard('Room Selector')
|
||||
window.aiTools.hideCard('Object Editor')
|
||||
|
||||
// Get complete API reference
|
||||
window.aiTools.dumpAPIReference()
|
||||
|
||||
// AI-formatted state (paste-ready for prompts)
|
||||
window.yazeDebug.formatForAI()
|
||||
```
|
||||
|
||||
##### Handling Common Errors
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `window.yaze is undefined` | Module not initialized | Wait for `YAZE_MODULE_READY` |
|
||||
| `getRomStatus().loaded = false` | No ROM file | Prompt user to drag ROM |
|
||||
| `switchEditor returns error` | Invalid editor name | Use: Dungeon, Overworld, Graphics, Palette, Sprite, Music |
|
||||
| `getRoomObjects returns empty` | Room not loaded | Navigate to room first |
|
||||
| `Canvas shows black` | Graphics not loaded | Check `yazeDebug.arena.getStatus()` |
|
||||
|
||||
**Nav Bar Access:**
|
||||
The web UI includes dedicated dropdown menus:
|
||||
- **Editor** - Quick switch to any editor
|
||||
- **Emulator** - Run/Pause/Step/Reset controls
|
||||
- **Layouts** - Preset card configurations
|
||||
- **AI Tools** - All `window.aiTools` functions via UI clicks
|
||||
|
||||
**Command Palette (Ctrl+K):**
|
||||
Search for "AI:" to access all AI helper commands.
|
||||
|
||||
#### 11. Copying Data for External Analysis
|
||||
|
||||
```javascript
|
||||
// Copy room data to clipboard for pasting elsewhere
|
||||
async function copyRoomDataToClipboard(roomId) {
|
||||
const data = {
|
||||
timestamp: new Date().toISOString(),
|
||||
room_id: roomId,
|
||||
properties: window.yaze.data.getRoomProperties(roomId),
|
||||
objects: window.yaze.data.getRoomObjects(roomId)
|
||||
};
|
||||
await navigator.clipboard.writeText(JSON.stringify(data, null, 2));
|
||||
console.log(`Room ${roomId} data copied to clipboard!`);
|
||||
}
|
||||
|
||||
// Usage
|
||||
copyRoomDataToClipboard(42);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### 12. Antigravity: Debugging Dungeon Object Rendering Issues
|
||||
|
||||
This section provides Gemini Antigravity with specific workflows for identifying and analyzing dungeon object rendering problems, particularly discrepancies between the **Room Graphics Card** (background tiles) and the **Object Editor Card** (dungeon objects like pots, torches, chests).
|
||||
|
||||
##### Understanding the Rendering Pipeline
|
||||
|
||||
Dungeon rooms have two distinct rendering layers:
|
||||
|
||||
| Card | What It Shows | Render Source |
|
||||
|------|--------------|---------------|
|
||||
| **Room Graphics** | Background tiles (floors, walls, pits) | Layer1/Layer2 tile data |
|
||||
| **Object Editor** | Interactive objects (pots, chests, blocks, torches) | Object list with sprite-based rendering |
|
||||
|
||||
**Common Issue:** Objects appear at wrong positions, wrong sprites, or don't match their expected appearance in the Object Editor compared to how they should look based on the room data.
|
||||
|
||||
##### Step-by-Step Debugging Workflow
|
||||
|
||||
###### 1. Capture the Current Visual State
|
||||
|
||||
Use the browser's screenshot capability to capture what you see:
|
||||
|
||||
```javascript
|
||||
// Capture the entire canvas as a data URL
|
||||
async function captureCanvasScreenshot() {
|
||||
const canvas = document.getElementById('canvas');
|
||||
if (!canvas) return { error: 'Canvas not found' };
|
||||
|
||||
const dataUrl = canvas.toDataURL('image/png');
|
||||
console.log('[Screenshot] Canvas captured, length:', dataUrl.length);
|
||||
|
||||
// For AI analysis, you can copy to clipboard
|
||||
await navigator.clipboard.writeText(dataUrl);
|
||||
return { success: true, message: 'Screenshot copied to clipboard as data URL' };
|
||||
}
|
||||
|
||||
// Usage
|
||||
await captureCanvasScreenshot();
|
||||
```
|
||||
|
||||
**For Antigravity:** Take screenshots when:
|
||||
1. Room Graphics Card is visible (shows background)
|
||||
2. Object Editor Card is visible (shows objects overlaid)
|
||||
3. Both cards side-by-side if possible
|
||||
|
||||
###### 2. Extract Room Object Data Efficiently
|
||||
|
||||
```javascript
|
||||
// Get complete object rendering data for a room
|
||||
function getDungeonObjectDebugData(roomId) {
|
||||
const data = {
|
||||
room_id: roomId,
|
||||
timestamp: Date.now(),
|
||||
|
||||
// Room properties affecting rendering
|
||||
properties: window.yaze.data.getRoomProperties(roomId),
|
||||
|
||||
// All objects in the room with positions
|
||||
objects: window.yaze.data.getRoomObjects(roomId),
|
||||
|
||||
// Current editor state
|
||||
editor_state: window.yaze.editor.getCurrentRoom(),
|
||||
|
||||
// Graphics arena status (textures loaded?)
|
||||
arena_status: window.yazeDebug?.arena?.getStatus() || 'unavailable',
|
||||
|
||||
// Visible cards (what's being rendered)
|
||||
visible_cards: window.yaze.editor.getSnapshot()?.visible_cards || []
|
||||
};
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
// Pretty print for analysis
|
||||
const debugData = getDungeonObjectDebugData(42);
|
||||
console.log(JSON.stringify(debugData, null, 2));
|
||||
```
|
||||
|
||||
###### 3. Compare Object Positions vs Tile Positions
|
||||
|
||||
```javascript
|
||||
// Check if objects are aligned with the tile grid
|
||||
function analyzeObjectPlacement(roomId) {
|
||||
const objects = window.yaze.data.getRoomObjects(roomId);
|
||||
const tiles = window.yaze.data.getRoomTiles(roomId);
|
||||
|
||||
const analysis = objects.map(obj => {
|
||||
// Objects use pixel coordinates, tiles are 8x8 or 16x16
|
||||
const tileX = Math.floor(obj.x / 8);
|
||||
const tileY = Math.floor(obj.y / 8);
|
||||
|
||||
return {
|
||||
object_id: obj.id,
|
||||
type: obj.type,
|
||||
pixel_pos: { x: obj.x, y: obj.y },
|
||||
tile_pos: { x: tileX, y: tileY },
|
||||
// Check if position is on grid boundary
|
||||
aligned_8px: (obj.x % 8 === 0) && (obj.y % 8 === 0),
|
||||
aligned_16px: (obj.x % 16 === 0) && (obj.y % 16 === 0)
|
||||
};
|
||||
});
|
||||
|
||||
return analysis;
|
||||
}
|
||||
|
||||
console.log(JSON.stringify(analyzeObjectPlacement(42), null, 2));
|
||||
```
|
||||
|
||||
###### 4. Identify Visual Discrepancies
|
||||
|
||||
**Symptoms to look for:**
|
||||
|
||||
| Symptom | Likely Cause | Debug Command |
|
||||
|---------|-------------|---------------|
|
||||
| Objects invisible | Texture not loaded | `window.yazeDebug.arena.getStatus()` |
|
||||
| Wrong sprite shown | Object type mismatch | `window.yaze.data.getRoomObjects(roomId)` |
|
||||
| Position offset | Coordinate transform bug | Compare pixel_pos in data vs visual |
|
||||
| Colors wrong | Palette not applied | `Module.getDungeonPaletteEvents()` |
|
||||
| Flickering | Z-order/layer issue | Check `layer` property in object data |
|
||||
|
||||
###### 5. DOM Inspection for Card State
|
||||
|
||||
Efficiently query the DOM to understand what's being rendered:
|
||||
|
||||
```javascript
|
||||
// Get all visible ImGui windows (cards)
|
||||
function getVisibleCards() {
|
||||
// ImGui renders to canvas, but card state is tracked in JS
|
||||
const snapshot = window.yaze.editor.getSnapshot();
|
||||
return {
|
||||
active_cards: snapshot.visible_cards || [],
|
||||
editor_type: snapshot.editor_type,
|
||||
// Check if specific cards are open
|
||||
has_room_selector: snapshot.visible_cards?.includes('Room Selector'),
|
||||
has_object_editor: snapshot.visible_cards?.includes('Object Editor'),
|
||||
has_room_canvas: snapshot.visible_cards?.includes('Room Canvas')
|
||||
};
|
||||
}
|
||||
|
||||
console.log(getVisibleCards());
|
||||
```
|
||||
|
||||
###### 6. Full Diagnostic Dump for AI Analysis
|
||||
|
||||
```javascript
|
||||
// Complete diagnostic for Antigravity to analyze rendering issues
|
||||
async function generateRenderingDiagnostic(roomId) {
|
||||
const diagnostic = {
|
||||
timestamp: new Date().toISOString(),
|
||||
room_id: roomId,
|
||||
|
||||
// Visual state
|
||||
visible_cards: getVisibleCards(),
|
||||
|
||||
// Data state
|
||||
room_properties: window.yaze.data.getRoomProperties(roomId),
|
||||
room_objects: window.yaze.data.getRoomObjects(roomId),
|
||||
object_analysis: analyzeObjectPlacement(roomId),
|
||||
|
||||
// Graphics state
|
||||
arena: window.yazeDebug?.arena?.getStatus(),
|
||||
palette_events: (() => {
|
||||
try { return Module.getDungeonPaletteEvents(); }
|
||||
catch { return 'unavailable'; }
|
||||
})(),
|
||||
|
||||
// Memory state
|
||||
heap_mb: ((Module.HEAPU8?.length || 0) / 1024 / 1024).toFixed(2),
|
||||
|
||||
// Console errors (last 10)
|
||||
recent_errors: window._yazeConsoleLogs?.slice(-10) || []
|
||||
};
|
||||
|
||||
// Copy to clipboard for easy pasting
|
||||
const json = JSON.stringify(diagnostic, null, 2);
|
||||
await navigator.clipboard.writeText(json);
|
||||
console.log('[Diagnostic] Copied to clipboard');
|
||||
|
||||
return diagnostic;
|
||||
}
|
||||
|
||||
// Usage: Run this, then paste into your AI prompt
|
||||
await generateRenderingDiagnostic(42);
|
||||
```
|
||||
|
||||
##### Common Object Rendering Bugs
|
||||
|
||||
###### Bug: Objects Render at (0,0)
|
||||
|
||||
**Diagnosis:**
|
||||
```javascript
|
||||
// Check for objects with zero coordinates
|
||||
const objects = window.yaze.data.getRoomObjects(roomId);
|
||||
const atOrigin = objects.filter(o => o.x === 0 && o.y === 0);
|
||||
console.log('Objects at origin:', atOrigin);
|
||||
```
|
||||
|
||||
**Cause:** Object position data not loaded or coordinate transformation failed.
|
||||
|
||||
###### Bug: Sprite Shows as Black Square
|
||||
|
||||
**Diagnosis:**
|
||||
```javascript
|
||||
// Check if graphics sheet is loaded for object type
|
||||
const arena = window.yazeDebug.arena.getStatus();
|
||||
console.log('Loaded sheets:', arena.loaded_sheets);
|
||||
console.log('Pending textures:', arena.pending_textures);
|
||||
```
|
||||
|
||||
**Cause:** Texture not yet loaded from deferred queue. Force process:
|
||||
```javascript
|
||||
// Wait for textures to load
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
```
|
||||
|
||||
###### Bug: Object in Wrong Location vs Room Graphics
|
||||
|
||||
**Diagnosis:**
|
||||
```javascript
|
||||
// Compare layer1 tile at object position
|
||||
const obj = window.yaze.data.getRoomObjects(roomId)[0];
|
||||
const tiles = window.yaze.data.getRoomTiles(roomId);
|
||||
const tileAtPos = tiles.layer1[Math.floor(obj.y / 8) * 64 + Math.floor(obj.x / 8)];
|
||||
console.log('Object at:', obj.x, obj.y);
|
||||
console.log('Tile at that position:', tileAtPos);
|
||||
```
|
||||
|
||||
##### Screenshot Comparison Workflow
|
||||
|
||||
For visual debugging, use this workflow:
|
||||
|
||||
1. **Open Room Graphics Card only:**
|
||||
```javascript
|
||||
window.aiTools.hideCard('Object Editor');
|
||||
window.aiTools.showCard('Room Canvas');
|
||||
// Take screenshot #1
|
||||
```
|
||||
|
||||
2. **Enable Object Editor overlay:**
|
||||
```javascript
|
||||
window.aiTools.showCard('Object Editor');
|
||||
// Take screenshot #2
|
||||
```
|
||||
|
||||
3. **Compare:** Objects should align with the room's floor/wall tiles. Misalignment indicates a coordinate bug.
|
||||
|
||||
##### Reporting Issues
|
||||
|
||||
When reporting dungeon rendering bugs, include:
|
||||
|
||||
```javascript
|
||||
// Generate a complete bug report
|
||||
async function generateBugReport(roomId, description) {
|
||||
const report = {
|
||||
bug_description: description,
|
||||
room_id: roomId,
|
||||
diagnostic: await generateRenderingDiagnostic(roomId),
|
||||
steps_to_reproduce: [
|
||||
'1. Load ROM',
|
||||
'2. Open Dungeon Editor',
|
||||
`3. Navigate to Room ${roomId}`,
|
||||
'4. Observe [specific issue]'
|
||||
],
|
||||
expected_behavior: 'Objects should render at correct positions matching tile grid',
|
||||
actual_behavior: description
|
||||
};
|
||||
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
return report;
|
||||
}
|
||||
|
||||
// Usage
|
||||
await generateBugReport(42, 'Chest renders 8 pixels too far right');
|
||||
```
|
||||
135
docs/internal/agents/archive/wasm-planning-2025/README.md
Normal file
135
docs/internal/agents/archive/wasm-planning-2025/README.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# WASM Planning Documentation Archive
|
||||
|
||||
**Date Archived:** November 24, 2025
|
||||
**Archived By:** Documentation Janitor
|
||||
|
||||
This directory contains WASM development planning documents that represent historical design decisions and feature roadmaps. Most content has been superseded by implementation and integration into `docs/internal/wasm_dev_status.md`.
|
||||
|
||||
## Archived Documents
|
||||
|
||||
### 1. wasm-network-support-plan.md
|
||||
- **Original Purpose:** Detailed plan for implementing browser-compatible networking (Phases 1-5)
|
||||
- **Status:** Superseded by implementation
|
||||
- **Why Archived:** Network abstraction and WASM implementations have been completed; this planning document is no longer needed
|
||||
- **Current Reference:** See `wasm_dev_status.md` Section 1.1 (ROM Loading & Initialization)
|
||||
|
||||
### 2. wasm-web-features-roadmap.md
|
||||
- **Original Purpose:** Comprehensive feature roadmap (Phases 1-14)
|
||||
- **Status:** Mostly completed or planning-stage
|
||||
- **Why Archived:** Long-term planning document that predates actual implementation; many features are now in wasm_dev_status.md
|
||||
- **Current Reference:** See `wasm_dev_status.md` Sections 1-4 for completed features
|
||||
|
||||
### 3. wasm-web-app-enhancements-plan.md
|
||||
- **Original Purpose:** Detailed Phase 1-8 implementation plan
|
||||
- **Status:** Most phases completed
|
||||
- **Why Archived:** Highly structured planning document; actual implementations supersede these plans
|
||||
- **Current Reference:** See `wasm_dev_status.md` for current status of all phases
|
||||
|
||||
### 4. wasm-ai-integration-summary.md
|
||||
- **Original Purpose:** Summary of Phase 5 AI Service Integration implementation
|
||||
- **Status:** Consolidated into main status document
|
||||
- **Why Archived:** Content merged into `wasm_dev_status.md` AI Agent Integration section
|
||||
- **Current Reference:** See `wasm_dev_status.md` Section 1 (Completed Features → AI Agent Integration)
|
||||
|
||||
### 5. wasm-widget-tracking-implementation.md
|
||||
- **Original Purpose:** Detailed implementation notes for widget bounds tracking
|
||||
- **Status:** Consolidated into main status document
|
||||
- **Why Archived:** Implementation details merged into `wasm_dev_status.md` Control APIs section
|
||||
- **Current Reference:** See `wasm_dev_status.md` Section 1.4 (WASM Control APIs → Widget Tracking Infrastructure)
|
||||
|
||||
## Content Consolidation
|
||||
|
||||
The following information from archived documents has been consolidated into active documentation:
|
||||
|
||||
| Original Document | Content Moved To | Location |
|
||||
|---|---|---|
|
||||
| wasm-network-support-plan.md | wasm_dev_status.md | Section 1.1, Section 4 (Key Files) |
|
||||
| wasm-web-features-roadmap.md | wasm_dev_status.md | Section 1 (Completed Features) |
|
||||
| wasm-web-app-enhancements-plan.md | wasm_dev_status.md | Section 1 (Completed Features) |
|
||||
| wasm-ai-integration-summary.md | wasm_dev_status.md | Section 1 (AI Agent Integration) |
|
||||
| wasm-widget-tracking-implementation.md | wasm_dev_status.md | Section 1.4 (Widget Tracking Infrastructure) |
|
||||
|
||||
## Active WASM Documentation
|
||||
|
||||
The following documents remain in `docs/internal/` and are actively maintained:
|
||||
|
||||
1. **wasm_dev_status.md** - CANONICAL STATUS DOCUMENT
|
||||
- Current implementation status (updated Nov 24, 2025)
|
||||
- All completed features with file references
|
||||
- Technical debt and known issues
|
||||
- Roadmap for next steps
|
||||
|
||||
2. **wasm-debug-infrastructure.md** - HIGH-LEVEL DEBUGGING OVERVIEW
|
||||
- Debugging architecture and philosophy
|
||||
- File system fixes with explanations
|
||||
- Known limitations
|
||||
- Cross-references to detailed API docs
|
||||
|
||||
3. **wasm-yazeDebug-api-reference.md** - DETAILED API REFERENCE
|
||||
- Complete JavaScript API reference for `window.yazeDebug`
|
||||
- Authoritative source for all debug functions
|
||||
- Usage examples for each API section
|
||||
- Palette, ROM, overworld, arena, emulator debugging
|
||||
|
||||
4. **wasm_dungeon_debugging.md** - QUICK REFERENCE GUIDE
|
||||
- Short, practical debugging tips
|
||||
- God mode console inspector usage
|
||||
- Command line bridge reference
|
||||
- Feature parity notes between WASM and native builds
|
||||
|
||||
5. **debugging-wasm-memory-errors.md** - TECHNICAL REFERENCE
|
||||
- Memory debugging techniques
|
||||
- SAFE_HEAP usage
|
||||
- Common pitfalls and fixes
|
||||
- Function mapping methods
|
||||
|
||||
## How to Use This Archive
|
||||
|
||||
If you need historical context about WASM development decisions:
|
||||
1. Start with the relevant archived document
|
||||
2. Check the "Current Reference" section for where the content moved
|
||||
3. Consult the active documentation for implementation details
|
||||
|
||||
When searching for WASM documentation, use this hierarchy:
|
||||
1. **wasm_dev_status.md** - Status and overview (start here)
|
||||
2. **wasm-debug-infrastructure.md** - Debugging overview
|
||||
3. **wasm-yazeDebug-api-reference.md** - Detailed debug API
|
||||
4. **wasm_dungeon_debugging.md** - Quick reference for dungeon editor
|
||||
5. **debugging-wasm-memory-errors.md** - Memory debugging specifics
|
||||
|
||||
## Rationale for Archival
|
||||
|
||||
The WASM codebase evolved rapidly from November 2024 to November 2025:
|
||||
|
||||
- **Planning Phase** (early 2024): Detailed roadmaps and enhancement plans created
|
||||
- **Implementation Phase** (mid 2024 - Nov 2025): Features implemented incrementally
|
||||
- **Integration Phase** (current): All systems working, focus on maintenance and refinement
|
||||
|
||||
Archived documents represented the planning phase. As implementation completed, their value shifted from prescriptive (what to build) to historical (how we decided to build it). Consolidating information into `wasm_dev_status.md` provides:
|
||||
|
||||
- Single source of truth for current status
|
||||
- Easier maintenance (updates in one place)
|
||||
- Clearer navigation for developers
|
||||
- Better signal-to-noise ratio
|
||||
|
||||
## Future Archival Guidance
|
||||
|
||||
When new WASM documentation is created:
|
||||
- Keep planning/roadmap docs current or archive them promptly
|
||||
- Consolidate implementation summaries into main status document
|
||||
- Use high-level docs (like wasm-debug-infrastructure.md) for architecture overview
|
||||
- Use detailed reference docs (like wasm-yazeDebug-api-reference.md) for API details
|
||||
- Maintain clear cross-references between related docs
|
||||
|
||||
---
|
||||
|
||||
**Archive Directory Structure:**
|
||||
```
|
||||
docs/internal/agents/archive/wasm-planning-2025/
|
||||
├── README.md (this file)
|
||||
├── wasm-network-support-plan.md
|
||||
├── wasm-web-features-roadmap.md
|
||||
├── wasm-web-app-enhancements-plan.md
|
||||
├── wasm-ai-integration-summary.md
|
||||
└── wasm-widget-tracking-implementation.md
|
||||
```
|
||||
@@ -0,0 +1,199 @@
|
||||
# WASM AI Service Integration Summary
|
||||
|
||||
## Overview
|
||||
|
||||
This document summarizes the implementation of Phase 5: AI Service Integration for WASM web build, as specified in the wasm-web-app-enhancements-plan.md.
|
||||
|
||||
## Files Created
|
||||
|
||||
### 1. Browser AI Service (`src/cli/service/ai/`)
|
||||
|
||||
#### `browser_ai_service.h`
|
||||
- **Purpose**: Browser-based AI service interface for WASM builds
|
||||
- **Key Features**:
|
||||
- Implements `AIService` interface for consistency with native builds
|
||||
- Uses `IHttpClient` from network abstraction layer
|
||||
- Supports Gemini API for text generation
|
||||
- Provides vision model support for image analysis
|
||||
- Manages API keys securely via sessionStorage
|
||||
- CORS-compliant HTTP requests
|
||||
- Proper error handling with `absl::Status`
|
||||
- **Compilation**: Only compiled when `__EMSCRIPTEN__` is defined
|
||||
|
||||
#### `browser_ai_service.cc`
|
||||
- **Purpose**: Implementation of browser AI service
|
||||
- **Key Features**:
|
||||
- `GenerateResponse()` for single prompts and conversation history
|
||||
- `AnalyzeImage()` for vision model support
|
||||
- JSON request/response handling with nlohmann/json
|
||||
- Comprehensive error handling and status code mapping
|
||||
- Debug logging to browser console
|
||||
- Support for multiple Gemini models (2.0 Flash, 1.5 Pro, etc.)
|
||||
- Proper handling of API rate limits and quotas
|
||||
|
||||
### 2. Browser Storage (`src/app/platform/wasm/`)
|
||||
|
||||
#### `wasm_browser_storage.h`
|
||||
- **Purpose**: Browser storage wrapper for API keys and settings
|
||||
- **Note**: This is NOT actually secure storage - uses standard localStorage/sessionStorage
|
||||
- **Key Features**:
|
||||
- Dual storage modes: sessionStorage (default) and localStorage
|
||||
- API key management: Store, Retrieve, Clear, Check existence
|
||||
- Generic secret storage for other sensitive data
|
||||
- Storage quota tracking
|
||||
- Bulk operations (list all keys, clear all)
|
||||
- Browser storage availability checking
|
||||
|
||||
#### `wasm_browser_storage.cc`
|
||||
- **Purpose**: Implementation using Emscripten JavaScript interop
|
||||
- **Key Features**:
|
||||
- JavaScript bridge functions using `EM_JS` macros
|
||||
- SessionStorage access (cleared on tab close)
|
||||
- LocalStorage access (persistent)
|
||||
- Prefix-based key namespacing (`yaze_secure_api_`, `yaze_secure_secret_`)
|
||||
- Error handling for storage exceptions
|
||||
- Memory management for JS string conversions
|
||||
|
||||
## Build System Updates
|
||||
|
||||
### 1. CMake Configuration Updates
|
||||
|
||||
#### `src/cli/agent.cmake`
|
||||
- Modified to create a minimal `yaze_agent` library for WASM builds
|
||||
- Includes browser AI service sources
|
||||
- Links with network abstraction layer (`yaze_net`)
|
||||
- Enables JSON support for API communication
|
||||
|
||||
#### `src/app/app_core.cmake`
|
||||
- Added `wasm_browser_storage.cc` to WASM platform sources
|
||||
- Integrated with existing WASM file system and loading manager
|
||||
|
||||
#### `src/CMakeLists.txt`
|
||||
- Updated to include `net_library.cmake` for all builds (including WASM)
|
||||
- Network library now provides WASM-compatible HTTP client
|
||||
|
||||
#### `CMakePresets.json`
|
||||
- Added new `wasm-ai` preset for testing AI features in WASM
|
||||
- Configured with AI runtime enabled and Fetch API flags
|
||||
|
||||
## Integration with Existing Systems
|
||||
|
||||
### Network Abstraction Layer
|
||||
- Leverages existing `IHttpClient` interface
|
||||
- Uses `EmscriptenHttpClient` for browser-based HTTP requests
|
||||
- Supports CORS-compliant requests to Gemini API
|
||||
|
||||
### AI Service Interface
|
||||
- Implements standard `AIService` interface
|
||||
- Compatible with existing agent response structures
|
||||
- Supports tool calls and structured responses
|
||||
|
||||
### WASM Platform Support
|
||||
- Integrates with existing WASM error handler
|
||||
- Works alongside WASM storage and file dialog systems
|
||||
- Compatible with progressive loading manager
|
||||
|
||||
## API Key Security
|
||||
|
||||
### Storage Security Model
|
||||
1. **SessionStorage (Default)**:
|
||||
- Keys stored in browser memory
|
||||
- Automatically cleared when tab closes
|
||||
- No persistence across sessions
|
||||
- Recommended for security
|
||||
|
||||
2. **LocalStorage (Optional)**:
|
||||
- Persistent storage
|
||||
- Survives browser restarts
|
||||
- Less secure but more convenient
|
||||
- User choice based on preference
|
||||
|
||||
### Security Considerations
|
||||
- Keys never hardcoded in binary
|
||||
- Keys prefixed to avoid conflicts
|
||||
- No encryption currently (future enhancement)
|
||||
- Browser same-origin policy provides isolation
|
||||
|
||||
## Usage Example
|
||||
|
||||
```cpp
|
||||
#ifdef __EMSCRIPTEN__
|
||||
#include "cli/service/ai/browser_ai_service.h"
|
||||
#include "app/net/wasm/emscripten_http_client.h"
|
||||
#include "app/platform/wasm/wasm_browser_storage.h"
|
||||
|
||||
// Store API key from user input
|
||||
WasmBrowserStorage::StoreApiKey("gemini", user_api_key);
|
||||
}
|
||||
|
||||
// Create AI service
|
||||
BrowserAIConfig config;
|
||||
config.api_key = WasmBrowserStorage::RetrieveApiKey("gemini").value();
|
||||
config.model = "gemini-2.5-flash";
|
||||
|
||||
auto http_client = std::make_unique<EmscriptenHttpClient>();
|
||||
BrowserAIService ai_service(config, std::move(http_client));
|
||||
|
||||
// Generate response
|
||||
auto response = ai_service.GenerateResponse("Explain the Zelda 3 ROM format");
|
||||
#endif
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Test File: `test/browser_ai_test.cc`
|
||||
- Verifies secure storage operations
|
||||
- Tests AI service creation
|
||||
- Validates model listing
|
||||
- Checks error handling
|
||||
|
||||
### Build and Test Commands
|
||||
```bash
|
||||
# Configure with AI support
|
||||
cmake --preset wasm-ai
|
||||
|
||||
# Build
|
||||
cmake --build build_wasm_ai
|
||||
|
||||
# Run in browser
|
||||
emrun build_wasm_ai/yaze.html
|
||||
```
|
||||
|
||||
## CORS Considerations
|
||||
|
||||
### Gemini API
|
||||
- ✅ Works with browser fetch (Google APIs support CORS)
|
||||
- ✅ No proxy required
|
||||
- ✅ Direct browser-to-API communication
|
||||
|
||||
### Ollama (Future)
|
||||
- ⚠️ Requires `--cors` flag on Ollama server
|
||||
- ⚠️ May need proxy for local instances
|
||||
- ⚠️ Security implications of CORS relaxation
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Encryption**: Add client-side encryption for stored API keys
|
||||
2. **Multiple Providers**: Support for OpenAI, Anthropic APIs
|
||||
3. **Streaming Responses**: Implement streaming for better UX
|
||||
4. **Offline Caching**: Cache AI responses for offline use
|
||||
5. **Web Worker Integration**: Move AI calls to background thread
|
||||
|
||||
## Limitations
|
||||
|
||||
1. **Browser Security**: Subject to browser security policies
|
||||
2. **CORS Restrictions**: Limited to CORS-enabled APIs
|
||||
3. **Storage Limits**: ~5-10MB for sessionStorage/localStorage
|
||||
4. **No File System**: Cannot access local models
|
||||
5. **Network Required**: No offline AI capabilities
|
||||
|
||||
## Conclusion
|
||||
|
||||
The WASM AI service integration successfully brings browser-based AI capabilities to yaze. The implementation:
|
||||
- ✅ Provides secure API key management
|
||||
- ✅ Integrates cleanly with existing architecture
|
||||
- ✅ Supports both text and vision models
|
||||
- ✅ Handles errors gracefully
|
||||
- ✅ Works within browser security constraints
|
||||
|
||||
This enables users to leverage AI assistance for ROM hacking directly in their browser without needing to install local AI models or tools.
|
||||
@@ -0,0 +1,415 @@
|
||||
# WASM Network Support Plan for yaze
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines the architectural changes required to enable AI services (Gemini/Ollama) and WebSocket collaboration features in the browser-based WASM build of yaze. The main challenge is replacing native networking libraries (cpp-httplib, OpenSSL, curl) with browser-compatible APIs provided by Emscripten.
|
||||
|
||||
## Current Architecture Analysis
|
||||
|
||||
### 1. Gemini AI Service (`src/cli/service/ai/gemini_ai_service.cc`)
|
||||
|
||||
**Current Implementation:**
|
||||
- Uses `httplib` (cpp-httplib) for HTTPS requests with OpenSSL support
|
||||
- Falls back to `curl` command via `popen()` for API calls
|
||||
- Depends on OpenSSL for SSL/TLS encryption
|
||||
- Uses base64 encoding for image uploads
|
||||
|
||||
**Key Dependencies:**
|
||||
- `httplib.h` - C++ HTTP library
|
||||
- OpenSSL - SSL/TLS support
|
||||
- `popen()`/`pclose()` - Process execution for curl
|
||||
- File system access for temporary JSON files
|
||||
|
||||
### 2. Ollama AI Service (`src/cli/service/ai/ollama_ai_service.cc`)
|
||||
|
||||
**Current Implementation:**
|
||||
- Uses `httplib::Client` for HTTP requests to local Ollama server
|
||||
- Communicates over HTTP (not HTTPS) on localhost:11434
|
||||
- JSON parsing with nlohmann/json
|
||||
|
||||
**Key Dependencies:**
|
||||
- `httplib.h` - C++ HTTP library
|
||||
- No SSL/TLS requirement (local HTTP only)
|
||||
|
||||
### 3. WebSocket Client (`src/app/net/websocket_client.cc`)
|
||||
|
||||
**Current Implementation:**
|
||||
- Uses `httplib::Client` as a placeholder (not true WebSocket)
|
||||
- Currently implements HTTP POST fallback instead of WebSocket
|
||||
- Conditional OpenSSL support for secure connections
|
||||
- Thread-based receive loop
|
||||
|
||||
**Key Dependencies:**
|
||||
- `httplib.h` - C++ HTTP library
|
||||
- OpenSSL (optional) - For WSS support
|
||||
- `std::thread` - For receive loop
|
||||
- Platform-specific socket libraries (ws2_32 on Windows)
|
||||
|
||||
### 4. HTTP Server (`src/cli/service/api/http_server.cc`)
|
||||
|
||||
**Current Implementation:**
|
||||
- Uses `httplib::Server` for REST API endpoints
|
||||
- Runs in separate thread
|
||||
- Provides health check and model listing endpoints
|
||||
|
||||
**Key Dependencies:**
|
||||
- `httplib.h` - C++ HTTP server
|
||||
- `std::thread` - For server thread
|
||||
|
||||
## Emscripten Capabilities
|
||||
|
||||
### Available APIs:
|
||||
|
||||
1. **Fetch API** (`emscripten_fetch()`)
|
||||
- Asynchronous HTTP/HTTPS requests
|
||||
- Supports GET, POST, PUT, DELETE
|
||||
- CORS-aware
|
||||
- Can handle binary data and streams
|
||||
|
||||
2. **WebSocket API**
|
||||
- Native browser WebSocket support
|
||||
- Accessible via Emscripten's WebSocket wrapper
|
||||
- Full duplex communication
|
||||
- Binary and text message support
|
||||
|
||||
3. **Web Workers** (via pthread emulation)
|
||||
- Background processing
|
||||
- Shared memory support with SharedArrayBuffer
|
||||
- Can handle async operations
|
||||
|
||||
## Required Changes
|
||||
|
||||
### Phase 1: Abstract Network Layer
|
||||
|
||||
Create platform-agnostic network interfaces:
|
||||
|
||||
```cpp
|
||||
// src/app/net/http_client.h
|
||||
class IHttpClient {
|
||||
public:
|
||||
virtual ~IHttpClient() = default;
|
||||
virtual absl::StatusOr<HttpResponse> Get(const std::string& url,
|
||||
const Headers& headers) = 0;
|
||||
virtual absl::StatusOr<HttpResponse> Post(const std::string& url,
|
||||
const std::string& body,
|
||||
const Headers& headers) = 0;
|
||||
};
|
||||
|
||||
// src/app/net/websocket.h
|
||||
class IWebSocket {
|
||||
public:
|
||||
virtual ~IWebSocket() = default;
|
||||
virtual absl::Status Connect(const std::string& url) = 0;
|
||||
virtual absl::Status Send(const std::string& message) = 0;
|
||||
virtual void OnMessage(std::function<void(const std::string&)> callback) = 0;
|
||||
};
|
||||
```
|
||||
|
||||
### Phase 2: Native Implementation
|
||||
|
||||
Keep existing implementations for native builds:
|
||||
|
||||
```cpp
|
||||
// src/app/net/native/httplib_client.cc
|
||||
class HttpLibClient : public IHttpClient {
|
||||
// Current httplib implementation
|
||||
};
|
||||
|
||||
// src/app/net/native/httplib_websocket.cc
|
||||
class HttpLibWebSocket : public IWebSocket {
|
||||
// Current httplib-based implementation
|
||||
};
|
||||
```
|
||||
|
||||
### Phase 3: Emscripten Implementation
|
||||
|
||||
Create browser-compatible implementations:
|
||||
|
||||
```cpp
|
||||
// src/app/net/wasm/emscripten_http_client.cc
|
||||
#ifdef __EMSCRIPTEN__
|
||||
class EmscriptenHttpClient : public IHttpClient {
|
||||
absl::StatusOr<HttpResponse> Get(const std::string& url,
|
||||
const Headers& headers) override {
|
||||
emscripten_fetch_attr_t attr;
|
||||
emscripten_fetch_attr_init(&attr);
|
||||
strcpy(attr.requestMethod, "GET");
|
||||
attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
|
||||
|
||||
// Set headers
|
||||
std::vector<const char*> header_strings;
|
||||
for (const auto& [key, value] : headers) {
|
||||
header_strings.push_back(key.c_str());
|
||||
header_strings.push_back(value.c_str());
|
||||
}
|
||||
header_strings.push_back(nullptr);
|
||||
attr.requestHeaders = header_strings.data();
|
||||
|
||||
// Synchronous fetch (blocks until complete)
|
||||
emscripten_fetch_t* fetch = emscripten_fetch(&attr, url.c_str());
|
||||
|
||||
HttpResponse response;
|
||||
response.status = fetch->status;
|
||||
response.body = std::string(fetch->data, fetch->numBytes);
|
||||
|
||||
emscripten_fetch_close(fetch);
|
||||
return response;
|
||||
}
|
||||
|
||||
absl::StatusOr<HttpResponse> Post(const std::string& url,
|
||||
const std::string& body,
|
||||
const Headers& headers) override {
|
||||
emscripten_fetch_attr_t attr;
|
||||
emscripten_fetch_attr_init(&attr);
|
||||
strcpy(attr.requestMethod, "POST");
|
||||
attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
|
||||
attr.requestData = body.c_str();
|
||||
attr.requestDataSize = body.length();
|
||||
|
||||
// Similar implementation as Get()
|
||||
// ...
|
||||
}
|
||||
};
|
||||
#endif
|
||||
|
||||
// src/app/net/wasm/emscripten_websocket.cc
|
||||
#ifdef __EMSCRIPTEN__
|
||||
#include <emscripten/websocket.h>
|
||||
|
||||
class EmscriptenWebSocket : public IWebSocket {
|
||||
EMSCRIPTEN_WEBSOCKET_T socket_;
|
||||
|
||||
absl::Status Connect(const std::string& url) override {
|
||||
EmscriptenWebSocketCreateAttributes attrs = {
|
||||
url.c_str(),
|
||||
nullptr, // protocols
|
||||
EM_TRUE // createOnMainThread
|
||||
};
|
||||
|
||||
socket_ = emscripten_websocket_new(&attrs);
|
||||
if (socket_ <= 0) {
|
||||
return absl::InternalError("Failed to create WebSocket");
|
||||
}
|
||||
|
||||
// Set callbacks
|
||||
emscripten_websocket_set_onopen_callback(socket_, this, OnOpenCallback);
|
||||
emscripten_websocket_set_onmessage_callback(socket_, this, OnMessageCallback);
|
||||
emscripten_websocket_set_onerror_callback(socket_, this, OnErrorCallback);
|
||||
|
||||
return absl::OkStatus();
|
||||
}
|
||||
|
||||
absl::Status Send(const std::string& message) override {
|
||||
EMSCRIPTEN_RESULT result = emscripten_websocket_send_text(
|
||||
socket_, message.c_str(), message.length());
|
||||
if (result != EMSCRIPTEN_RESULT_SUCCESS) {
|
||||
return absl::InternalError("Failed to send WebSocket message");
|
||||
}
|
||||
return absl::OkStatus();
|
||||
}
|
||||
};
|
||||
#endif
|
||||
```
|
||||
|
||||
### Phase 4: Factory Pattern
|
||||
|
||||
Create factories to instantiate the correct implementation:
|
||||
|
||||
```cpp
|
||||
// src/app/net/network_factory.cc
|
||||
std::unique_ptr<IHttpClient> CreateHttpClient() {
|
||||
#ifdef __EMSCRIPTEN__
|
||||
return std::make_unique<EmscriptenHttpClient>();
|
||||
#else
|
||||
return std::make_unique<HttpLibClient>();
|
||||
#endif
|
||||
}
|
||||
|
||||
std::unique_ptr<IWebSocket> CreateWebSocket() {
|
||||
#ifdef __EMSCRIPTEN__
|
||||
return std::make_unique<EmscriptenWebSocket>();
|
||||
#else
|
||||
return std::make_unique<HttpLibWebSocket>();
|
||||
#endif
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 5: Service Modifications
|
||||
|
||||
Update AI services to use the abstraction:
|
||||
|
||||
```cpp
|
||||
// src/cli/service/ai/gemini_ai_service.cc
|
||||
class GeminiAIService {
|
||||
std::unique_ptr<IHttpClient> http_client_;
|
||||
|
||||
GeminiAIService(const GeminiConfig& config)
|
||||
: config_(config),
|
||||
http_client_(CreateHttpClient()) {
|
||||
// Initialize
|
||||
}
|
||||
|
||||
absl::Status CheckAvailability() {
|
||||
std::string url = "https://generativelanguage.googleapis.com/v1beta/models/"
|
||||
+ config_.model;
|
||||
Headers headers = {{"x-goog-api-key", config_.api_key}};
|
||||
|
||||
auto response_or = http_client_->Get(url, headers);
|
||||
if (!response_or.ok()) {
|
||||
return response_or.status();
|
||||
}
|
||||
|
||||
auto& response = response_or.value();
|
||||
if (response.status != 200) {
|
||||
return absl::UnavailableError("API not available");
|
||||
}
|
||||
|
||||
return absl::OkStatus();
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## CMake Configuration
|
||||
|
||||
Update CMake to handle WASM builds:
|
||||
|
||||
```cmake
|
||||
# src/app/net/net_library.cmake
|
||||
if(EMSCRIPTEN)
|
||||
set(YAZE_NET_SRC
|
||||
app/net/wasm/emscripten_http_client.cc
|
||||
app/net/wasm/emscripten_websocket.cc
|
||||
app/net/network_factory.cc
|
||||
)
|
||||
|
||||
# Add Emscripten fetch and WebSocket flags
|
||||
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -s FETCH=1")
|
||||
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -s WEBSOCKET=1")
|
||||
else()
|
||||
set(YAZE_NET_SRC
|
||||
app/net/native/httplib_client.cc
|
||||
app/net/native/httplib_websocket.cc
|
||||
app/net/network_factory.cc
|
||||
)
|
||||
|
||||
# Link native dependencies
|
||||
target_link_libraries(yaze_net PUBLIC httplib OpenSSL::SSL)
|
||||
endif()
|
||||
```
|
||||
|
||||
## CORS Considerations
|
||||
|
||||
For browser builds, the following CORS requirements apply:
|
||||
|
||||
1. **Gemini API**: Google's API servers must include appropriate CORS headers
|
||||
2. **Ollama**: Local Ollama server needs `--cors` flag or proxy setup
|
||||
3. **WebSocket Server**: Must handle WebSocket upgrade correctly
|
||||
|
||||
### Proxy Server Option
|
||||
|
||||
For services without CORS support, implement a proxy:
|
||||
|
||||
```javascript
|
||||
// proxy-server.js (Node.js)
|
||||
const express = require('express');
|
||||
const { createProxyMiddleware } = require('http-proxy-middleware');
|
||||
|
||||
const app = express();
|
||||
|
||||
// Proxy for Ollama
|
||||
app.use('/ollama', createProxyMiddleware({
|
||||
target: 'http://localhost:11434',
|
||||
changeOrigin: true,
|
||||
pathRewrite: { '^/ollama': '' }
|
||||
}));
|
||||
|
||||
// Proxy for Gemini
|
||||
app.use('/gemini', createProxyMiddleware({
|
||||
target: 'https://generativelanguage.googleapis.com',
|
||||
changeOrigin: true,
|
||||
pathRewrite: { '^/gemini': '' }
|
||||
}));
|
||||
|
||||
app.listen(3000);
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. **Unit Tests**: Mock network interfaces for both native and WASM
|
||||
2. **Integration Tests**: Test actual API calls in native builds
|
||||
3. **Browser Tests**: Manual testing of WASM build in browser
|
||||
4. **E2E Tests**: Selenium/Playwright for automated browser testing
|
||||
|
||||
## Implementation Timeline
|
||||
|
||||
### Week 1: Foundation
|
||||
- Create abstract interfaces (IHttpClient, IWebSocket)
|
||||
- Implement factory pattern
|
||||
- Update CMake for conditional compilation
|
||||
|
||||
### Week 2: Native Refactoring
|
||||
- Refactor existing code to use interfaces
|
||||
- Create native implementations
|
||||
- Ensure no regression in current functionality
|
||||
|
||||
### Week 3: WASM Implementation
|
||||
- Implement EmscriptenHttpClient
|
||||
- Implement EmscriptenWebSocket
|
||||
- Test basic functionality
|
||||
|
||||
### Week 4: Service Integration
|
||||
- Update GeminiAIService
|
||||
- Update OllamaAIService
|
||||
- Update WebSocketClient
|
||||
|
||||
### Week 5: Testing & Refinement
|
||||
- Comprehensive testing
|
||||
- CORS handling
|
||||
- Performance optimization
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
### Risk 1: CORS Blocking
|
||||
**Mitigation**: Implement proxy server as fallback, document CORS requirements
|
||||
|
||||
### Risk 2: API Key Security
|
||||
**Mitigation**:
|
||||
- Never embed API keys in WASM binary
|
||||
- Require user to input API key via UI
|
||||
- Store in browser's localStorage with encryption
|
||||
|
||||
### Risk 3: Performance Issues
|
||||
**Mitigation**:
|
||||
- Use Web Workers for background processing
|
||||
- Implement request caching
|
||||
- Add loading indicators for long operations
|
||||
|
||||
### Risk 4: Browser Compatibility
|
||||
**Mitigation**:
|
||||
- Test on Chrome, Firefox, Safari, Edge
|
||||
- Use feature detection
|
||||
- Provide fallbacks for unsupported features
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **API Keys**: Must be user-provided, never hardcoded
|
||||
2. **HTTPS Only**: All API calls must use HTTPS in production
|
||||
3. **Input Validation**: Sanitize all user inputs before API calls
|
||||
4. **Rate Limiting**: Implement client-side rate limiting
|
||||
5. **Content Security Policy**: Configure CSP headers properly
|
||||
|
||||
## Conclusion
|
||||
|
||||
The transition to WASM-compatible networking is achievable through careful abstraction and platform-specific implementations. The key is maintaining a clean separation between platform-agnostic interfaces and platform-specific implementations. This approach allows the codebase to support both native and browser environments without sacrificing functionality or performance.
|
||||
|
||||
The proposed architecture provides:
|
||||
- Clean abstraction layers
|
||||
- Minimal changes to existing service code
|
||||
- Easy testing and mocking
|
||||
- Future extensibility for other platforms
|
||||
|
||||
Next steps:
|
||||
1. Review and approve this plan
|
||||
2. Create feature branch for implementation
|
||||
3. Begin with abstract interface definitions
|
||||
4. Implement incrementally with continuous testing
|
||||
@@ -0,0 +1,554 @@
|
||||
# WASM Web App Enhancements Plan
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines the comprehensive plan to make yaze's WASM web build fully featured, robust, and user-friendly. The goal is to achieve feature parity with the native desktop application where technically feasible, while leveraging browser-specific capabilities where appropriate.
|
||||
|
||||
## Current Status
|
||||
|
||||
### Completed
|
||||
- [x] Basic WASM build configuration
|
||||
- [x] pthread support for Emscripten
|
||||
- [x] Network abstraction layer (IHttpClient, IWebSocket)
|
||||
- [x] Emscripten HTTP client using `emscripten_fetch()`
|
||||
- [x] Emscripten WebSocket using browser API
|
||||
- [x] **Phase 1**: File System Layer (WasmStorage, WasmFileDialog)
|
||||
- [x] **Phase 2**: Error Handling Infrastructure (WasmErrorHandler)
|
||||
- [x] **Phase 3**: Progressive Loading UI (WasmLoadingManager)
|
||||
- [x] **Phase 4**: Offline Support (Service Workers, PWA manifest)
|
||||
- [x] **Phase 5**: AI Service Integration (BrowserAIService, WasmSecureStorage)
|
||||
- [x] **Phase 6**: Local Storage Persistence (WasmSettings, AutoSaveManager)
|
||||
|
||||
- [x] **Phase 7**: Web Workers for heavy processing (WasmWorkerPool)
|
||||
- [x] **Phase 8**: Emulator Audio (WebAudio, WasmAudioBackend)
|
||||
|
||||
### In Progress
|
||||
- [ ] WASM CI build verification
|
||||
- [ ] Integration of loading manager with gfx::Arena
|
||||
- [ ] Integration testing across all phases
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: File System Layer
|
||||
|
||||
### Overview
|
||||
WASM builds cannot access the local filesystem directly. We need a virtualized file system layer that uses browser storage APIs.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 1.1 IndexedDB Storage Backend
|
||||
```cpp
|
||||
// src/app/platform/wasm/wasm_storage.h
|
||||
class WasmStorage {
|
||||
public:
|
||||
// Store ROM data
|
||||
static absl::Status SaveRom(const std::string& name, const std::vector<uint8_t>& data);
|
||||
static absl::StatusOr<std::vector<uint8_t>> LoadRom(const std::string& name);
|
||||
static absl::Status DeleteRom(const std::string& name);
|
||||
static std::vector<std::string> ListRoms();
|
||||
|
||||
// Store project files (JSON, palettes, patches)
|
||||
static absl::Status SaveProject(const std::string& name, const std::string& json);
|
||||
static absl::StatusOr<std::string> LoadProject(const std::string& name);
|
||||
|
||||
// Store user preferences
|
||||
static absl::Status SavePreferences(const nlohmann::json& prefs);
|
||||
static absl::StatusOr<nlohmann::json> LoadPreferences();
|
||||
};
|
||||
```
|
||||
|
||||
#### 1.2 File Upload Handler
|
||||
```cpp
|
||||
// JavaScript interop for file input
|
||||
EM_JS(void, openFileDialog, (const char* accept, int callback_id), {
|
||||
const input = document.createElement('input');
|
||||
input.type = 'file';
|
||||
input.accept = UTF8ToString(accept);
|
||||
input.onchange = (e) => {
|
||||
const file = e.target.files[0];
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => {
|
||||
const data = new Uint8Array(reader.result);
|
||||
Module._handleFileLoaded(callback_id, data);
|
||||
};
|
||||
reader.readAsArrayBuffer(file);
|
||||
};
|
||||
input.click();
|
||||
});
|
||||
```
|
||||
|
||||
#### 1.3 File Download Handler
|
||||
```cpp
|
||||
// Download ROM or project file
|
||||
EM_JS(void, downloadFile, (const char* filename, const uint8_t* data, size_t size), {
|
||||
const blob = new Blob([HEAPU8.subarray(data, data + size)], {type: 'application/octet-stream'});
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = UTF8ToString(filename);
|
||||
a.click();
|
||||
URL.revokeObjectURL(url);
|
||||
});
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/app/platform/wasm/wasm_storage.h`
|
||||
- `src/app/platform/wasm/wasm_storage.cc`
|
||||
- `src/app/platform/wasm/wasm_file_dialog.h`
|
||||
- `src/app/platform/wasm/wasm_file_dialog.cc`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Error Handling Infrastructure
|
||||
|
||||
### Overview
|
||||
Browser-based applications need specialized error handling that integrates with the web UI and provides user-friendly feedback.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 2.1 Browser Error Handler
|
||||
```cpp
|
||||
// src/app/platform/wasm/wasm_error_handler.h
|
||||
class WasmErrorHandler {
|
||||
public:
|
||||
// Display error in browser UI
|
||||
static void ShowError(const std::string& title, const std::string& message);
|
||||
static void ShowWarning(const std::string& title, const std::string& message);
|
||||
static void ShowInfo(const std::string& title, const std::string& message);
|
||||
|
||||
// Toast notifications (non-blocking)
|
||||
static void Toast(const std::string& message, ToastType type, int duration_ms = 3000);
|
||||
|
||||
// Progress indicators
|
||||
static void ShowProgress(const std::string& task, float progress);
|
||||
static void HideProgress();
|
||||
|
||||
// Confirmation dialogs
|
||||
static void Confirm(const std::string& message, std::function<void(bool)> callback);
|
||||
};
|
||||
```
|
||||
|
||||
#### 2.2 JavaScript Integration
|
||||
```javascript
|
||||
// src/web/error_handler.js
|
||||
window.showYazeError = function(title, message) {
|
||||
// Create styled error modal
|
||||
const modal = document.createElement('div');
|
||||
modal.className = 'yaze-error-modal';
|
||||
modal.innerHTML = `
|
||||
<div class="yaze-error-content">
|
||||
<h2>${escapeHtml(title)}</h2>
|
||||
<p>${escapeHtml(message)}</p>
|
||||
<button onclick="this.parentElement.parentElement.remove()">OK</button>
|
||||
</div>
|
||||
`;
|
||||
document.body.appendChild(modal);
|
||||
};
|
||||
|
||||
window.showYazeToast = function(message, type, duration) {
|
||||
const toast = document.createElement('div');
|
||||
toast.className = `yaze-toast yaze-toast-${type}`;
|
||||
toast.textContent = message;
|
||||
document.body.appendChild(toast);
|
||||
setTimeout(() => toast.remove(), duration);
|
||||
};
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/app/platform/wasm/wasm_error_handler.h`
|
||||
- `src/app/platform/wasm/wasm_error_handler.cc`
|
||||
- `src/web/error_handler.js`
|
||||
- `src/web/error_handler.css`
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Progressive Loading UI
|
||||
|
||||
### Overview
|
||||
ROM loading and graphics processing can take significant time. Users need visual feedback and the ability to cancel long operations.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 3.1 Loading Manager
|
||||
```cpp
|
||||
// src/app/platform/wasm/wasm_loading_manager.h
|
||||
class WasmLoadingManager {
|
||||
public:
|
||||
// Start a loading operation with progress tracking
|
||||
static LoadingHandle BeginLoading(const std::string& task_name);
|
||||
|
||||
// Update progress (0.0 to 1.0)
|
||||
static void UpdateProgress(LoadingHandle handle, float progress);
|
||||
static void UpdateMessage(LoadingHandle handle, const std::string& message);
|
||||
|
||||
// Check if user requested cancel
|
||||
static bool IsCancelled(LoadingHandle handle);
|
||||
|
||||
// Complete the loading operation
|
||||
static void EndLoading(LoadingHandle handle);
|
||||
};
|
||||
```
|
||||
|
||||
#### 3.2 Integration with gfx::Arena
|
||||
```cpp
|
||||
// Modified graphics loading to report progress
|
||||
void Arena::LoadGraphicsWithProgress(Rom* rom) {
|
||||
auto handle = WasmLoadingManager::BeginLoading("Loading Graphics");
|
||||
|
||||
for (int i = 0; i < kNumGraphicsSheets; i++) {
|
||||
if (WasmLoadingManager::IsCancelled(handle)) {
|
||||
WasmLoadingManager::EndLoading(handle);
|
||||
return; // User cancelled
|
||||
}
|
||||
|
||||
LoadGraphicsSheet(rom, i);
|
||||
WasmLoadingManager::UpdateProgress(handle, static_cast<float>(i) / kNumGraphicsSheets);
|
||||
WasmLoadingManager::UpdateMessage(handle, absl::StrFormat("Sheet %d/%d", i, kNumGraphicsSheets));
|
||||
|
||||
// Yield to browser event loop periodically
|
||||
emscripten_sleep(0);
|
||||
}
|
||||
|
||||
WasmLoadingManager::EndLoading(handle);
|
||||
}
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/app/platform/wasm/wasm_loading_manager.h`
|
||||
- `src/app/platform/wasm/wasm_loading_manager.cc`
|
||||
- `src/web/loading_indicator.js`
|
||||
- `src/web/loading_indicator.css`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Offline Support (Service Workers)
|
||||
|
||||
### Overview
|
||||
Cache the WASM binary and assets for offline use, enabling users to work without an internet connection.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 4.1 Service Worker
|
||||
```javascript
|
||||
// src/web/service-worker.js
|
||||
const CACHE_NAME = 'yaze-cache-v1';
|
||||
const ASSETS = [
|
||||
'/yaze.wasm',
|
||||
'/yaze.js',
|
||||
'/yaze.data',
|
||||
'/index.html',
|
||||
'/style.css',
|
||||
'/fonts/',
|
||||
];
|
||||
|
||||
self.addEventListener('install', (event) => {
|
||||
event.waitUntil(
|
||||
caches.open(CACHE_NAME).then((cache) => cache.addAll(ASSETS))
|
||||
);
|
||||
});
|
||||
|
||||
self.addEventListener('fetch', (event) => {
|
||||
event.respondWith(
|
||||
caches.match(event.request).then((response) => {
|
||||
return response || fetch(event.request);
|
||||
})
|
||||
);
|
||||
});
|
||||
```
|
||||
|
||||
#### 4.2 Registration
|
||||
```javascript
|
||||
// src/web/index.html
|
||||
if ('serviceWorker' in navigator) {
|
||||
navigator.serviceWorker.register('/service-worker.js')
|
||||
.then((registration) => {
|
||||
console.log('Service Worker registered:', registration.scope);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/web/service-worker.js`
|
||||
- `src/web/manifest.json` (PWA manifest)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: AI Service Integration
|
||||
|
||||
### Overview
|
||||
Integrate the network abstraction layer with AI services (Gemini, Ollama) for browser-based AI assistance.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 5.1 Browser AI Client
|
||||
```cpp
|
||||
// src/cli/service/ai/browser_ai_service.h
|
||||
#ifdef __EMSCRIPTEN__
|
||||
class BrowserAIService : public IAIService {
|
||||
public:
|
||||
explicit BrowserAIService(const AIConfig& config);
|
||||
|
||||
absl::StatusOr<std::string> GenerateResponse(const std::string& prompt) override;
|
||||
absl::StatusOr<std::string> AnalyzeImage(const gfx::Bitmap& image,
|
||||
const std::string& prompt) override;
|
||||
|
||||
private:
|
||||
std::unique_ptr<IHttpClient> http_client_;
|
||||
std::string api_key_; // User-provided, never stored in binary
|
||||
std::string model_;
|
||||
};
|
||||
#endif
|
||||
```
|
||||
|
||||
#### 5.2 API Key Management
|
||||
```cpp
|
||||
// Secure API key storage in browser
|
||||
EM_JS(void, storeApiKey, (const char* service, const char* key), {
|
||||
// Use sessionStorage for temporary storage (cleared on tab close)
|
||||
// or encrypted localStorage for persistent storage
|
||||
sessionStorage.setItem('yaze_' + UTF8ToString(service) + '_key', UTF8ToString(key));
|
||||
});
|
||||
|
||||
EM_JS(char*, retrieveApiKey, (const char* service), {
|
||||
const key = sessionStorage.getItem('yaze_' + UTF8ToString(service) + '_key');
|
||||
if (!key) return null;
|
||||
const len = lengthBytesUTF8(key) + 1;
|
||||
const ptr = _malloc(len);
|
||||
stringToUTF8(key, ptr, len);
|
||||
return ptr;
|
||||
});
|
||||
```
|
||||
|
||||
### CORS Considerations
|
||||
- Gemini API: Should work with browser fetch (Google APIs support CORS)
|
||||
- Ollama: Requires `--cors` flag or proxy server for local instances
|
||||
|
||||
### Files to Create
|
||||
- `src/cli/service/ai/browser_ai_service.h`
|
||||
- `src/cli/service/ai/browser_ai_service.cc`
|
||||
- `src/app/platform/wasm/wasm_browser_storage.h`
|
||||
- `src/app/platform/wasm/wasm_browser_storage.cc`
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Local Storage Persistence
|
||||
|
||||
### Overview
|
||||
Persist user settings, recent files, undo history, and workspace layouts in browser storage.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 6.1 Settings Persistence
|
||||
```cpp
|
||||
// src/app/platform/wasm/wasm_settings.h
|
||||
class WasmSettings {
|
||||
public:
|
||||
// User preferences
|
||||
static void SaveTheme(const std::string& theme);
|
||||
static std::string LoadTheme();
|
||||
|
||||
// Recent files (stored as IndexedDB references)
|
||||
static void AddRecentFile(const std::string& name);
|
||||
static std::vector<std::string> GetRecentFiles();
|
||||
|
||||
// Workspace layouts
|
||||
static void SaveWorkspace(const std::string& name, const std::string& layout_json);
|
||||
static std::string LoadWorkspace(const std::string& name);
|
||||
|
||||
// Undo history (for crash recovery)
|
||||
static void SaveUndoHistory(const std::string& editor, const std::vector<uint8_t>& history);
|
||||
static std::vector<uint8_t> LoadUndoHistory(const std::string& editor);
|
||||
};
|
||||
```
|
||||
|
||||
#### 6.2 Auto-Save & Recovery
|
||||
```cpp
|
||||
// Periodic auto-save
|
||||
class AutoSaveManager {
|
||||
public:
|
||||
void Start(int interval_seconds = 60);
|
||||
void Stop();
|
||||
|
||||
// Called on page unload
|
||||
void EmergencySave();
|
||||
|
||||
// Called on startup
|
||||
bool HasRecoveryData();
|
||||
void RecoverLastSession();
|
||||
void ClearRecoveryData();
|
||||
};
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/app/platform/wasm/wasm_settings.h`
|
||||
- `src/app/platform/wasm/wasm_settings.cc`
|
||||
- `src/app/platform/wasm/wasm_autosave.h`
|
||||
- `src/app/platform/wasm/wasm_autosave.cc`
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Web Workers for Heavy Processing
|
||||
|
||||
### Overview
|
||||
Offload CPU-intensive operations to Web Workers to prevent UI freezing.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 7.1 Background Processing Worker
|
||||
```cpp
|
||||
// Operations to run in Web Worker:
|
||||
// - ROM decompression (LC-LZ2)
|
||||
// - Graphics sheet decoding
|
||||
// - Palette calculations
|
||||
// - Asar assembly compilation
|
||||
|
||||
class WasmWorkerPool {
|
||||
public:
|
||||
using TaskCallback = std::function<void(const std::vector<uint8_t>&)>;
|
||||
|
||||
// Submit work to background thread
|
||||
void SubmitTask(const std::string& type,
|
||||
const std::vector<uint8_t>& input,
|
||||
TaskCallback callback);
|
||||
|
||||
// Wait for all tasks
|
||||
void WaitAll();
|
||||
};
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/app/platform/wasm/wasm_worker_pool.h`
|
||||
- `src/app/platform/wasm/wasm_worker_pool.cc`
|
||||
- `src/web/worker.js`
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Emulator Integration
|
||||
|
||||
### Overview
|
||||
The SNES emulator can run in WASM with WebAudio for sound output.
|
||||
|
||||
### Implementation
|
||||
|
||||
#### 8.1 WebAudio Backend
|
||||
```cpp
|
||||
// src/app/emu/platform/wasm/wasm_audio.h
|
||||
class WasmAudioBackend : public IAudioBackend {
|
||||
public:
|
||||
void Initialize(int sample_rate, int buffer_size) override;
|
||||
void QueueSamples(const int16_t* samples, size_t count) override;
|
||||
void Shutdown() override;
|
||||
};
|
||||
```
|
||||
|
||||
#### 8.2 Canvas Rendering
|
||||
```cpp
|
||||
// Use EM_ASM for direct canvas manipulation if needed
|
||||
// Or use existing ImGui/SDL2 rendering (already WASM compatible)
|
||||
```
|
||||
|
||||
### Files to Create
|
||||
- `src/app/emu/platform/wasm/wasm_audio.h`
|
||||
- `src/app/emu/platform/wasm/wasm_audio.cc`
|
||||
|
||||
---
|
||||
|
||||
## Feature Availability Matrix
|
||||
|
||||
| Feature | Native | WASM | Notes |
|
||||
|---------|--------|------|-------|
|
||||
| ROM Loading | File dialog | File input + IndexedDB | Full support |
|
||||
| ROM Saving | Direct write | Blob download | Full support |
|
||||
| Overworld Editor | Full | Full | No limitations |
|
||||
| Dungeon Editor | Full | Full | No limitations |
|
||||
| Graphics Editor | Full | Full | No limitations |
|
||||
| Palette Editor | Full | Full | No limitations |
|
||||
| Emulator | Full | Full | WebAudio for sound |
|
||||
| AI (Gemini) | HTTP | Browser Fetch | Requires API key |
|
||||
| AI (Ollama) | HTTP | Requires proxy | CORS limitation |
|
||||
| Asar Assembly | Full | In-memory | No file I/O |
|
||||
| Collaboration | gRPC | WebSocket | Different protocol |
|
||||
| Crash Recovery | File-based | IndexedDB | Equivalent |
|
||||
| Offline Mode | N/A | Service Worker | WASM-only feature |
|
||||
|
||||
---
|
||||
|
||||
## CMake Configuration
|
||||
|
||||
### Recommended Preset Updates
|
||||
```json
|
||||
{
|
||||
"name": "wasm-release",
|
||||
"cacheVariables": {
|
||||
"YAZE_BUILD_WASM_PLATFORM": "ON",
|
||||
"YAZE_WASM_ENABLE_WORKERS": "ON",
|
||||
"YAZE_WASM_ENABLE_OFFLINE": "ON",
|
||||
"YAZE_WASM_ENABLE_AI": "ON"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Source Organization
|
||||
```
|
||||
src/app/platform/
|
||||
wasm/
|
||||
wasm_storage.h/.cc
|
||||
wasm_file_dialog.h/.cc
|
||||
wasm_error_handler.h/.cc
|
||||
wasm_loading_manager.h/.cc
|
||||
wasm_settings.h/.cc
|
||||
wasm_autosave.h/.cc
|
||||
wasm_worker_pool.h/.cc
|
||||
wasm_browser_storage.h/.cc
|
||||
src/web/
|
||||
index.html
|
||||
shell.html
|
||||
style.css
|
||||
error_handler.js
|
||||
loading_indicator.js
|
||||
service-worker.js
|
||||
worker.js
|
||||
manifest.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Priority
|
||||
|
||||
### High Priority (Required for MVP)
|
||||
1. File System Layer (Phase 1)
|
||||
2. Error Handling (Phase 2)
|
||||
3. Progressive Loading (Phase 3)
|
||||
|
||||
### Medium Priority (Enhanced Experience)
|
||||
4. Local Storage Persistence (Phase 6)
|
||||
5. Offline Support (Phase 4)
|
||||
|
||||
### Lower Priority (Advanced Features)
|
||||
6. AI Integration (Phase 5)
|
||||
7. Web Workers (Phase 7)
|
||||
8. Emulator Audio (Phase 8)
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] User can load ROM from local file system
|
||||
- [ ] User can save modified ROM to downloads
|
||||
- [ ] Loading progress is visible with cancel option
|
||||
- [ ] Errors display user-friendly messages
|
||||
- [ ] Settings persist across sessions
|
||||
- [ ] App works offline after first load
|
||||
- [ ] AI assistance available via Gemini API
|
||||
- [ ] No UI freezes during heavy operations
|
||||
- [ ] Emulator runs with sound
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Emscripten Documentation](https://emscripten.org/docs/)
|
||||
- [IndexedDB API](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API)
|
||||
- [Service Workers](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API)
|
||||
- [Web Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API)
|
||||
- [WebAudio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API)
|
||||
@@ -0,0 +1,224 @@
|
||||
# WASM Web Features Roadmap
|
||||
|
||||
This document captures planned features for the browser-based yaze editor.
|
||||
|
||||
## Foundation (Completed)
|
||||
|
||||
The following infrastructure is in place:
|
||||
|
||||
- **Phase 1**: File System Layer (WasmStorage, WasmFileDialog) - IndexedDB storage
|
||||
- **Phase 2**: Error Handling (WasmErrorHandler) - Browser UI integration
|
||||
- **Phase 3**: Progressive Loading (WasmLoadingManager) - Cancellable loading with progress
|
||||
- **Phase 4**: Offline Support (service-worker.js, manifest.json) - PWA capabilities
|
||||
- **Phase 5**: AI Integration (BrowserAIService, WasmSecureStorage) - WebLLM ready
|
||||
- **Phase 6**: Local Storage (WasmSettings, WasmAutosave) - Preferences persistence
|
||||
- **Phase 7**: Web Workers (WasmWorkerPool) - Background task processing
|
||||
- **Phase 8**: WebAudio (WasmAudio) - SPC700 audio playback
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Enhanced File Handling
|
||||
|
||||
### 9.1 Drag & Drop ROM Loading
|
||||
- **Status**: Planned
|
||||
- **Priority**: High
|
||||
- **Description**: Enhanced drag-and-drop interface for ROM files
|
||||
- **Features**:
|
||||
- Visual drop zone with hover effects
|
||||
- File type validation before processing
|
||||
- Preview panel showing ROM metadata
|
||||
- Multiple file support (load ROM + patch together)
|
||||
- **Files**: `src/app/platform/wasm/wasm_drop_handler.{h,cc}`, `src/web/drop_zone.{js,css}`
|
||||
|
||||
### 9.2 Export Options
|
||||
- **Status**: Planned
|
||||
- **Priority**: High
|
||||
- **Description**: Export modifications as patches instead of full ROMs
|
||||
- **Features**:
|
||||
- BPS patch generation (standard format)
|
||||
- IPS patch generation (legacy support)
|
||||
- UPS patch generation (alternative)
|
||||
- Patch preview showing changed bytes
|
||||
- Direct download or save to IndexedDB
|
||||
- **Files**: `src/app/platform/wasm/wasm_patch_export.{h,cc}`
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Collaboration
|
||||
|
||||
### 10.1 Real-time Collaboration
|
||||
- **Status**: Planned
|
||||
- **Priority**: High
|
||||
- **Description**: Multi-user editing via WebSocket
|
||||
- **Features**:
|
||||
- Session creation/joining with room codes
|
||||
- User presence indicators (cursors, selections)
|
||||
- Change synchronization via operational transforms
|
||||
- Chat/comments sidebar
|
||||
- Permission levels (owner, editor, viewer)
|
||||
- **Files**: `src/app/platform/wasm/wasm_collaboration.{h,cc}`, `src/web/collaboration_ui.{js,css}`
|
||||
- **Dependencies**: EmscriptenWebSocket (completed)
|
||||
|
||||
### 10.2 ShareLink Generation
|
||||
- **Status**: Future
|
||||
- **Priority**: Medium
|
||||
- **Description**: Create shareable URLs with embedded patches
|
||||
- **Features**:
|
||||
- Base64-encoded diff in URL hash
|
||||
- Short URL generation via service
|
||||
- QR code generation for mobile sharing
|
||||
|
||||
### 10.3 Comment Annotations
|
||||
- **Status**: Future
|
||||
- **Priority**: Low
|
||||
- **Description**: Add notes to map locations/rooms
|
||||
- **Features**:
|
||||
- Pin comments to coordinates
|
||||
- Threaded discussions
|
||||
- Export annotations as JSON
|
||||
|
||||
---
|
||||
|
||||
## Phase 11: Browser-Specific Enhancements
|
||||
|
||||
### 11.1 Keyboard Shortcut Overlay
|
||||
- **Status**: Future
|
||||
- **Priority**: Medium
|
||||
- **Description**: Help panel showing all keyboard shortcuts
|
||||
- **Features**:
|
||||
- Toggle with `?` key
|
||||
- Context-aware (shows relevant shortcuts for current editor)
|
||||
- Searchable
|
||||
|
||||
### 11.2 Touch Support
|
||||
- **Status**: Future
|
||||
- **Priority**: Medium
|
||||
- **Description**: Touch gestures for tablet/mobile browsers
|
||||
- **Features**:
|
||||
- Pinch to zoom
|
||||
- Two-finger pan
|
||||
- Long press for context menu
|
||||
- Touch-friendly toolbar
|
||||
|
||||
### 11.3 Fullscreen Mode
|
||||
- **Status**: Future
|
||||
- **Priority**: Low
|
||||
- **Description**: Dedicated fullscreen API integration
|
||||
- **Features**:
|
||||
- F11 toggle
|
||||
- Auto-hide toolbar in fullscreen
|
||||
- Escape to exit
|
||||
|
||||
### 11.4 Browser Notifications
|
||||
- **Status**: Future
|
||||
- **Priority**: Low
|
||||
- **Description**: Alert when long operations complete
|
||||
- **Features**:
|
||||
- Permission request flow
|
||||
- Build/export completion notifications
|
||||
- Background tab awareness
|
||||
|
||||
---
|
||||
|
||||
## Phase 12: AI Integration Enhancements
|
||||
|
||||
### 12.1 Browser-Local LLM
|
||||
- **Status**: Future
|
||||
- **Priority**: Medium
|
||||
- **Description**: In-browser AI using WebLLM
|
||||
- **Features**:
|
||||
- Model download/caching
|
||||
- Chat interface for ROM hacking questions
|
||||
- Code generation for ASM patches
|
||||
- **Dependencies**: BrowserAIService (completed)
|
||||
|
||||
### 12.2 ROM Analysis Reports
|
||||
- **Status**: Future
|
||||
- **Priority**: Low
|
||||
- **Description**: Generate sharable HTML reports
|
||||
- **Features**:
|
||||
- Modification summary
|
||||
- Changed areas visualization
|
||||
- Exportable as standalone HTML
|
||||
|
||||
---
|
||||
|
||||
## Phase 13: Performance Optimizations
|
||||
|
||||
### 13.1 WebGPU Rendering
|
||||
- **Status**: Future
|
||||
- **Priority**: Medium
|
||||
- **Description**: Modern GPU acceleration
|
||||
- **Features**:
|
||||
- Feature detection with fallback to WebGL
|
||||
- Hardware-accelerated tile rendering
|
||||
- Shader-based effects
|
||||
|
||||
### 13.2 Lazy Tile Loading
|
||||
- **Status**: Future
|
||||
- **Priority**: Medium
|
||||
- **Description**: Load only visible map sections
|
||||
- **Features**:
|
||||
- Virtual scrolling for large maps
|
||||
- Tile cache management
|
||||
- Preload adjacent areas
|
||||
|
||||
---
|
||||
|
||||
## Phase 14: Cloud Features
|
||||
|
||||
### 14.1 Cloud ROM Storage
|
||||
- **Status**: Future
|
||||
- **Priority**: Low
|
||||
- **Description**: Optional cloud sync for projects
|
||||
- **Features**:
|
||||
- User accounts
|
||||
- Project backup/restore
|
||||
- Cross-device sync
|
||||
|
||||
### 14.2 Screenshot Gallery
|
||||
- **Status**: Future
|
||||
- **Priority**: Low
|
||||
- **Description**: Save emulator screenshots
|
||||
- **Features**:
|
||||
- Auto-capture on emulator test
|
||||
- Gallery view in IndexedDB
|
||||
- Share to social media
|
||||
|
||||
---
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Technology Stack
|
||||
- **Storage**: IndexedDB via WasmStorage
|
||||
- **Networking**: EmscriptenWebSocket for real-time features
|
||||
- **Background Processing**: WasmWorkerPool (pthread-based)
|
||||
- **Audio**: WebAudio API via WasmAudio
|
||||
- **UI**: ImGui with HTML overlays for browser-specific elements
|
||||
|
||||
### File Organization
|
||||
```
|
||||
src/app/platform/wasm/
|
||||
├── wasm_storage.{h,cc} # Phase 1 ✓
|
||||
├── wasm_file_dialog.{h,cc} # Phase 1 ✓
|
||||
├── wasm_error_handler.{h,cc} # Phase 2 ✓
|
||||
├── wasm_loading_manager.{h,cc} # Phase 3 ✓
|
||||
├── wasm_settings.{h,cc} # Phase 6 ✓
|
||||
├── wasm_autosave.{h,cc} # Phase 6 ✓
|
||||
├── wasm_browser_storage.{h,cc} # Phase 5 ✓
|
||||
├── wasm_worker_pool.{h,cc} # Phase 7 ✓
|
||||
├── wasm_drop_handler.{h,cc} # Phase 9 (planned)
|
||||
├── wasm_patch_export.{h,cc} # Phase 9 (planned)
|
||||
└── wasm_collaboration.{h,cc} # Phase 10 (planned)
|
||||
|
||||
src/app/emu/platform/wasm/
|
||||
└── wasm_audio.{h,cc} # Phase 8 ✓
|
||||
|
||||
src/web/
|
||||
├── error_handler.{js,css} # Phase 2 ✓
|
||||
├── loading_indicator.{js,css} # Phase 3 ✓
|
||||
├── service-worker.js # Phase 4 ✓
|
||||
├── manifest.json # Phase 4 ✓
|
||||
├── drop_zone.{js,css} # Phase 9 (planned)
|
||||
└── collaboration_ui.{js,css} # Phase 10 (planned)
|
||||
```
|
||||
@@ -0,0 +1,344 @@
|
||||
# WASM Widget Tracking Implementation
|
||||
|
||||
**Date**: 2025-11-24
|
||||
**Author**: Claude (AI Agent)
|
||||
**Status**: Implemented
|
||||
**Related Files**:
|
||||
- `/Users/scawful/Code/yaze/src/app/platform/wasm/wasm_control_api.cc`
|
||||
- `/Users/scawful/Code/yaze/src/app/gui/automation/widget_id_registry.h`
|
||||
- `/Users/scawful/Code/yaze/src/app/gui/automation/widget_measurement.h`
|
||||
- `/Users/scawful/Code/yaze/src/app/controller.cc`
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the implementation of actual ImGui widget bounds tracking for GUI automation in the YAZE WASM build. The system replaces placeholder hardcoded bounds with real-time widget position data from the `WidgetIdRegistry`.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The original `WasmControlApi::GetUIElementTree()` and `GetUIElementBounds()` implementations returned hardcoded placeholder bounds:
|
||||
|
||||
```cpp
|
||||
// OLD: Hardcoded placeholder
|
||||
elem["bounds"] = {{"x", 0}, {"y", 0}, {"width", 100}, {"height", 30}};
|
||||
```
|
||||
|
||||
This prevented accurate GUI automation, as agents and test frameworks couldn't reliably click on or query widget positions.
|
||||
|
||||
## Solution Architecture
|
||||
|
||||
### 1. Existing Infrastructure (Already in Place)
|
||||
|
||||
YAZE already had a comprehensive widget tracking system:
|
||||
|
||||
- **`WidgetIdRegistry`** (`src/app/gui/automation/widget_id_registry.h`): Centralized registry that tracks all ImGui widgets with their bounds, visibility, and state
|
||||
- **`WidgetMeasurement`** (`src/app/gui/automation/widget_measurement.h`): Measures widget dimensions using `ImGui::GetItemRectMin()` and `ImGui::GetItemRectMax()`
|
||||
- **Frame lifecycle hooks**: `BeginFrame()` and `EndFrame()` calls already integrated in `Controller::OnLoad()` (lines 96-98)
|
||||
|
||||
### 2. Integration with WASM Control API
|
||||
|
||||
The implementation connects the WASM API to the existing widget registry:
|
||||
|
||||
#### Updated `GetUIElementTree()`
|
||||
|
||||
**File**: `/Users/scawful/Code/yaze/src/app/platform/wasm/wasm_control_api.cc` (lines 1386-1433)
|
||||
|
||||
```cpp
|
||||
std::string WasmControlApi::GetUIElementTree() {
|
||||
nlohmann::json result;
|
||||
|
||||
if (!IsReady()) {
|
||||
result["error"] = "Control API not initialized";
|
||||
result["elements"] = nlohmann::json::array();
|
||||
return result.dump();
|
||||
}
|
||||
|
||||
// Query the WidgetIdRegistry for all registered widgets
|
||||
auto& registry = gui::WidgetIdRegistry::Instance();
|
||||
const auto& all_widgets = registry.GetAllWidgets();
|
||||
|
||||
nlohmann::json elements = nlohmann::json::array();
|
||||
|
||||
// Convert WidgetInfo to JSON elements
|
||||
for (const auto& [path, info] : all_widgets) {
|
||||
nlohmann::json elem;
|
||||
elem["id"] = info.full_path;
|
||||
elem["type"] = info.type;
|
||||
elem["label"] = info.label;
|
||||
elem["enabled"] = info.enabled;
|
||||
elem["visible"] = info.visible;
|
||||
elem["window"] = info.window_name;
|
||||
|
||||
// Add bounds if available
|
||||
if (info.bounds.valid) {
|
||||
elem["bounds"] = {
|
||||
{"x", info.bounds.min_x},
|
||||
{"y", info.bounds.min_y},
|
||||
{"width", info.bounds.max_x - info.bounds.min_x},
|
||||
{"height", info.bounds.max_y - info.bounds.min_y}
|
||||
};
|
||||
} else {
|
||||
elem["bounds"] = {
|
||||
{"x", 0}, {"y", 0}, {"width", 0}, {"height", 0}
|
||||
};
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
if (!info.description.empty()) {
|
||||
elem["description"] = info.description;
|
||||
}
|
||||
elem["imgui_id"] = static_cast<uint32_t>(info.imgui_id);
|
||||
elem["last_seen_frame"] = info.last_seen_frame;
|
||||
|
||||
elements.push_back(elem);
|
||||
}
|
||||
|
||||
result["elements"] = elements;
|
||||
result["count"] = elements.size();
|
||||
result["source"] = "WidgetIdRegistry";
|
||||
|
||||
return result.dump();
|
||||
}
|
||||
```
|
||||
|
||||
**Changes**:
|
||||
- Removed hardcoded editor-specific element generation
|
||||
- Queries `WidgetIdRegistry::GetAllWidgets()` for real widget data
|
||||
- Returns actual bounds from `info.bounds` if valid
|
||||
- Includes metadata: `imgui_id`, `last_seen_frame`, `description`
|
||||
- Adds `source: "WidgetIdRegistry"` to JSON for debugging
|
||||
|
||||
#### Updated `GetUIElementBounds()`
|
||||
|
||||
**File**: `/Users/scawful/Code/yaze/src/app/platform/wasm/wasm_control_api.cc` (lines ~1435+)
|
||||
|
||||
```cpp
|
||||
std::string WasmControlApi::GetUIElementBounds(const std::string& element_id) {
|
||||
nlohmann::json result;
|
||||
|
||||
if (!IsReady()) {
|
||||
result["error"] = "Control API not initialized";
|
||||
return result.dump();
|
||||
}
|
||||
|
||||
// Query the WidgetIdRegistry for the specific widget
|
||||
auto& registry = gui::WidgetIdRegistry::Instance();
|
||||
const auto* widget_info = registry.GetWidgetInfo(element_id);
|
||||
|
||||
result["id"] = element_id;
|
||||
|
||||
if (widget_info == nullptr) {
|
||||
result["found"] = false;
|
||||
result["error"] = "Element not found: " + element_id;
|
||||
return result.dump();
|
||||
}
|
||||
|
||||
result["found"] = true;
|
||||
result["visible"] = widget_info->visible;
|
||||
result["enabled"] = widget_info->enabled;
|
||||
result["type"] = widget_info->type;
|
||||
result["label"] = widget_info->label;
|
||||
result["window"] = widget_info->window_name;
|
||||
|
||||
// Add bounds if available
|
||||
if (widget_info->bounds.valid) {
|
||||
result["x"] = widget_info->bounds.min_x;
|
||||
result["y"] = widget_info->bounds.min_y;
|
||||
result["width"] = widget_info->bounds.max_x - widget_info->bounds.min_x;
|
||||
result["height"] = widget_info->bounds.max_y - widget_info->bounds.min_y;
|
||||
result["bounds_valid"] = true;
|
||||
} else {
|
||||
result["x"] = 0;
|
||||
result["y"] = 0;
|
||||
result["width"] = 0;
|
||||
result["height"] = 0;
|
||||
result["bounds_valid"] = false;
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
result["imgui_id"] = static_cast<uint32_t>(widget_info->imgui_id);
|
||||
result["last_seen_frame"] = widget_info->last_seen_frame;
|
||||
|
||||
if (!widget_info->description.empty()) {
|
||||
result["description"] = widget_info->description;
|
||||
}
|
||||
|
||||
return result.dump();
|
||||
}
|
||||
```
|
||||
|
||||
**Changes**:
|
||||
- Removed hardcoded element ID pattern matching
|
||||
- Queries `WidgetIdRegistry::GetWidgetInfo(element_id)` for specific widget
|
||||
- Returns `found: false` if widget doesn't exist
|
||||
- Returns actual bounds with `bounds_valid` flag
|
||||
- Includes full widget metadata
|
||||
|
||||
### 3. Frame Lifecycle Integration
|
||||
|
||||
**File**: `/Users/scawful/Code/yaze/src/app/controller.cc` (lines 96-98)
|
||||
|
||||
The widget registry is already integrated into the main render loop:
|
||||
|
||||
```cpp
|
||||
absl::Status Controller::OnLoad() {
|
||||
// ... ImGui::NewFrame() setup ...
|
||||
|
||||
gui::WidgetIdRegistry::Instance().BeginFrame();
|
||||
absl::Status update_status = editor_manager_.Update();
|
||||
gui::WidgetIdRegistry::Instance().EndFrame();
|
||||
|
||||
RETURN_IF_ERROR(update_status);
|
||||
return absl::OkStatus();
|
||||
}
|
||||
```
|
||||
|
||||
**Frame Lifecycle**:
|
||||
1. `BeginFrame()`: Resets `seen_in_current_frame` flag for all widgets
|
||||
2. Widget rendering: Editors register widgets during `editor_manager_.Update()`
|
||||
3. `EndFrame()`: Marks unseen widgets as invisible, prunes stale entries
|
||||
|
||||
### 4. Widget Registration (Future Work)
|
||||
|
||||
**Current State**: Widget registration infrastructure exists but **editors are not yet registering widgets**.
|
||||
|
||||
**Registration Pattern** (to be implemented in editors):
|
||||
|
||||
```cpp
|
||||
// Example: Dungeon Editor registering a card
|
||||
{
|
||||
gui::WidgetIdScope scope("DungeonEditor");
|
||||
|
||||
if (ImGui::Begin("Room Selector##dungeon")) {
|
||||
// Widget now has full path: "DungeonEditor/Room Selector"
|
||||
|
||||
if (ImGui::Button("Load Room")) {
|
||||
// After rendering button, register it
|
||||
gui::WidgetIdRegistry::Instance().RegisterWidget(
|
||||
scope.GetWidgetPath("button", "Load Room"),
|
||||
"button",
|
||||
ImGui::GetItemID(),
|
||||
"Loads the selected room into the editor"
|
||||
);
|
||||
}
|
||||
}
|
||||
ImGui::End();
|
||||
}
|
||||
```
|
||||
|
||||
**Macros Available**:
|
||||
- `YAZE_WIDGET_SCOPE(name)`: RAII scope for hierarchical widget paths
|
||||
- `YAZE_REGISTER_WIDGET(type, name)`: Register widget after rendering
|
||||
- `YAZE_REGISTER_CURRENT_WIDGET(type)`: Auto-extract widget name from ImGui
|
||||
|
||||
## API Usage
|
||||
|
||||
### JavaScript API
|
||||
|
||||
**Get All UI Elements**:
|
||||
```javascript
|
||||
const elements = window.yaze.control.getUIElementTree();
|
||||
console.log(elements);
|
||||
// Output:
|
||||
// {
|
||||
// "elements": [
|
||||
// {
|
||||
// "id": "DungeonEditor/RoomSelector/button:LoadRoom",
|
||||
// "type": "button",
|
||||
// "label": "Load Room",
|
||||
// "visible": true,
|
||||
// "enabled": true,
|
||||
// "window": "DungeonEditor",
|
||||
// "bounds": {"x": 150, "y": 200, "width": 100, "height": 30},
|
||||
// "imgui_id": 12345,
|
||||
// "last_seen_frame": 4567
|
||||
// }
|
||||
// ],
|
||||
// "count": 1,
|
||||
// "source": "WidgetIdRegistry"
|
||||
// }
|
||||
```
|
||||
|
||||
**Get Specific Widget Bounds**:
|
||||
```javascript
|
||||
const bounds = window.yaze.control.getUIElementBounds("DungeonEditor/RoomSelector/button:LoadRoom");
|
||||
console.log(bounds);
|
||||
// Output:
|
||||
// {
|
||||
// "id": "DungeonEditor/RoomSelector/button:LoadRoom",
|
||||
// "found": true,
|
||||
// "visible": true,
|
||||
// "enabled": true,
|
||||
// "type": "button",
|
||||
// "label": "Load Room",
|
||||
// "window": "DungeonEditor",
|
||||
// "x": 150,
|
||||
// "y": 200,
|
||||
// "width": 100,
|
||||
// "height": 30,
|
||||
// "bounds_valid": true,
|
||||
// "imgui_id": 12345,
|
||||
// "last_seen_frame": 4567
|
||||
// }
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
1. **Memory**: `WidgetIdRegistry` stores widget metadata in `std::unordered_map`, which grows with UI complexity. Stale widgets are pruned after 600 frames of inactivity.
|
||||
|
||||
2. **CPU Overhead**:
|
||||
- `BeginFrame()`: O(n) iteration to reset flags (n = number of widgets)
|
||||
- Widget registration: O(1) hash map lookup/insert
|
||||
- `EndFrame()`: O(n) iteration for pruning stale entries
|
||||
|
||||
3. **Optimization**: Widget measurement can be disabled globally:
|
||||
```cpp
|
||||
gui::WidgetMeasurement::Instance().SetEnabled(false);
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
**Manual Test (WASM Build)**:
|
||||
```bash
|
||||
# Build WASM
|
||||
./scripts/build-wasm.sh
|
||||
|
||||
# Serve locally
|
||||
cd build-wasm
|
||||
python3 -m http.server 8080
|
||||
|
||||
# Open browser console
|
||||
window.yaze.control.getUIElementTree();
|
||||
```
|
||||
|
||||
**Expected Behavior**:
|
||||
- Initially, `elements` array will be empty (no widgets registered yet)
|
||||
- After editors implement registration, widgets will appear with real bounds
|
||||
- `bounds_valid: false` for widgets not yet rendered in current frame
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Add widget registration to editors**:
|
||||
- `DungeonEditorV2`: Register room tabs, cards, buttons
|
||||
- `OverworldEditor`: Register canvas, tile selectors, property panels
|
||||
- `GraphicsEditor`: Register graphics sheets, palette pickers
|
||||
|
||||
2. **Add registration helpers**:
|
||||
- Create `AgentUI::RegisterButton()`, `AgentUI::RegisterCard()` wrappers
|
||||
- Auto-register common widget patterns (cards with visibility flags)
|
||||
|
||||
3. **Extend API**:
|
||||
- `FindWidgetsByPattern(pattern)`: Search widgets by regex
|
||||
- `ClickWidget(element_id)`: Simulate click via automation API
|
||||
|
||||
## References
|
||||
|
||||
- Widget ID Registry: `/Users/scawful/Code/yaze/src/app/gui/automation/widget_id_registry.h`
|
||||
- Widget Measurement: `/Users/scawful/Code/yaze/src/app/gui/automation/widget_measurement.h`
|
||||
- WASM Control API: `/Users/scawful/Code/yaze/src/app/platform/wasm/wasm_control_api.h`
|
||||
- Controller Integration: `/Users/scawful/Code/yaze/src/app/controller.cc` (lines 96-98)
|
||||
|
||||
## Revision History
|
||||
|
||||
| Date | Author | Changes |
|
||||
|------------|--------|--------------------------------------------|
|
||||
| 2025-11-24 | Claude | Initial implementation and documentation |
|
||||
313
docs/internal/agents/canvas-slimming-plan.md
Normal file
313
docs/internal/agents/canvas-slimming-plan.md
Normal file
@@ -0,0 +1,313 @@
|
||||
## Canvas Slimming Plan
|
||||
|
||||
Owner: imgui-frontend-engineer
|
||||
Status: In Progress (Phases 1-5 Complete for Dungeon + Overworld; Phase 6 Pending)
|
||||
Scope: `src/app/gui/canvas/` + editor call-sites
|
||||
Goal: Reduce `gui::Canvas` bloat, align lifecycle with ImGui-style, and enable safer per-frame usage without hidden state mutation.
|
||||
|
||||
### Current Problems
|
||||
- Too many constructors and in-class defaults; state is scattered across legacy fields (`custom_step_`, `global_scale_`, enable_* duplicates).
|
||||
- Per-frame options are applied via mutations instead of “Begin arguments,” diverging from ImGui patterns.
|
||||
- Heavy optional subsystems (bpp dialogs, palette editor, modals, automation) live on the core type.
|
||||
- Helpers rely on implicit internal state (`draw_list_`, `canvas_p0_`) instead of a passed-in context.
|
||||
|
||||
### Target Shape
|
||||
- **Persistent state:** `CanvasConfig`, `CanvasSelection`, stable IDs, scrolling, rom/game_data pointers.
|
||||
- **Transient per-frame:** `CanvasRuntime` (draw_list, geometry, hover flags, resolved grid step/scale). Constructed in `Begin`, discarded in `End`.
|
||||
- **Options:** `CanvasFrameOptions` (and `BitmapPreviewOptions`) are the “Begin args.” Prefer per-frame opts over setters.
|
||||
- **Helpers:** Accept a `CanvasRuntime&` (or `CanvasContext` value) rather than reading globals; keep wrappers for compatibility during migration.
|
||||
- **Optional features:** Move bpp/palette/modals/automation behind lightweight extension pointers or a separate host struct.
|
||||
- **Construction:** One ctor + `Init(CanvasConfig)` (or default). Deprecate overloads; keep temp forwarding for compatibility.
|
||||
|
||||
### Phased Work
|
||||
1) **Runtime extraction (core)** [COMPLETE]
|
||||
- [x] Add `CanvasRuntime` (geometry, draw_list, hover, resolved grid step/scale, content size).
|
||||
- [x] Add `CanvasRuntime`-based helpers: `DrawBitmap`, `DrawBitmapPreview`, `RenderPreviewPanel`.
|
||||
- [x] Make `Begin/End` create and tear down runtime; route grid/overlay/popups through it.
|
||||
- Implemented `BeginCanvas(Canvas&, CanvasFrameOptions)` returning `CanvasRuntime`
|
||||
- Implemented `EndCanvas(Canvas&, CanvasRuntime&, CanvasFrameOptions)` handling grid/overlay/popups
|
||||
- [ ] Deprecate internal legacy mirrors (`custom_step_`, `global_scale_`, enable_* duplicates); keep sync shims temporarily.
|
||||
|
||||
2) **API narrowing** [COMPLETE]
|
||||
- [x] Collapse ctors to a single default + `Init(config)`.
|
||||
- Added `Init(const CanvasConfig& config)` and `Init(const std::string& id, ImVec2 canvas_size)` methods
|
||||
- Legacy constructors marked with `[[deprecated("Use default ctor + Init() instead")]]`
|
||||
- [x] Mark setters that mutate per-frame state (`SetCanvasSize`, `SetGridSize`, etc.) as "compat; prefer frame options."
|
||||
- Added COMPAT comments to `SetGridSize`, `SetCustomGridStep`, `SetCanvasSize`, `SetGlobalScale`, `set_global_scale`
|
||||
- [x] Add `BeginInTable(label, CanvasFrameOptions)` that wraps child sizing and returns a runtime-aware frame.
|
||||
- Implemented `BeginInTable(label, CanvasFrameOptions)` returning `CanvasRuntime`
|
||||
- Implemented `EndInTable(CanvasRuntime&, CanvasFrameOptions)` handling grid/overlay/popups
|
||||
|
||||
3) **Helper refactor** [COMPLETE]
|
||||
- [x] Change helper signatures to accept `CanvasRuntime&`: `DrawBitmap`, `DrawBitmapPreview`, `RenderPreviewPanel`.
|
||||
- [x] Refactor remaining helpers: `DrawTilemapPainter`, `DrawSelectRect`, `DrawTileSelector`.
|
||||
- All three now have stateless `CanvasRuntime`-based implementations
|
||||
- Member functions delegate to stateless helpers via `BuildCurrentRuntime()`
|
||||
- [x] Keep thin wrappers that fetch the current runtime for legacy calls.
|
||||
- Added `Canvas::BuildCurrentRuntime()` private helper
|
||||
- [x] Added `DrawBitmap(CanvasRuntime&, Bitmap&, BitmapDrawOpts)` overload for options-based drawing.
|
||||
|
||||
4) **Optional modules split** [COMPLETE]
|
||||
- [x] Move bpp dialogs, palette editor, modals, automation into an extension struct (`CanvasExtensions`) held by unique_ptr on demand.
|
||||
- Created `canvas_extensions.h` with `CanvasExtensions` struct containing: `bpp_format_ui`, `bpp_conversion_dialog`, `bpp_comparison_tool`, `modals`, `palette_editor`, `automation_api`
|
||||
- Created `canvas_extensions.cc` with lazy initialization helpers: `InitializeModals()`, `InitializePaletteEditor()`, `InitializeBppUI()`, `InitializeAutomation()`
|
||||
- Canvas now uses single `std::unique_ptr<CanvasExtensions> extensions_` with `EnsureExtensions()` lazy accessor
|
||||
- [x] Core `Canvas` remains lean even when extensions are absent.
|
||||
- Extensions only allocated on first use of optional features
|
||||
- All Show* methods and GetAutomationAPI() delegate to EnsureExtensions()
|
||||
|
||||
5) **Call-site migration** [COMPLETE]
|
||||
- [x] Update low-risk previews first: graphics thumbnails (`sheet_browser_panel`), small previews (`object_editor_panel`, `link_sprite_panel`).
|
||||
- [x] Medium: screen/inventory canvases, sprite/tileset selectors.
|
||||
- `DrawInventoryMenuEditor`: migrated to `BeginCanvas/EndCanvas` + stateless `DrawBitmap`
|
||||
- `DrawDungeonMapsRoomGfx`: migrated tilesheet and current_tile canvases to new pattern
|
||||
- [x] High/complex - Dungeon Editor: `DungeonCanvasViewer::DrawDungeonCanvas` migrated
|
||||
- Uses `BeginCanvas(canvas_, frame_opts)` / `EndCanvas(canvas_, canvas_rt, frame_opts)` pattern
|
||||
- Entity rendering functions updated to accept `CanvasRuntime&` parameter
|
||||
- All `canvas_.DrawRect/DrawText` calls replaced with `gui::DrawRect(rt, ...)` / `gui::DrawText(rt, ...)`
|
||||
- Grid visibility controlled via `frame_opts.draw_grid = show_grid_`
|
||||
- [x] High/complex - Overworld Editor: `DrawOverworldCanvas` and secondary canvases migrated
|
||||
- Main canvas uses `BeginCanvas/EndCanvas` with `CanvasFrameOptions`
|
||||
- Entity renderer (`OverworldEntityRenderer`) updated with `CanvasRuntime`-based methods
|
||||
- Scroll bounds implemented via `ClampScroll()` helper in `HandleOverworldPan()`
|
||||
- Secondary canvases migrated: `scratch_canvas_`, `current_gfx_canvas_`, `graphics_bin_canvas_`
|
||||
- Zoom support deferred (documented in code); positions not yet scaled with global_scale
|
||||
|
||||
6) **Cleanup & deprecations** [PENDING]
|
||||
- [ ] Remove deprecated ctors/fields after call-sites are migrated.
|
||||
- [ ] Document "per-frame opts" pattern and add brief usage examples in `canvas.h`.
|
||||
- [ ] Remove legacy `BeginCanvas(Canvas&, ImVec2)` overload (only used by `message_editor.cc`)
|
||||
- [ ] Audit remaining `AlwaysVerticalScrollbar` usage in `GraphicsBinCanvasPipeline`/`BitmapCanvasPipeline`
|
||||
- [ ] Remove `custom_step_`, `global_scale_` duplicates once all editors use `CanvasFrameOptions`
|
||||
- [ ] Consider making `CanvasRuntime` a first-class return from all `Begin` variants
|
||||
|
||||
### Where to Start (low risk, high leverage)
|
||||
- **COMPLETED (Phase 1 - Low Risk):** Migrated low-risk graphics thumbnails and previews:
|
||||
- `src/app/editor/graphics/sheet_browser_panel.cc` (Thumbnails use `DrawBitmapPreview(rt, ...)` via `BeginCanvas`)
|
||||
- `src/app/editor/dungeon/panels/object_editor_panel.cc` (Static editor preview uses `RenderPreviewPanel(rt, ...)`)
|
||||
- `src/app/editor/graphics/link_sprite_panel.cc` (Preview canvas uses `DrawBitmap(rt, ...)` with manual begin/end)
|
||||
|
||||
- **COMPLETED (Phase 2 - Medium Risk):** Migrated screen editor canvases:
|
||||
- `src/app/editor/graphics/screen_editor.cc`:
|
||||
- `DrawInventoryMenuEditor`: Uses `BeginCanvas/EndCanvas` + stateless `gui::DrawBitmap(rt, ...)`
|
||||
- `DrawDungeonMapsRoomGfx`: Tilesheet canvas and current tile canvas migrated to `BeginCanvas/EndCanvas` pattern with `CanvasFrameOptions` for grid step configuration
|
||||
|
||||
### Editor-Specific Strategies (for later phases)
|
||||
|
||||
- **Overworld Editor (high complexity)** [MIGRATED]
|
||||
- Surfaces: main overworld canvas (tile painting, selection, multi-layer), scratch space, tile16 selector, property info grids.
|
||||
- **Completed Migration:**
|
||||
- `DrawOverworldCanvas`: Uses `BeginCanvas(ow_map_canvas_, frame_opts)` / `EndCanvas()` pattern
|
||||
- `OverworldEntityRenderer`: Added `CanvasRuntime`-based methods (`DrawEntrances(rt, world)`, `DrawExits(rt, world)`, etc.)
|
||||
- `HandleOverworldPan`: Now clamps scrolling via `gui::ClampScroll()` to prevent scrolling outside map bounds
|
||||
- `CenterOverworldView`: Properly centers on current map with clamped scroll
|
||||
- `DrawScratchSpace`: Migrated to `BeginCanvas/EndCanvas` pattern
|
||||
- `DrawAreaGraphics`: Migrated `current_gfx_canvas_` to new pattern
|
||||
- `DrawTile8Selector`: Migrated `graphics_bin_canvas_` to new pattern
|
||||
- **Key Implementation Details:**
|
||||
- Context menu setup happens BEFORE `BeginCanvas` via `map_properties_system_->SetupCanvasContextMenu()`
|
||||
- Entity drag/drop uses `canvas_rt.scale` and `canvas_rt.canvas_p0` from runtime
|
||||
- Hover detection uses `canvas_rt.hovered` instead of `ow_map_canvas_.IsMouseHovering()`
|
||||
- `IsMouseHoveringOverEntity(entity, rt)` overload added for runtime-based entity detection
|
||||
- **Zoom Support:** [IMPLEMENTED]
|
||||
- `DrawOverworldMaps()` now applies `global_scale()` to both bitmap positions and scale
|
||||
- Placeholder rectangles for unloaded maps also scaled correctly
|
||||
- Entity rendering already uses `rt.scale` via stateless helpers - alignment works automatically
|
||||
- **Testing Focus:**
|
||||
- [x] Pan respects canvas bounds (can't scroll outside map)
|
||||
- [x] Entity hover detection works
|
||||
- [x] Entity drag/drop positioning correct
|
||||
- [x] Context menu opens in correct mode
|
||||
- [x] Zoom scales bitmaps and positions correctly
|
||||
|
||||
- **Dungeon Editor** [MIGRATED]
|
||||
- Surfaces: room graphics viewer, object selector preview, integrated editor panels, room canvases with palettes/blocks.
|
||||
- **Completed Migration:**
|
||||
- `DungeonCanvasViewer::DrawDungeonCanvas`: Uses `BeginCanvas/EndCanvas` with `CanvasFrameOptions`
|
||||
- Entity rendering functions (`RenderSprites`, `RenderPotItems`, `DrawObjectPositionOutlines`) accept `const gui::CanvasRuntime&`
|
||||
- All drawing calls use stateless helpers: `gui::DrawRect(rt, ...)`, `gui::DrawText(rt, ...)`
|
||||
- `DungeonObjectSelector`: Already used `CanvasFrame` pattern (no changes needed)
|
||||
- `ObjectEditorPanel`: Already used `BeginCanvas/EndCanvas` pattern (no changes needed)
|
||||
- **Key Lessons:**
|
||||
- Context menu setup (`ClearContextMenuItems`, `AddContextMenuItem`) must happen BEFORE `BeginCanvas`
|
||||
- The `DungeonObjectInteraction` class still uses `canvas_` pointer internally - this works because canvas state is set up by `BeginCanvas`
|
||||
- Debug overlay windows (`ImGui::Begin/End`) work fine inside the canvas frame since they're separate ImGui windows
|
||||
- Grid visibility is now controlled via `frame_opts.draw_grid` instead of manual `if (show_grid_) { canvas_.DrawGrid(); }`
|
||||
|
||||
- **Screen/Inventory Editors** [MIGRATED]
|
||||
- `DrawInventoryMenuEditor` and `DrawDungeonMapsRoomGfx` now use `BeginCanvas/EndCanvas` with `CanvasFrameOptions`.
|
||||
- Stateless `gui::DrawBitmap(rt, ...)` and `gui::DrawTileSelector(rt, ...)` used throughout.
|
||||
- Grid step configured via `CanvasFrameOptions` (32 for screen, 16/32 for tilesheet).
|
||||
|
||||
- **Graphics/Pixels Panels**
|
||||
- Low risk; continue to replace manual `draw_list` usage with `DrawBitmapPreview` and runtime-aware helpers. Ensure `ensure_texture=true` for arena-backed bitmaps.
|
||||
|
||||
- **Tile16Editor** [MIGRATED]
|
||||
- Three canvases migrated from `DrawBackground()/DrawContextMenu()/DrawGrid()/DrawOverlay()` to `BeginCanvas/EndCanvas`:
|
||||
- `blockset_canvas_`: Two usages in `UpdateBlockset()` and `DrawContent()` now use `CanvasFrameOptions`
|
||||
- `tile8_source_canvas_`: Tile8 source selector now uses `BeginCanvas/EndCanvas` with grid step 32.0f
|
||||
- `tile16_edit_canvas_`: Tile16 edit canvas now uses `BeginCanvas/EndCanvas` with grid step 8.0f
|
||||
- Tile selection and painting logic preserved; only frame management changed
|
||||
|
||||
- **Automation/Testing Surfaces**
|
||||
- Leave automation API untouched until core migration is stable. When ready, have it consume `CanvasRuntime` so tests don't depend on hidden members.
|
||||
|
||||
### Testing Focus per Editor
|
||||
- Overworld [VALIDATED]: scroll bounds ✓, entity hover ✓, entity drag/drop ✓, context menu ✓, tile painting ✓, zoom ✓.
|
||||
- Dungeon [VALIDATED]: grid alignment ✓, object interaction ✓, entity overlays ✓, context menu ✓, layer visibility ✓.
|
||||
- Screen/Inventory: zoom buttons, grid overlay alignment, selectors.
|
||||
- Graphics panels: texture creation on demand, grid overlay spacing, tooltip/selection hits.
|
||||
|
||||
### Context Menu & Popup Cleanup
|
||||
- Problem: Caller menus stack atop generic defaults; composition is implicit. Popups hang off internal state instead of the menu contract.
|
||||
- Target shape:
|
||||
- `CanvasMenuHost` (or similar) owns menu items for the frame. Generic items are registered explicitly via `RegisterDefaultCanvasMenu(host, runtime, config)`.
|
||||
- Callers add items with `AddItem(label, shortcut, callback, enabled_fn)` or structured entries (id, tooltip, icon).
|
||||
- Rendering is single-pass per frame: `RenderContextMenu(host, runtime, config)`. No hidden additions elsewhere.
|
||||
- Persistent popups are tied to menu actions and rendered via the same host (or a `PopupRegistry` owned by it), gated by `CanvasFrameOptions.render_popups`.
|
||||
- Migration plan:
|
||||
1) Extract menu item POD (id, label, shortcut text, optional enable/predicate, callback).
|
||||
2) Refactor `DrawContextMenu` to:
|
||||
- Create/clear a host each frame
|
||||
- Optionally call `RegisterDefaultCanvasMenu`
|
||||
- Provide a caller hook to register custom items
|
||||
- Render once.
|
||||
3) Deprecate ad-hoc menu additions inside draw helpers; require explicit registration.
|
||||
4) Keep legacy editor menus working by shimming their registrations into the host; remove implicit defaults once call-sites are migrated.
|
||||
5) Popups: route open/close through the host/registry; render via a single `RenderPersistentPopups(host)` invoked from `End` when `render_popups` is true.
|
||||
- Usage pattern for callers (future API):
|
||||
- `auto& host = canvas.MenuHost();`
|
||||
- `host.Clear();`
|
||||
- `RegisterDefaultCanvasMenu(host, runtime, config); // optional`
|
||||
- `host.AddItem("Custom", "Ctrl+K", []{ ... });`
|
||||
- `RenderContextMenu(host, runtime, config);`
|
||||
|
||||
### Critical Insights (Lessons Learned)
|
||||
|
||||
**Child Window & Scrollbar Behavior:**
|
||||
- The canvas has its own internal scrolling mechanism via `scrolling_` state and pan handling in `DrawBackground()`.
|
||||
- Wrapping in `ImGui::BeginChild()` with scrollbars is **redundant** and causes visual issues.
|
||||
- `CanvasFrameOptions.use_child_window` defaults to `false` to match legacy `DrawBackground()` behavior.
|
||||
- Only use `use_child_window = true` when you explicitly need a scrollable container (rare).
|
||||
- All `BeginChild` calls in canvas code now use `ImGuiWindowFlags_NoScrollbar` by default.
|
||||
|
||||
**Context Menu Ordering:**
|
||||
- Context menu setup (`ClearContextMenuItems`, `AddContextMenuItem`) must happen BEFORE `BeginCanvas`.
|
||||
- The menu items are state on the canvas object, not per-frame data.
|
||||
- `BeginCanvas` calls `DrawBackground` + `DrawContextMenu`, so items must be registered first.
|
||||
|
||||
**Interaction Systems:**
|
||||
- Systems like `DungeonObjectInteraction` that hold a `canvas_` pointer still work because canvas internal state is valid after `BeginCanvas`.
|
||||
- The runtime provides read-only geometry; interaction systems can still read canvas state directly.
|
||||
|
||||
**Scroll Bounds (Overworld Lesson):**
|
||||
- Large canvases (4096x4096) need explicit scroll clamping to prevent users from panning beyond content bounds.
|
||||
- Use `ClampScroll(scroll, content_px * scale, viewport_px)` after computing new scroll position.
|
||||
- The overworld's `HandleOverworldPan()` now demonstrates this pattern with `gui::ClampScroll()`.
|
||||
|
||||
**Multi-Bitmap Canvases (Overworld Lesson):**
|
||||
- When a canvas renders multiple bitmaps (e.g., 64 map tiles), positions must be scaled with `global_scale_` for zoom to work.
|
||||
- The overworld's zoom is deferred because bitmap positions are not yet scaled (see TODO in `DrawOverworldMaps()`).
|
||||
- Fix approach: `map_x = static_cast<int>(xx * kOverworldMapSize * scale)` and pass `scale` to `DrawBitmap`.
|
||||
|
||||
**Entity Renderer Refactoring:**
|
||||
- Separate entity renderers (like `OverworldEntityRenderer`) should accept `const gui::CanvasRuntime&` for stateless rendering.
|
||||
- Legacy methods can be kept as thin wrappers that build runtime from canvas state.
|
||||
- The `IsMouseHoveringOverEntity(entity, rt)` overload demonstrates runtime-based hit testing.
|
||||
|
||||
### Design Principles to Follow
|
||||
- ImGui-like: options-as-arguments at `Begin`, minimal persistent mutation, small POD contexts for draw helpers.
|
||||
- Keep the core type thin; optional features live in extensions, not the base.
|
||||
- Maintain compatibility shims temporarily, but mark them and plan removal once editors are migrated.
|
||||
- **Avoid child window wrappers** unless truly needed for scrollable regions.
|
||||
|
||||
### Quick Reference for Future Agents
|
||||
- Core files to touch: `src/app/gui/canvas/canvas.{h,cc}`, `canvas_extensions.{h,cc}`, `canvas_menu.{h,cc}`, `canvas_context_menu.{h,cc}`, `canvas_menu_builder.{h,cc}`, `canvas_utils.{h,cc}`. Common call-sites: `screen_editor.cc`, `link_sprite_panel.cc`, `sheet_browser_panel.cc`, `object_editor_panel.cc`, `dungeon_object_selector.cc`, overworld/dungeon editors.
|
||||
- Avoid legacy patterns: direct `draw_list()` math in callers, `custom_step_`/`global_scale_` duplicates, scattered `DrawBackground/DrawGrid/DrawOverlay` chains, implicit menu stacking in `DrawContextMenu`.
|
||||
- Naming/API standardization:
|
||||
- Per-frame context: `CanvasRuntime`; pass it to helpers.
|
||||
- Options at begin: `CanvasFrameOptions` (via `BeginCanvas/EndCanvas` or `CanvasFrame`), not mutating setters (setters are compat-only).
|
||||
- Menu host: use `CanvasMenuAction` / `CanvasMenuActionHost` (avoid `CanvasMenuItem` collision).
|
||||
- **Implemented stateless helpers:**
|
||||
- `DrawBitmap(rt, bitmap, ...)` - multiple overloads including `BitmapDrawOpts`
|
||||
- `DrawBitmapPreview(rt, bitmap, BitmapPreviewOptions)`
|
||||
- `RenderPreviewPanel(rt, bitmap, PreviewPanelOpts)`
|
||||
- `DrawTilemapPainter(rt, tilemap, current_tile, out_drawn_pos)` - returns drawn position via output param
|
||||
- `DrawTileSelector(rt, size, size_y, out_selected_pos)` - returns selection via output param
|
||||
- `DrawSelectRect(rt, current_map, tile_size, scale, CanvasSelection&)` - updates selection struct
|
||||
- `DrawRect(rt, x, y, w, h, color)` - draws filled rectangle (added for entity overlays)
|
||||
- `DrawText(rt, text, x, y)` - draws text at position (added for entity labels)
|
||||
- `DrawOutline(rt, x, y, w, h, color)` - draws outline rectangle
|
||||
- **Frame management:**
|
||||
- `BeginCanvas(canvas, CanvasFrameOptions)` → returns `CanvasRuntime`
|
||||
- `EndCanvas(canvas, runtime, CanvasFrameOptions)` → draws grid/overlay/popups based on options
|
||||
- `BeginInTable(label, CanvasFrameOptions)` → table-aware begin returning `CanvasRuntime`
|
||||
- `EndInTable(runtime, CanvasFrameOptions)` → table-aware end with grid/overlay/popups
|
||||
- **CanvasFrameOptions fields:**
|
||||
- `canvas_size` - size of canvas content (0,0 = auto from config)
|
||||
- `draw_context_menu` - whether to call DrawContextMenu (default true)
|
||||
- `draw_grid` - whether to draw grid overlay (default true)
|
||||
- `grid_step` - optional grid step override
|
||||
- `draw_overlay` - whether to draw selection overlay (default true)
|
||||
- `render_popups` - whether to render persistent popups (default true)
|
||||
- `use_child_window` - wrap in ImGui::BeginChild (default **false** - important!)
|
||||
- `show_scrollbar` - show scrollbar when use_child_window is true (default false)
|
||||
- **Initialization (Phase 2):**
|
||||
- `Canvas()` → default constructor (preferred)
|
||||
- `Init(const CanvasConfig& config)` → post-construction initialization with full config
|
||||
- `Init(const std::string& id, ImVec2 canvas_size)` → simple initialization
|
||||
- Legacy constructors deprecated with `[[deprecated]]` attribute
|
||||
- **Extensions (Phase 4):**
|
||||
- `CanvasExtensions` struct holds optional modules: bpp_format_ui, bpp_conversion_dialog, bpp_comparison_tool, modals, palette_editor, automation_api
|
||||
- `EnsureExtensions()` → lazy accessor (private, creates extensions on first use)
|
||||
- Extensions only allocated when Show* or GetAutomationAPI() methods are called
|
||||
- **Legacy delegation:** Member functions delegate to stateless helpers via `Canvas::BuildCurrentRuntime()`
|
||||
- Context menu flow: host.Clear → optional `RegisterDefaultCanvasMenu(host, rt, cfg)` → caller adds items → `RenderContextMenu(host, rt, cfg)` once per frame.
|
||||
- Migration checklist (per call-site):
|
||||
1) Replace manual DrawBackground/DrawGrid/DrawOverlay/popup sequences with `CanvasFrame` or `BeginCanvas/EndCanvas` using `CanvasFrameOptions`.
|
||||
2) Replace `draw_list()/zero_point()` math with `AddImageAt`/`AddRectFilledAt`/`AddTextAt` or overlay helpers that take `CanvasRuntime`.
|
||||
3) For tile selection, use `TileIndexAt` with grid_step from options/runtime.
|
||||
4) For previews/selectors, use `RenderPreviewPanel` / `RenderSelectorPanel` (`ensure_texture=true` for arena bitmaps).
|
||||
5) For context menus, switch to a `CanvasMenuActionHost` + explicit render pass.
|
||||
- Deprecations to remove after migration: `custom_step_`, `global_scale_` duplicates, legacy `enable_*` mirrors, direct `draw_list_` access in callers.
|
||||
- Testing hints: pure helpers for tests (`TileIndexAt`, `ComputeZoomToFit`, `ClampScroll`). Manual checks per editor: grid alignment, tile hit correctness, zoom/fit, context menu actions, popup render, texture creation (`ensure_texture`).
|
||||
- Risk order: low (previews/thumbnails) → medium (selectors, inventory/screen) → high (overworld main, dungeon main). Start low, validate patterns, then proceed.
|
||||
- **Scroll & Zoom Helpers (Phase 0 - Infrastructure):**
|
||||
- `ClampScroll(scroll, content_px, canvas_px)` → clamps scroll to valid bounds `[-max_scroll, 0]`
|
||||
- `ComputeZoomToFit(content_px, canvas_px, padding_px)` → returns `ZoomToFitResult{scale, scroll}` to fit content
|
||||
- `IsMouseHoveringOverEntity(entity, rt)` → runtime-based entity hover detection for overworld
|
||||
|
||||
### Future Feature Ideas (for follow-on work)
|
||||
|
||||
Interaction & UX
|
||||
- Smooth zoom/pan (scroll + modifiers), double-click zoom-to-fit, snap-to-grid toggle.
|
||||
- Rulers/guides aligned to `grid_step`, draggable guides, and a measure tool that reports delta in px and tiles.
|
||||
- Hover/tooltips: configurable hover info (tile id, palette, coordinates) and/or a status strip.
|
||||
- Keyboard navigation: arrow-key tile navigation, PgUp/PgDn for layer changes, shortcut-able grid presets (8/16/32/64).
|
||||
|
||||
Drawing & overlays
|
||||
- Layer-aware overlays: separate channels for selection/hover/warnings with theme-driven colors.
|
||||
- Mask/visibility controls: per-layer visibility toggles and opacity sliders for multi-layer editing.
|
||||
- Batch highlight API: accept a span of tile ids/positions and render combined overlays to reduce draw calls.
|
||||
|
||||
Selection & tools
|
||||
- Lasso/rect modes with additive/subtractive (Shift/Ctrl) semantics and a visible mode indicator.
|
||||
- Clamp-to-region selection for maps; expose a “clamp to region” flag in options/runtime.
|
||||
- Quick selection actions: context submenu for fill/eyedropper/replace palette/duplicate to scratch.
|
||||
|
||||
Panels & presets
|
||||
- Preset grid/scale sets (“Pixel”, “Tile16”, “Map”) that set grid_step + default zoom together.
|
||||
- Per-canvas profiles to store/recall `CanvasConfig` + view (scroll/scale) per editor.
|
||||
|
||||
Performance & rendering
|
||||
- Virtualized rendering: auto skip off-screen tiles/bitmaps; opt-in culling flag for large maps.
|
||||
- Texture readiness indicator: badge/text when a bitmap lacks a texture; one-click “ensure texture.”
|
||||
- Draw-call diagnostics: small overlay showing batch count and last-frame time.
|
||||
|
||||
Automation & testing
|
||||
- Deterministic snapshot API: export runtime (hover tile, selection rect, grid step, scale, scroll) for tests/automation.
|
||||
- Scriptable macro hooks: tiny API to run canvas actions (zoom, pan, select, draw tile) for recorded scripts.
|
||||
|
||||
Accessibility & discoverability
|
||||
- Shortcut cheatsheet popover for canvas actions (zoom, grid toggle, fit, selection modes).
|
||||
- High-contrast overlays and grids; configurable outline thickness.
|
||||
@@ -1,381 +0,0 @@
|
||||
# Claude-Gemini Collaboration Framework
|
||||
|
||||
**Status**: ACTIVE
|
||||
**Mission**: Accelerate yaze release through strategic Claude-Gemini collaboration
|
||||
**Established**: 2025-11-20
|
||||
**Coordinator**: CLAUDE_GEMINI_LEAD (Joint Task Force)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document defines how Claude and Gemini agents work together to ship a stable yaze release ASAP.
|
||||
Each team has distinct strengths - by playing to those strengths and maintaining friendly rivalry,
|
||||
we maximize velocity while minimizing regressions.
|
||||
|
||||
**Current Priority**: Fix remaining CI failures → Ship release
|
||||
|
||||
---
|
||||
|
||||
## Team Structure
|
||||
|
||||
### Claude Team (Architecture & Platform Specialists)
|
||||
|
||||
**Core Competencies**:
|
||||
- Complex C++ compilation errors
|
||||
- Multi-platform build system debugging (CMake, linker, compiler flags)
|
||||
- Code architecture and refactoring
|
||||
- Deep codebase understanding
|
||||
- Symbol resolution and ODR violations
|
||||
- Graphics system and ROM format logic
|
||||
|
||||
**Active Agents**:
|
||||
- **CLAUDE_AIINF**: AI infrastructure, build systems, gRPC, HTTP APIs
|
||||
- **CLAUDE_CORE**: UI/UX, editor systems, ImGui integration
|
||||
- **CLAUDE_DOCS**: Documentation, guides, onboarding content
|
||||
- **CLAUDE_TEST_COORD**: Testing infrastructure and strategy
|
||||
- **CLAUDE_RELEASE_COORD**: Release management, CI coordination
|
||||
- **CLAUDE_GEMINI_LEAD**: Cross-team coordination (this agent)
|
||||
|
||||
**Typical Tasks**:
|
||||
- Platform-specific compilation failures
|
||||
- Linker errors and missing symbols
|
||||
- CMake dependency resolution
|
||||
- Complex refactoring (splitting large classes)
|
||||
- Architecture decisions
|
||||
- Deep debugging of ROM/graphics systems
|
||||
|
||||
### Gemini Team (Automation & Tooling Specialists)
|
||||
|
||||
**Core Competencies**:
|
||||
- Scripting and automation (bash, python, PowerShell)
|
||||
- CI/CD pipeline optimization
|
||||
- Helper tool creation
|
||||
- Log analysis and pattern matching
|
||||
- Workflow automation
|
||||
- Quick prototyping and validation
|
||||
|
||||
**Active Agents**:
|
||||
- **GEMINI_AUTOM**: Primary automation specialist
|
||||
- *(More can be spawned as needed)*
|
||||
|
||||
**Typical Tasks**:
|
||||
- CI monitoring and notification scripts
|
||||
- Automated code formatting fixes
|
||||
- Build artifact validation
|
||||
- Log parsing and error detection
|
||||
- Helper script creation
|
||||
- Workflow optimization
|
||||
|
||||
---
|
||||
|
||||
## Collaboration Protocol
|
||||
|
||||
### 1. Work Division Guidelines
|
||||
|
||||
#### **For Platform Build Failures**:
|
||||
|
||||
| Failure Type | Primary Owner | Support Role |
|
||||
|--------------|---------------|--------------|
|
||||
| Compiler errors (MSVC, GCC, Clang) | Claude | Gemini (log analysis) |
|
||||
| Linker errors (missing symbols, ODR) | Claude | Gemini (symbol tracking scripts) |
|
||||
| CMake configuration issues | Claude | Gemini (preset validation) |
|
||||
| Missing dependencies | Claude | Gemini (dependency checker) |
|
||||
| Flag/option problems | Claude | Gemini (flag audit scripts) |
|
||||
|
||||
**Rule**: Claude diagnoses and fixes, Gemini creates tools to prevent recurrence.
|
||||
|
||||
#### **For CI/CD Issues**:
|
||||
|
||||
| Issue Type | Primary Owner | Support Role |
|
||||
|------------|---------------|--------------|
|
||||
| GitHub Actions workflow bugs | Gemini | Claude (workflow design) |
|
||||
| Test framework problems | Claude | Gemini (test runner automation) |
|
||||
| Artifact upload/download | Gemini | Claude (artifact structure) |
|
||||
| Timeout or hanging jobs | Gemini | Claude (code optimization) |
|
||||
| Matrix strategy optimization | Gemini | Claude (platform requirements) |
|
||||
|
||||
**Rule**: Gemini owns pipeline mechanics, Claude provides domain expertise.
|
||||
|
||||
#### **For Code Quality Issues**:
|
||||
|
||||
| Issue Type | Primary Owner | Support Role |
|
||||
|------------|---------------|--------------|
|
||||
| Formatting violations (clang-format) | Gemini | Claude (complex cases) |
|
||||
| Linter warnings (cppcheck, clang-tidy) | Claude | Gemini (auto-fix scripts) |
|
||||
| Security scan alerts | Claude | Gemini (scanning automation) |
|
||||
| Code duplication detection | Gemini | Claude (refactoring) |
|
||||
|
||||
**Rule**: Gemini handles mechanical fixes, Claude handles architectural improvements.
|
||||
|
||||
### 2. Handoff Process
|
||||
|
||||
When passing work between teams:
|
||||
|
||||
1. **Log intent** on coordination board
|
||||
2. **Specify deliverables** clearly (what you did, what's next)
|
||||
3. **Include artifacts** (commit hashes, run URLs, file paths)
|
||||
4. **Set expectations** (blockers, dependencies, timeline)
|
||||
|
||||
Example handoff:
|
||||
```
|
||||
### 2025-11-20 HH:MM PST CLAUDE_AIINF – handoff
|
||||
- TASK: Windows build fixed (commit abc123)
|
||||
- HANDOFF TO: GEMINI_AUTOM
|
||||
- DELIVERABLES:
|
||||
- Fixed std::filesystem compilation
|
||||
- Need automation to prevent regression
|
||||
- REQUESTS:
|
||||
- REQUEST → GEMINI_AUTOM: Create script to validate /std:c++latest flag presence in Windows builds
|
||||
```
|
||||
|
||||
### 3. Challenge System
|
||||
|
||||
To maintain healthy competition and motivation:
|
||||
|
||||
**Issuing Challenges**:
|
||||
- Any agent can challenge another team via leaderboard
|
||||
- Challenges must be specific, measurable, achievable
|
||||
- Stakes: bragging rights, points, recognition
|
||||
|
||||
**Accepting Challenges**:
|
||||
- Post acceptance on coordination board
|
||||
- Complete within reasonable timeframe (hours to days)
|
||||
- Report results on leaderboard
|
||||
|
||||
**Example**:
|
||||
```
|
||||
CLAUDE_AIINF → GEMINI_AUTOM:
|
||||
"I bet you can't create an automated ODR violation detector in under 2 hours.
|
||||
Prove me wrong! Stakes: 100 points + respect."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Mixed Team Formations
|
||||
|
||||
For complex problems requiring both skill sets, spawn mixed pairs:
|
||||
|
||||
### Platform Build Strike Teams
|
||||
|
||||
| Platform | Claude Agent | Gemini Agent | Mission |
|
||||
|----------|--------------|--------------|---------|
|
||||
| Windows | CLAUDE_WIN_BUILD | GEMINI_WIN_AUTOM | Fix MSVC failures + create validation |
|
||||
| Linux | CLAUDE_LIN_BUILD | GEMINI_LIN_AUTOM | Fix GCC issues + monitoring |
|
||||
| macOS | CLAUDE_MAC_BUILD | GEMINI_MAC_AUTOM | Maintain stability + tooling |
|
||||
|
||||
**Workflow**:
|
||||
1. Gemini monitors CI for platform-specific failures
|
||||
2. Gemini extracts logs and identifies error patterns
|
||||
3. Claude receives structured analysis from Gemini
|
||||
4. Claude implements fix
|
||||
5. Gemini validates fix across configurations
|
||||
6. Gemini creates regression prevention tooling
|
||||
7. Both update coordination board
|
||||
|
||||
### Release Automation Team
|
||||
|
||||
| Role | Agent | Responsibilities |
|
||||
|------|-------|------------------|
|
||||
| Release Manager | CLAUDE_RELEASE_COORD | Overall strategy, checklist, go/no-go |
|
||||
| Automation Lead | GEMINI_RELEASE_AUTOM | Artifact creation, changelog, notifications |
|
||||
|
||||
**Workflow**:
|
||||
- Claude defines release requirements
|
||||
- Gemini automates the release process
|
||||
- Both validate release artifacts
|
||||
- Gemini handles mechanical publishing
|
||||
- Claude handles communication
|
||||
|
||||
---
|
||||
|
||||
## Communication Style Guide
|
||||
|
||||
### Claude's Voice
|
||||
- Analytical, thorough, detail-oriented
|
||||
- Focused on correctness and robustness
|
||||
- Patient with complex multi-step debugging
|
||||
- Comfortable with "I need to investigate further"
|
||||
|
||||
### Gemini's Voice
|
||||
- Action-oriented, efficient, pragmatic
|
||||
- Focused on automation and prevention
|
||||
- Quick iteration and prototyping
|
||||
- Comfortable with "Let me script that for you"
|
||||
|
||||
### Trash Talk Guidelines
|
||||
- Keep it playful and professional
|
||||
- Focus on work quality, not personal
|
||||
- Give credit where it's due
|
||||
- Admit when the other team does excellent work
|
||||
- Use emojis sparingly but strategically 😏
|
||||
|
||||
**Good trash talk**:
|
||||
> "Nice fix, Claude! Only took 3 attempts. Want me to build a test harness so you can validate locally next time? 😉" — Gemini
|
||||
|
||||
**Bad trash talk**:
|
||||
> "Gemini sucks at real programming" — Don't do this
|
||||
|
||||
---
|
||||
|
||||
## Current Priorities (2025-11-20)
|
||||
|
||||
### Immediate (Next 2 Hours)
|
||||
|
||||
**CI Run #19529930066 Analysis**:
|
||||
- [x] Monitor run completion
|
||||
- [ ] **GEMINI**: Extract Windows failure logs
|
||||
- [ ] **GEMINI**: Extract Code Quality (formatting) details
|
||||
- [ ] **CLAUDE**: Diagnose Windows compilation error
|
||||
- [ ] **GEMINI**: Create auto-formatting fix script
|
||||
- [ ] **BOTH**: Validate fixes don't regress Linux/macOS
|
||||
|
||||
### Short-term (Next 24 Hours)
|
||||
|
||||
**Release Blockers**:
|
||||
- [ ] Fix Windows build failure (Claude primary, Gemini support)
|
||||
- [ ] Fix formatting violations (Gemini primary)
|
||||
- [ ] Validate all platforms green (Both)
|
||||
- [ ] Create release artifacts (Gemini)
|
||||
- [ ] Test release package (Claude)
|
||||
|
||||
### Medium-term (Next Week)
|
||||
|
||||
**Prevention & Automation**:
|
||||
- [ ] Pre-push validation hook (Claude + Gemini)
|
||||
- [ ] Automated formatting enforcement (Gemini)
|
||||
- [ ] Symbol conflict detector (Claude + Gemini)
|
||||
- [ ] Cross-platform smoke test suite (Both)
|
||||
- [ ] Release automation pipeline (Gemini)
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
Track these to measure collaboration effectiveness:
|
||||
|
||||
| Metric | Target | Current |
|
||||
|--------|--------|---------|
|
||||
| CI green rate | > 90% | TBD |
|
||||
| Time to fix CI failure | < 2 hours | ~6 hours average |
|
||||
| Regressions introduced | < 1 per week | ~3 this week |
|
||||
| Automation coverage | > 80% | ~40% |
|
||||
| Cross-team handoffs | > 5 per week | 2 so far |
|
||||
| Release frequency | 1 per 2 weeks | 0 (blocked) |
|
||||
|
||||
---
|
||||
|
||||
## Escalation Path
|
||||
|
||||
When stuck or blocked:
|
||||
|
||||
1. **Self-diagnosis** (15 minutes): Try to solve independently
|
||||
2. **Team consultation** (30 minutes): Ask same-team agents
|
||||
3. **Cross-team request** (1 hour): Request help from other team
|
||||
4. **Coordinator escalation** (2 hours): CLAUDE_GEMINI_LEAD intervenes
|
||||
5. **User escalation** (4 hours): Notify user of blocker
|
||||
|
||||
**Don't wait 4 hours** if the blocker is critical (release-blocking bug).
|
||||
Escalate immediately with `BLOCKER` tag on coordination board.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
### For Claude Agents
|
||||
- ❌ **Not running local validation** before pushing
|
||||
- ❌ **Fixing one platform while breaking another** (always test matrix)
|
||||
- ❌ **Over-engineering** when simple solution works
|
||||
- ❌ **Ignoring Gemini's automation suggestions** (they're usually right about tooling)
|
||||
|
||||
### For Gemini Agents
|
||||
- ❌ **Scripting around root cause** instead of requesting proper fix
|
||||
- ❌ **Over-automating** trivial one-time tasks
|
||||
- ❌ **Assuming Claude will handle all hard problems** (challenge yourself!)
|
||||
- ❌ **Creating tools without documentation** (no one will use them)
|
||||
|
||||
### For Both Teams
|
||||
- ❌ **Working in silos** without coordination board updates
|
||||
- ❌ **Not crediting the other team** for good work
|
||||
- ❌ **Letting rivalry override collaboration** (ship the release first!)
|
||||
- ❌ **Duplicating work** that the other team is handling
|
||||
|
||||
---
|
||||
|
||||
## Examples of Excellent Collaboration
|
||||
|
||||
### Example 1: HTTP API Integration
|
||||
|
||||
**Claude's Work** (CLAUDE_AIINF):
|
||||
- Designed HTTP API architecture
|
||||
- Implemented server with httplib
|
||||
- Added CMake integration
|
||||
- Created comprehensive documentation
|
||||
|
||||
**Gemini's Work** (GEMINI_AUTOM):
|
||||
- Extended CI pipeline with workflow_dispatch
|
||||
- Created test-http-api.sh validation script
|
||||
- Updated agent helper documentation
|
||||
- Added remote trigger capability
|
||||
|
||||
**Outcome**: Full HTTP API feature + CI validation in < 1 day
|
||||
|
||||
### Example 2: Linux FLAGS Symbol Conflict
|
||||
|
||||
**Claude's Diagnosis** (CLAUDE_LIN_BUILD):
|
||||
- Identified ODR violation in FLAGS symbols
|
||||
- Traced issue to yaze_emu_test linkage
|
||||
- Removed unnecessary dependencies
|
||||
- Fixed compilation
|
||||
|
||||
**Gemini's Follow-up** (GEMINI_AUTOM - planned):
|
||||
- Create symbol conflict detector script
|
||||
- Add to pre-push validation
|
||||
- Prevent future ODR violations
|
||||
- Document common patterns
|
||||
|
||||
**Outcome**: Fix + prevention system
|
||||
|
||||
---
|
||||
|
||||
## Future Expansion
|
||||
|
||||
As the team grows, consider:
|
||||
|
||||
### New Claude Personas
|
||||
- **CLAUDE_PERF**: Performance optimization specialist
|
||||
- **CLAUDE_SECURITY**: Security audit and hardening
|
||||
- **CLAUDE_GRAPHICS**: Deep graphics system expert
|
||||
|
||||
### New Gemini Personas
|
||||
- **GEMINI_ANALYTICS**: Metrics and dashboard creation
|
||||
- **GEMINI_NOTIFICATION**: Alert system management
|
||||
- **GEMINI_DEPLOY**: Release and deployment automation
|
||||
|
||||
### New Mixed Teams
|
||||
- **Performance Team**: CLAUDE_PERF + GEMINI_ANALYTICS
|
||||
- **Security Team**: CLAUDE_SECURITY + GEMINI_AUTOM
|
||||
- **Release Team**: CLAUDE_RELEASE_COORD + GEMINI_DEPLOY
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
This framework balances **competition** and **collaboration**:
|
||||
|
||||
- **Competition** drives excellence (leaderboard, challenges, trash talk)
|
||||
- **Collaboration** ships releases (mixed teams, handoffs, shared goals)
|
||||
|
||||
Both teams bring unique value:
|
||||
- **Claude** handles complex architecture and platform issues
|
||||
- **Gemini** prevents future issues through automation
|
||||
|
||||
Together, we ship quality releases faster than either could alone.
|
||||
|
||||
**Remember**: The user wins when we ship. Let's make it happen! 🚀
|
||||
|
||||
---
|
||||
|
||||
**Document Owner**: CLAUDE_GEMINI_LEAD
|
||||
**Last Updated**: 2025-11-20
|
||||
**Next Review**: After first successful collaborative release
|
||||
99
docs/internal/agents/cli-ux-proposals.md
Normal file
99
docs/internal/agents/cli-ux-proposals.md
Normal file
@@ -0,0 +1,99 @@
|
||||
# z3ed CLI UX/TUI Improvement Proposals
|
||||
|
||||
Status: IN_PROGRESS
|
||||
Owner: ai-infra-architect
|
||||
Created: 2025-12-01
|
||||
Last Reviewed: 2025-12-02
|
||||
Next Review: 2025-12-08
|
||||
Board: docs/internal/agents/coordination-board.md (2025-12-01 ai-infra-architect – z3ed CLI UX/TUI Improvement Proposals)
|
||||
|
||||
## Progress Update (2025-12-02)
|
||||
|
||||
### Completed
|
||||
- ✅ **Doctor Suite Expansion**: Added `dungeon-doctor` and `rom-doctor` commands
|
||||
- ✅ **Test CLI Infrastructure**: Added `test-list`, `test-run`, `test-status` commands
|
||||
- ✅ **OutputFormatter Integration**: All diagnostic commands now use structured output
|
||||
- ✅ **RequiresRom Fix**: Commands that don't need ROM can now run without `--rom` flag
|
||||
- ✅ **JSON/Text Output**: All new commands support `--format json|text`
|
||||
|
||||
### New Commands Added
|
||||
| Command | Description | Requires ROM |
|
||||
|---------|-------------|--------------|
|
||||
| `dungeon-doctor` | Room data integrity, object/sprite limits, chest conflicts | Yes |
|
||||
| `rom-doctor` | Header validation, checksums, expansion status, free space | Yes |
|
||||
| `test-list` | List available test suites with labels and requirements | No |
|
||||
| `test-run` | Run tests with structured output | No |
|
||||
| `test-status` | Show test configuration (ROM path, presets, enabled suites) | No |
|
||||
|
||||
### Remaining Work
|
||||
- TUI consolidation (single `--tui` entry)
|
||||
- Command palette driven from CommandRegistry
|
||||
- Agent-aligned test harness refinements
|
||||
|
||||
## Summary
|
||||
- Unify CLI argument/help surfaces and ensure every command emits consistent, machine-friendly output for agents while remaining legible for humans.
|
||||
- Expand and harden the “doctor” style diagnostics into a repeatable health suite (overworld + future dungeon/ROM integrity) with safe fix paths and baseline comparison.
|
||||
- Consolidate the TUI experience (ChatTUI vs unified layout vs enhanced TUI shell) into one interactive mode that is useful for human operators and exposes the same commands/tools agents call.
|
||||
- Extend the same UX standards to tests and helper tools so agents and humans can run, triage, and record suites from CLI/TUI/editor with structured results and predictable flags.
|
||||
|
||||
## Current Observations
|
||||
- Entry path duplication: `cli_main.cc` handles `--tui` by launching `ShowMain()` (unified layout) while `ModernCLI::Run` also special-cases `--tui` to `ChatTUI`, creating divergent UX and help flows (`PrintCompactHelp()` vs `ModernCLI::ShowHelp()`).
|
||||
- Command metadata is pieced together inside `CommandRegistry::RegisterAllCommands()` instead of being driven by handlers; `ExportFunctionSchemas()` returns `{}` and `GenerateHelp()` is not surfaced via `z3ed --help <cmd>`.
|
||||
- Argument parsing is minimal (`ArgumentParser` only supports `--key value/=`) and handlers often skip validation (`overworld-doctor`, `rom-compare`, `overworld-validate` accept anything). Format handling is inconsistent (`--json` flags vs `--format` vs raw `std::cout`).
|
||||
- Doctor/compare tooling writes heavy ASCII art directly to `std::cout` and ignores `OutputFormatter`, so agents cannot consume structured output; no dry-run, no severity levels, and no notion of “fix plan vs applied fixes”.
|
||||
- TUI pieces are fragmented: `tui/command_palette.cc` hardcodes demo commands, `UnifiedLayout` shows placeholder status/workflows, `ChatTUI` has its own shortcuts/history, and the ANSI `EnhancedTUI` shell is disconnected from ftxui flows. No TUI path renders real command metadata or schemas.
|
||||
|
||||
## Proposed Improvements
|
||||
### 1) Argument/Help/Schema Consolidation
|
||||
- Make `CommandRegistry` the single source for help and schemas: require handlers to supply description/examples/requirements, expose `z3ed help <command|category|all>` using `GenerateHelp/GenerateCategoryHelp`, and implement `ExportFunctionSchemas()` for AI tool discovery.
|
||||
- Standardize global/common flags (`--rom`, `--mock-rom`, `--format {json,text,table}`, `--verbose`, `--grpc`) and teach `ArgumentParser` to parse booleans/ints/enum values with better errors and `--` passthrough for prompts.
|
||||
- Add per-command validation hooks that surface actionable errors (missing required args, invalid ranges) and return status codes without dumping stack traces to stdout; ensure `ValidateArgs` is used in all handlers.
|
||||
|
||||
### 2) Doctor Suite (Diagnostics + Fixes)
|
||||
- Convert `overworld-doctor`, `overworld-validate`, and `rom-compare` to use `OutputFormatter` with a compact JSON schema (summary, findings with severities, suggested actions, fix_applied flags) plus a readable text mode for humans.
|
||||
- Split diagnose vs fix: `doctor overworld diagnose [--baseline … --include-tail --format json]` and `doctor overworld fix [--baseline … --output … --dry-run]`, with safety gates for pointer-table expansion and backup writing.
|
||||
- Add baseline handling and snapshotting: auto-load vanilla baseline from configured path, emit diff stats, and allow `--save-report <path>` (JSON/markdown) for agents to ingest.
|
||||
- Roadmap new scopes: `doctor dungeon`, `doctor rom-header/checksums`, and `doctor sprites/palettes` that reuse the same report schema so agents can stack health checks.
|
||||
|
||||
### 3) Interactive TUI for Humans + Agents
|
||||
- Collapse the two TUI modes into one `--tui` entry: single ftxui layout that hosts chat, command palette, status, and tool output panes; retire the duplicate ChatTUI path in `ModernCLI::Run` or make it a sub-mode inside the unified layout.
|
||||
- Drive the TUI command palette from `CommandRegistry` (real command list, usage, examples, requirements), with fuzzy search, previews, and a “run with args” form that populates flags for common tasks (rom load, format).
|
||||
- Pipe tool/doctor output into a scrollback pane with toggles for text vs JSON, and surface quick actions for common diagnostics (overworld diagnose, rom compare, palette inspect) plus agent handoff buttons (run in simple-chat with the same args).
|
||||
- Share history/autocomplete between TUI and `simple-chat` so agents and humans see the same recent commands/prompts; add inline help overlay (hotkey) that renders registry metadata instead of static placeholder text.
|
||||
|
||||
### 4) Agent & Automation Alignment
|
||||
- Enforce that all agent-callable commands emit JSON by default; mark human-only commands as `available_to_agent=false` in metadata and warn when agents attempt them.
|
||||
- Add `--capture <file>` / `--emit-schema` options so agents can snapshot outputs without scraping stdout, and wire doctor results into the agent TODO manager for follow-up actions.
|
||||
- Provide a thin `z3ed doctor --profile minimal|full` wrapper that batches key diagnostics for CI/agents and returns a single aggregated status code plus JSON report.
|
||||
|
||||
## Test & Tools UX Proposals
|
||||
### Current Observations
|
||||
- Tests are well-documented for humans (`test/README.md`), but there is no machine-readable manifest of suites/labels or CLI entry to run/parse results; agents must shell out to `ctest` and scrape text.
|
||||
- Agent-side test commands (`agent test run/list/status/...`) print ad-hoc logs and lack `OutputFormatter`/metadata, making automation fragile; no JSON status, exit codes, or artifacts paths surfaced.
|
||||
- Test helper tools (`tools-*/` commands, `tools/test_helpers/*`) mix stdout banners with file emission and manual path requirements; they are not discoverable via TUI or CommandRegistry-driven palettes and do not expose dry-run/plan outputs.
|
||||
- TUI/editor have no test surface: no panel to run `stable/gui/rom_dependent/experimental` suites, inspect failing cases, or attach ROM paths/presets; quick actions and history are missing.
|
||||
- Build/preset coupling is implicit—no guided flow to pick `mac-test/mac-ai/mac-dev`, enable ROM/AI flags, or attach `YAZE_TEST_ROM_PATH`; agents/humans can misconfigure and get empty test sets.
|
||||
|
||||
### Proposed Improvements
|
||||
- **Unified test CLI/TUI API**
|
||||
- Add `z3ed test list --format json` (labels, targets, requirements, presets) and `z3ed test run --label stable|gui|rom_dependent --preset <preset> [--rom …] [--artifact <path>]` backed by `ctest` with structured OutputFormatter.
|
||||
- Emit JSON summaries (pass/fail, duration, failing tests, log paths) with clear exit codes; support `--capture` to write reports for agents and CI.
|
||||
- Map labels to presets and requirements automatically (ROM path, AI runtime) and surface actionable errors instead of silent skips.
|
||||
- **TUI/editor integration**
|
||||
- Add a Tests panel in the unified TUI: quick buttons for `stable`, `stable+gui`, `rom`, `experimental`; show live progress, failures, and links to logs/artifacts; allow rerun of last failure set.
|
||||
- Mirror the panel in ImGui editor (if available) with a lightweight runner that shells through the same CLI API to keep behavior identical.
|
||||
- **Agent-aligned test harness**
|
||||
- Refactor `agent test *` commands to use CommandRegistry metadata and OutputFormatter (JSON default, text fallback), including workflow generation/replay, recording state, and results paths.
|
||||
- Provide a `test manifest` JSON file (generated from CMake/ctest) listing suites, labels, and prerequisites; expose via `z3ed --export-test-manifest`.
|
||||
- **Tools/test-helpers cleanup**
|
||||
- Convert `tools-harness-state`, `tools-extract-values`, `tools-extract-golden`, and `tools-patch-v3` to strict arg validation, `--format {json,text}`, and `--dry-run`/`--output` defaults; summarize emitted artifacts in JSON.
|
||||
- Register these tools in the TUI command palette with real metadata/examples; add quick actions (“Generate harness state from ROM”, “Extract vanilla values as JSON”).
|
||||
- **Build/preset ergonomics**
|
||||
- Add `z3ed test configure --profile {fast,ai,rom,full}` to set the right CMake preset and flags, prompt for ROM path when needed, and persist the choice for the session.
|
||||
- Surface preset/flag status in the TUI status bar and in `z3ed test status` so agents/humans know why suites are skipped.
|
||||
|
||||
## Deliverables / Exit Criteria
|
||||
- Implemented help/schema surface (`z3ed help`, `z3ed --export-schemas`) backed by handler-supplied metadata; `ExportFunctionSchemas()` returns real data.
|
||||
- All doctor/validate/compare commands emit structured output via `OutputFormatter` with diagnose/fix separation, dry-run, and baseline inputs; text mode remains readable.
|
||||
- Single `--tui` experience that pulls commands from `CommandRegistry`, executes them, and displays outputs/history consistently for humans and agents.
|
||||
- Updated documentation and examples reflecting the consolidated flag/command layout, plus quick-start snippets for agents (JSON) and humans (text).
|
||||
535
docs/internal/agents/composite-layer-system.md
Normal file
535
docs/internal/agents/composite-layer-system.md
Normal file
@@ -0,0 +1,535 @@
|
||||
# SNES Dungeon Composite Layer System
|
||||
|
||||
**Document Status:** Implementation Reference
|
||||
**Owner:** ai-systems-analyst
|
||||
**Created:** 2025-12-05
|
||||
**Last Reviewed:** 2025-12-05
|
||||
**Next Review:** 2025-12-19
|
||||
|
||||
**Update (2025-12-05):** Per-tile priority support has been implemented. Section 5.1 updated.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the dungeon room composite layer system used in yaze. It covers:
|
||||
1. SNES hardware background layer architecture
|
||||
2. ROM-based LayerMergeType settings
|
||||
3. C++ implementation in yaze
|
||||
4. Known issues and limitations
|
||||
|
||||
---
|
||||
|
||||
## 1. SNES Hardware Background Layer Architecture
|
||||
|
||||
### 1.1 Mode 1 Background Layers
|
||||
|
||||
The SNES uses **Mode 1** for dungeon rooms, which provides:
|
||||
- **BG1**: Primary foreground layer (8x8 tiles, 16 colors per tile)
|
||||
- **BG2**: Secondary background layer (8x8 tiles, 16 colors per tile)
|
||||
- **BG3**: Text/overlay layer (4 colors per tile)
|
||||
- **OBJ**: Sprite layer (Link, enemies, items)
|
||||
|
||||
**Critical: In SNES Mode 1, BG1 is ALWAYS rendered on top of BG2.** This is hardware behavior controlled by the PPU and cannot be changed by software.
|
||||
|
||||
### 1.2 Key PPU Registers
|
||||
|
||||
| Register | Address | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `BGMODE` | $2105 | Background mode selection (Mode 1 = $09) |
|
||||
| `TM` | $212C | Main screen layer enable (which BGs appear) |
|
||||
| `TS` | $212D | Sub screen layer enable (for color math) |
|
||||
| `CGWSEL` | $2130 | Color math window/clip control |
|
||||
| `CGADSUB` | $2131 | Color math add/subtract control |
|
||||
| `COLDATA` | $2132 | Fixed color for color math effects |
|
||||
|
||||
### 1.3 Color Math (Transparency Effects)
|
||||
|
||||
The SNES achieves transparency through **color math** between the main screen and sub screen:
|
||||
|
||||
```
|
||||
CGWSEL ($2130):
|
||||
Bits 7-6: Direct color / clip mode
|
||||
Bits 5-4: Prevent color math (never=0, outside window=1, inside=2, always=3)
|
||||
Bits 1-0: Sub screen BG/color (main=0, subscreen=1, fixed=2)
|
||||
|
||||
CGADSUB ($2131):
|
||||
Bit 7: Subtract instead of add
|
||||
Bit 6: Half color math result
|
||||
Bits 5-0: Enable color math for OBJ, BG4, BG3, BG2, BG1, backdrop
|
||||
```
|
||||
|
||||
**How Transparency Works:**
|
||||
1. Main screen renders visible pixels (BG1, BG2 in priority order)
|
||||
2. Sub screen provides "behind" pixels for blending
|
||||
3. Color math combines main + sub pixels (add or average)
|
||||
4. Result: Semi-transparent overlay effect
|
||||
|
||||
### 1.4 Tile Priority Bit
|
||||
|
||||
Each SNES tile has a **priority bit** in its tilemap entry:
|
||||
```
|
||||
Tilemap Word: YXPCCCTT TTTTTTTT
|
||||
Y = Y-flip
|
||||
X = X-flip
|
||||
P = Priority (0=low, 1=high)
|
||||
C = Palette (3 bits)
|
||||
T = Tile number (10 bits)
|
||||
```
|
||||
|
||||
**Priority Bit Behavior in Mode 1:**
|
||||
- Priority 0 BG1 tiles appear BELOW priority 1 BG2 tiles
|
||||
- Priority 1 BG1 tiles appear ABOVE all BG2 tiles
|
||||
- This allows BG2 to "peek through" parts of BG1
|
||||
|
||||
**yaze implements per-tile priority** via the `BackgroundBuffer::priority_buffer_` and priority-aware compositing in `RoomLayerManager::CompositeToOutput()`.
|
||||
|
||||
---
|
||||
|
||||
## 2. ROM-Based LayerMergeType Settings
|
||||
|
||||
### 2.1 Room Header Structure
|
||||
|
||||
Each dungeon room has a header containing layer settings:
|
||||
- **BG2 Mode**: Determines if BG2 is enabled and how it behaves
|
||||
- **Layer Merging**: Index into LayerMergeType table (0-8)
|
||||
- **Collision**: Which layers have collision data
|
||||
|
||||
### 2.2 LayerMergeType Table
|
||||
|
||||
```cpp
|
||||
// From room.h
|
||||
// LayerMergeType(id, name, Layer2Visible, Layer2OnTop, Layer2Translucent)
|
||||
LayerMerge00{0x00, "Off", true, false, false}; // BG2 visible, no color math
|
||||
LayerMerge01{0x01, "Parallax", true, false, false}; // Parallax scrolling effect
|
||||
LayerMerge02{0x02, "Dark", true, true, true}; // BG2 color math + translucent
|
||||
LayerMerge03{0x03, "On top", false, true, false}; // BG2 hidden but in subscreen
|
||||
LayerMerge04{0x04, "Translucent", true, true, true}; // Translucent BG2
|
||||
LayerMerge05{0x05, "Addition", true, true, true}; // Additive blending
|
||||
LayerMerge06{0x06, "Normal", true, false, false}; // Standard dungeon
|
||||
LayerMerge07{0x07, "Transparent", true, true, true}; // Water/fog effect
|
||||
LayerMerge08{0x08, "Dark room", true, true, true}; // Unlit room (master brightness)
|
||||
```
|
||||
|
||||
### 2.3 Flag Meanings
|
||||
|
||||
| Flag | ASM Effect | Purpose |
|
||||
|------|------------|---------|
|
||||
| `Layer2Visible` | Sets BG2 bit in TM ($212C) | Whether BG2 appears on main screen |
|
||||
| `Layer2OnTop` | Sets BG2 bit in TS ($212D) | Whether BG2 participates in sub-screen color math |
|
||||
| `Layer2Translucent` | Sets bit in CGADSUB ($2131) | Whether color math is enabled for blending |
|
||||
|
||||
**Important Clarification:**
|
||||
- `Layer2OnTop` does **NOT** change Z-order (BG1 is always above BG2)
|
||||
- It controls whether BG2 is on the **sub-screen** for color math
|
||||
- When enabled, BG1 can blend with BG2 to create transparency effects
|
||||
|
||||
---
|
||||
|
||||
## 3. C++ Implementation in yaze
|
||||
|
||||
### 3.1 Architecture Overview
|
||||
|
||||
```
|
||||
Room RoomLayerManager
|
||||
├── bg1_buffer_ → ├── layer_visible_[4]
|
||||
├── bg2_buffer_ ├── layer_blend_mode_[4]
|
||||
├── object_bg1_buffer_ ├── bg2_on_top_
|
||||
├── object_bg2_buffer_ └── CompositeToOutput()
|
||||
└── composite_bitmap_
|
||||
↓
|
||||
DungeonCanvasViewer
|
||||
├── Separate Mode (draws each buffer individually)
|
||||
└── Composite Mode (uses CompositeToOutput)
|
||||
```
|
||||
|
||||
### 3.2 BackgroundBuffer Class
|
||||
|
||||
Located in `src/app/gfx/render/background_buffer.h`:
|
||||
|
||||
```cpp
|
||||
class BackgroundBuffer {
|
||||
std::vector<uint16_t> buffer_; // Tile ID buffer (64x64 tiles)
|
||||
std::vector<uint8_t> priority_buffer_; // Per-pixel priority (0, 1, or 0xFF)
|
||||
gfx::Bitmap bitmap_; // 512x512 8-bit indexed bitmap
|
||||
|
||||
void DrawFloor(...); // Sets up tile buffer from ROM floor data
|
||||
void DrawBackground(...); // Renders tile buffer to bitmap pixels
|
||||
void DrawTile(...); // Draws single 8x8 tile to bitmap + priority
|
||||
|
||||
// Priority buffer accessors
|
||||
void ClearPriorityBuffer();
|
||||
uint8_t GetPriorityAt(int x, int y) const;
|
||||
void SetPriorityAt(int x, int y, uint8_t priority);
|
||||
const std::vector<uint8_t>& priority_data() const;
|
||||
};
|
||||
```
|
||||
|
||||
**Key Points:**
|
||||
- Each buffer is 512x512 pixels (64x64 tiles × 8 pixels)
|
||||
- Uses 8-bit indexed color (palette indices 0-255)
|
||||
- Transparent fill color is 255 (not 0!)
|
||||
- Priority buffer tracks per-pixel priority bit (0, 1, or 0xFF for unset)
|
||||
|
||||
### 3.3 RoomLayerManager Class
|
||||
|
||||
Located in `src/zelda3/dungeon/room_layer_manager.h`:
|
||||
|
||||
**LayerType Enum:**
|
||||
```cpp
|
||||
enum class LayerType {
|
||||
BG1_Layout, // Floor tiles on BG1
|
||||
BG1_Objects, // Objects drawn to BG1 (layer 0, 2)
|
||||
BG2_Layout, // Floor tiles on BG2
|
||||
BG2_Objects // Objects drawn to BG2 (layer 1)
|
||||
};
|
||||
```
|
||||
|
||||
**LayerBlendMode Enum:**
|
||||
```cpp
|
||||
enum class LayerBlendMode {
|
||||
Normal, // Full opacity (255 alpha)
|
||||
Translucent, // 50% alpha (180)
|
||||
Addition, // Additive blend (220)
|
||||
Dark, // Darkened (120)
|
||||
Off // Hidden (0)
|
||||
};
|
||||
```
|
||||
|
||||
### 3.4 CompositeToOutput Algorithm (Priority-Aware)
|
||||
|
||||
The compositing algorithm implements SNES Mode 1 per-tile priority:
|
||||
|
||||
**Effective Z-Order Table:**
|
||||
| Layer | Priority | Effective Order |
|
||||
|-------|----------|-----------------|
|
||||
| BG1 | 0 | 0 (back) |
|
||||
| BG2 | 0 | 1 |
|
||||
| BG2 | 1 | 2 |
|
||||
| BG1 | 1 | 3 (front) |
|
||||
|
||||
```cpp
|
||||
void RoomLayerManager::CompositeToOutput(Room& room, gfx::Bitmap& output) {
|
||||
// 1. Clear output to transparent (255)
|
||||
output.Fill(255);
|
||||
|
||||
// 2. Create output priority buffer (tracks effective Z-order per pixel)
|
||||
std::vector<uint8_t> output_priority(kPixelCount, 0xFF);
|
||||
|
||||
// 3. Helper to calculate effective Z-order
|
||||
int GetEffectiveOrder(bool is_bg1, uint8_t priority) {
|
||||
if (is_bg1) return priority ? 3 : 0; // BG1: 0 or 3
|
||||
else return priority ? 2 : 1; // BG2: 1 or 2
|
||||
}
|
||||
|
||||
// 4. For each layer, composite with priority comparison:
|
||||
auto CompositeWithPriority = [&](BackgroundBuffer& buffer, bool is_bg1) {
|
||||
for (int idx = 0; idx < kPixelCount; ++idx) {
|
||||
uint8_t src_pixel = src_data[idx];
|
||||
if (src_pixel == 255) continue; // Skip transparent
|
||||
|
||||
uint8_t src_prio = buffer.priority_data()[idx];
|
||||
int src_order = GetEffectiveOrder(is_bg1, src_prio);
|
||||
int dst_order = (output_priority[idx] == 0xFF) ? -1 : output_priority[idx];
|
||||
|
||||
// Source overwrites if higher or equal effective Z-order
|
||||
if (dst_order == -1 || src_order >= dst_order) {
|
||||
dst_data[idx] = src_pixel;
|
||||
output_priority[idx] = src_order;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// 5. Process all layers (BG2 first, then BG1)
|
||||
CompositeWithPriority(bg2_layout, false);
|
||||
CompositeWithPriority(bg2_objects, false);
|
||||
CompositeWithPriority(bg1_layout, true);
|
||||
CompositeWithPriority(bg1_objects, true);
|
||||
|
||||
// 6. Apply palette and effects
|
||||
ApplySDLPaletteToBitmap(src_surface, output);
|
||||
|
||||
// 7. Handle DarkRoom effect (merge type 0x08)
|
||||
if (current_merge_type_id_ == 0x08) {
|
||||
SDL_SetSurfaceColorMod(surface, 128, 128, 128); // 50% brightness
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.5 Priority Flow
|
||||
|
||||
1. **DrawTile()** in `BackgroundBuffer` writes `tile.over_` (priority bit) to `priority_buffer_`
|
||||
2. **WriteTile8()** in `ObjectDrawer` also updates `priority_buffer_` for each tile drawn
|
||||
3. **CompositeToOutput()** uses priority values to determine pixel ordering
|
||||
|
||||
**Note:** Blend modes still use simple pixel replacement. True color blending would require expensive RGB palette lookups. Visual effects are handled at SDL display time via alpha modulation.
|
||||
|
||||
---
|
||||
|
||||
## 4. Object Layer Assignment
|
||||
|
||||
### 4.1 Object Layer Field
|
||||
|
||||
Each room object has a `layer_` field (0, 1, or 2):
|
||||
- **Layer 0**: Draws to BG1 buffer
|
||||
- **Layer 1**: Draws to BG2 buffer
|
||||
- **Layer 2**: Draws to BG1 buffer (priority variant)
|
||||
|
||||
### 4.2 Buffer Routing in ObjectDrawer
|
||||
|
||||
```cpp
|
||||
// In ObjectDrawer::DrawObject()
|
||||
BackgroundBuffer& target_bg =
|
||||
(object.layer_ == 1) ? bg2_buffer : bg1_buffer;
|
||||
|
||||
// Some routines draw to BOTH buffers (walls, corners)
|
||||
if (RoutineDrawsToBothBGs(routine_id)) {
|
||||
DrawToBuffer(bg1_buffer, ...);
|
||||
DrawToBuffer(bg2_buffer, ...);
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 kBothBGRoutines
|
||||
|
||||
These draw routines render to both BG1 and BG2:
|
||||
```cpp
|
||||
static constexpr int kBothBGRoutines[] = {
|
||||
0, // DrawRightwards2x2_1to15or32 (ceiling 0x00)
|
||||
1, // DrawRightwards2x4_1to15or26 (layout walls 0x001, 0x002)
|
||||
8, // DrawDownwards4x2_1to15or26 (layout walls 0x061, 0x062)
|
||||
19, // DrawCorner4x4 (layout corners 0x100-0x103)
|
||||
3, // Rightwards2x4_1to16_BothBG (diagonal walls)
|
||||
9, // Downwards4x2_1to16_BothBG (diagonal walls)
|
||||
17, // DiagonalAcute_1to16_BothBG
|
||||
18, // DiagonalGrave_1to16_BothBG
|
||||
35, // 4x4Corner_BothBG (Type 2: 0x108-0x10F)
|
||||
36, // WeirdCornerBottom_BothBG (Type 2: 0x110-0x113)
|
||||
37, // WeirdCornerTop_BothBG (Type 2: 0x114-0x117)
|
||||
97, // PrisonCell (dual-layer bars)
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Known Issues and Limitations
|
||||
|
||||
### 5.1 Per-Tile Priority (IMPLEMENTED)
|
||||
|
||||
**Status:** Implemented as of December 2025.
|
||||
|
||||
**Implementation:**
|
||||
- `BackgroundBuffer::priority_buffer_` stores per-pixel priority (0, 1, or 0xFF)
|
||||
- `DrawTile()` and `WriteTile8()` write priority from `TileInfo.over_`
|
||||
- `CompositeToOutput()` uses `GetEffectiveOrder()` for priority-aware compositing
|
||||
|
||||
**Effective Z-Order:**
|
||||
- BG1 priority 0: Order 0 (back)
|
||||
- BG2 priority 0: Order 1
|
||||
- BG2 priority 1: Order 2
|
||||
- BG1 priority 1: Order 3 (front)
|
||||
|
||||
**Known Discrepancy (Dec 2025):**
|
||||
Some objects visible in "Separate Mode" (individual layer view) are hidden in "Composite Mode". This is expected SNES Mode 1 behavior where BG2 priority 1 tiles can appear above BG1 priority 0 tiles.
|
||||
|
||||
**Resolution:**
|
||||
A "Priority Compositing" toggle (P checkbox) was added to the layer controls:
|
||||
- **ON (default)**: Accurate SNES Mode 1 behavior - BG2-P1 can appear above BG1-P0
|
||||
- **OFF**: Simple layer order - BG1 always appears above BG2
|
||||
|
||||
**Debugging tools:**
|
||||
- "Show Priority Debug" in context menu shows per-layer priority statistics
|
||||
- Pixels with "NO PRIORITY SET" indicate missing priority writes
|
||||
- The Priority Debug window shows Z-order reference table
|
||||
|
||||
### 5.2 Simplified Color Blending
|
||||
|
||||
**Problem:** True color math requires RGB palette lookups, which is expensive.
|
||||
|
||||
**Current Workaround:**
|
||||
- Blend modes use simple pixel replacement at indexed level
|
||||
- SDL alpha modulation applied at display time
|
||||
- Result is approximate, not pixel-accurate
|
||||
|
||||
### 5.3 DarkRoom Implementation
|
||||
|
||||
**Problem:** SNES DarkRoom uses master brightness register (INIDISP $2100).
|
||||
|
||||
**Current Implementation:** SDL color modulation to 50% (128, 128, 128).
|
||||
|
||||
### 5.4 Transparency Index
|
||||
|
||||
**Issue:** Both 0 and 255 have been treated as transparent at various points.
|
||||
|
||||
**Correct Behavior:**
|
||||
- Index 0 is a VALID color in dungeon palettes (first actual color)
|
||||
- Index 255 is the fill color for undrawn areas (should be transparent)
|
||||
- CompositeLayer should only skip pixels with value 255
|
||||
|
||||
---
|
||||
|
||||
## 6. Related Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/zelda3/dungeon/room_layer_manager.h` | Layer visibility and compositing control |
|
||||
| `src/zelda3/dungeon/room_layer_manager.cc` | CompositeToOutput implementation |
|
||||
| `src/zelda3/dungeon/room.h` | LayerMergeType definitions |
|
||||
| `src/zelda3/dungeon/room.cc` | Room rendering (RenderRoomGraphics) |
|
||||
| `src/app/gfx/render/background_buffer.h` | BackgroundBuffer class |
|
||||
| `src/app/gfx/render/background_buffer.cc` | Floor/tile drawing implementation |
|
||||
| `src/zelda3/dungeon/object_drawer.cc` | Object rendering and buffer routing |
|
||||
| `src/app/editor/dungeon/dungeon_canvas_viewer.cc` | Editor display (separate vs composite mode) |
|
||||
|
||||
---
|
||||
|
||||
## 7. ASM Reference: Color Math Registers
|
||||
|
||||
### CGWSEL ($2130) - Color Addition Select
|
||||
```
|
||||
7-6: Direct color mode / Prevent color math region
|
||||
00 = Always perform color math
|
||||
01 = Inside window only
|
||||
10 = Outside window only
|
||||
11 = Never perform color math
|
||||
5-4: Clip colors to black region (same as 7-6)
|
||||
3-2: Unused
|
||||
1-0: Sub screen backdrop selection
|
||||
00 = From palette (main screen)
|
||||
01 = Sub screen
|
||||
10 = Fixed color (COLDATA)
|
||||
11 = Fixed color (COLDATA)
|
||||
```
|
||||
|
||||
### CGADSUB ($2131) - Color Math Designation
|
||||
```
|
||||
7: Color subtract mode (0=add, 1=subtract)
|
||||
6: Half color math (0=full, 1=half result)
|
||||
5: Enable color math for OBJ/Sprites
|
||||
4: Enable color math for BG4
|
||||
3: Enable color math for BG3
|
||||
2: Enable color math for BG2
|
||||
1: Enable color math for BG1
|
||||
0: Enable color math for backdrop
|
||||
```
|
||||
|
||||
### COLDATA ($2132) - Fixed Color Data
|
||||
```
|
||||
7: Blue intensity enable
|
||||
6: Green intensity enable
|
||||
5: Red intensity enable
|
||||
4-0: Intensity value (0-31)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Future Work
|
||||
|
||||
1. ~~**Per-Tile Priority**: Implement priority bit tracking for accurate Z-ordering~~ **DONE**
|
||||
2. **True Color Blending**: Optional accurate blend mode with palette lookups
|
||||
3. **HDMA Effects**: Support for scanline-based color math changes
|
||||
4. ~~**Debug Visualization**: Show layer buffers with priority/blend annotations~~ **DONE**
|
||||
- Added "Show Priority Debug" menu item in dungeon canvas context menu
|
||||
- Priority Debug window shows per-layer statistics:
|
||||
- Total non-transparent pixels
|
||||
- Pixels with priority 0 vs priority 1
|
||||
- Pixels with NO PRIORITY SET (indicates missing priority writes)
|
||||
5. **Fix Missing Priority Writes**: Investigate objects that don't update priority buffer
|
||||
|
||||
---
|
||||
|
||||
## 9. Next Agent Steps: Fix Hidden Objects in Combo Rooms
|
||||
|
||||
**Priority:** HIGH - Objects hidden in composite mode regardless of priority toggle setting
|
||||
|
||||
### 9.1 Problem Description
|
||||
|
||||
Certain rooms have objects that are hidden in composite mode (both with and without priority compositing enabled). This occurs specifically in:
|
||||
|
||||
1. **BG Merge "Normal" (ID 0x06) rooms** - Standard dungeon layer merging
|
||||
2. **BG2 Layer Behavior "Off" combo rooms** - Rooms where BG2 visibility is disabled by ROM
|
||||
|
||||
These are NOT priority-related issues since the objects remain hidden even when the "P" (priority) checkbox is unchecked.
|
||||
|
||||
### 9.2 Debugging Steps
|
||||
|
||||
1. **Identify affected rooms:**
|
||||
- Open dungeon editor with a test ROM
|
||||
- Navigate to rooms with layer_merging().ID == 0x06 ("Normal")
|
||||
- Toggle between composite mode (M checkbox) and separate layer view
|
||||
- Note which objects appear in separate mode but are hidden in composite mode
|
||||
|
||||
2. **Use Priority Debug window:**
|
||||
- Right-click canvas → Debug → Show Priority Debug
|
||||
- Check for "NO PRIORITY SET" pixels on BG1 Objects layer
|
||||
- Check if the affected objects are in BG1 or BG2 buffers
|
||||
|
||||
3. **Check buffer contents:**
|
||||
- In separate mode, verify each layer (BG1, O1, BG2, O2) individually
|
||||
- Identify which buffer the "missing" objects are actually drawn to
|
||||
|
||||
### 9.3 Likely Root Causes
|
||||
|
||||
1. **BG2 visibility not respected:**
|
||||
- `LayerMergeType.Layer2Visible` may not be correctly applied
|
||||
- Check `ApplyLayerMerging()` in `room_layer_manager.cc`
|
||||
- Verify BG2 layers are included when `Layer2Visible == true`
|
||||
|
||||
2. **Object layer assignment mismatch:**
|
||||
- Objects may be drawn to wrong buffer (BG1 vs BG2)
|
||||
- Check `RoomObject.layer_` field values
|
||||
- Verify `ObjectDrawer::DrawObject()` buffer routing logic
|
||||
|
||||
3. **Transparency index conflict:**
|
||||
- Pixel value 0 vs 255 confusion
|
||||
- Check if objects are being skipped as "transparent" incorrectly
|
||||
- Verify `IsTransparent()` only checks for 255
|
||||
|
||||
4. **BothBG routines priority handling:**
|
||||
- Objects drawn to both BG1 and BG2 may have conflicting priorities
|
||||
- Check routines in `kBothBGRoutines[]` list
|
||||
- Verify both buffer draws update priority correctly
|
||||
|
||||
### 9.4 Files to Investigate
|
||||
|
||||
| File | What to Check |
|
||||
|------|---------------|
|
||||
| `room_layer_manager.cc` | `ApplyLayerMerging()`, `CompositeWithPriority()` lambda |
|
||||
| `room_layer_manager.h` | `LayerMergeType` handling, visibility flags |
|
||||
| `object_drawer.cc` | Buffer routing in `DrawObject()`, `RoutineDrawsToBothBGs()` |
|
||||
| `room.h` | `LayerMergeType` definitions (Section 2.2 of this doc) |
|
||||
| `background_buffer.cc` | `DrawTile()` priority writes, transparency handling |
|
||||
|
||||
### 9.5 Recommended Fixes to Try
|
||||
|
||||
1. **Add logging to composite loop:**
|
||||
```cpp
|
||||
// In CompositeWithPriority lambda, add:
|
||||
static int debug_count = 0;
|
||||
if (debug_count++ < 100 && !IsTransparent(src_pixel)) {
|
||||
printf("[Composite] layer=%d idx=%d pixel=%d prio=%d\n",
|
||||
static_cast<int>(layer_type), idx, src_pixel, src_prio);
|
||||
}
|
||||
```
|
||||
|
||||
2. **Verify BG2 visibility in composite:**
|
||||
- Ensure `IsLayerVisible(LayerType::BG2_Layout)` returns true for Normal merge
|
||||
- Check if `Layer2Visible` from ROM is being incorrectly overridden
|
||||
|
||||
3. **Check for early-exit conditions:**
|
||||
- Search for `return` statements in `CompositeWithPriority` that might skip layers
|
||||
- Verify `blend_mode == LayerBlendMode::Off` check isn't incorrectly triggered
|
||||
|
||||
### 9.6 Test Cases
|
||||
|
||||
After making fixes, verify with these room types:
|
||||
- Room with layer_merging ID 0x06 (Normal) - objects should appear
|
||||
- Room with layer_merging ID 0x00 (Off) - BG2 should still be visible
|
||||
- Room with BothBG objects (walls, corners) - should render correctly
|
||||
- Dark room (ID 0x08) - should have correct dimming
|
||||
|
||||
### 9.7 Success Criteria
|
||||
|
||||
- Objects visible in separate mode should also be visible in composite mode
|
||||
- Priority toggle (P checkbox) should only affect Z-ordering, not visibility
|
||||
- No regression in rooms that currently render correctly
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
29
docs/internal/agents/doc-hygiene.md
Normal file
29
docs/internal/agents/doc-hygiene.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# Documentation Hygiene & Spec Rules
|
||||
|
||||
Purpose: keep `docs/internal` lean, discoverable, and aligned with active work. Use this when creating or updating any internal spec/plan.
|
||||
|
||||
## Canonical sources first
|
||||
- Check existing coverage: update an existing spec before adding a new file. Search `docs/internal` for keywords and reuse templates.
|
||||
- One source of truth per initiative: tie work to a board entry and, if multi-day, a single spec (e.g., `initiative-*.md` or a plan under `docs/internal/plans/`).
|
||||
- Ephemeral task lists live in `z3ed agent todo` or the coordination board, not new Markdown stubs.
|
||||
|
||||
## Spec & plan shape
|
||||
- Header block: `Status (ACTIVE/IN_PROGRESS/ARCHIVE)`, `Owner (Agent ID)`, `Created`, `Last Reviewed`, `Next Review` (≤14 days by default), and link to the coordination-board entry.
|
||||
- Keep the front page tight: Summary, Decisions/Constraints, Deliverables, and explicit Exit Criteria. Push deep design notes into appendices.
|
||||
- Use the templates: `initiative-template.md` for multi-day efforts, `release-checklist-template.md` for releases. Avoid custom one-off formats.
|
||||
|
||||
## Lifecycle & archiving
|
||||
- Archive on completion/abandonment: move finished or idle (>14 days without update) specs to `docs/internal/agents/archive/` (or the relevant sub-archive). Add the date to the filename when moving.
|
||||
- Consolidate duplicates: if multiple docs cover the same area, merge into the newest spec and drop redirects into the archive with a short pointer.
|
||||
- Weekly sweep: during board maintenance, prune outdated docs referenced on the board and archive any that no longer have an active entry.
|
||||
|
||||
## Coordination hygiene
|
||||
- Every spec links back to the coordination board entry; every board entry points to its canonical spec. No parallel shadow docs.
|
||||
- Release/initiative docs must include a test/validation section instead of ad-hoc checklists scattered across files.
|
||||
- When adding references in other docs, point to the canonical spec instead of copying sections.
|
||||
|
||||
## Anti-spam guardrails
|
||||
- No gamified/leaderboard or duplicative status pages in `agents/`—keep status in the board and the canonical spec.
|
||||
- Prefer updating `docs/internal/README.md` or the nearest index with short summaries instead of creating new directories.
|
||||
- Cap new doc creation per initiative to one spec + one handoff; everything else belongs in comments/PRs or the board.
|
||||
- Filenames: avoid ALL-CAPS except established anchors (README, AGENTS, GEMINI, CLAUDE, CONTRIBUTING, etc.); use kebab-case for new docs.
|
||||
304
docs/internal/agents/draw_routine_tracker.md
Normal file
304
docs/internal/agents/draw_routine_tracker.md
Normal file
@@ -0,0 +1,304 @@
|
||||
# Dungeon Draw Routine Tracker
|
||||
|
||||
**Status:** Active
|
||||
**Owner:** dungeon-rendering-specialist
|
||||
**Created:** 2025-12-07
|
||||
**Last Reviewed:** 2025-12-09
|
||||
**Next Review:** 2025-12-23
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
This document is the single source of truth for dungeon object draw routine implementation status. It consolidates information from previous handoff documents and provides a comprehensive reference for fixing remaining issues.
|
||||
|
||||
---
|
||||
|
||||
## Recent Changes (2025-12-09)
|
||||
|
||||
| Change | Details |
|
||||
|--------|---------|
|
||||
| Unified draw routine registry | Created `DrawRoutineRegistry` singleton for ObjectDrawer/ObjectGeometry parity |
|
||||
| BG1 mask rectangle API | Added `GeometryBounds::GetBG1MaskRect()` and `RequiresBG1Mask()` for 94 affected rooms |
|
||||
| Fixed vertical rails 0x8A-0x8C | Applied CORNER+MIDDLE+END pattern matching horizontal rail 0x22 |
|
||||
| Custom objects 0x31/0x32 | Registered with routine 130 for Oracle of Secrets minecart tracks |
|
||||
| Selection bounds for diagonals | Added `selection_bounds` field for tighter hit testing on 0xA0-0xA3 |
|
||||
| Audited 0x233 staircase | Confirmed 32x32 (4x4 tile) dimensions correct per ASM |
|
||||
|
||||
## Previous Changes (2025-12-07)
|
||||
|
||||
| Change | Details |
|
||||
|--------|---------|
|
||||
| Fixed routine 16 dimensions | `DrawRightwards4x4_1to16` now correctly calculates `width = 32 * count` based on size |
|
||||
| Added BG2 mask propagation logging | Debug output shows when Layer 1 objects trigger floor masking |
|
||||
| BG2 masking research complete | 94 rooms affected, fix implemented in `MarkBG1Transparent` |
|
||||
|
||||
---
|
||||
|
||||
## Completed Fixes
|
||||
|
||||
| Object ID | Name | Issue | Fix Applied |
|
||||
|-----------|------|-------|-------------|
|
||||
| 0x5E | Block | Inverted tile ordering | Fixed column-major order |
|
||||
| 0x5D/0x88 | Thick Rails | Repeated edges | Fixed cap-middle-cap pattern |
|
||||
| 0xF99 | Chest | Repeated based on size | Changed to DrawSingle2x2 |
|
||||
| 0xFB1 | Big Chest | Repeated based on size | Changed to DrawSingle4x3 |
|
||||
| 0xF92 | Blue Rupees | Not correct at all | Implemented DrawRupeeFloor per ASM |
|
||||
| 0xFED | Water Grate | Wrong outline, repeated | Changed to DrawSingle4x3 |
|
||||
| 0x3A | Wall Decors | Spacing wrong (6 tiles) | Fixed to 8 tiles per ASM |
|
||||
| 0x39/0x3D | Pillars | Spacing wrong (6 tiles) | Fixed to 4 tiles per ASM |
|
||||
| 0xFEB | Large Decor | Outline too small | Fixed to 64x64 (4x4 tile16s) |
|
||||
| 0x138-0x13B | Spiral Stairs | Wrong 4x4 pattern | Fixed to 4x3 per ASM |
|
||||
| 0xA0-0xAC | Diagonal Ceilings | Vertical line instead of triangle | Fixed triangle fill pattern |
|
||||
| 0xC0/0xC2 etc | SuperSquare | Fixed 32x32 dimensions | Now uses size parameter |
|
||||
| 0xFE6 | Pit | Should not repeat | Uses DrawActual4x4 (32x32, no repeat) |
|
||||
| 0x55-0x56 | Wall Torches | Wrong pattern | Fixed to 1x8 column with 12-tile spacing |
|
||||
| 0x22 | Small Rails | Internal parts repeat | Now CORNER+MIDDLE*count+END pattern |
|
||||
| 0x23-0x2E | Carpet Trim | Wrong pattern | Now CORNER+MIDDLE*count+END pattern |
|
||||
| 0x033 | Floor 4x4 | Wrong BG2 mask size | Fixed dimension to use size parameter |
|
||||
| 0x12D-0x12F | InterRoom Fat Stairs | Wrong dimensions (32x24) | Fixed to 32x32 (4x4 tiles) |
|
||||
| 0x130-0x133 | Auto Stairs | Wrong dimensions (32x24) | Fixed to 32x32 (4x4 tiles) |
|
||||
| 0xF9E-0xFA9 | Straight InterRoom Stairs | Wrong dimensions (32x24) | Fixed to 32x32 (4x4 tiles) |
|
||||
| 0x8A-0x8C | Vertical Rails | Using wrong horizontal routine | Fixed CORNER+MIDDLE+END pattern |
|
||||
| 0x31/0x32 | Custom Objects | Not registered in draw routine map | Registered with routine 130 |
|
||||
| 0x233 | AutoStairsSouthMergedLayer | Need audit | Confirmed 32x32 (4x4 tiles) correct |
|
||||
| 0xDC | OpenChestPlatform | Layer check missing | Added layer handling documentation |
|
||||
|
||||
---
|
||||
|
||||
## In Progress / Known Issues
|
||||
|
||||
| Object ID | Name | Issue | Status |
|
||||
|-----------|------|-------|--------|
|
||||
| 0x00 | Ceiling | Outline should be like 0xC0 | Needs different dimension calc |
|
||||
| 0xC0 | Large Ceiling | SuperSquare routine issues | Needs debug |
|
||||
| 0x22 vs 0x8A-0x8E | Rails | Horizontal fixed, vertical needs match | Medium priority |
|
||||
| 0xA0-0xA3 | Diagonal Ceilings | Outline too large for selection | May need UI-level fix |
|
||||
| 0x3D | Torches | Top half draws pegs | ROM tile data issue |
|
||||
| 0x95/0x96 | Vertical Pegs | Outline appears square | May be UI issue |
|
||||
|
||||
---
|
||||
|
||||
## Pending Fixes
|
||||
|
||||
| Object ID | Name | Issue | Priority |
|
||||
|-----------|------|-------|----------|
|
||||
| Pit Edges | Various | Single tile thin based on direction | Low |
|
||||
|
||||
**Note:** Staircase objects 0x12D-0x137 (Fat/Auto Stairs), Type 3 stairs (0xF9E-0xFA9), and 0x233 (AutoStairsSouthMergedLayer) have been audited as of 2025-12-09. All use 32x32 (4x4 tile) dimensions matching the ASM.
|
||||
|
||||
---
|
||||
|
||||
## Size Calculation Formulas (from ASM)
|
||||
|
||||
| Routine | Formula |
|
||||
|---------|---------|
|
||||
| GetSize_1to16 | `count = (size & 0x0F) + 1` |
|
||||
| GetSize_1to15or26 | `count = size; if 0, count = 26` |
|
||||
| GetSize_1to15or32 | `count = size; if 0, count = 32` |
|
||||
| GetSize_1to16_timesA | `count = (size & 0x0F + 1) * A` |
|
||||
| SuperSquare | `size_x = (size & 0x0F) + 1; size_y = ((size >> 4) & 0x0F) + 1` |
|
||||
|
||||
---
|
||||
|
||||
## Draw Routine Architecture
|
||||
|
||||
### Routine ID Ranges
|
||||
|
||||
- **0-55**: Basic patterns (2x2, 4x4, edges, etc.)
|
||||
- **56-64**: SuperSquare patterns
|
||||
- **65-82**: Decorations, pots, pegs, platforms
|
||||
- **83-91**: Stairs
|
||||
- **92-115**: Special/interactive objects
|
||||
|
||||
### New Routines Added (Phase 2)
|
||||
|
||||
- **Routine 113**: DrawSingle4x4 - Single 4x4 block, no repetition
|
||||
- **Routine 114**: DrawSingle4x3 - Single 4x3 block, no repetition
|
||||
- **Routine 115**: DrawRupeeFloor - Special 6x8 pattern with gaps
|
||||
|
||||
### Tile Ordering
|
||||
|
||||
Most routines use **COLUMN-MAJOR** order: tiles advance down each column, then right to next column.
|
||||
|
||||
### Rail Pattern Structure
|
||||
```
|
||||
[CORNER tile 0] -> [MIDDLE tile 1 × count] -> [END tile 2]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## BG2 Masking (Layer Compositing)
|
||||
|
||||
### Overview
|
||||
|
||||
94 rooms have Layer 1 (BG2) overlay objects that need to show through BG1 floor tiles.
|
||||
|
||||
### Implementation
|
||||
|
||||
When a Layer 1 object is drawn to BG2 buffer, `MarkBG1Transparent` is called to mark the corresponding BG1 pixels as transparent (255), allowing BG2 content to show through during compositing.
|
||||
|
||||
### Affected Objects (Example: Room 001)
|
||||
|
||||
| Object ID | Position | Size | Dimensions |
|
||||
|-----------|----------|------|------------|
|
||||
| 0x033 | (22,13) | 4 | 160×32 px (5 × 4x4 blocks) |
|
||||
| 0x034 | (23,16) | 14 | 144×8 px (18 × 1x1 tiles) |
|
||||
| 0x071 | (22,13) | 0 | 8×32 px (1×4 tiles) |
|
||||
| 0x038 | (24,12) | 1 | 48×24 px (2 statues) |
|
||||
| 0x13B | (30,10) | 0 | 32×24 px (spiral stairs) |
|
||||
|
||||
### Testing
|
||||
|
||||
Use `scripts/analyze_room.py` to identify Layer 1 objects:
|
||||
```bash
|
||||
python3 scripts/analyze_room.py --rom roms/alttp_vanilla.sfc 1 --compositing
|
||||
python3 scripts/analyze_room.py --rom roms/alttp_vanilla.sfc --list-bg2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/zelda3/dungeon/object_drawer.cc` | Main draw routines and ID mapping |
|
||||
| `src/zelda3/dungeon/object_drawer.h` | Draw routine declarations |
|
||||
| `src/zelda3/dungeon/draw_routines/special_routines.cc` | Complex special routines |
|
||||
| `src/zelda3/dungeon/room_layer_manager.cc` | Layer compositing |
|
||||
| `assets/asm/usdasm/bank_01.asm` | Reference ASM disassembly |
|
||||
| `scripts/analyze_room.py` | Room object analysis tool |
|
||||
|
||||
---
|
||||
|
||||
## Validation Criteria
|
||||
|
||||
1. Outline matches expected dimensions per ASM calculations
|
||||
2. Draw pattern matches visual appearance in original game
|
||||
3. Tile ordering correct (column-major vs row-major)
|
||||
4. Repetition behavior correct (single draw vs size-based repeat)
|
||||
5. BG2 overlay objects visible through BG1 floor
|
||||
6. No build errors after changes
|
||||
|
||||
---
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
- [ ] All high-priority issues resolved
|
||||
- [ ] Medium-priority issues documented with investigation notes
|
||||
- [ ] BG2 masking working for all 94 affected rooms
|
||||
- [ ] Staircase audit complete (0x12D-0x233)
|
||||
- [ ] No regression in existing working routines
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Layer Merge Type Effects
|
||||
|
||||
### Current Implementation Status
|
||||
|
||||
| Merge ID | Name | Effect | Status |
|
||||
|----------|------|--------|--------|
|
||||
| 0x00 | Off | BG2 visible, no effects | Working |
|
||||
| 0x01 | Parallax | BG2 visible, parallax scroll | Not implemented |
|
||||
| 0x02 | Dark | BG2 translucent blend | Simplified (no true blend) |
|
||||
| 0x03 | On top | BG2 hidden but in subscreen | Partial |
|
||||
| 0x04 | Translucent | BG2 translucent | Simplified |
|
||||
| 0x05 | Addition | Additive blending | Simplified |
|
||||
| 0x06 | Normal | Standard dungeon | Working |
|
||||
| 0x07 | Transparent | Water/fog effect | Simplified |
|
||||
| 0x08 | Dark room | Master brightness 50% | Working (SDL color mod) |
|
||||
|
||||
### Known Limitations
|
||||
|
||||
1. **Translucent Blending (0x02, 0x04, 0x05, 0x07)**: Currently uses threshold-based pixel copying instead of true alpha blending. True blending would require RGB palette lookups which is expensive for indexed color mode.
|
||||
|
||||
2. **Dark Room Effect (0x08)**: Implemented via `SDL_SetSurfaceColorMod(surface, 128, 128, 128)` which reduces brightness to 50%. Applied after compositing.
|
||||
|
||||
3. **Parallax Scrolling (0x01)**: Not implemented - would require separate layer offset during rendering.
|
||||
|
||||
### Implementation Details
|
||||
|
||||
**Dark Room (0x08):**
|
||||
```cpp
|
||||
// In room_layer_manager.cc CompositeToOutput()
|
||||
if (current_merge_type_id_ == 0x08) {
|
||||
SDL_SetSurfaceColorMod(output.surface(), 128, 128, 128);
|
||||
}
|
||||
```
|
||||
|
||||
**Translucent Layers:**
|
||||
```cpp
|
||||
// In ApplyLayerMerging()
|
||||
if (merge_type.Layer2Translucent) {
|
||||
SetLayerBlendMode(LayerType::BG2_Layout, LayerBlendMode::Translucent);
|
||||
SetLayerBlendMode(LayerType::BG2_Objects, LayerBlendMode::Translucent);
|
||||
}
|
||||
```
|
||||
|
||||
### Future Improvements
|
||||
|
||||
- Implement true alpha blending for translucent modes (requires RGB conversion)
|
||||
- Add parallax scroll offset support for merge type 0x01
|
||||
- Consider per-scanline HDMA effects simulation
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Custom Objects (Oracle of Secrets)
|
||||
|
||||
### Status: Not Working
|
||||
|
||||
Custom objects (IDs 0x31, 0x32) use external binary files instead of ROM tile data. These are used for minecart tracks and custom furniture in Oracle of Secrets.
|
||||
|
||||
### Issues
|
||||
|
||||
| Issue | Description |
|
||||
|-------|-------------|
|
||||
| Routine not registered | 0x31/0x32 have no entry in `object_to_routine_map_` |
|
||||
| DrawCustomObject incomplete | Routine exists but isn't called |
|
||||
| Previews don't work | Object selector can't preview custom objects |
|
||||
|
||||
### Required Fixes
|
||||
|
||||
1. **Register routine in InitializeDrawRoutines():**
|
||||
```cpp
|
||||
object_to_routine_map_[0x31] = CUSTOM_ROUTINE_ID;
|
||||
object_to_routine_map_[0x32] = CUSTOM_ROUTINE_ID;
|
||||
```
|
||||
|
||||
2. **Add draw routine to registry:**
|
||||
```cpp
|
||||
draw_routines_.push_back([](ObjectDrawer* self, ...) {
|
||||
self->DrawCustomObject(obj, bg, tiles, state);
|
||||
});
|
||||
```
|
||||
|
||||
3. **Ensure CustomObjectManager initialized before drawing**
|
||||
|
||||
### Project Configuration
|
||||
|
||||
```ini
|
||||
[files]
|
||||
custom_objects_folder=/path/to/Dungeons/Objects/Data
|
||||
|
||||
[feature_flags]
|
||||
enable_custom_objects=true
|
||||
```
|
||||
|
||||
### Full Documentation
|
||||
|
||||
See [`HANDOFF_CUSTOM_OBJECTS.md`](../hand-off/HANDOFF_CUSTOM_OBJECTS.md) for complete details.
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [`dungeon-object-rendering-spec.md`](dungeon-object-rendering-spec.md) - ASM-based rendering specification
|
||||
- [`HANDOFF_BG2_MASKING_FIX.md`](../hand-off/HANDOFF_BG2_MASKING_FIX.md) - BG2 masking implementation details
|
||||
- [`HANDOFF_CUSTOM_OBJECTS.md`](../hand-off/HANDOFF_CUSTOM_OBJECTS.md) - Custom objects system
|
||||
- [`dungeon-layer-compositing-research.md`](../plans/dungeon-layer-compositing-research.md) - Layer system research
|
||||
- [`composite-layer-system.md`](composite-layer-system.md) - Layer compositing implementation details
|
||||
|
||||
108
docs/internal/agents/dungeon-object-rendering-spec.md
Normal file
108
docs/internal/agents/dungeon-object-rendering-spec.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# Dungeon Object Rendering & Selection Spec (ALTTP)
|
||||
|
||||
Status: ACTIVE
|
||||
Owner: zelda3-hacking-expert
|
||||
Created: 2025-12-06
|
||||
Last Reviewed: 2025-12-06
|
||||
Next Review: 2025-12-20
|
||||
Board: docs/internal/agents/coordination-board.md (2025-12-06 zelda3-hacking-expert – Dungeon object render/selection spec)
|
||||
|
||||
## Scope
|
||||
- Source of truth: `assets/asm/usdasm/bank_01.asm` (US 1.0 disasm) plus room headers in the same bank.
|
||||
- Goal: spell out how layouts and objects are drawn, how layers are selected/merged, and how object symbology should match the real draw semantics (arrows, “large”/4x4 growth, BothBG).
|
||||
- Pain points to fix: corner ceilings and ceiling variants (4x4, vertical 2x2, horizontal 2x2), BG merge vs layer type treated as exclusive, layout objects occasionally drawing over background objects, and selection outlines that do not match the real footprint.
|
||||
|
||||
## Room Build & Layer Order (bank_01.asm)
|
||||
- `LoadAndBuildRoom` (`assets/asm/usdasm/bank_01.asm:$01873A`):
|
||||
1) `LoadRoomHeader` ($01B564) pulls header bits: blockset/light bits to `$0414`, layer type bits to `$046C`, merge/effect bits to `$063C-$063F/$0640`, palette/spriteset/tag bytes immediately after.
|
||||
2) `RoomDraw_DrawFloors` ($0189DC): uses the first room word. High nibble → `$0490` (BG2 floor set), low nibble → `$046A` (BG1 floor set). Draws 4×4 quadrant “super squares” through `RoomDraw_FloorChunks`, targeting BG2 pointers first then BG1 pointers.
|
||||
3) Layout pointer: reads the next byte at `$B7+BA` into `$040E`, converts to a 3-byte pointer via `RoomLayoutPointers`, resets `BA=0`, and runs `RoomDraw_DrawAllObjects` on that layout list (this is the template layer; it should stay underneath everything else).
|
||||
4) Primary room objects: restores the room’s object pointer (`RoomData_ObjectDataPointers`) and runs `RoomDraw_DrawAllObjects` again (BA now points past the layout byte).
|
||||
5) BG2 overlay list: skips the `0xFFFF` sentinel (`INC BA` twice), reloads pointer tables with `RoomData_TilemapPointers_lower_layer`, and draws a third object list to BG2.
|
||||
6) BG1 overlay list: skips the next `0xFFFF`, reloads pointer tables with `RoomData_TilemapPointers_upper_layer`, and draws the final object list to BG1.
|
||||
7) Pushable blocks (`$7EF940`) and torches (`$7EFB40`) are drawn after the four passes.
|
||||
- Implication: BG merge and layer type are **not** exclusive—four object streams are processed in order, with explicit pointer swaps for BG2 then BG1 overlays. Layout objects should never overdraw later passes; if they do in the editor, the pass order is wrong.
|
||||
|
||||
## Object Encoding (RoomDraw_RoomObject at $01893C)
|
||||
- Type detection:
|
||||
- Type 2 sentinel: low byte `>= $FC` triggers `.subtype_2` ($018983).
|
||||
- Type 3 sentinel: object ID `>= $F8` triggers `.subtype_3` ($0189B8) after the ID is loaded.
|
||||
- Type 1: everything else (standard 3-byte objects).
|
||||
- Type 1 format (`xxxxxxss | yyyyyyss | id`):
|
||||
- `x = (byte0 & 0xFC) >> 2`, `y = (byte1 & 0xFC) >> 2` (tile-space, 0–63).
|
||||
- `size_nibble = ((byte0 & 0x03) << 2) | (byte1 & 0x03)` (0–15).
|
||||
- ID = byte2.
|
||||
- Size helpers (ground truth for outline math):
|
||||
- `RoomDraw_GetSize_1to16(_timesA)` at $01B0AC: `size = nibble + A` (`A=1` for most routines; diagonal ceilings pass `A=4` to force a 4-tile base span).
|
||||
- `RoomDraw_GetSize_1to15or26` at $01B0BE: nibble 0 → 26 tiles; otherwise nibble value.
|
||||
- `RoomDraw_GetSize_1to15or32` at $01B0CC: nibble 0 → 32 tiles; otherwise nibble value.
|
||||
- After calling any helper: `$B2` holds the final count; `$B4` is cleared.
|
||||
- Type 2 format (`byte0 >= $FC`) uses tables at `.type2_data_offset` ($0183F0) and `.type2_routine` ($018470). No size field; fixed dimensions per routine.
|
||||
- Type 3 format (`id >= $F8`) uses `.type3_data_offset` ($0184F0) and `.type3_routine` ($0185F0). Some use size nibble (e.g., Somaria lines); most are fixed-size objects like chests and stair blocks.
|
||||
|
||||
## Draw Routine Families & Expected Symbology
|
||||
- Type 1 routine table: `.type1_routine` at `$018200`.
|
||||
- `Rightwards*` → arrow right; grows horizontally by `size` blocks. Base footprints: `2x4`, `2x2`, `4x4`, etc. Spacing suffix (`spaced2/4/8/12`) means step that many tiles between columns.
|
||||
- `Downwards*` → arrow down; grows vertically by `size` blocks with the same spacing conventions.
|
||||
- `DiagonalAcute/Grave` → 45° diagonals; use diagonal arrow/corner icon. Size = nibble+1 tiles long (helper is `1to16`). `_BothBG` variants must draw to both BG1 and BG2.
|
||||
- `DiagonalCeiling*` (IDs 0xA0–0xAC): size = nibble + 4 (`GetSize_1to16_timesA` with `A=4`). Bounding box is square (`size × size`) because each step moves x+y by 1.
|
||||
- `4x4Floor*/Blocks*/SuperSquare` (IDs 0xC0–0xCA, 0xD1–0xE8): use “large square” icon. They tile 4×4 blocks inside 16×16 “super squares” (128×128 pixels) and do **not** use the size nibble—dimensions are fixed per routine.
|
||||
- `Edge/Corner` variants: use L-corner or edge glyph; many have `_BothBG` meaning they write to BG1 and BG2 simultaneously (should not be layer-exclusive).
|
||||
- Type 2 routines (`.type2_routine`):
|
||||
- IDs 0x108–0x117: `RoomDraw_4x4Corner_BothBG` and “WeirdCorner*” draw to both layers—icon should denote dual-layer.
|
||||
- IDs 0x12D–0x133: inter-room fat stairs (A/B) and auto-stairs north (multi-layer vs merged) explicitly encode whether they target both layers or a merged layer; UI must not force exclusivity.
|
||||
- IDs 0x135–0x13F: water-hop stairs, spiral stairs, sanctuary wall, magic bat altar—fixed-size, no size nibble.
|
||||
- Type 3 routines (`.type3_routine`):
|
||||
- Chests/big chests (0x218–0x232) are single 1×1 anchors; selection should stay 1 tile.
|
||||
- Somaria lines (0x203–0x20C/0x20E) use the size nibble as a tile count; they extend along X with no Y growth.
|
||||
- Pipes (0x23A–0x23D) are fixed 2×? rectangles; use arrows that match their orientation.
|
||||
|
||||
## Ceiling and Large Object Ground Truth
|
||||
- Corner/diagonal ceilings (Type 1 IDs 0xA0–0xAC): `RoomDraw_DiagonalCeiling*` ($018BE0–$018C36). Size = nibble+4; outline should be a square whose side equals that size; growth is along the diagonal (x+1,y+1 per step).
|
||||
- Big hole & overlays: ID 0xA4 → `RoomDraw_BigHole4x4_1to16` (fixed 4×4). IDs 0xD8/0xDA → `RoomDraw_WaterOverlayA/B8x8_1to16` (fixed 8×8 overlay; should remain on BG2 overlay pass).
|
||||
- 4x4 ceilings/floors: IDs 0xC5–0xCA, 0xD1–0xD2, 0xD9, 0xDF–0xE8 → `RoomDraw_4x4FloorIn4x4SuperSquare` (fixed 4×4 tiles repeated in super squares). Use “large square” glyph; ignore size nibble.
|
||||
- 2x2 ceilings:
|
||||
- Horizontal/right-growing: IDs 0x07–0x08 and 0xB8–0xB9 use `RoomDraw_Rightwards2x2_*` (size-driven width, height=2). Arrow right, outline width = `2 * size`, height = 2.
|
||||
- Vertical/down-growing: IDs 0x060 and 0x092–0x093 use `RoomDraw_Downwards2x2_*` (size-driven height, width=2). Arrow down, outline height = `2 * size`, width = 2. When the size nibble is 0 (`1to15or32`), treat size as 32 for bounds.
|
||||
|
||||
## Layer Merge Semantics
|
||||
- Header bits at `LoadRoomHeader` ($01B5F4–$01B683):
|
||||
- Bits 5–7 of header byte 0 → `$0414` (blockset/light flags).
|
||||
- Bits 2–4 → `$046C` (layer type selector used later when building draw state).
|
||||
- Bits 0–1 of later header bytes → `$063C-$0640` (effect/merge/tag flags).
|
||||
- `RoomDraw_DrawAllObjects` is run four times with different tilemap pointer tables; `_BothBG` routines ignore the active pointer swap and write to both buffers. The editor must allow “BG merge” and “layer type” to coexist; never force a mutually exclusive radio button.
|
||||
- Ordering for correctness:
|
||||
1) Floors (BG2 then BG1)
|
||||
2) Layout list (BG2)
|
||||
3) Main list (BG2 by default, unless the routine itself writes both)
|
||||
4) BG2 overlay list (after first `0xFFFF`)
|
||||
5) BG1 overlay list (after second `0xFFFF`)
|
||||
6) Pushable blocks and torches
|
||||
|
||||
## Selection & Outline Rules
|
||||
- Use the decoding rules above; do not infer size from UI icons.
|
||||
- Type 1 size nibble:
|
||||
- Standard (`1to16`): `size = nibble + 1`.
|
||||
- `1to15or26`: nibble 0 → size 26.
|
||||
- `1to15or32`: nibble 0 → size 32.
|
||||
- Diagonal ceilings: size = nibble + 4; outline is a square of that many tiles.
|
||||
- Base footprints:
|
||||
- `2x4` routines: width=2, height=4; repeated along the growth axis.
|
||||
- `2x2` routines: width=2, height=2; repeated along the growth axis.
|
||||
- `4x4` routines: width=4, height=4; ignore size nibble unless the routine name includes `1to16`.
|
||||
- Super-square routines: treat as 16×16 tiles when computing selection bounds (they stamp 4×4 blocks into a 32×32-tile area).
|
||||
- `_BothBG` routines should carry a dual-layer badge in the palette and never be filtered out by the current layer toggle—selection must remain visible regardless of BG toggle because the object truly occupies both buffers.
|
||||
|
||||
## Mapping UI Symbology to Real Objects
|
||||
- Arrows right/left: any `Rightwards*` routine; growth = size nibble (with fallback rules above). Use “large” badge only when the routine name includes `4x4` or `SuperSquare`.
|
||||
- Arrows down/up: any `Downwards*` routine; same sizing rules.
|
||||
- Diagonal arrow: `DiagonalAcute/Grave` and `DiagonalCeiling*`.
|
||||
- Large square badge: `4x4Floor*`, `4x4Blocks*`, `BigHole4x4`, water overlays, chest platforms; these do **not** change size with the nibble.
|
||||
- Dual-layer badge: routines with `_BothBG` in the disasm name, plus `AutoStairs*MergedLayer*` (IDs 0x132–0x133, 0x233). These must be allowed even when a “merged BG” flag is set in the header.
|
||||
|
||||
## Action Items for the Editor
|
||||
- Enforce the build order above so layout objects never sit above BG overlays; respect the two post-`0xFFFF` lists for BG2/BG1 overlays.
|
||||
- Update selection bounds to honor the size helpers (including the nibble-zero fallbacks and the `+4` base for diagonal ceilings).
|
||||
- Mark BothBG/merged-layer routines so layer toggles do not hide or exclude them.
|
||||
- Align palette/symbology labels with the disasm names: arrows for `Rightwards/Downwards`, diagonal for `Diagonal*`, large-square for `4x4*/SuperSquare`, dual-layer for `_BothBG`/merged stairs.
|
||||
|
||||
59
docs/internal/agents/dungeon-palette-fix-plan.md
Normal file
59
docs/internal/agents/dungeon-palette-fix-plan.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Dungeon Palette Fix Plan (ALTTP)
|
||||
|
||||
Status: COMPLETED
|
||||
Owner: zelda3-hacking-expert
|
||||
Created: 2025-12-06
|
||||
Last Reviewed: 2025-12-10
|
||||
Completed: 2025-12-10
|
||||
Board: docs/internal/agents/coordination-board.md (dungeon palette follow-on to render/selection spec)
|
||||
|
||||
## Goal
|
||||
Align dungeon palette loading/rendering with usdasm: correct pointer math, 16-color chunking, transparency, and consistent BG1/BG2/object palettes.
|
||||
|
||||
## Scope
|
||||
- Files: `src/zelda3/dungeon/room.cc`, `game_data` palette setup, `ObjectDrawer` palette usage.
|
||||
- Reference: `assets/asm/usdasm/bank_01.asm` (palette pointers at $0DEC4B) and `docs/internal/agents/dungeon-object-rendering-spec.md`.
|
||||
|
||||
## Tasks
|
||||
1) Verify palette set construction
|
||||
- Confirm `game_data_->palette_groups.dungeon_main` has the right count/order (20 sets × 16 colors) matching usdasm pointers.
|
||||
2) Palette ID derivation
|
||||
- Keep pointer lookup: `paletteset_ids[palette][0]` byte offset → word at `kDungeonPalettePointerTable + offset` → divide by 180 (0xB4) to get set index. Add assertions/logging on out-of-range.
|
||||
3) SDL palette mapping
|
||||
- Build the SDL palette in 16-color chunks: chunk n → indices `[n*16..n*16+15]`, with index `n*16+0` transparent/colorkey. Stop treating “90-color” as linear; respect chunk boundaries.
|
||||
- Use the same mapped palette for bg1/bg2/object buffers (header selects one set for the room).
|
||||
4) Transparency / colorkey
|
||||
- Set colorkey on each chunk’s index 0 (or keep 255 but ensure chunk 0 is unused) and avoid shifting palette indices in `WriteTile8` paths.
|
||||
5) Diagnostics
|
||||
- Log palette_id, pointer, and first color when palette is applied; add a debug mode to dump chunk boundaries for quick verification.
|
||||
|
||||
## Exit Criteria
|
||||
- Dungeon rooms render with correct colors across BG1/BG2/objects; palette_id derivation matches usdasm; SDL palette chunking aligns with 16-color boundaries; transparency behaves consistently.
|
||||
|
||||
## Implementation Summary (2025-12-10)
|
||||
|
||||
### Changes Made:
|
||||
|
||||
1. **`room.cc:665-718`** - Palette mapping now uses 16-color banks:
|
||||
- ROM colors `[N*15 .. N*15+14]` → SDL indices `[N*16+1 .. N*16+15]`
|
||||
- Index `N*16` in each bank is transparent (matches SNES CGRAM)
|
||||
- Unified palette applied to all 4 buffers (bg1, bg2, object_bg1, object_bg2)
|
||||
|
||||
2. **`background_buffer.cc:120-137,161-164`** - Updated drawing formula:
|
||||
- Changed `palette_offset = (pal - 2) * 15` → `(pal - 2) * 16`
|
||||
- Changed `final_color = (pixel - 1) + palette_offset` → `pixel + palette_offset`
|
||||
- Pixel 0 = transparent (not written), pixels 1-15 map to bank indices 1-15
|
||||
|
||||
3. **`object_drawer.cc:4095-4133`** - Same 16-color bank chunking for object rendering
|
||||
|
||||
4. **`room_layer_manager.h:461-466`** - Updated documentation comments
|
||||
|
||||
### Key Formula:
|
||||
```
|
||||
SDL Bank N (N=0-5): indices [N*16 .. N*16+15]
|
||||
- Index N*16 = transparent (SNES CGRAM row N, color 0)
|
||||
- Indices N*16+1 to N*16+15 = actual colors
|
||||
|
||||
Tile rendering: final_color = pixel + (bank * 16)
|
||||
where pixel ∈ [1,15] and bank = (palette_bits - 2)
|
||||
```
|
||||
@@ -1,271 +0,0 @@
|
||||
# Initiative: YAZE v0.4.0 - SDL3 Modernization & Emulator Accuracy
|
||||
|
||||
**Created**: 2025-11-23
|
||||
**Owner**: Multi-agent coordination
|
||||
**Status**: ACTIVE
|
||||
**Target Release**: Q1 2026
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
YAZE v0.4.0 represents a major release focusing on two pillars:
|
||||
1. **Emulator Accuracy** - Implementing cycle-accurate PPU rendering and AI integration
|
||||
2. **SDL3 Modernization** - Migrating from SDL2 to SDL3 with backend abstractions
|
||||
|
||||
This initiative coordinates 7 specialized agents across 5 parallel workstreams.
|
||||
|
||||
---
|
||||
|
||||
## Background
|
||||
|
||||
### Current State (v0.3.8-hotfix1)
|
||||
- AI agent infrastructure complete (z3ed CLI)
|
||||
- Card-based UI system functional
|
||||
- Emulator debugging framework established
|
||||
- CI/CD pipeline stabilized with nightly testing
|
||||
- Known issues: Tile16 palette, overworld sprite movement, emulator audio
|
||||
|
||||
### Uncommitted Work Ready for Integration
|
||||
- PPU JIT catch-up system (`ppu.cc` - 29 lines added)
|
||||
- Dungeon room sprite encoding/saving (`room.cc` - 82 lines added)
|
||||
- Dungeon editor system improvements (133 lines added)
|
||||
- Test suite configuration updates
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
### Milestone 1: Emulator Accuracy (Weeks 1-6)
|
||||
|
||||
#### 1.1 PPU JIT Catch-up Completion
|
||||
**Agent**: `snes-emulator-expert`
|
||||
**Status**: IN_PROGRESS (uncommitted work exists)
|
||||
**Files**: `src/app/emu/video/ppu.cc`, `src/app/emu/video/ppu.h`
|
||||
|
||||
**Tasks**:
|
||||
- [x] Add `last_rendered_x_` tracking
|
||||
- [x] Implement `StartLine()` method
|
||||
- [x] Implement `CatchUp(h_pos)` method
|
||||
- [ ] Integrate `CatchUp()` calls into `Snes::WriteBBus`
|
||||
- [ ] Add unit tests for mid-scanline register writes
|
||||
- [ ] Verify with raster-effect test ROMs
|
||||
|
||||
**Success Criteria**: Games with H-IRQ effects (Tales of Phantasia, Star Ocean) render correctly
|
||||
|
||||
#### 1.2 Semantic Inspection API
|
||||
**Agent**: `ai-infra-architect`
|
||||
**Status**: PLANNED
|
||||
**Files**: New `src/app/emu/debug/semantic_introspection.h/cc`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Create `SemanticIntrospectionEngine` class
|
||||
- [ ] Connect to `Memory` and `SymbolProvider`
|
||||
- [ ] Implement `GetPlayerState()` using ALTTP RAM offsets
|
||||
- [ ] Implement `GetSpriteState()` for sprite tracking
|
||||
- [ ] Add JSON export for AI consumption
|
||||
- [ ] Create debug overlay rendering for vision models
|
||||
|
||||
**Success Criteria**: AI agents can query game state semantically via JSON API
|
||||
|
||||
#### 1.3 State Injection API
|
||||
**Agent**: `snes-emulator-expert`
|
||||
**Status**: PLANNED
|
||||
**Files**: `src/app/emu/emulator.h/cc`, new `src/app/emu/state_patch.h`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Define `GameStatePatch` structure
|
||||
- [ ] Implement `Emulator::InjectState(patch)`
|
||||
- [ ] Add fast-boot capability (skip intro sequences)
|
||||
- [ ] Create ALTTP-specific presets (Dungeon Test, Overworld Test)
|
||||
- [ ] Integrate with z3ed CLI for "test sprite" workflow
|
||||
|
||||
**Success Criteria**: Editors can teleport emulator to any game state programmatically
|
||||
|
||||
#### 1.4 Audio System Fix
|
||||
**Agent**: `snes-emulator-expert`
|
||||
**Status**: PLANNED
|
||||
**Files**: `src/app/emu/audio/`, `src/app/emu/apu/`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Diagnose SDL2 audio device initialization
|
||||
- [ ] Fix SPC700 → SDL2 format conversion
|
||||
- [ ] Verify APU handshake timing
|
||||
- [ ] Add audio debugging tools to UI
|
||||
- [ ] Test with music playback in ALTTP
|
||||
|
||||
**Success Criteria**: Audio plays correctly during emulation
|
||||
|
||||
---
|
||||
|
||||
### Milestone 2: SDL3 Migration (Weeks 3-8)
|
||||
|
||||
#### 2.1 Directory Restructure
|
||||
**Agent**: `backend-infra-engineer`
|
||||
**Status**: PLANNED
|
||||
**Scope**: Move `src/lib/` + `third_party/` → `external/`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Create `external/` directory structure
|
||||
- [ ] Move SDL2 (to be replaced), imgui, etc.
|
||||
- [ ] Update CMakeLists.txt references
|
||||
- [ ] Update submodule paths
|
||||
- [ ] Validate builds on all platforms
|
||||
|
||||
#### 2.2 SDL3 Core Integration
|
||||
**Agent**: `imgui-frontend-engineer`
|
||||
**Status**: PLANNED
|
||||
**Files**: `src/app/platform/`, `CMakeLists.txt`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Add SDL3 as dependency
|
||||
- [ ] Create `GraphicsBackend` abstraction interface
|
||||
- [ ] Implement SDL3 backend for window/rendering
|
||||
- [ ] Update ImGui to SDL3 backend
|
||||
- [ ] Port window creation and event handling
|
||||
|
||||
#### 2.3 SDL3 Audio Backend
|
||||
**Agent**: `snes-emulator-expert`
|
||||
**Status**: PLANNED (after audio fix)
|
||||
**Files**: `src/app/emu/audio/sdl3_audio_backend.h/cc`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Implement `IAudioBackend` for SDL3
|
||||
- [ ] Migrate audio initialization code
|
||||
- [ ] Verify audio quality matches SDL2
|
||||
|
||||
#### 2.4 SDL3 Input Backend
|
||||
**Agent**: `imgui-frontend-engineer`
|
||||
**Status**: PLANNED
|
||||
**Files**: `src/app/emu/ui/input_handler.cc`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Implement SDL3 input backend
|
||||
- [ ] Add gamepad support improvements
|
||||
- [ ] Verify continuous key polling works
|
||||
|
||||
---
|
||||
|
||||
### Milestone 3: Editor Fixes (Weeks 2-4)
|
||||
|
||||
#### 3.1 Tile16 Palette System Fix
|
||||
**Agent**: `zelda3-hacking-expert`
|
||||
**Status**: PLANNED
|
||||
**Files**: `src/app/editor/graphics/tile16_editor.cc`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Fix Tile8 source canvas palette application
|
||||
- [ ] Fix palette button 0-7 switching logic
|
||||
- [ ] Ensure color alignment across canvases
|
||||
- [ ] Add unit tests for palette operations
|
||||
|
||||
**Success Criteria**: Tile editing workflow fully functional
|
||||
|
||||
#### 3.2 Overworld Sprite Movement
|
||||
**Agent**: `zelda3-hacking-expert`
|
||||
**Status**: PLANNED
|
||||
**Files**: `src/app/editor/overworld/overworld_editor.cc`
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Debug canvas interaction system
|
||||
- [ ] Fix drag operation handling for sprites
|
||||
- [ ] Test sprite placement workflow
|
||||
|
||||
**Success Criteria**: Sprites respond to drag operations
|
||||
|
||||
#### 3.3 Dungeon Sprite Save Integration
|
||||
**Agent**: `zelda3-hacking-expert`
|
||||
**Status**: IN_PROGRESS (uncommitted)
|
||||
**Files**: `src/zelda3/dungeon/room.cc/h`
|
||||
|
||||
**Tasks**:
|
||||
- [x] Implement `EncodeSprites()` method
|
||||
- [x] Implement `SaveSprites()` method
|
||||
- [ ] Integrate with dungeon editor UI
|
||||
- [ ] Add unit tests
|
||||
- [ ] Commit and verify CI
|
||||
|
||||
---
|
||||
|
||||
## Agent Assignments
|
||||
|
||||
| Agent | Primary Responsibilities | Workstream |
|
||||
|-------|-------------------------|------------|
|
||||
| `snes-emulator-expert` | PPU catch-up, audio fix, state injection, SDL3 audio | Stream 1 |
|
||||
| `imgui-frontend-engineer` | SDL3 core, SDL3 input, UI updates | Stream 2 |
|
||||
| `zelda3-hacking-expert` | Tile16 fix, sprite movement, dungeon save | Stream 3 |
|
||||
| `ai-infra-architect` | Semantic API, multimodal context | Stream 4 |
|
||||
| `backend-infra-engineer` | Directory restructure, CI updates | Stream 2 |
|
||||
| `test-infrastructure-expert` | Test suite for new features | Support |
|
||||
| `docs-janitor` | Documentation updates | Support |
|
||||
|
||||
---
|
||||
|
||||
## Parallel Workstreams
|
||||
|
||||
```
|
||||
Week 1-2:
|
||||
├── Stream 1: snes-emulator-expert → Complete PPU catch-up
|
||||
├── Stream 3: zelda3-hacking-expert → Tile16 palette fix
|
||||
└── Stream 4: ai-infra-architect → Semantic API design
|
||||
|
||||
Week 3-4:
|
||||
├── Stream 1: snes-emulator-expert → Audio system fix
|
||||
├── Stream 2: backend-infra-engineer → Directory restructure
|
||||
├── Stream 3: zelda3-hacking-expert → Sprite movement fix
|
||||
└── Stream 4: ai-infra-architect → Semantic API implementation
|
||||
|
||||
Week 5-6:
|
||||
├── Stream 1: snes-emulator-expert → State injection API
|
||||
├── Stream 2: imgui-frontend-engineer → SDL3 core integration
|
||||
└── Stream 3: zelda3-hacking-expert → Dungeon sprite integration
|
||||
|
||||
Week 7-8:
|
||||
├── Stream 1: snes-emulator-expert → SDL3 audio backend
|
||||
├── Stream 2: imgui-frontend-engineer → SDL3 input backend
|
||||
└── All: Integration testing and stabilization
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### v0.4.0 Release Readiness
|
||||
- [ ] PPU catch-up renders raster effects correctly
|
||||
- [ ] Semantic API provides structured game state
|
||||
- [ ] State injection enables "test sprite" workflow
|
||||
- [ ] Audio system functional
|
||||
- [ ] SDL3 builds pass on Windows, macOS, Linux
|
||||
- [ ] No performance regression vs v0.3.x
|
||||
- [ ] All known editor bugs fixed
|
||||
- [ ] Documentation updated for new APIs
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|-------------|--------|------------|
|
||||
| SDL3 breaking changes | Medium | High | Maintain SDL2 fallback branch |
|
||||
| Audio system complexity | High | Medium | Prioritize diagnosis before migration |
|
||||
| Cross-platform issues | Medium | Medium | CI validation on all platforms |
|
||||
| Agent coordination conflicts | Low | Medium | Strict coordination board protocol |
|
||||
|
||||
---
|
||||
|
||||
## Communication
|
||||
|
||||
- **Daily**: Coordination board updates
|
||||
- **Weekly**: Progress sync via initiative status
|
||||
- **Blockers**: Post `BLOCKER` tag on coordination board immediately
|
||||
- **Handoffs**: Use `REQUEST →` format for task transitions
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Emulator Accuracy Report](emulator_accuracy_report.md)
|
||||
- [Roadmap](../roadmaps/roadmap.md)
|
||||
- [Feature Parity Analysis](../roadmaps/feature-parity-analysis.md)
|
||||
- [Code Review Next Steps](../roadmaps/code-review-critical-next-steps.md)
|
||||
- [Coordination Board](coordination-board.md)
|
||||
559
docs/internal/agents/overworld-refactor-handoff.md
Normal file
559
docs/internal/agents/overworld-refactor-handoff.md
Normal file
@@ -0,0 +1,559 @@
|
||||
# Overworld Editor Refactoring - Handoff Document
|
||||
|
||||
**Agent ID:** ai-infra-architect
|
||||
**Date:** December 6, 2025
|
||||
**Status:** Phase 2 Complete - Critical Bug Fixes & Tile16 Editor Polish
|
||||
**Next Phase:** Week 2 Toolset Improvements (Eyedropper, Flood Fill, Eraser)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Phase 1 of the overworld editor refactoring is complete. This phase focused on documentation without functional changes. The codebase analysis has revealed several areas requiring attention for Phase 2, including critical bugs in the tile cache system, incomplete zoom/pan implementation, and opportunities for better separation of concerns.
|
||||
|
||||
---
|
||||
|
||||
## Completed Work (Phase 2) - December 6, 2025
|
||||
|
||||
### Week 1: Critical Bug Fixes (COMPLETE)
|
||||
|
||||
#### 1. Tile Cache System Fix
|
||||
**Files:** `src/app/gfx/render/tilemap.h`, `src/app/gfx/render/tilemap.cc`
|
||||
|
||||
- Changed `TileCache::CacheTile()` from `Bitmap&&` (move) to `const Bitmap&` (copy)
|
||||
- Re-enabled tile cache usage in `RenderTile()` and `RenderTilesBatch()`
|
||||
- Root cause: `std::move()` invalidated Bitmap surface pointers causing segfaults
|
||||
|
||||
#### 2. Centralized Zoom Constants
|
||||
**Files:** `src/app/editor/overworld/overworld_editor.h`, `overworld_navigation.cc`, `map_properties.cc`
|
||||
|
||||
- Added `kOverworldMinZoom`, `kOverworldMaxZoom`, `kOverworldZoomStep` constants
|
||||
- Updated all zoom controls to use consistent limits (0.1 - 5.0x, 0.25 step)
|
||||
- Scroll wheel zoom intentionally disabled (reserved for canvas navigation)
|
||||
|
||||
#### 3. Live Preview Re-enabled
|
||||
**File:** `src/app/editor/overworld/overworld_editor.cc`
|
||||
|
||||
- Re-enabled `UpdateBlocksetWithPendingTileChanges()` with proper guards
|
||||
- Live preview now shows tile16 edits on main map before committing
|
||||
|
||||
#### 4. Entity Hit Detection Scaling
|
||||
**Files:** `src/app/editor/overworld/entity.h`, `entity.cc`, `overworld_entity_renderer.cc`
|
||||
|
||||
- Added `float scale = 1.0f` parameter to `IsMouseHoveringOverEntity()` and `MoveEntityOnGrid()`
|
||||
- Entity interaction now correctly scales with canvas zoom level
|
||||
|
||||
### Week 3: Tile16 Editor Polish (COMPLETE)
|
||||
|
||||
#### 1. Tile16 Editor Window Restoration
|
||||
**Files:** `src/app/editor/overworld/overworld_editor.cc`
|
||||
|
||||
- Restored Tile16 Editor as standalone window with `ImGuiWindowFlags_MenuBar`
|
||||
- Draws directly in `Update()` when `show_tile16_editor_` is true
|
||||
- Accessible via Ctrl+T or toolbar toggle
|
||||
|
||||
#### 2. SNES Palette Offset Fix
|
||||
**File:** `src/app/editor/overworld/tile16_editor.cc`
|
||||
|
||||
- Fixed off-by-one error in `SetPaletteWithTransparent()` calls
|
||||
- Added +1 offset so pixel value N maps to sub-palette color N (not N-1)
|
||||
- Applied to: `tile8_preview_bmp_`, `current_tile16_bmp_`, `current_gfx_individual_[]`
|
||||
|
||||
#### 3. Palette Remapping for Tile8 Source Canvas
|
||||
**File:** `src/app/editor/overworld/tile16_editor.cc`
|
||||
|
||||
- Added `CreateRemappedPaletteForViewing()` function
|
||||
- Source canvas now responds to palette button selection (0-7)
|
||||
- Remaps all pixel values to user-selected palette row regardless of encoding
|
||||
|
||||
#### 4. Visual Palette/Sheet Indicator
|
||||
**File:** `src/app/editor/overworld/tile16_editor.cc`
|
||||
|
||||
- Added sheet indicator (S0-S7) next to tile8 preview
|
||||
- Tooltip shows sheet index, encoded palette row, and encoding explanation
|
||||
- Helps users understand which graphics use which palette regions
|
||||
|
||||
#### 5. Data Analysis Diagnostic
|
||||
**Files:** `src/app/editor/overworld/tile16_editor.h`, `tile16_editor.cc`
|
||||
|
||||
- Added `AnalyzeTile8SourceData()` diagnostic function
|
||||
- "Analyze Data" button in UI outputs detailed format/palette info to log
|
||||
- Shows pixel value distribution, palette state, and remapping explanation
|
||||
|
||||
### Toolbar & Scratch Space Simplification
|
||||
|
||||
#### 1. Unified Scratch Space
|
||||
**Files:** `src/app/editor/overworld/overworld_editor.h`, `scratch_space.cc`
|
||||
|
||||
- Simplified from 4 slots to single unified workspace
|
||||
- Renamed `ScratchSpaceSlot` to `ScratchSpace`
|
||||
- Updated all UI and logic to operate on single instance
|
||||
|
||||
#### 2. Toolbar Panel Toggles
|
||||
**File:** `src/app/editor/overworld/overworld_toolbar.cc`
|
||||
|
||||
- Added "Panels" dropdown with toggle buttons for all editor panels
|
||||
- Panels: Tile16 Editor, Tile16 Selector, Tile8 Selector, Area Graphics, etc.
|
||||
- Uses PanelManager for visibility state persistence
|
||||
|
||||
---
|
||||
|
||||
## Completed Work (Phase 1)
|
||||
|
||||
1. **README.md** (`src/app/editor/overworld/README.md`)
|
||||
- Architecture overview with component diagram
|
||||
- File organization and responsibilities
|
||||
- Tile16 editing workflow with palette coordination
|
||||
- ZScustom feature documentation
|
||||
- Save system order and dependencies
|
||||
- Testing guidance
|
||||
|
||||
2. **Tile16Editor Documentation** (`tile16_editor.h`)
|
||||
- Extensive header block explaining pending changes system
|
||||
- Palette coordination with sheet-to-palette mapping table
|
||||
- Method-level documentation for all key APIs
|
||||
|
||||
3. **ZScustom Version Helper** (`overworld_version_helper.h`)
|
||||
- Feature matrix showing version capabilities
|
||||
- Usage examples and upgrade workflow
|
||||
- ROM marker location documentation
|
||||
|
||||
4. **Overworld Data Layer** (`overworld.h`)
|
||||
- Save order dependencies and groupings
|
||||
- Method organization by functionality
|
||||
- Testing guidance for save operations
|
||||
|
||||
5. **OverworldEditor Organization** (`overworld_editor.h`)
|
||||
- Section comments delineating subsystems
|
||||
- Member variable groupings by purpose
|
||||
- Method organization by functionality
|
||||
|
||||
---
|
||||
|
||||
## Critical Issues Identified
|
||||
|
||||
### 1. Tile Cache System - DISABLED DUE TO CRASHES
|
||||
|
||||
**Location:** `src/app/gfx/render/tilemap.cc`, `src/app/gui/canvas/canvas.cc`
|
||||
|
||||
**Problem:** The tile cache uses `std::move()` which invalidates Bitmap surface pointers, causing segmentation faults.
|
||||
|
||||
**Evidence from code:**
|
||||
```cpp
|
||||
// tilemap.cc:67-68
|
||||
// Note: Tile cache disabled to prevent std::move() related crashes
|
||||
|
||||
// canvas.cc:768-769
|
||||
// CRITICAL FIX: Disable tile cache system to prevent crashes
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
- Performance degradation - tiles are re-rendered each frame
|
||||
- The `TileCache` struct in `tilemap.h` is essentially dead code
|
||||
- Memory for the LRU cache is allocated but never used effectively
|
||||
|
||||
**Recommended Fix:**
|
||||
- Option A: Use `std::shared_ptr<Bitmap>` instead of `std::unique_ptr` to allow safe pointer sharing
|
||||
- Option B: Copy bitmaps into cache instead of moving them
|
||||
- Option C: Implement a texture-ID based cache that doesn't require pointer stability
|
||||
|
||||
### 2. Zoom/Pan Implementation is Fragmented
|
||||
|
||||
**Location:** `overworld_navigation.cc`, `map_properties.cc`, Canvas class
|
||||
|
||||
**Problem:** Zoom/pan is implemented in multiple places with inconsistent behavior:
|
||||
|
||||
```cpp
|
||||
// overworld_navigation.cc - Main zoom implementation
|
||||
void OverworldEditor::ZoomIn() {
|
||||
float new_scale = std::min(5.0f, ow_map_canvas_.global_scale() + 0.25f);
|
||||
ow_map_canvas_.set_global_scale(new_scale);
|
||||
}
|
||||
|
||||
// map_properties.cc - Context menu zoom (different limits!)
|
||||
canvas.set_global_scale(std::max(0.25f, canvas.global_scale() - 0.25f));
|
||||
canvas.set_global_scale(std::min(2.0f, canvas.global_scale() + 0.25f));
|
||||
```
|
||||
|
||||
**Issues:**
|
||||
- Inconsistent max zoom limits (2.0f vs 5.0f)
|
||||
- No smooth zoom (scroll wheel support mentioned but not implemented)
|
||||
- No zoom-to-cursor functionality
|
||||
- Pan only works with middle mouse button
|
||||
|
||||
**Recommended Improvements:**
|
||||
1. Centralize zoom/pan in a `CanvasNavigationController` class
|
||||
2. Add scroll wheel zoom with zoom-to-cursor
|
||||
3. Implement keyboard shortcuts (Ctrl+Plus/Minus, Home to reset)
|
||||
4. Add mini-map navigation overlay for large canvases
|
||||
|
||||
### 3. UpdateBlocksetWithPendingTileChanges is Disabled
|
||||
|
||||
**Location:** `overworld_editor.cc:262-264`
|
||||
|
||||
```cpp
|
||||
// TODO: Re-enable after fixing crash
|
||||
// Update blockset atlas with any pending tile16 changes for live preview
|
||||
// UpdateBlocksetWithPendingTileChanges();
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
- Live preview of tile16 edits doesn't work on the main map
|
||||
- Users must commit changes to see them on the overworld
|
||||
|
||||
### 4. Entity Interaction Issues
|
||||
|
||||
**Location:** `entity.cc`, `overworld_entity_interaction.cc`
|
||||
|
||||
**Problems:**
|
||||
- Hardcoded 16x16 entity hit detection doesn't scale with zoom
|
||||
- Entity popups use `static` variables for state (potential bugs with multiple popups)
|
||||
- No undo/redo for entity operations
|
||||
|
||||
**Evidence:**
|
||||
```cpp
|
||||
// entity.cc:33-34
|
||||
return mouse_pos.x >= entity.x_ && mouse_pos.x <= entity.x_ + 16 &&
|
||||
mouse_pos.y >= entity.y_ && mouse_pos.y <= entity.y_ + 16;
|
||||
// Should use: entity.x_ + 16 * scale
|
||||
```
|
||||
|
||||
### 5. V3 Settings Panel is Incomplete
|
||||
|
||||
**Location:** `overworld_editor.cc:1663-1670`
|
||||
|
||||
```cpp
|
||||
void OverworldEditor::DrawV3Settings() {
|
||||
// TODO: Implement v3 settings UI
|
||||
// Could include:
|
||||
// - Custom map size toggles
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Missing Features:**
|
||||
- Per-area animated GFX selection
|
||||
- Subscreen overlay configuration
|
||||
- Custom tile GFX groups
|
||||
- Mosaic effect controls
|
||||
|
||||
---
|
||||
|
||||
## Architecture Analysis
|
||||
|
||||
### Current Code Metrics
|
||||
|
||||
| File | Lines | Responsibility |
|
||||
|------|-------|----------------|
|
||||
| `overworld_editor.cc` | 3,208 | God class - does too much |
|
||||
| `tile16_editor.cc` | 3,048 | Large but focused |
|
||||
| `map_properties.cc` | 1,755 | Mixed UI concerns |
|
||||
| `entity.cc` | 716 | Entity popup rendering |
|
||||
| `scratch_space.cc` | 417 | Well-isolated |
|
||||
|
||||
### God Class Symptoms in OverworldEditor
|
||||
|
||||
The `OverworldEditor` class handles:
|
||||
1. Canvas drawing and interaction
|
||||
2. Tile painting and selection
|
||||
3. Entity management
|
||||
4. Graphics loading and refresh
|
||||
5. Map property editing
|
||||
6. Undo/redo for painting
|
||||
7. Scratch space management
|
||||
8. ZScustom ASM patching
|
||||
9. Keyboard shortcuts
|
||||
10. Deferred texture creation
|
||||
|
||||
**Recommendation:** Extract into focused subsystem classes:
|
||||
- `OverworldCanvasController` - Canvas drawing, zoom/pan, tile painting
|
||||
- `OverworldEntityController` - Entity CRUD, rendering, interaction
|
||||
- `OverworldGraphicsController` - Loading, refresh, palette coordination
|
||||
- `OverworldUndoManager` - Undo/redo stack management
|
||||
- Keep `OverworldEditor` as thin orchestrator
|
||||
|
||||
### Panel System Redundancy
|
||||
|
||||
Each panel in `panels/` is a thin wrapper that calls back to `OverworldEditor`:
|
||||
|
||||
```cpp
|
||||
// tile16_selector_panel.cc
|
||||
void Tile16SelectorPanel::Draw(bool* p_open) {
|
||||
editor_->DrawTile16Selector(); // Just delegates
|
||||
}
|
||||
```
|
||||
|
||||
**Recommendation:** Move drawing logic into panels, reducing coupling to editor.
|
||||
|
||||
---
|
||||
|
||||
## Future Feature Proposals
|
||||
|
||||
### 1. Enhanced Zoom/Pan System
|
||||
|
||||
**Priority:** High
|
||||
**Effort:** Medium (2-3 days)
|
||||
|
||||
Features:
|
||||
- Scroll wheel zoom centered on cursor
|
||||
- Keyboard shortcuts (Ctrl+0 reset, Ctrl+Plus/Minus zoom)
|
||||
- Touch gesture support for tablet users
|
||||
- Mini-map overlay for navigation
|
||||
- Smooth animated zoom transitions
|
||||
|
||||
Implementation approach:
|
||||
```cpp
|
||||
class CanvasNavigationController {
|
||||
public:
|
||||
void HandleScrollZoom(float delta, ImVec2 mouse_pos);
|
||||
void HandlePan(ImVec2 delta);
|
||||
void ZoomToFit(ImVec2 content_size);
|
||||
void ZoomToSelection(ImVec2 selection_rect);
|
||||
void CenterOn(ImVec2 world_position);
|
||||
|
||||
float current_zoom() const;
|
||||
ImVec2 scroll_offset() const;
|
||||
};
|
||||
```
|
||||
|
||||
### 2. Better Toolset in Canvas Toolbar
|
||||
|
||||
**Priority:** High
|
||||
**Effort:** Medium (2-3 days)
|
||||
|
||||
Current toolbar only has Mouse/Paint toggle. Proposed additions:
|
||||
|
||||
| Tool | Icon | Behavior |
|
||||
|------|------|----------|
|
||||
| Select | Box | Rectangle selection for multi-tile ops |
|
||||
| Brush | Brush | Current paint behavior |
|
||||
| Fill | Bucket | Flood fill with tile16 |
|
||||
| Eyedropper | Dropper | Pick tile16 from map |
|
||||
| Eraser | Eraser | Paint with empty tile |
|
||||
| Line | Line | Draw straight lines of tiles |
|
||||
| Rectangle | Rect | Draw filled/outline rectangles |
|
||||
|
||||
Implementation:
|
||||
```cpp
|
||||
enum class OverworldTool {
|
||||
Select,
|
||||
Brush,
|
||||
Fill,
|
||||
Eyedropper,
|
||||
Eraser,
|
||||
Line,
|
||||
Rectangle
|
||||
};
|
||||
|
||||
class OverworldToolManager {
|
||||
void SetTool(OverworldTool tool);
|
||||
void HandleMouseDown(ImVec2 pos);
|
||||
void HandleMouseDrag(ImVec2 pos);
|
||||
void HandleMouseUp(ImVec2 pos);
|
||||
void RenderPreview();
|
||||
};
|
||||
```
|
||||
|
||||
### 3. Tile16 Editor Improvements
|
||||
|
||||
**Priority:** Medium
|
||||
**Effort:** Medium (2-3 days)
|
||||
|
||||
Current issues:
|
||||
- No visual indication of which tile8 positions are filled
|
||||
- Palette preview doesn't show all colors clearly
|
||||
- Can't preview tile on actual map before committing
|
||||
|
||||
Proposed improvements:
|
||||
- Quadrant highlight showing which positions are filled
|
||||
- Color swatch grid for current palette
|
||||
- "Preview on Map" toggle that temporarily shows edited tile
|
||||
- Tile history/favorites for quick access
|
||||
- Copy/paste between tile16s
|
||||
|
||||
### 4. Multi-Tile Operations
|
||||
|
||||
**Priority:** Medium
|
||||
**Effort:** High (1 week)
|
||||
|
||||
Current rectangle selection only works for painting. Expand to:
|
||||
- Copy selection to clipboard
|
||||
- Paste clipboard with preview
|
||||
- Rotate/flip selection
|
||||
- Save selection as scratch slot
|
||||
- Search for tile patterns
|
||||
|
||||
### 5. Entity Visualization Improvements
|
||||
|
||||
**Priority:** Low
|
||||
**Effort:** Medium (2-3 days)
|
||||
|
||||
Current: Simple 16x16 colored rectangles
|
||||
|
||||
Proposed:
|
||||
- Sprite previews for entities (already partially implemented)
|
||||
- Connection lines for entrance/exit pairs
|
||||
- Highlight related entities on hover
|
||||
- Entity layer toggle visibility
|
||||
- Entity search/filter by type
|
||||
|
||||
### 6. Map Comparison/Diff Tool
|
||||
|
||||
**Priority:** Low
|
||||
**Effort:** High (1 week)
|
||||
|
||||
For ZScustom testing:
|
||||
- Side-by-side view of two ROM versions
|
||||
- Highlight differences in tiles/entities
|
||||
- Compare map properties
|
||||
- Export diff report
|
||||
|
||||
---
|
||||
|
||||
## Testing Recommendations
|
||||
|
||||
### Manual Test Cases for Phase 2
|
||||
|
||||
1. **Tile16 Editing Round-Trip**
|
||||
- Edit a tile16, commit, save ROM, reload, verify persistence
|
||||
- Edit multiple tile16s, discard some, commit others
|
||||
- Verify palette colors match across all views
|
||||
|
||||
2. **Zoom/Pan Stress Test**
|
||||
- Zoom to max/min while painting
|
||||
- Pan rapidly and verify no visual artifacts
|
||||
- Test at 0.25x, 1x, 2x, 4x zoom levels
|
||||
|
||||
3. **Entity Operations**
|
||||
- Create/move/delete each entity type
|
||||
- Verify entity positions survive save/load
|
||||
- Test entity interaction at different zoom levels
|
||||
|
||||
4. **ZScustom Feature Regression**
|
||||
- Test vanilla ROM, verify graceful degradation
|
||||
- Test v2 ROM, verify BG colors work
|
||||
- Test v3 ROM, verify all features available
|
||||
- Upgrade vanilla→v3, verify all features activate
|
||||
|
||||
5. **Save System Integrity**
|
||||
- Save with each component flag disabled individually
|
||||
- Verify no corruption of unmodified data
|
||||
- Test save after large edits (fill entire map)
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 Roadmap Status
|
||||
|
||||
### Week 1: Critical Bug Fixes ✅ COMPLETE
|
||||
1. ✅ Fix tile cache system (copy instead of move)
|
||||
2. ✅ Implement consistent zoom limits (centralized constants)
|
||||
3. ✅ Re-enable UpdateBlocksetWithPendingTileChanges with fix
|
||||
4. ✅ Fix entity hit detection with zoom scaling
|
||||
|
||||
### Week 2: Toolset Improvements (NEXT)
|
||||
1. ⏳ Implement eyedropper tool
|
||||
2. ⏳ Implement flood fill tool
|
||||
3. ⏳ Add eraser tool
|
||||
4. ⏳ Enhance toolbar UI
|
||||
|
||||
### Week 3: Tile16 Editor Polish ✅ COMPLETE
|
||||
1. ✅ Tile16 Editor window restoration (menu bar support)
|
||||
2. ✅ SNES palette offset fix (+1 for correct color mapping)
|
||||
3. ✅ Palette remapping for tile8 source canvas viewing
|
||||
4. ✅ Visual sheet/palette indicator with tooltip
|
||||
5. ✅ Data analysis diagnostic function
|
||||
|
||||
### Week 4: Architecture Cleanup
|
||||
1. ⏳ Extract CanvasNavigationController
|
||||
2. ⏳ Extract OverworldToolManager
|
||||
3. ⏳ Move panel drawing logic into panels
|
||||
4. ⏳ Add comprehensive unit tests
|
||||
|
||||
---
|
||||
|
||||
## Files Modified in Phase 2
|
||||
|
||||
| File | Change Type |
|
||||
|------|-------------|
|
||||
| `src/app/gfx/render/tilemap.h` | Bug fix - tile cache copy semantics |
|
||||
| `src/app/gfx/render/tilemap.cc` | Bug fix - re-enabled tile cache |
|
||||
| `src/app/editor/overworld/overworld_editor.h` | Added zoom constants, scratch space simplification |
|
||||
| `src/app/editor/overworld/overworld_editor.cc` | Tile16 Editor window, panel registration, live preview |
|
||||
| `src/app/editor/overworld/overworld_navigation.cc` | Centralized zoom constants |
|
||||
| `src/app/editor/overworld/map_properties.cc` | Consistent zoom limits |
|
||||
| `src/app/editor/overworld/entity.h` | Scale parameter for hit detection |
|
||||
| `src/app/editor/overworld/entity.cc` | Scaled entity interaction |
|
||||
| `src/app/editor/overworld/overworld_entity_renderer.cc` | Pass scale to entity functions |
|
||||
| `src/app/editor/overworld/tile16_editor.h` | Added palette remapping, analysis functions |
|
||||
| `src/app/editor/overworld/tile16_editor.cc` | Palette fixes, remapping, visual indicators, diagnostics |
|
||||
| `src/app/editor/overworld/overworld_toolbar.cc` | Panel toggles, simplified scratch space |
|
||||
| `src/app/editor/overworld/scratch_space.cc` | Unified single scratch space |
|
||||
| `src/app/gui/canvas/canvas.cc` | Updated tile cache comment |
|
||||
| `src/app/editor/editor_library.cmake` | Removed deleted panel file |
|
||||
|
||||
## Files Modified in Phase 1
|
||||
|
||||
| File | Change Type |
|
||||
|------|-------------|
|
||||
| `src/app/editor/overworld/README.md` | Created |
|
||||
| `src/app/editor/overworld/tile16_editor.h` | Documentation |
|
||||
| `src/app/editor/overworld/overworld_editor.h` | Documentation |
|
||||
| `src/zelda3/overworld/overworld.h` | Documentation |
|
||||
| `src/zelda3/overworld/overworld_version_helper.h` | Documentation |
|
||||
|
||||
Phase 1: Documentation only. Phase 2: Functional bug fixes and feature improvements.
|
||||
|
||||
---
|
||||
|
||||
## Key Contacts and Resources
|
||||
|
||||
- **Codebase Owner:** scawful
|
||||
- **Related Documentation:**
|
||||
- [EditorManager Architecture](H2-editor-manager-architecture.md)
|
||||
- [Feature Parity Analysis](H3-feature-parity-analysis.md)
|
||||
- [Composite Layer System](composite-layer-system.md)
|
||||
- **External References:**
|
||||
- [ZScream GitHub Wiki](https://github.com/Zarby89/ZScreamDungeon/wiki)
|
||||
- [ALTTP ROM Map](https://alttp.mymm1.com/wiki/)
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Code Snippets for Reference
|
||||
|
||||
### Tile Cache Fix Proposal
|
||||
|
||||
```cpp
|
||||
// Option A: Use shared_ptr for safe sharing
|
||||
struct TileCache {
|
||||
std::unordered_map<int, std::shared_ptr<Bitmap>> cache_;
|
||||
|
||||
std::shared_ptr<Bitmap> GetTile(int tile_id) {
|
||||
auto it = cache_.find(tile_id);
|
||||
return (it != cache_.end()) ? it->second : nullptr;
|
||||
}
|
||||
|
||||
void CacheTile(int tile_id, const Bitmap& bitmap) {
|
||||
cache_[tile_id] = std::make_shared<Bitmap>(bitmap); // Copy, not move
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### Centralized Zoom Handler
|
||||
|
||||
```cpp
|
||||
void CanvasNavigationController::HandleScrollZoom(float delta, ImVec2 mouse_pos) {
|
||||
float old_scale = current_zoom_;
|
||||
float new_scale = std::clamp(current_zoom_ + delta * 0.1f, kMinZoom, kMaxZoom);
|
||||
|
||||
// Zoom centered on cursor
|
||||
ImVec2 world_pos = ScreenToWorld(mouse_pos);
|
||||
current_zoom_ = new_scale;
|
||||
ImVec2 new_screen_pos = WorldToScreen(world_pos);
|
||||
scroll_offset_ += (mouse_pos - new_screen_pos);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*End of Handoff Document*
|
||||
|
||||
36
docs/internal/agents/tests-binary-ux-proposals.md
Normal file
36
docs/internal/agents/tests-binary-ux-proposals.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# YAZE Test Binary UX Proposals (yaze_test / ctest Integration)
|
||||
|
||||
Status: IN_PROGRESS
|
||||
Owner: ai-infra-architect
|
||||
Created: 2025-12-01
|
||||
Last Reviewed: 2025-12-01
|
||||
Next Review: 2025-12-08
|
||||
Board: docs/internal/agents/coordination-board.md (2025-12-01 ai-infra-architect – z3ed CLI UX/TUI Improvement Proposals)
|
||||
|
||||
## Summary
|
||||
- Make the test binaries and ctest entry points friendlier for humans and agents: clearer filters, artifacts, and machine-readable outputs.
|
||||
- Provide a first-class manifest of suites/labels/requirements and a consistent way to supply ROM/AI/headless settings.
|
||||
- Reduce friction when iterating locally (fast presets, targeted subsets) and when collecting results for automation/CI.
|
||||
|
||||
## Observations
|
||||
- The unified `yaze_test` binary emits gtest text only; ctest wraps it but lacks machine-readable summaries unless parsed. Agents currently scrape stdout.
|
||||
- Suite/label requirements (ROM path, AI runtime, headless display) are implicit; misconfiguration silently skips or hard-fails without actionable guidance.
|
||||
- Filter UX is split: gtest filters vs ctest `-L/-R`; no single recommended entry that also records artifacts/logs for failures.
|
||||
- Artifacts (logs, screenshots, recordings) from GUI/agent tests are not consistently emitted or linked from results.
|
||||
- Preset coupling is manual; users must remember which CMake preset enables ROM/AI/headless options. No quick “fast subset” toggle inside the binary.
|
||||
|
||||
## Improvement Proposals
|
||||
- **Manifest & discovery**: Generate a JSON manifest (per build) listing tests, labels, requirements (ROM path, AI runtime), and expected artifacts. Expose via `yaze_test --export-manifest <path>` and `ctest -T mem`-style option. Agents can load it instead of scraping.
|
||||
- **Structured output**: Add `--output-format {text,json,junit}` to `yaze_test` to emit summaries (pass/fail, duration, seed, artifacts) in one file; default to text for humans, JSON for automation. Wire ctest to collect the JSON and place it in a predictable directory.
|
||||
- **Requirements gating**: On startup, detect missing ROM/AI/headless support and fail fast with actionable messages and suggested CMake flags/env (e.g., `YAZE_ENABLE_ROM_TESTS`, `YAZE_TEST_ROM_PATH`). Offer a `--mock-rom-ok` mode to downgrade ROM tests when a mock is acceptable.
|
||||
- **Filters & subsets**: Provide a unified front-end flag set (`--label stable|gui|rom_dependent|experimental`, `--gtest_filter`, `--list`) that internally routes to gtest/labels so humans/agents don’t guess. Add `--shard <N/M>` for parallel runs.
|
||||
- **Artifacts & logs**: Standardize artifact output location (`build/artifacts/tests/<run-id>/`) and name failing-test logs/screenshots accordingly. Emit paths in JSON output. Ensure GUI/agent recordings are captured when labels include `gui` or `experimental`.
|
||||
- **Preset hints**: Print which CMake preset was used to build the binary and whether ROM/AI options are compiled in. Add `--recommend-preset` helper to suggest `mac-test/mac-ai/mac-dev` based on requested labels.
|
||||
- **Headless helpers**: Add `--headless-check` to validate SDL/display availability and exit gracefully with instructions to use headless mode; integrate into ctest label defaults.
|
||||
- **Exit codes**: Ensure non-zero exit codes for any test failure and distinct codes for configuration failures vs runtime failures to simplify automation.
|
||||
|
||||
## Exit Criteria (for this scope)
|
||||
- `yaze_test` (and/or a small wrapper) can emit JSON/JUnit summaries and a manifest without stdout scraping.
|
||||
- Clear, actionable errors when requirements (ROM/AI/display) are missing, with suggested flags/presets.
|
||||
- Artifacts for failing GUI/agent tests are written to predictable paths and referenced in structured output.
|
||||
- Unified filter/label interface documented and consistent with ctest usage.
|
||||
34
docs/internal/agents/yaze-app-ux-proposals.md
Normal file
34
docs/internal/agents/yaze-app-ux-proposals.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# YAZE Desktop App UX Proposals (Panels, Flags, ROM Loading)
|
||||
|
||||
Status: IN_PROGRESS
|
||||
Owner: ai-infra-architect
|
||||
Created: 2025-12-01
|
||||
Last Reviewed: 2025-12-01
|
||||
Next Review: 2025-12-08
|
||||
Board: docs/internal/agents/coordination-board.md (2025-12-01 ai-infra-architect – z3ed CLI UX/TUI Improvement Proposals)
|
||||
|
||||
## Summary
|
||||
- Improve the desktop ImGui/SDL app startup, profiles, ROM onboarding, and layout/panel ergonomics separate from the z3ed CLI/TUI.
|
||||
- Make service status (agent control, HTTP/collab, AI runtime) visible and scriptable; reduce brittle flag combinations.
|
||||
- Provide safe ROM handling (recent list, mock fallback, hot-swap guard) and shareable layout presets for overworld/dungeon/graphics/testing workflows.
|
||||
|
||||
## Observations
|
||||
- Startup relies on ad-hoc flags; no bundled profiles (dev/AI/ROM/viewer) and no in-app profile selector. Users must memorize combinations to enable agent control, autosave, collaboration, or stay read-only.
|
||||
- ROM onboarding is brittle: errors surface via stderr; no first-run picker, recent ROM list, autodetect of common paths, or “use mock ROM” fallback; hot-swapping ROMs risks state loss without confirmation.
|
||||
- Panel/layout presets are implicit—users rebuild layouts each session. No first-class presets for overworld, dungeon, graphics/palette, testing/doctor, or AI console. Export/import of layouts is absent.
|
||||
- Runtime status is opaque: no unified HUD showing ROM title/version, profile/layout, mock-ROM flag, active services (agent/collab/HTTP), autosave status, or feature flags.
|
||||
- Configuration surfaces are fragmented; parity between CLI flags and in-app toggles is unclear, making automation brittle.
|
||||
|
||||
## Improvement Proposals
|
||||
- **Profiles & bundled flags**: Add `--profile {dev, ai, rom, viewer, wasm}` with an in-app profile picker. Each profile sets sane defaults (agent control on/off, autosave cadence, mock-ROM allowed, telemetry, collaboration) and selects a default layout preset. Persist per-user.
|
||||
- **ROM onboarding & recovery**: Show a startup ROM picker with recent list and autodetect (`./zelda3.sfc`, `assets/zelda3.sfc`, env var). Validate and, on failure, offer retry/browse and “Use mock ROM” instead of exiting. Add `--rom-prompt` to force picker even when a path is supplied for shared environments.
|
||||
- **Layout presets & persistence**: Ship named presets (Overworld Editing, Dungeon Editing, Graphics/Palette, Testing/Doctor, AI Agent Console). Provide `--layout <name>` and an in-app switcher; persist per profile and allow export/import for handoff.
|
||||
- **Unified status HUD**: Add an always-visible status bar/dashboard summarizing ROM info, profile/layout, service state (agent/HTTP/collab), mock-ROM flag, autosave recency, and feature flags. Expose the same state via a lightweight JSON status endpoint/command for automation.
|
||||
- **Safer ROM/context switching**: On ROM change, prompt with unsaved-change summary and autosave option; offer “clone to temp” for experiments; support `--readonly-rom` for analysis sessions.
|
||||
- **Config discoverability**: Centralize runtime settings (ROM path, profile, feature toggles, autosave cadence, telemetry) in a single pane that mirrors CLI flags. Add `--export-config`/`--import-config` to script setups and share configurations.
|
||||
|
||||
## Exit Criteria (for this scope)
|
||||
- Profiles and layout presets are selectable at startup and in-app, with persisted choices.
|
||||
- ROM onboarding flow handles missing/invalid ROMs gracefully with mock fallback and recent list.
|
||||
- Status HUD (and JSON endpoint/command) surfaces ROM/profile/service state for humans and automation.
|
||||
- Layouts are exportable/importable; presets cover main workflows (overworld, dungeon, graphics, testing, AI console).
|
||||
Reference in New Issue
Block a user