backend-infra-engineer: Post v0.3.9-hotfix7 snapshot (build cleanup)
This commit is contained in:
scawful
@@ -0,0 +1,424 @@
|
||||
# WASM Debug Infrastructure for AI Integration
|
||||
|
||||
**Date:** November 25, 2025 (Updated)
|
||||
**Status:** Current - Active debugging API
|
||||
**Version:** 2.3.0
|
||||
**Purpose:** Comprehensive debug API for Gemini/Antigravity AI integration to analyze rendering issues and game state in the yaze web application.
|
||||
|
||||
**Note:** This document is the high-level overview for WASM debugging. For detailed API reference, see `wasm-yazeDebug-api-reference.md`. For general WASM status and control APIs, see `wasm_dev_status.md`.
|
||||
|
||||
## Overview
|
||||
|
||||
The WASM debug infrastructure provides JavaScript access to internal yaze data structures for AI-powered debugging of dungeon palette rendering issues and other visual artifacts.
|
||||
|
||||
## Memory Configuration
|
||||
|
||||
The WASM build uses optimized memory settings configured in `src/app/app.cmake`:
|
||||
|
||||
| Setting | Value | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `INITIAL_MEMORY` | 256MB | Reduces heap resizing during ROM load (~200MB needed for overworld) |
|
||||
| `MAXIMUM_MEMORY` | 1GB | Prevents runaway allocations |
|
||||
| `STACK_SIZE` | 8MB | Handles recursive operations during asset decompression |
|
||||
| `ALLOW_MEMORY_GROWTH` | 1 | Enables dynamic heap expansion |
|
||||
|
||||
**Emscripten Flags:**
|
||||
```
|
||||
-s INITIAL_MEMORY=268435456 -s ALLOW_MEMORY_GROWTH=1 -s MAXIMUM_MEMORY=1073741824 -s STACK_SIZE=8388608
|
||||
```
|
||||
|
||||
## ROM Loading Progress
|
||||
|
||||
The WASM build reports loading progress through C++ `WasmLoadingManager`. Progress can be monitored in the browser UI or console.
|
||||
|
||||
### Loading Stages
|
||||
|
||||
| Progress | Stage |
|
||||
|----------|-------|
|
||||
| 0% | Reading ROM file... |
|
||||
| 5% | Loading ROM data... |
|
||||
| 10% | Initializing editors... |
|
||||
| 18% | Loading graphics sheets... |
|
||||
| 26% | Loading overworld... |
|
||||
| 34% | Loading dungeons... |
|
||||
| 42% | Loading screen editor... |
|
||||
| 50%+ | Loading remaining editors... |
|
||||
| 100% | Complete |
|
||||
|
||||
### Monitoring Loading in Console
|
||||
|
||||
```javascript
|
||||
// Check ROM loading status
|
||||
window.yaze.control.getRomStatus()
|
||||
|
||||
// Check graphics loading status
|
||||
window.yazeDebug.arena.getStatus()
|
||||
|
||||
// Get editor state after loading
|
||||
window.yaze.editor.getSnapshot()
|
||||
```
|
||||
|
||||
### Loading Indicator JavaScript API
|
||||
|
||||
```javascript
|
||||
// Called by C++ via WasmLoadingManager
|
||||
window.createLoadingIndicator(id, taskName) // Creates loading overlay
|
||||
window.updateLoadingProgress(id, progress, msg) // Updates progress (0.0-1.0)
|
||||
window.removeLoadingIndicator(id) // Removes loading overlay
|
||||
window.isLoadingCancelled(id) // Check if user cancelled
|
||||
```
|
||||
|
||||
## Files Modified/Created
|
||||
|
||||
### Core Debug Inspector
|
||||
- **`src/web/yaze_debug_inspector.cc`** (renamed from `palette_inspector.cpp`)
|
||||
- Main WASM debug inspector providing JavaScript bindings via Emscripten
|
||||
- Exports functions for palette, ROM, overworld, arena, and emulator debugging
|
||||
|
||||
### JavaScript API
|
||||
- **`src/web/shell.html`**
|
||||
- Added `window.yazeDebug` API object for browser console access
|
||||
- Enhanced `paletteInspector` with better error handling
|
||||
- Added pixel inspector overlay for visual debugging
|
||||
|
||||
### File System Fixes
|
||||
- **`src/web/app.js`**
|
||||
- Fixed race condition in `initPersistentFS()`
|
||||
- Added detection for C++ runtime-initialized IDBFS
|
||||
- Improved user feedback when file system is initializing
|
||||
|
||||
### Build System
|
||||
- **`src/app/app.cmake`**
|
||||
- Updated to include `web/yaze_debug_inspector.cc` for WASM builds
|
||||
|
||||
## Debug API Reference
|
||||
|
||||
### JavaScript API (`window.yazeDebug`)
|
||||
|
||||
```javascript
|
||||
// Check if API is ready
|
||||
window.yazeDebug.isReady() // Returns: boolean
|
||||
|
||||
// Capabilities
|
||||
window.yazeDebug.capabilities // ['palette', 'arena', 'timeline', 'pixel-inspector', 'rom', 'overworld']
|
||||
|
||||
// Palette Debugging
|
||||
window.yazeDebug.palette.getEvents() // Get palette debug events
|
||||
window.yazeDebug.palette.getFullState() // Full palette state with metadata
|
||||
window.yazeDebug.palette.getData() // Raw palette data
|
||||
window.yazeDebug.palette.getComparisons() // Color comparison data
|
||||
window.yazeDebug.palette.samplePixel(x,y) // Sample pixel at coordinates
|
||||
window.yazeDebug.palette.clear() // Clear debug events
|
||||
|
||||
// ROM Debugging
|
||||
window.yazeDebug.rom.getStatus() // ROM load state, size, title
|
||||
window.yazeDebug.rom.readBytes(address, count) // Read up to 256 bytes from ROM
|
||||
window.yazeDebug.rom.getPaletteGroup(groupName, idx) // Get palette group by name
|
||||
|
||||
// Overworld Debugging
|
||||
window.yazeDebug.overworld.getMapInfo(mapId) // Map properties (0-159)
|
||||
window.yazeDebug.overworld.getTileInfo(mapId, x, y) // Tile data at coordinates
|
||||
|
||||
// Arena (Graphics) Debugging
|
||||
window.yazeDebug.arena.getStatus() // Texture queue size, active sheets
|
||||
window.yazeDebug.arena.getSheetInfo(idx) // Details for specific graphics sheet
|
||||
|
||||
// Timeline Analysis
|
||||
window.yazeDebug.timeline.get() // Ordered event timeline
|
||||
|
||||
// AI Analysis Helpers
|
||||
window.yazeDebug.analysis.getSummary() // Diagnostic summary
|
||||
window.yazeDebug.analysis.getHypothesis() // AI hypothesis analysis
|
||||
window.yazeDebug.analysis.getFullState() // Combined full state
|
||||
|
||||
// Utility Functions
|
||||
window.yazeDebug.dumpAll() // Complete state dump (JSON)
|
||||
window.yazeDebug.formatForAI() // Human-readable format for AI
|
||||
```
|
||||
|
||||
### C++ EMSCRIPTEN_BINDINGS
|
||||
|
||||
```cpp
|
||||
EMSCRIPTEN_BINDINGS(yaze_debug_inspector) {
|
||||
// Palette debug functions
|
||||
function("getDungeonPaletteEvents", &getDungeonPaletteEvents);
|
||||
function("getColorComparisons", &getColorComparisons);
|
||||
function("samplePixelAt", &samplePixelAt);
|
||||
function("clearPaletteDebugEvents", &clearPaletteDebugEvents);
|
||||
function("getFullPaletteState", &getFullPaletteState);
|
||||
function("getPaletteData", &getPaletteData);
|
||||
function("getEventTimeline", &getEventTimeline);
|
||||
function("getDiagnosticSummary", &getDiagnosticSummary);
|
||||
function("getHypothesisAnalysis", &getHypothesisAnalysis);
|
||||
|
||||
// Arena debug functions
|
||||
function("getArenaStatus", &getArenaStatus);
|
||||
function("getGfxSheetInfo", &getGfxSheetInfo);
|
||||
|
||||
// ROM debug functions
|
||||
function("getRomStatus", &getRomStatus);
|
||||
function("readRomBytes", &readRomBytes);
|
||||
function("getRomPaletteGroup", &getRomPaletteGroup);
|
||||
|
||||
// Overworld debug functions
|
||||
function("getOverworldMapInfo", &getOverworldMapInfo);
|
||||
function("getOverworldTileInfo", &getOverworldTileInfo);
|
||||
|
||||
// Emulator debug functions
|
||||
function("getEmulatorStatus", &getEmulatorStatus);
|
||||
function("readEmulatorMemory", &readEmulatorMemory);
|
||||
function("getEmulatorVideoState", &getEmulatorVideoState);
|
||||
|
||||
// Combined state
|
||||
function("getFullDebugState", &getFullDebugState);
|
||||
}
|
||||
```
|
||||
|
||||
## File System Issues & Fixes
|
||||
|
||||
### Problem: Silent File Opening Failure
|
||||
|
||||
**Symptoms:**
|
||||
- Clicking "Open ROM" did nothing
|
||||
- No error messages shown to user
|
||||
- Console showed `FS not ready yet; still initializing` even after `IDBFS synced successfully`
|
||||
|
||||
**Root Cause:**
|
||||
The application had two separate IDBFS initialization paths:
|
||||
1. **C++ runtime** (`main.cc` → `MountFilesystems()`) - Initializes IDBFS and logs `IDBFS synced successfully`
|
||||
2. **JavaScript** (`app.js` → `initPersistentFS()`) - Tried to mount IDBFS again, failed silently
|
||||
|
||||
The JavaScript code never set `fsReady = true` because:
|
||||
- It waited for IDBFS to be available
|
||||
- C++ already mounted IDBFS
|
||||
- The JS `FS.syncfs()` callback never fired (already synced)
|
||||
- `fsReady` stayed `false`, blocking all file operations
|
||||
|
||||
**Fix Applied:**
|
||||
|
||||
```javascript
|
||||
function initPersistentFS() {
|
||||
// ... existing code ...
|
||||
|
||||
// Check if /roms already exists (C++ may have already set up the FS)
|
||||
var romsExists = false;
|
||||
try {
|
||||
FS.stat('/roms');
|
||||
romsExists = true;
|
||||
} catch (e) {
|
||||
// Directory doesn't exist
|
||||
}
|
||||
|
||||
if (romsExists) {
|
||||
// C++ already mounted IDBFS, just mark as ready
|
||||
console.log('[WASM] FS already initialized by C++ runtime');
|
||||
fsReady = true;
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
|
||||
// ... continue with JS mounting if needed ...
|
||||
}
|
||||
```
|
||||
|
||||
### Problem: No User Feedback During FS Init
|
||||
|
||||
**Symptoms:**
|
||||
- User clicked "Open ROM" during initialization
|
||||
- Nothing happened, no error shown
|
||||
- Only console warning logged
|
||||
|
||||
**Fix Applied:**
|
||||
|
||||
```javascript
|
||||
function ensureFSReady(showAlert = true) {
|
||||
if (fsReady && typeof FS !== 'undefined') return true;
|
||||
if (fsInitPromise) {
|
||||
console.warn('FS not ready yet; still initializing.');
|
||||
if (showAlert) {
|
||||
// Show status message in header
|
||||
var status = document.getElementById('header-status');
|
||||
if (status) {
|
||||
status.textContent = 'File system initializing... please wait';
|
||||
status.style.color = '#ffaa00';
|
||||
setTimeout(function() {
|
||||
status.textContent = 'Ready';
|
||||
status.style.color = '';
|
||||
}, 3000);
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
// ... rest of function
|
||||
}
|
||||
```
|
||||
|
||||
### Problem: JSON Parse Errors in Problems Panel
|
||||
|
||||
**Symptoms:**
|
||||
- Console error: `Failed to parse palette events JSON: unexpected character at line 1 column 2`
|
||||
- Problems panel showed errors when no palette events existed
|
||||
|
||||
**Root Cause:**
|
||||
- `Module.getDungeonPaletteEvents()` returned empty string `""` instead of `"[]"`
|
||||
- JSON.parse failed on empty string
|
||||
|
||||
**Fix Applied:**
|
||||
|
||||
```javascript
|
||||
// Handle empty or invalid responses
|
||||
if (!eventsStr || eventsStr.length === 0 || eventsStr === '[]') {
|
||||
var list = document.getElementById('problems-list');
|
||||
if (list) {
|
||||
list.innerHTML = '<div class="problems-empty">No palette events yet.</div>';
|
||||
}
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
## Valid Palette Group Names
|
||||
|
||||
For `getRomPaletteGroup(groupName, paletteIndex)`:
|
||||
|
||||
| Group Name | Description |
|
||||
|------------|-------------|
|
||||
| `ow_main` | Overworld main palettes |
|
||||
| `ow_aux` | Overworld auxiliary palettes |
|
||||
| `ow_animated` | Overworld animated palettes |
|
||||
| `hud` | HUD/UI palettes |
|
||||
| `global_sprites` | Global sprite palettes |
|
||||
| `armors` | Link armor palettes |
|
||||
| `swords` | Sword palettes |
|
||||
| `shields` | Shield palettes |
|
||||
| `sprites_aux1` | Sprite auxiliary 1 |
|
||||
| `sprites_aux2` | Sprite auxiliary 2 |
|
||||
| `sprites_aux3` | Sprite auxiliary 3 |
|
||||
| `dungeon_main` | Dungeon main palettes |
|
||||
| `grass` | Grass palettes |
|
||||
| `3d_object` | 3D object palettes |
|
||||
| `ow_mini_map` | Overworld minimap palettes |
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
# Build WASM with debug infrastructure
|
||||
./scripts/build-wasm.sh debug
|
||||
|
||||
# Serve locally (sets COOP/COEP headers for SharedArrayBuffer)
|
||||
./scripts/serve-wasm.sh 8080
|
||||
```
|
||||
|
||||
**Important:** The dev server must set COOP/COEP headers for SharedArrayBuffer support. Use `./scripts/serve-wasm.sh` which handles this automatically.
|
||||
|
||||
## Testing the API
|
||||
|
||||
Open browser console after loading the application:
|
||||
|
||||
```javascript
|
||||
// Verify API is loaded
|
||||
window.yazeDebug.isReady()
|
||||
|
||||
// Get ROM status (after loading a ROM)
|
||||
window.yazeDebug.rom.getStatus()
|
||||
|
||||
// Read bytes from ROM address 0x10000
|
||||
window.yazeDebug.rom.readBytes(0x10000, 32)
|
||||
|
||||
// Get dungeon palette group
|
||||
window.yazeDebug.rom.getPaletteGroup('dungeon_main', 0)
|
||||
|
||||
// Get overworld map info for Light World map 0
|
||||
window.yazeDebug.overworld.getMapInfo(0)
|
||||
|
||||
// Full debug dump for AI analysis
|
||||
window.yazeDebug.dumpAll()
|
||||
```
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **Emulator debug functions** require emulator to be initialized and running
|
||||
2. **Overworld tile info** requires overworld to be loaded in editor
|
||||
3. **Palette sampling** works on the visible canvas area only
|
||||
4. **ROM byte reading** limited to 256 bytes per call to prevent large responses
|
||||
5. **Memory reading** from emulator limited to 256 bytes per call
|
||||
6. **Loading indicator** managed by C++ `WasmLoadingManager` - don't create separate JS indicators
|
||||
|
||||
## Quick Start for AI Agents
|
||||
|
||||
1. **Load a ROM**: Use the file picker or drag-and-drop a `.sfc` file
|
||||
2. **Wait for loading**: Monitor progress via loading overlay or `window.yaze.control.getRomStatus()`
|
||||
3. **Verify ready state**: `window.yaze.control.isReady()` should return `true`
|
||||
4. **Start debugging**: Use `window.yazeDebug.dumpAll()` for full state or specific APIs
|
||||
|
||||
```javascript
|
||||
// Complete verification sequence
|
||||
if (window.yaze.control.isReady()) {
|
||||
const status = window.yaze.control.getRomStatus();
|
||||
if (status.loaded) {
|
||||
console.log('ROM loaded:', status.filename);
|
||||
console.log('AI-ready dump:', window.yazeDebug.formatForAI());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Gemini Antigravity AI Integration
|
||||
|
||||
The web interface includes dedicated tools for AI assistants that struggle to discover ImGui elements.
|
||||
|
||||
### window.aiTools API
|
||||
|
||||
High-level helper functions with console output for AI readability:
|
||||
|
||||
```javascript
|
||||
// Get full application state (ROM, editor, cards, layouts)
|
||||
window.aiTools.getAppState()
|
||||
|
||||
// Get current editor snapshot
|
||||
window.aiTools.getEditorState()
|
||||
|
||||
// Card management
|
||||
window.aiTools.getVisibleCards()
|
||||
window.aiTools.getAvailableCards()
|
||||
window.aiTools.showCard('Room Selector')
|
||||
window.aiTools.hideCard('Object Editor')
|
||||
|
||||
// Navigation
|
||||
window.aiTools.navigateTo('room:0') // Go to dungeon room
|
||||
window.aiTools.navigateTo('map:5') // Go to overworld map
|
||||
window.aiTools.navigateTo('Dungeon') // Switch editor
|
||||
|
||||
// Data access
|
||||
window.aiTools.getRoomData(0) // Dungeon room data
|
||||
window.aiTools.getMapData(0) // Overworld map data
|
||||
|
||||
// Documentation
|
||||
window.aiTools.dumpAPIReference() // Complete API reference
|
||||
```
|
||||
|
||||
### Nav Bar Dropdowns
|
||||
|
||||
The web UI includes four dedicated dropdown menus:
|
||||
|
||||
| Dropdown | Purpose |
|
||||
|----------|---------|
|
||||
| **Editor** | Quick switch between all 13 editors |
|
||||
| **Emulator** | Show/Run/Pause/Step/Reset + Memory Viewer |
|
||||
| **Layouts** | Preset card configurations |
|
||||
| **AI Tools** | All `window.aiTools` functions via UI |
|
||||
|
||||
### Command Palette (Ctrl+K)
|
||||
|
||||
All AI tools accessible via palette:
|
||||
- `Editor: <name>` - Switch editors
|
||||
- `Emulator: <action>` - Control emulator
|
||||
- `AI: Get App State` - Application state
|
||||
- `AI: API Reference` - Full API documentation
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] Add dungeon room state debugging
|
||||
- [ ] Add sprite debugging
|
||||
- [ ] Add memory watch points
|
||||
- [ ] Add breakpoint support for emulator
|
||||
- [ ] Add texture atlas visualization
|
||||
- [ ] Add palette history tracking
|
||||
286
docs/internal/agents/archive/wasm-docs-2025/wasm_dev_status.md
Normal file
286
docs/internal/agents/archive/wasm-docs-2025/wasm_dev_status.md
Normal file
@@ -0,0 +1,286 @@
|
||||
# WASM / Web Agent Integration Status
|
||||
|
||||
**Last Updated:** November 25, 2025
|
||||
**Status:** Functional MVP with Agent APIs (ROM loading fixed, loading progress added, control APIs implemented, performance optimizations applied)
|
||||
|
||||
## Overview
|
||||
This document tracks the development state of the `yaze` WASM web application, specifically focusing on the AI Agent integration (`z3ed` console) and the modern UI overhaul.
|
||||
|
||||
## 1. Completed Features
|
||||
|
||||
### ROM Loading & Initialization (November 2025 Fixes)
|
||||
* **ROM File Validation (`rom_file_manager.cc`):**
|
||||
* Fixed minimum ROM size check from 1MB to 512KB (was rejecting valid 1MB Zelda 3 ROMs)
|
||||
* **CMake WASM Configuration (`app.cmake`):**
|
||||
* Added `MODULARIZE=1` and `EXPORT_NAME='createYazeModule'` to match `app.js` expectations
|
||||
* Added missing exports: `_yazeHandleDroppedFile`, `_yazeHandleDropError`, `_yazeHandleDragEnter`, `_yazeHandleDragLeave`, `_malloc`, `_free`
|
||||
* Added missing runtime methods: `lengthBytesUTF8`, `IDBFS`, `allocateUTF8`
|
||||
* **JavaScript Fixes (`filesystem_manager.js`):**
|
||||
* Fixed `Module.ccall` return type from `'null'` (string) to `null`
|
||||
* Fixed direct function fallback to properly allocate/free memory for string parameters
|
||||
* **Drop Zone (`drop_zone.js`):**
|
||||
* Disabled duplicate auto-initialization (conflicted with C++ handler)
|
||||
* Now delegates to `FilesystemManager.handleRomUpload` instead of calling non-existent function
|
||||
* **Loading Progress (`editor_manager.cc`):**
|
||||
* Added `WasmLoadingManager` integration to `LoadAssets()`
|
||||
* Shows progress for each editor: "Loading overworld...", "Loading dungeons...", etc.
|
||||
* **UI Streamlining (`shell.html`, `app.js`):**
|
||||
* Removed HTML welcome screen - canvas is always visible
|
||||
* Loading overlay shows during initialization with status messages
|
||||
|
||||
### AI Agent Integration
|
||||
* **Core Bridge (`wasm_terminal_bridge.cc`):**
|
||||
* Exposes `Z3edProcessCommand` to JavaScript.
|
||||
* Exposes `GetGlobalBrowserAIService()` and `GetGlobalRom()` to C++ handlers.
|
||||
* **Browser Agent (`browser_agent.cc`):**
|
||||
* **`agent chat`**: Fully functional with conversation history. Uses `std::thread` for non-blocking AI calls.
|
||||
* **`agent plan`**: Generates text-based implementation plans (asynchronous).
|
||||
* **`agent diff`**: Shows the "pending plan" (conceptual diff).
|
||||
* **`agent list/describe`**: Introspects ROM resources via `ResourceCatalog`.
|
||||
* **`agent todo`**: Fully implemented with persistent storage.
|
||||
* **Browser AI Service** (`src/cli/service/ai/browser_ai_service.cc`):
|
||||
* Implements `AIService` interface for browser-based AI calls
|
||||
* Uses `IHttpClient` from network abstraction layer (CORS-compatible)
|
||||
* Supports Gemini API (text and vision models)
|
||||
* Secures API keys via sessionStorage (cleared on tab close)
|
||||
* Comprehensive error handling with `absl::Status`
|
||||
* **Browser Storage** (`src/app/platform/wasm/wasm_browser_storage.cc`):
|
||||
* Non-hardcoded API key management via sessionStorage/localStorage
|
||||
* User-provided keys, never embedded in binary
|
||||
* Namespaced storage to avoid conflicts
|
||||
* **Persistence (`todo_manager.cc`):**
|
||||
* Updated to use `WasmStorage` (IndexedDB) when compiled for Emscripten. TODOs persist across reloads.
|
||||
|
||||
### UI & UX
|
||||
* **Drag & Drop (`wasm_drop_handler.cc`):**
|
||||
* Supports `.sfc`, `.smc`, `.zip`.
|
||||
* Automatically writes to `/roms/` in MEMFS and loads the ROM.
|
||||
* Stubbed support for `.pal` / `.tpl`.
|
||||
* **Modern Interface:**
|
||||
* **`main.css`**: Unified design system (VS Code dark theme variables).
|
||||
* **`app.js`**: Extracted logic from `shell.html`. Handles terminal resize, zoom, and PWA updates.
|
||||
* **Components**: `terminal.css`, `collab_console.css`, etc., updated to use CSS variables.
|
||||
|
||||
### WASM Control APIs (November 2025)
|
||||
|
||||
The WASM build now exposes comprehensive JavaScript APIs for programmatic control, enabling LLM agents with DOM access to interact with the editor.
|
||||
|
||||
#### Editor State APIs (`window.yaze.editor`)
|
||||
* **`getSnapshot()`**: Get current editor state (type, ROM status, active data)
|
||||
* **`getCurrentRoom()`**: Get dungeon room info (room_id, active_rooms, visible_cards)
|
||||
* **`getCurrentMap()`**: Get overworld map info (map_id, world, world_name)
|
||||
* **`getSelection()`**: Get current selection in active editor
|
||||
|
||||
#### Read-only Data APIs (`window.yaze.data`)
|
||||
* **Dungeon Data:**
|
||||
* `getRoomTiles(roomId)` - Get room tile data (layer1, layer2)
|
||||
* `getRoomObjects(roomId)` - Get objects in a room
|
||||
* `getRoomProperties(roomId)` - Get room properties (music, palette, tileset)
|
||||
* **Overworld Data:**
|
||||
* `getMapTiles(mapId)` - Get map tile data
|
||||
* `getMapEntities(mapId)` - Get entities (entrances, exits, items, sprites)
|
||||
* `getMapProperties(mapId)` - Get map properties (gfx_group, palette, area_size)
|
||||
* **Palette Data:**
|
||||
* `getPalette(group, id)` - Get palette colors
|
||||
* `getPaletteGroups()` - List available palette groups
|
||||
|
||||
#### GUI Automation APIs (`window.yaze.gui`)
|
||||
* **Element Discovery:**
|
||||
* `discover()` - List all interactive UI elements with metadata
|
||||
* `getElementBounds(id)` - Get element position and dimensions (backed by `WidgetIdRegistry`)
|
||||
* `waitForElement(id, timeout)` - Async wait for element to appear
|
||||
* **Interaction:**
|
||||
* `click(target)` - Click by element ID or {x, y} coordinates
|
||||
* `doubleClick(target)` - Double-click
|
||||
* `drag(from, to, steps)` - Drag operation
|
||||
* `pressKey(key, modifiers)` - Send keyboard input
|
||||
* `type(text, delay)` - Type text string
|
||||
* `scroll(dx, dy)` - Scroll canvas
|
||||
* **Utility:**
|
||||
* `takeScreenshot(format)` - Capture canvas as base64
|
||||
* `getCanvasInfo()` - Get canvas dimensions
|
||||
* `isReady()` - Check if GUI API is ready
|
||||
|
||||
**Widget Tracking Infrastructure** (November 2025):
|
||||
The `WidgetIdRegistry` system tracks all ImGui widget bounds in real-time:
|
||||
- **Real-time Bounds**: `GetUIElementTree()` and `GetUIElementBounds()` query live widget positions via `WidgetIdRegistry`
|
||||
- **Frame Lifecycle**: Integrated into `Controller::OnLoad()` with `BeginFrame()` and `EndFrame()` hooks
|
||||
- **Bounds Data**: Includes `min_x`, `min_y`, `max_x`, `max_y` for accurate GUI automation
|
||||
- **Metadata**: Returns `imgui_id`, `last_seen_frame`, widget type, visibility, enabled state
|
||||
- **Key Files**: `src/app/gui/automation/widget_id_registry.h`, `src/app/gui/automation/widget_measurement.h`
|
||||
|
||||
#### Control APIs (`window.yaze.control`)
|
||||
* **Editor Control:** `switchEditor()`, `getCurrentEditor()`, `getAvailableEditors()`
|
||||
* **Card Control:** `openCard()`, `closeCard()`, `toggleCard()`, `getVisibleCards()`
|
||||
* **Layout Control:** `setCardLayout()`, `getAvailableLayouts()`, `saveCurrentLayout()`
|
||||
* **Menu Actions:** `triggerMenuAction()`, `getAvailableMenuActions()`
|
||||
* **Session Control:** `getSessionInfo()`, `createSession()`, `switchSession()`
|
||||
* **ROM Control:** `getRomStatus()`, `readRomBytes()`, `writeRomBytes()`, `saveRom()`
|
||||
|
||||
#### Extended UI Control APIs (November 2025)
|
||||
|
||||
**Async Editor Switching (`yazeDebug.switchToEditorAsync`)**:
|
||||
Promise-based editor switching with operation tracking for reliable LLM automation.
|
||||
* Returns `Promise<{success, editor, session_id, error}>` after editor transition completes
|
||||
* Supports all 14 editor types: Assembly, Dungeon, Graphics, Music, Overworld, Palette, Screen, Sprite, Message, Hex, Agent, Settings, World, Map
|
||||
* 5-second timeout with proper error reporting
|
||||
|
||||
**Card Control API (`yazeDebug.cards`)**:
|
||||
* `show(cardId)` - Show a specific card by ID (e.g., "dungeon.room_selector")
|
||||
* `hide(cardId)` - Hide a specific card
|
||||
* `toggle(cardId)` - Toggle card visibility
|
||||
* `getState()` - Get visibility state of all cards
|
||||
* `getInCategory(category)` - List cards in a category (dungeon, overworld, etc.)
|
||||
* `showGroup(groupName)` - Show predefined card groups (dungeon_editing, overworld_editing, etc.)
|
||||
* `hideGroup(groupName)` - Hide predefined card groups
|
||||
* `getGroups()` - List available card groups
|
||||
|
||||
**Sidebar Control API (`yazeDebug.sidebar`)**:
|
||||
* `isTreeView()` - Check if tree view mode is active
|
||||
* `setTreeView(enabled)` - Switch between tree view (200px) and icon mode (48px)
|
||||
* `toggle()` - Toggle between view modes
|
||||
* `getState()` - Get sidebar state (mode, width, collapsed)
|
||||
|
||||
**Right Panel Control API (`yazeDebug.rightPanel`)**:
|
||||
* `open(panelName)` - Open specific panel: properties, agent, proposals, settings, help
|
||||
* `close()` - Close current panel
|
||||
* `toggle(panelName)` - Toggle panel visibility
|
||||
* `getState()` - Get panel state (active, expanded, width)
|
||||
* `openProperties()` - Convenience method for properties panel
|
||||
* `openAgent()` - Convenience method for agent chat panel
|
||||
|
||||
**Tree View Sidebar**:
|
||||
New hierarchical sidebar mode (200px wide) with:
|
||||
* Category icons and expandable tree nodes
|
||||
* Checkboxes for each card with visibility toggles
|
||||
* Visible count badges per category
|
||||
* "Show All" / "Hide All" buttons per category
|
||||
* Toggle button to switch to icon mode
|
||||
|
||||
**Selection Properties Panel**:
|
||||
New right-side panel for editing selected entities:
|
||||
* Context-aware property display based on selection type
|
||||
* Supports dungeon rooms, objects, sprites, entrances
|
||||
* Supports overworld maps, tiles, sprites, entrances, exits, items
|
||||
* Supports graphics sheets and palettes
|
||||
* Position/size editors with clamping
|
||||
* Byte/word property editors with hex display
|
||||
* Flag property editors with checkboxes
|
||||
* Advanced and raw data toggles
|
||||
|
||||
**Key Files:**
|
||||
* `src/app/platform/wasm/wasm_control_api.cc` - C++ implementation
|
||||
* `src/app/platform/wasm/wasm_control_api.h` - API declarations
|
||||
* `src/web/core/agent_automation.js` - GUI automation layer
|
||||
* `src/web/debug/yaze_debug_inspector.cc` - Extended WASM bindings
|
||||
* `src/app/editor/system/editor_card_registry.cc` - Tree view sidebar implementation
|
||||
* `src/app/editor/ui/right_panel_manager.cc` - Right panel management
|
||||
* `src/app/editor/ui/selection_properties_panel.cc` - Properties panel implementation
|
||||
|
||||
### Performance Optimizations & Bug Fixes (November 2025)
|
||||
|
||||
A comprehensive audit and fix of the WASM web layer was performed to address performance issues, memory leaks, and race conditions.
|
||||
|
||||
#### JavaScript Performance Fixes (`app.js`)
|
||||
* **Event Sanitization Optimization:**
|
||||
* Removed redundant document-level event listeners (canvas-only now)
|
||||
* Added WeakMap caching to avoid re-sanitizing the same event objects
|
||||
* Optimized to check only relevant properties per event type category
|
||||
* ~50% reduction in sanitization overhead
|
||||
* **Console Log Buffer:**
|
||||
* Replaced O(n) `Array.shift()` with O(1) circular buffer implementation
|
||||
* Uses modulo arithmetic for constant-time log rotation
|
||||
* **Polling Cleanup:**
|
||||
* Added timeout tracking and max retry limits for module initialization
|
||||
* Proper interval cleanup when components are destroyed
|
||||
* Added `window.YAZE_MODULE_READY` flag for reliable initialization detection
|
||||
|
||||
#### Memory Leak Fixes
|
||||
* **Service Worker Cache (`service-worker.js`):**
|
||||
* Added `MAX_RUNTIME_CACHE_SIZE` (50 entries) with LRU eviction
|
||||
* New `trimRuntimeCache()` function enforces size limits
|
||||
* `addToRuntimeCacheWithEviction()` wrapper for cache operations
|
||||
* **Confirmation Callbacks (`wasm_error_handler.cc`):**
|
||||
* Added `CallbackEntry` struct with timestamps for timeout tracking
|
||||
* Auto-cleanup of callbacks older than 5 minutes
|
||||
* Page unload handler via `js_register_cleanup_handler()`
|
||||
* **Loading Indicators (`loading_indicator.js`):**
|
||||
* Added try-catch error handling to ensure cleanup on errors
|
||||
* Stale indicator cleanup (5-minute timeout)
|
||||
* Periodic cleanup interval with proper lifecycle management
|
||||
|
||||
#### Race Condition Fixes
|
||||
* **Module Initialization (`app.js`):**
|
||||
* Added `window.YAZE_MODULE_READY` flag set AFTER promise resolves
|
||||
* Updated `waitForModule()` to check both Module existence AND ready flag
|
||||
* Prevents code from seeing incomplete Module state
|
||||
* **FS Ready State (`filesystem_manager.js`):**
|
||||
* Restructured `initPersistentFS()` with synchronous lock pattern
|
||||
* Promise created immediately before async operations
|
||||
* Eliminates race where two calls could create duplicate promises
|
||||
* **Redundant FS Exposure:**
|
||||
* Added `fsExposed` flag to prevent wasteful redundant calls
|
||||
* Reduced from 3 setTimeout calls to 1 conditional retry
|
||||
|
||||
#### C++ WASM Fixes
|
||||
* **Memory Safety (`wasm_storage.cc`):**
|
||||
* Added `free(data_ptr)` in error paths of `LoadRom()` to prevent memory leaks
|
||||
* Ensures allocated memory is freed even when operations fail
|
||||
* **Cleanup Handlers (`wasm_error_handler.cc`):**
|
||||
* Added `cleanupConfirmCallbacks()` function for page unload
|
||||
* Registered via `js_register_cleanup_handler()` in `Initialize()`
|
||||
|
||||
#### Drop Zone Optimization (`drop_zone.js`, `filesystem_manager.js`)
|
||||
* **Eliminated Double File Reading:**
|
||||
* Added new `FilesystemManager.handleRomData(filename, data)` method
|
||||
* Accepts pre-read `Uint8Array` instead of `File` object
|
||||
* Drop zone now passes already-read data instead of re-reading
|
||||
* Reduces CPU and memory usage for ROM uploads
|
||||
|
||||
**Key Files Modified:**
|
||||
* `src/web/app.js` - Event sanitization, console buffer, module init
|
||||
* `src/web/core/filesystem_manager.js` - FS init race fix, handleRomData
|
||||
* `src/web/core/loading_indicator.js` - Stale cleanup, error handling
|
||||
* `src/web/components/drop_zone.js` - Use handleRomData
|
||||
* `src/web/pwa/service-worker.js` - Cache eviction
|
||||
* `src/app/platform/wasm/wasm_storage.cc` - Memory free on error
|
||||
* `src/app/platform/wasm/wasm_error_handler.cc` - Callback cleanup
|
||||
|
||||
## 2. Technical Debt & Known Issues
|
||||
|
||||
* **`SimpleChatSession`**: This C++ class relies on `VimMode` and raw TTY input, which is incompatible with WASM. We bypassed this by implementing a custom `HandleChatCommand` in `browser_agent.cc`. The original `SimpleChatSession` remains unused in the browser build.
|
||||
* **Emscripten Fetch Blocking**: The `EmscriptenHttpClient` implementation contains a `cv.wait()` which blocks the main thread. We worked around this by spawning `std::thread` in the command handlers, but the HTTP client itself remains synchronous-blocking if called directly on the main thread.
|
||||
* **Single-Threaded Rendering**: Dungeon graphics loading happens on the main thread (`DungeonEditorV2::DrawRoomTab`), causing UI freezes on large ROMs.
|
||||
|
||||
## 3. Next Steps / Roadmap
|
||||
|
||||
### Short Term
|
||||
1. **Palette Import**: Implement the logic in `wasm_drop_handler.cc` (or `main.cc` callback) to parse `.pal` files and apply them to `PaletteManager`.
|
||||
2. **Deep Linking**: Add logic to `app.js` and `main.cc` to parse URL query parameters (e.g., `?rom=url`) for easy sharing.
|
||||
|
||||
### Medium Term
|
||||
1. **In-Memory Proposal Registry**:
|
||||
* Implement a `WasmProposalRegistry` that mimics the file-based `ProposalRegistry`.
|
||||
* Store "sandboxes" as `Rom` copies in memory (or IndexedDB blobs).
|
||||
* Enable `agent apply` to execute the plans generated by `agent plan`.
|
||||
2. **Multithreaded Graphics**:
|
||||
* Refactor `DungeonEditorV2` to use `WasmWorkerPool` for `LoadRoomGraphics`.
|
||||
* Requires decoupling `Room` data structures from the loading logic to pass data across threads safely.
|
||||
|
||||
## 4. Key Files
|
||||
|
||||
* **C++ Logic**:
|
||||
* `src/cli/handlers/agent/browser_agent.cc` (Agent commands)
|
||||
* `src/cli/wasm_terminal_bridge.cc` (JS <-> C++ Bridge)
|
||||
* `src/app/platform/wasm/wasm_drop_handler.cc` (File drag & drop)
|
||||
* `src/app/platform/wasm/wasm_control_api.cc` (Control API implementation)
|
||||
* `src/app/platform/wasm/wasm_control_api.h` (Control API declarations)
|
||||
* `src/cli/service/agent/todo_manager.cc` (Persistence logic)
|
||||
|
||||
* **Web Frontend**:
|
||||
* `src/web/shell.html` (Entry point)
|
||||
* `src/web/app.js` (Main UI logic)
|
||||
* `src/web/core/agent_automation.js` (GUI Automation layer)
|
||||
* `src/web/styles/main.css` (Theme definitions)
|
||||
* `src/web/components/terminal.js` (Console UI component)
|
||||
* `src/web/components/collaboration_ui.js` (Collaboration UI)
|
||||
@@ -0,0 +1,687 @@
|
||||
### WASM Debugging Guide: Dungeon Editor
|
||||
|
||||
**Status:** Current (November 2025)
|
||||
**Last Updated:** 2025-11-25
|
||||
**Version:** 2.3.0
|
||||
|
||||
The WASM build includes a powerful, hidden "Debug Inspector" that bypasses the need for GDB/LLDB by exposing C++ state directly to the browser console.
|
||||
|
||||
**Cross-Reference:** For comprehensive debug API reference, see `wasm-debug-infrastructure.md` and `wasm-yazeDebug-api-reference.md`.
|
||||
|
||||
#### 1. The "God Mode" Console Inspector
|
||||
The file `src/web/yaze_debug_inspector.cc` binds C++ functions to the global `Module` object. You can invoke these directly from Chrome/Firefox DevTools.
|
||||
|
||||
**Status & State:**
|
||||
* `Module.getEmulatorStatus()`: Returns JSON with CPU registers (A, X, Y, PC), Flags, and PPU state.
|
||||
* `Module.getFullDebugState()`: Returns a massive JSON dump suitable for pasting into an AI prompt for analysis.
|
||||
* `Module.getArenaStatus()`: Checks the memory arena used for dungeon rendering (vital for "out of memory" rendering glitches).
|
||||
|
||||
**Memory Inspection:**
|
||||
* `Module.readEmulatorMemory(addr, length)`: Reads WRAM/SRAM.
|
||||
* *Example:* Check Link's X-Coordinate ($20): `Module.readEmulatorMemory(0x7E0020, 2)`
|
||||
* `Module.readRom(addr, length)`: Verifies if your ROM patch actually applied in memory.
|
||||
|
||||
**Graphics & Palette:**
|
||||
* `Module.getDungeonPaletteEvents()`: Returns a log of recent palette uploads. Use this if colors look wrong or "flashy" in the editor.
|
||||
|
||||
#### 2. The Command Line Bridge
|
||||
The terminal you see in the web app isn't just a UI toy; it's a direct bridge to the C++ backend.
|
||||
|
||||
* **Architecture**: `src/web/terminal.js` captures your keystrokes and calls `Module.ccall('Z3edProcessCommand', ...)` which routes to `src/cli/wasm_terminal_bridge.cc`.
|
||||
* **Debug Tip**: If the editor UI freezes, the terminal often remains responsive (running on a separate event cadence). You can use it to:
|
||||
1. Save your work: `save`
|
||||
2. Dump state: (If a custom command exists)
|
||||
3. Reset the emulator.
|
||||
|
||||
#### 3. The Hidden "Debug Controls" Card
|
||||
The code in `src/app/editor/dungeon/dungeon_editor_v2.cc` contains a function `DrawDebugControlsCard()`, controlled by the boolean `show_debug_controls_`.
|
||||
|
||||
* **Current Status**: This is currently **hidden** by default and likely has no UI toggle in the public build.
|
||||
* **Recommended Task**: Create a `z3ed` CLI command to toggle this boolean.
|
||||
* *Implementation*: Add a command `editor debug toggle` in `wasm_terminal_bridge.cc` that finds the active `DungeonEditorV2` instance and flips `show_debug_controls_ = !show_debug_controls_`. This would give you on-screen access to render passes and layer toggles.
|
||||
|
||||
#### 4. Feature Parity
|
||||
There are **NO** `__EMSCRIPTEN__` checks inside the `src/app/editor/dungeon/` logic.
|
||||
* **Implication**: If a logic bug exists in WASM, it likely exists in the Native macOS/Linux build too. Reproduce bugs on Desktop first for easier debugging (breakpoints, etc.), then verify the fix on Web.
|
||||
* **Exception**: Rendering glitches are likely WASM-specific due to the single-threaded `TickFrame` loop vs. the multi-threaded desktop renderer.
|
||||
|
||||
#### 5. Thread Pool Configuration
|
||||
The WASM build uses a fixed thread pool (`PTHREAD_POOL_SIZE=8` in CMakePresets.json).
|
||||
* **Warning Signs**: If you see "Tried to spawn a new thread, but the thread pool is exhausted", heavy parallel operations are exceeding the pool size.
|
||||
* **Fix**: Increase `PTHREAD_POOL_SIZE` in CMakePresets.json and rebuild with `--clean`
|
||||
* **Root Cause**: Often happens during ROM loading when multiple graphics sheets are decompressed in parallel.
|
||||
|
||||
#### 6. Memory Configuration
|
||||
The WASM build uses optimized memory settings to reduce heap resize operations:
|
||||
|
||||
| Setting | Value | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `INITIAL_MEMORY` | 256MB | Reduces heap resizing during ROM load |
|
||||
| `MAXIMUM_MEMORY` | 1GB | Prevents runaway allocations |
|
||||
| `STACK_SIZE` | 8MB | Handles recursive asset decompression |
|
||||
|
||||
* **Warning Signs**: Console shows `_emscripten_resize_heap` calls during loading
|
||||
* **Common Causes**: Overworld map loading (~160MB for 160 maps), sprite preview buffers, dungeon object emulator
|
||||
* **Optimization Applied**: Lazy initialization for SNES emulator instances and sprite preview buffers
|
||||
|
||||
#### 7. ROM Loading Progress
|
||||
The C++ `WasmLoadingManager` controls loading progress display. Monitor loading status:
|
||||
|
||||
```javascript
|
||||
// Check if ROM is loaded
|
||||
window.yaze.control.getRomStatus()
|
||||
// Returns: { loaded: true/false, filename: "...", title: "...", size: ... }
|
||||
|
||||
// Check arena (graphics) status
|
||||
window.yazeDebug.arena.getStatus()
|
||||
|
||||
// Full editor state after loading
|
||||
window.yaze.editor.getSnapshot()
|
||||
```
|
||||
|
||||
**Loading Progress Stages:**
|
||||
| Progress | Stage |
|
||||
|----------|-------|
|
||||
| 10% | Initializing editors... |
|
||||
| 18% | Loading graphics sheets... |
|
||||
| 26% | Loading overworld... |
|
||||
| 34% | Loading dungeons... |
|
||||
| 42%+ | Loading remaining editors... |
|
||||
| 100% | Complete |
|
||||
|
||||
#### 8. Prompting AI Agents (Claude Code / Gemini Antigravity)
|
||||
|
||||
This section provides precise prompts and workflows for AI agents to interact with the YAZE WASM app.
|
||||
|
||||
##### Step 1: Verify the App is Ready
|
||||
|
||||
Before any operations, the AI must confirm the WASM module is initialized:
|
||||
|
||||
```javascript
|
||||
// Check module ready state
|
||||
window.YAZE_MODULE_READY // Should be true
|
||||
|
||||
// Check if APIs are available
|
||||
typeof window.yaze !== 'undefined' &&
|
||||
typeof window.yaze.control !== 'undefined' // Should be true
|
||||
```
|
||||
|
||||
**If not ready**, wait and retry:
|
||||
```javascript
|
||||
// Poll until ready (max 10 seconds)
|
||||
async function waitForYaze() {
|
||||
for (let i = 0; i < 100; i++) {
|
||||
if (window.YAZE_MODULE_READY && window.yaze?.control) return true;
|
||||
await new Promise(r => setTimeout(r, 100));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
await waitForYaze();
|
||||
```
|
||||
|
||||
##### Step 2: Load a ROM File
|
||||
|
||||
**Option A: User Drag-and-Drop (Recommended)**
|
||||
Prompt the user: *"Please drag and drop your Zelda 3 ROM (.sfc or .smc) onto the canvas."*
|
||||
|
||||
Then verify:
|
||||
```javascript
|
||||
// Check if ROM loaded successfully
|
||||
const status = window.yaze.control.getRomStatus();
|
||||
console.log(status);
|
||||
// Expected: { loaded: true, filename: "zelda3.sfc", title: "...", size: 1048576 }
|
||||
```
|
||||
|
||||
**Option B: Check if ROM Already Loaded**
|
||||
```javascript
|
||||
window.yaze.control.getRomStatus().loaded // true if ROM is present
|
||||
```
|
||||
|
||||
**Option C: Load from IndexedDB (if previously saved)**
|
||||
```javascript
|
||||
// List saved ROMs
|
||||
FilesystemManager.listSavedRoms && FilesystemManager.listSavedRoms();
|
||||
```
|
||||
|
||||
##### Step 3: Open the Dungeon Editor
|
||||
|
||||
```javascript
|
||||
// Switch to dungeon editor
|
||||
window.yaze.control.switchEditor('Dungeon');
|
||||
|
||||
// Verify switch was successful
|
||||
const snapshot = window.yaze.editor.getSnapshot();
|
||||
console.log(snapshot.editor_type); // Should be "Dungeon"
|
||||
```
|
||||
|
||||
##### Step 4: Navigate to a Specific Room
|
||||
|
||||
```javascript
|
||||
// Get current room info
|
||||
window.yaze.editor.getCurrentRoom();
|
||||
// Returns: { room_id: 0, active_rooms: [...], visible_cards: [...] }
|
||||
|
||||
// Navigate to room 42 (Hyrule Castle Entrance)
|
||||
window.aiTools.navigateTo('room:42');
|
||||
|
||||
// Or use control API
|
||||
window.yaze.control.openCard('Room 42');
|
||||
```
|
||||
|
||||
##### Step 5: Inspect Room Data
|
||||
|
||||
```javascript
|
||||
// Get room properties
|
||||
const props = window.yaze.data.getRoomProperties(42);
|
||||
console.log(props);
|
||||
// Returns: { music: 5, palette: 2, tileset: 7, ... }
|
||||
|
||||
// Get room objects (chests, torches, blocks, etc.)
|
||||
const objects = window.yaze.data.getRoomObjects(42);
|
||||
console.log(objects);
|
||||
|
||||
// Get room tile data
|
||||
const tiles = window.yaze.data.getRoomTiles(42);
|
||||
console.log(tiles.layer1, tiles.layer2);
|
||||
```
|
||||
|
||||
##### Example AI Prompt Workflow
|
||||
|
||||
**User asks:** "Show me the objects in room 42 of the dungeon editor"
|
||||
|
||||
**AI should execute:**
|
||||
```javascript
|
||||
// 1. Verify ready
|
||||
if (!window.YAZE_MODULE_READY) throw new Error("WASM not ready");
|
||||
|
||||
// 2. Check ROM
|
||||
const rom = window.yaze.control.getRomStatus();
|
||||
if (!rom.loaded) throw new Error("No ROM loaded - please drag a ROM file onto the canvas");
|
||||
|
||||
// 3. Switch to dungeon editor
|
||||
window.yaze.control.switchEditor('Dungeon');
|
||||
|
||||
// 4. Navigate to room
|
||||
window.aiTools.navigateTo('room:42');
|
||||
|
||||
// 5. Get and display data
|
||||
const objects = window.yaze.data.getRoomObjects(42);
|
||||
console.log("Room 42 Objects:", JSON.stringify(objects, null, 2));
|
||||
```
|
||||
|
||||
#### 9. JavaScript Tips for Browser Debugging
|
||||
|
||||
##### Console Shortcuts
|
||||
|
||||
```javascript
|
||||
// Alias for quick access
|
||||
const y = window.yaze;
|
||||
const yd = window.yazeDebug;
|
||||
|
||||
// Quick status check
|
||||
y.control.getRomStatus()
|
||||
y.editor.getSnapshot()
|
||||
yd.arena.getStatus()
|
||||
```
|
||||
|
||||
##### Error Handling Pattern
|
||||
|
||||
Always wrap API calls in try-catch when automating:
|
||||
```javascript
|
||||
function safeCall(fn, fallback = null) {
|
||||
try {
|
||||
return fn();
|
||||
} catch (e) {
|
||||
console.error('[YAZE API Error]', e.message);
|
||||
return fallback;
|
||||
}
|
||||
}
|
||||
|
||||
// Usage
|
||||
const status = safeCall(() => window.yaze.control.getRomStatus(), { loaded: false });
|
||||
```
|
||||
|
||||
##### Async Operations
|
||||
|
||||
Some operations are async. Use proper await patterns:
|
||||
```javascript
|
||||
// Wait for element to appear in GUI
|
||||
await window.yaze.gui.waitForElement('dungeon-room-canvas', 5000);
|
||||
|
||||
// Then interact
|
||||
window.yaze.gui.click('dungeon-room-canvas');
|
||||
```
|
||||
|
||||
##### Debugging State Issues
|
||||
|
||||
```javascript
|
||||
// Full state dump for debugging
|
||||
const debugState = {
|
||||
moduleReady: window.YAZE_MODULE_READY,
|
||||
romStatus: window.yaze?.control?.getRomStatus?.() || 'API unavailable',
|
||||
editorSnapshot: window.yaze?.editor?.getSnapshot?.() || 'API unavailable',
|
||||
arenaStatus: window.yazeDebug?.arena?.getStatus?.() || 'API unavailable',
|
||||
consoleErrors: window._yazeConsoleLogs?.filter(l => l.includes('[ERROR]')) || []
|
||||
};
|
||||
console.log(JSON.stringify(debugState, null, 2));
|
||||
```
|
||||
|
||||
##### Monitoring Loading Progress
|
||||
|
||||
```javascript
|
||||
// Set up a loading progress monitor
|
||||
let lastProgress = 0;
|
||||
const progressInterval = setInterval(() => {
|
||||
const status = window.yaze?.control?.getRomStatus?.();
|
||||
if (status?.loaded) {
|
||||
console.log('ROM loaded successfully!');
|
||||
clearInterval(progressInterval);
|
||||
}
|
||||
}, 500);
|
||||
|
||||
// Clear after 30 seconds timeout
|
||||
setTimeout(() => clearInterval(progressInterval), 30000);
|
||||
```
|
||||
|
||||
##### Inspecting Graphics Issues
|
||||
|
||||
```javascript
|
||||
// Check if graphics sheets are loaded
|
||||
const arenaStatus = window.yazeDebug.arena.getStatus();
|
||||
console.log('Loaded sheets:', arenaStatus.loaded_sheets);
|
||||
console.log('Pending textures:', arenaStatus.pending_textures);
|
||||
|
||||
// Check for palette issues
|
||||
const paletteEvents = Module.getDungeonPaletteEvents?.() || 'Not available';
|
||||
console.log('Recent palette changes:', paletteEvents);
|
||||
```
|
||||
|
||||
##### Memory Usage Check
|
||||
|
||||
```javascript
|
||||
// Check WASM memory usage
|
||||
const memInfo = {
|
||||
heapSize: Module.HEAPU8?.length || 0,
|
||||
heapSizeMB: ((Module.HEAPU8?.length || 0) / 1024 / 1024).toFixed(2) + ' MB'
|
||||
};
|
||||
console.log('Memory:', memInfo);
|
||||
```
|
||||
|
||||
#### 10. Gemini Antigravity AI Tools
|
||||
|
||||
For AI assistants using the Antigravity browser extension, use the high-level `window.aiTools` API:
|
||||
|
||||
##### Complete Workflow Example
|
||||
|
||||
```javascript
|
||||
// Step-by-step for Gemini/Antigravity
|
||||
async function inspectDungeonRoom(roomId) {
|
||||
// 1. Verify environment
|
||||
if (!window.YAZE_MODULE_READY) {
|
||||
return { error: "WASM module not ready. Please wait for initialization." };
|
||||
}
|
||||
|
||||
// 2. Check ROM
|
||||
const romStatus = window.yaze.control.getRomStatus();
|
||||
if (!romStatus.loaded) {
|
||||
return { error: "No ROM loaded. Please drag a Zelda 3 ROM onto the canvas." };
|
||||
}
|
||||
|
||||
// 3. Switch to dungeon editor
|
||||
window.yaze.control.switchEditor('Dungeon');
|
||||
|
||||
// 4. Navigate to room
|
||||
window.aiTools.navigateTo(`room:${roomId}`);
|
||||
|
||||
// 5. Gather all data
|
||||
return {
|
||||
room_id: roomId,
|
||||
properties: window.yaze.data.getRoomProperties(roomId),
|
||||
objects: window.yaze.data.getRoomObjects(roomId),
|
||||
tiles: window.yaze.data.getRoomTiles(roomId),
|
||||
editor_state: window.yaze.editor.getCurrentRoom()
|
||||
};
|
||||
}
|
||||
|
||||
// Usage
|
||||
const data = await inspectDungeonRoom(42);
|
||||
console.log(JSON.stringify(data, null, 2));
|
||||
```
|
||||
|
||||
##### Quick Reference Commands
|
||||
|
||||
```javascript
|
||||
// Get full application state with console output
|
||||
window.aiTools.getAppState()
|
||||
|
||||
// Get dungeon room data
|
||||
window.aiTools.getRoomData(0)
|
||||
|
||||
// Navigate directly to a room
|
||||
window.aiTools.navigateTo('room:42')
|
||||
|
||||
// Show/hide editor cards
|
||||
window.aiTools.showCard('Room Selector')
|
||||
window.aiTools.hideCard('Object Editor')
|
||||
|
||||
// Get complete API reference
|
||||
window.aiTools.dumpAPIReference()
|
||||
|
||||
// AI-formatted state (paste-ready for prompts)
|
||||
window.yazeDebug.formatForAI()
|
||||
```
|
||||
|
||||
##### Handling Common Errors
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `window.yaze is undefined` | Module not initialized | Wait for `YAZE_MODULE_READY` |
|
||||
| `getRomStatus().loaded = false` | No ROM file | Prompt user to drag ROM |
|
||||
| `switchEditor returns error` | Invalid editor name | Use: Dungeon, Overworld, Graphics, Palette, Sprite, Music |
|
||||
| `getRoomObjects returns empty` | Room not loaded | Navigate to room first |
|
||||
| `Canvas shows black` | Graphics not loaded | Check `yazeDebug.arena.getStatus()` |
|
||||
|
||||
**Nav Bar Access:**
|
||||
The web UI includes dedicated dropdown menus:
|
||||
- **Editor** - Quick switch to any editor
|
||||
- **Emulator** - Run/Pause/Step/Reset controls
|
||||
- **Layouts** - Preset card configurations
|
||||
- **AI Tools** - All `window.aiTools` functions via UI clicks
|
||||
|
||||
**Command Palette (Ctrl+K):**
|
||||
Search for "AI:" to access all AI helper commands.
|
||||
|
||||
#### 11. Copying Data for External Analysis
|
||||
|
||||
```javascript
|
||||
// Copy room data to clipboard for pasting elsewhere
|
||||
async function copyRoomDataToClipboard(roomId) {
|
||||
const data = {
|
||||
timestamp: new Date().toISOString(),
|
||||
room_id: roomId,
|
||||
properties: window.yaze.data.getRoomProperties(roomId),
|
||||
objects: window.yaze.data.getRoomObjects(roomId)
|
||||
};
|
||||
await navigator.clipboard.writeText(JSON.stringify(data, null, 2));
|
||||
console.log(`Room ${roomId} data copied to clipboard!`);
|
||||
}
|
||||
|
||||
// Usage
|
||||
copyRoomDataToClipboard(42);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### 12. Antigravity: Debugging Dungeon Object Rendering Issues
|
||||
|
||||
This section provides Gemini Antigravity with specific workflows for identifying and analyzing dungeon object rendering problems, particularly discrepancies between the **Room Graphics Card** (background tiles) and the **Object Editor Card** (dungeon objects like pots, torches, chests).
|
||||
|
||||
##### Understanding the Rendering Pipeline
|
||||
|
||||
Dungeon rooms have two distinct rendering layers:
|
||||
|
||||
| Card | What It Shows | Render Source |
|
||||
|------|--------------|---------------|
|
||||
| **Room Graphics** | Background tiles (floors, walls, pits) | Layer1/Layer2 tile data |
|
||||
| **Object Editor** | Interactive objects (pots, chests, blocks, torches) | Object list with sprite-based rendering |
|
||||
|
||||
**Common Issue:** Objects appear at wrong positions, wrong sprites, or don't match their expected appearance in the Object Editor compared to how they should look based on the room data.
|
||||
|
||||
##### Step-by-Step Debugging Workflow
|
||||
|
||||
###### 1. Capture the Current Visual State
|
||||
|
||||
Use the browser's screenshot capability to capture what you see:
|
||||
|
||||
```javascript
|
||||
// Capture the entire canvas as a data URL
|
||||
async function captureCanvasScreenshot() {
|
||||
const canvas = document.getElementById('canvas');
|
||||
if (!canvas) return { error: 'Canvas not found' };
|
||||
|
||||
const dataUrl = canvas.toDataURL('image/png');
|
||||
console.log('[Screenshot] Canvas captured, length:', dataUrl.length);
|
||||
|
||||
// For AI analysis, you can copy to clipboard
|
||||
await navigator.clipboard.writeText(dataUrl);
|
||||
return { success: true, message: 'Screenshot copied to clipboard as data URL' };
|
||||
}
|
||||
|
||||
// Usage
|
||||
await captureCanvasScreenshot();
|
||||
```
|
||||
|
||||
**For Antigravity:** Take screenshots when:
|
||||
1. Room Graphics Card is visible (shows background)
|
||||
2. Object Editor Card is visible (shows objects overlaid)
|
||||
3. Both cards side-by-side if possible
|
||||
|
||||
###### 2. Extract Room Object Data Efficiently
|
||||
|
||||
```javascript
|
||||
// Get complete object rendering data for a room
|
||||
function getDungeonObjectDebugData(roomId) {
|
||||
const data = {
|
||||
room_id: roomId,
|
||||
timestamp: Date.now(),
|
||||
|
||||
// Room properties affecting rendering
|
||||
properties: window.yaze.data.getRoomProperties(roomId),
|
||||
|
||||
// All objects in the room with positions
|
||||
objects: window.yaze.data.getRoomObjects(roomId),
|
||||
|
||||
// Current editor state
|
||||
editor_state: window.yaze.editor.getCurrentRoom(),
|
||||
|
||||
// Graphics arena status (textures loaded?)
|
||||
arena_status: window.yazeDebug?.arena?.getStatus() || 'unavailable',
|
||||
|
||||
// Visible cards (what's being rendered)
|
||||
visible_cards: window.yaze.editor.getSnapshot()?.visible_cards || []
|
||||
};
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
// Pretty print for analysis
|
||||
const debugData = getDungeonObjectDebugData(42);
|
||||
console.log(JSON.stringify(debugData, null, 2));
|
||||
```
|
||||
|
||||
###### 3. Compare Object Positions vs Tile Positions
|
||||
|
||||
```javascript
|
||||
// Check if objects are aligned with the tile grid
|
||||
function analyzeObjectPlacement(roomId) {
|
||||
const objects = window.yaze.data.getRoomObjects(roomId);
|
||||
const tiles = window.yaze.data.getRoomTiles(roomId);
|
||||
|
||||
const analysis = objects.map(obj => {
|
||||
// Objects use pixel coordinates, tiles are 8x8 or 16x16
|
||||
const tileX = Math.floor(obj.x / 8);
|
||||
const tileY = Math.floor(obj.y / 8);
|
||||
|
||||
return {
|
||||
object_id: obj.id,
|
||||
type: obj.type,
|
||||
pixel_pos: { x: obj.x, y: obj.y },
|
||||
tile_pos: { x: tileX, y: tileY },
|
||||
// Check if position is on grid boundary
|
||||
aligned_8px: (obj.x % 8 === 0) && (obj.y % 8 === 0),
|
||||
aligned_16px: (obj.x % 16 === 0) && (obj.y % 16 === 0)
|
||||
};
|
||||
});
|
||||
|
||||
return analysis;
|
||||
}
|
||||
|
||||
console.log(JSON.stringify(analyzeObjectPlacement(42), null, 2));
|
||||
```
|
||||
|
||||
###### 4. Identify Visual Discrepancies
|
||||
|
||||
**Symptoms to look for:**
|
||||
|
||||
| Symptom | Likely Cause | Debug Command |
|
||||
|---------|-------------|---------------|
|
||||
| Objects invisible | Texture not loaded | `window.yazeDebug.arena.getStatus()` |
|
||||
| Wrong sprite shown | Object type mismatch | `window.yaze.data.getRoomObjects(roomId)` |
|
||||
| Position offset | Coordinate transform bug | Compare pixel_pos in data vs visual |
|
||||
| Colors wrong | Palette not applied | `Module.getDungeonPaletteEvents()` |
|
||||
| Flickering | Z-order/layer issue | Check `layer` property in object data |
|
||||
|
||||
###### 5. DOM Inspection for Card State
|
||||
|
||||
Efficiently query the DOM to understand what's being rendered:
|
||||
|
||||
```javascript
|
||||
// Get all visible ImGui windows (cards)
|
||||
function getVisibleCards() {
|
||||
// ImGui renders to canvas, but card state is tracked in JS
|
||||
const snapshot = window.yaze.editor.getSnapshot();
|
||||
return {
|
||||
active_cards: snapshot.visible_cards || [],
|
||||
editor_type: snapshot.editor_type,
|
||||
// Check if specific cards are open
|
||||
has_room_selector: snapshot.visible_cards?.includes('Room Selector'),
|
||||
has_object_editor: snapshot.visible_cards?.includes('Object Editor'),
|
||||
has_room_canvas: snapshot.visible_cards?.includes('Room Canvas')
|
||||
};
|
||||
}
|
||||
|
||||
console.log(getVisibleCards());
|
||||
```
|
||||
|
||||
###### 6. Full Diagnostic Dump for AI Analysis
|
||||
|
||||
```javascript
|
||||
// Complete diagnostic for Antigravity to analyze rendering issues
|
||||
async function generateRenderingDiagnostic(roomId) {
|
||||
const diagnostic = {
|
||||
timestamp: new Date().toISOString(),
|
||||
room_id: roomId,
|
||||
|
||||
// Visual state
|
||||
visible_cards: getVisibleCards(),
|
||||
|
||||
// Data state
|
||||
room_properties: window.yaze.data.getRoomProperties(roomId),
|
||||
room_objects: window.yaze.data.getRoomObjects(roomId),
|
||||
object_analysis: analyzeObjectPlacement(roomId),
|
||||
|
||||
// Graphics state
|
||||
arena: window.yazeDebug?.arena?.getStatus(),
|
||||
palette_events: (() => {
|
||||
try { return Module.getDungeonPaletteEvents(); }
|
||||
catch { return 'unavailable'; }
|
||||
})(),
|
||||
|
||||
// Memory state
|
||||
heap_mb: ((Module.HEAPU8?.length || 0) / 1024 / 1024).toFixed(2),
|
||||
|
||||
// Console errors (last 10)
|
||||
recent_errors: window._yazeConsoleLogs?.slice(-10) || []
|
||||
};
|
||||
|
||||
// Copy to clipboard for easy pasting
|
||||
const json = JSON.stringify(diagnostic, null, 2);
|
||||
await navigator.clipboard.writeText(json);
|
||||
console.log('[Diagnostic] Copied to clipboard');
|
||||
|
||||
return diagnostic;
|
||||
}
|
||||
|
||||
// Usage: Run this, then paste into your AI prompt
|
||||
await generateRenderingDiagnostic(42);
|
||||
```
|
||||
|
||||
##### Common Object Rendering Bugs
|
||||
|
||||
###### Bug: Objects Render at (0,0)
|
||||
|
||||
**Diagnosis:**
|
||||
```javascript
|
||||
// Check for objects with zero coordinates
|
||||
const objects = window.yaze.data.getRoomObjects(roomId);
|
||||
const atOrigin = objects.filter(o => o.x === 0 && o.y === 0);
|
||||
console.log('Objects at origin:', atOrigin);
|
||||
```
|
||||
|
||||
**Cause:** Object position data not loaded or coordinate transformation failed.
|
||||
|
||||
###### Bug: Sprite Shows as Black Square
|
||||
|
||||
**Diagnosis:**
|
||||
```javascript
|
||||
// Check if graphics sheet is loaded for object type
|
||||
const arena = window.yazeDebug.arena.getStatus();
|
||||
console.log('Loaded sheets:', arena.loaded_sheets);
|
||||
console.log('Pending textures:', arena.pending_textures);
|
||||
```
|
||||
|
||||
**Cause:** Texture not yet loaded from deferred queue. Force process:
|
||||
```javascript
|
||||
// Wait for textures to load
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
```
|
||||
|
||||
###### Bug: Object in Wrong Location vs Room Graphics
|
||||
|
||||
**Diagnosis:**
|
||||
```javascript
|
||||
// Compare layer1 tile at object position
|
||||
const obj = window.yaze.data.getRoomObjects(roomId)[0];
|
||||
const tiles = window.yaze.data.getRoomTiles(roomId);
|
||||
const tileAtPos = tiles.layer1[Math.floor(obj.y / 8) * 64 + Math.floor(obj.x / 8)];
|
||||
console.log('Object at:', obj.x, obj.y);
|
||||
console.log('Tile at that position:', tileAtPos);
|
||||
```
|
||||
|
||||
##### Screenshot Comparison Workflow
|
||||
|
||||
For visual debugging, use this workflow:
|
||||
|
||||
1. **Open Room Graphics Card only:**
|
||||
```javascript
|
||||
window.aiTools.hideCard('Object Editor');
|
||||
window.aiTools.showCard('Room Canvas');
|
||||
// Take screenshot #1
|
||||
```
|
||||
|
||||
2. **Enable Object Editor overlay:**
|
||||
```javascript
|
||||
window.aiTools.showCard('Object Editor');
|
||||
// Take screenshot #2
|
||||
```
|
||||
|
||||
3. **Compare:** Objects should align with the room's floor/wall tiles. Misalignment indicates a coordinate bug.
|
||||
|
||||
##### Reporting Issues
|
||||
|
||||
When reporting dungeon rendering bugs, include:
|
||||
|
||||
```javascript
|
||||
// Generate a complete bug report
|
||||
async function generateBugReport(roomId, description) {
|
||||
const report = {
|
||||
bug_description: description,
|
||||
room_id: roomId,
|
||||
diagnostic: await generateRenderingDiagnostic(roomId),
|
||||
steps_to_reproduce: [
|
||||
'1. Load ROM',
|
||||
'2. Open Dungeon Editor',
|
||||
`3. Navigate to Room ${roomId}`,
|
||||
'4. Observe [specific issue]'
|
||||
],
|
||||
expected_behavior: 'Objects should render at correct positions matching tile grid',
|
||||
actual_behavior: description
|
||||
};
|
||||
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
return report;
|
||||
}
|
||||
|
||||
// Usage
|
||||
await generateBugReport(42, 'Chest renders 8 pixels too far right');
|
||||
```
|
||||
Reference in New Issue
Block a user