scawful/yaze
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

backend-infra-engineer: Release v0.3.3 snapshot

Browse Source
This commit is contained in:
scawful
2025-11-21 21:35:50 -05:00
parent 3d71417f62
commit 476dd1cd1c
818 changed files with 65706 additions and 35514 deletions
Download Patch File Download Diff File Expand all files Collapse all files

320
docs/public/build/build-from-source.md Normal file
View File

@@ -0,0 +1,320 @@
# Build Instructions
yaze uses a modern CMake build system with presets for easy configuration. This guide explains the
environment checks, dependencies, and platform-specific considerations. For concise per-platform
commands, always start with the [Build & Test Quick Reference](quick-reference.md).
## 1. Environment Verification
**Before your first build**, run the verification script to ensure your environment is configured correctly.
### Windows (PowerShell)
```powershell
.\scripts\verify-build-environment.ps1
# With automatic fixes
.\scripts\verify-build-environment.ps1 -FixIssues
```
> Tip: After verification, run `.\scripts\setup-vcpkg-windows.ps1` to bootstrap vcpkg, ensure `clang-cl`/Ninja are installed, and cache the `x64-windows` triplet.
### macOS & Linux (Bash)
```bash
./scripts/verify-build-environment.sh
# With automatic fixes
./scripts\verify-build-environment.sh --fix
```
The script checks for required tools like CMake, a C++23 compiler, and platform-specific dependencies.
## 2. Using Presets
- Pick the preset that matches your platform/workflow (debug: `mac-dbg` / `lin-dbg` / `win-dbg`,
AI-enabled: `mac-ai` / `win-ai`, release: `*-rel`, etc.).
- Configure with `cmake --preset <name>` and build with `cmake --build --preset <name> [--target …]`.
- Add `-v` to a preset name (e.g., `mac-dbg-v`) to surface compiler warnings.
- Need a full matrix? See the [CMake Presets Guide](presets.md) for every preset and the quick
reference for ready-to-run command snippets.
## Feature Toggles & Windows Profiles
### Windows Presets
| Preset | Purpose |
| --- | --- |
| `win-dbg`, `win-rel`, `ci-windows` | Core builds without agent UI, gRPC, or AI runtimes. Fastest option for MSVC/clang-cl. |
| `win-ai`, `win-vs-ai` | Full agent stack for local development (UI panels + remote automation + AI runtime). |
| `ci-windows-ai` | Nightly/weekly CI preset that exercises the entire automation stack on Windows. |
### Agent Feature Flags
| Option | Default | Effect |
| --- | --- | --- |
| `YAZE_BUILD_AGENT_UI` | `ON` when `YAZE_BUILD_GUI=ON` | Builds the ImGui widgets used by the chat/agent panels. |
| `YAZE_ENABLE_REMOTE_AUTOMATION` | `ON` for `*-ai` presets | Adds gRPC/protobuf services plus GUI automation clients. |
| `YAZE_ENABLE_AI_RUNTIME` | `ON` for `*-ai` presets | Enables Gemini/Ollama transports, proposal planning, and advanced routing logic. |
| `YAZE_ENABLE_AGENT_CLI` | `ON` when `YAZE_BUILD_CLI=ON` | Compiles the conversational agent stack consumed by `z3ed`. |
Combine these switches to match your workflow: keep everything `OFF` for lightweight GUI hacking or turn them `ON` for automation-heavy work with sketchybar/yabai/skhd, tmux, or remote runners.
## 3. Dependencies
- **Required**: CMake 3.16+, C++23 Compiler (GCC 13+, Clang 16+, MSVC 2019+), Git.
- **Bundled**: All other dependencies (SDL2, ImGui, Asar, nlohmann/json, cpp-httplib, GoogleTest, etc.) live under the `ext/` directory or are managed by CMake's `FetchContent`. No external package manager is required for a basic build.
- **Optional**:
- **gRPC**: For GUI test automation. Can be enabled with `-DYAZE_WITH_GRPC=ON`.
- **vcpkg (Windows)**: Can be used for faster gRPC builds on Windows (optional).
## 4. Platform Setup
### macOS
```bash
# Install Xcode Command Line Tools
xcode-select --install
# Recommended: Install build tools via Homebrew
brew install cmake pkg-config
# For sandboxed/offline builds: Install dependencies to avoid network fetch
brew install yaml-cpp googletest
```
**Note**: When building in sandboxed/offline environments (e.g., via Claude Code or restricted networks), install `yaml-cpp` and `googletest` via Homebrew to avoid GitHub fetch failures. The build system automatically detects Homebrew installations and uses them as fallback:
- yaml-cpp: `/opt/homebrew/opt/yaml-cpp`, `/usr/local/opt/yaml-cpp`
- googletest: `/opt/homebrew/opt/googletest`, `/usr/local/opt/googletest`
### Linux (Ubuntu/Debian)
```bash
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build pkg-config \
libgtk-3-dev libdbus-1-dev
```
### Windows
1. **Install Visual Studio 2022** with the “Desktop development with C++” workload (requires MSVC + MSBuild).
2. **Install Ninja** (recommended): `choco install ninja` or enable the “CMake tools for Windows” optional component.
3. Run the verifier: `.\scripts\verify-build-environment.ps1 -FixIssues` – this checks Visual Studio workloads, Ninja, clang-cl, Git settings, and vcpkg cache.
4. Bootstrap vcpkg once: `.\scripts\setup-vcpkg-windows.ps1` (prefetches SDL2, yaml-cpp, etc.).
5. Use the `win-*` presets (Ninja) or `win-vs-*` presets (Visual Studio generator) as needed. For AI/gRPC features, prefer `win-ai` / `win-vs-ai`.
6. For quick validation, run the PowerShell helper:
```powershell
pwsh -File scripts/agents/windows-smoke-build.ps1 -Preset win-ai -Target z3ed
```
## 5. Testing
The project uses CTest and GoogleTest. Tests are organized into categories using labels. See the [Testing Guide](../developer/testing-guide.md) for details.
### Running Tests with Presets
The easiest way to run tests is with `ctest` presets.
```bash
# Configure a development build (enables ROM-dependent tests)
cmake --preset mac-dev -DYAZE_TEST_ROM_PATH=/path/to/your/zelda3.sfc
# Build the tests
cmake --build --preset mac-dev --target yaze_test
# Run stable tests (fast, run in CI)
ctest --preset dev
# Run all tests, including ROM-dependent and experimental
ctest --preset all
```
### Running Tests Manually
You can also run tests by invoking the test executable directly or using CTest with labels.
```bash
# Run all tests via the executable
./build/bin/yaze_test
# Run only stable tests using CTest labels
ctest --test-dir build --label-regex "STABLE"
# Run tests matching a name
ctest --test-dir build -R "AsarWrapperTest"
# Exclude ROM-dependent tests
ctest --test-dir build --label-exclude "ROM_DEPENDENT"
```
## 6. IDE Integration
### VS Code (Recommended)
1. Install the **CMake Tools** extension.
2. Open the project folder.
3. Select a preset from the status bar (e.g., `mac-ai`). On Windows, choose the desired kit (e.g., “Visual Studio Build Tools 2022”) so the generator matches your preset (`win-*` uses Ninja, `win-vs-*` uses Visual Studio).
4. Press F5 to build and debug.
5. After changing presets, run `cp build/compile_commands.json .` to update IntelliSense.
### Visual Studio (Windows)
1. Select **File → Open → Folder** and choose the `yaze` directory.
2. Visual Studio will automatically detect `CMakePresets.json`.
3. Select the desired preset (e.g., `win-dbg` or `win-ai`) from the configuration dropdown.
4. Press F5 to build and run.
### Xcode (macOS)
```bash
# Generate an Xcode project from a preset
cmake --preset mac-dbg -G Xcode
# Open the project
open build/yaze.xcodeproj
```
## 7. Windows Build Optimization
### GitHub Actions / CI Builds
**Current Configuration (Optimized):**
- **Compilers**: Both clang-cl and MSVC supported (matrix)
- **vcpkg**: Only fast packages (SDL2, yaml-cpp) - 2 minutes
- **gRPC**: Built via FetchContent (v1.75.1) - cached after first build
- **Caching**: Aggressive multi-tier caching (vcpkg + FetchContent + sccache)
- **Agent matrix**: A dedicated `ci-windows-ai` job runs outside pull requests to exercise the full gRPC + AI runtime stack.
- **Expected time**:
- First build: ~10-15 minutes
- Cached build: ~3-5 minutes
**Why FetchContent for gRPC in CI?**
- vcpkg's latest gRPC (v1.71.0) has no pre-built binaries
- Building from source via vcpkg: 45-90 minutes
- FetchContent with caching: 10-15 minutes first time, <1 min cached
- Better control over gRPC version (v1.75.1 - latest stable)
- BoringSSL ASM disabled on Windows for clang-cl compatibility
- zlib conflict: gRPC's FetchContent builds its own zlib, conflicts with vcpkg's
### Desktop Development: Faster builds with vcpkg (optional)
For desktop development, you can use vcpkg for faster gRPC builds:
```powershell
# Bootstrap vcpkg and prefetch packages
.\scripts\setup-vcpkg-windows.ps1
# Configure with vcpkg
cmake -B build -DYAZE_USE_VCPKG_GRPC=ON -DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake
```
**Benefits:**
- Pre-compiled gRPC packages: ~5 minutes vs ~10-15 minutes
- No need to build gRPC from source
- Faster iteration during development
**Note:** CI/CD workflows use FetchContent by default for reliability.
### Local Development
#### Fast Build (Recommended)
Use FetchContent for all dependencies (matches CI):
```powershell
# Configure (first time: ~15 min, subsequent: ~2 min)
cmake -B build -G "Visual Studio 17 2022" -A x64
# Build
cmake --build build --config RelWithDebInfo --parallel
```
#### Using vcpkg (Optional)
If you prefer vcpkg for local development:
```powershell
# Install ONLY the fast packages
vcpkg install sdl2:x64-windows yaml-cpp:x64-windows
# Let CMake use FetchContent for gRPC
cmake -B build -DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake
```
**DO NOT** install grpc or zlib via vcpkg:
- gRPC v1.71.0 has no pre-built binaries (45-90 min build)
- zlib conflicts with gRPC's bundled zlib
### Compiler Support
**clang-cl (Recommended):**
- Used in both CI and release workflows
- Better diagnostics than MSVC
- Fully compatible with MSVC libraries
**MSVC:**
- Also tested in CI matrix
- Fallback option if clang-cl issues occur
**Compiler Flags (Applied Automatically):**
- `/bigobj` - Large object files (required for gRPC)
- `/permissive-` - Standards conformance
- `/wd4267 /wd4244` - Suppress harmless conversion warnings
- `/constexpr:depth2048` - Template instantiation depth
## 8. Troubleshooting
Build issues, especially on Windows, often stem from environment misconfiguration. Before anything else, run the verification script.
```powershell
# Run the verification script in PowerShell
.\scripts\verify-build-environment.ps1
```
This script is your primary diagnostic tool and can detect most common problems.
### Automatic Fixes
If the script finds issues, you can often fix them automatically by running it with the `-FixIssues` flag. This can:
- Synchronize Git submodules.
- Correct Git `core.autocrlf` and `core.longpaths` settings, which are critical for cross-platform compatibility on Windows.
- Prompt to clean stale CMake caches.
```powershell
# Attempt to fix detected issues automatically
.\scripts\verify-build-environment.ps1 -FixIssues
```
### Cleaning Stale Builds
After pulling major changes or switching branches, your build directory can become "stale," leading to strange compiler or linker errors. The verification script will warn you about old build files. You can clean them manually or use the `-CleanCache` flag.
**This will delete all `build*` and `out` directories.**
```powershell
# Clean all build artifacts to start fresh
.\scripts\verify-build-environment.ps1 -CleanCache
```
### Common Issues
#### "nlohmann/json.hpp: No such file or directory"
**Cause**: You are building code that requires AI features without using an AI-enabled preset, or your Git submodules are not initialized.
**Solution**:
1. Use an AI preset like `win-ai` or `mac-ai`.
2. Ensure submodules are present by running `git submodule update --init --recursive`.
#### "Cannot open file 'yaze.exe': Permission denied"
**Cause**: A previous instance of `yaze.exe` is still running in the background.
**Solution**: Close it using Task Manager or run:
```cmd
taskkill /F /IM yaze.exe
```
#### "C++ standard 'cxx_std_23' not supported"
**Cause**: Your compiler is too old.
**Solution**: Update your tools. You need Visual Studio 2022 17.4+, GCC 13+, or Clang 16+. The verification script checks this.
#### Visual Studio Can't Find Presets
**Cause**: VS failed to parse `CMakePresets.json` or its cache is corrupt.
**Solution**:
1. Close and reopen the folder (`File -> Close Folder`).
2. Check the "CMake" pane in the Output window for specific JSON parsing errors.
3. Delete the hidden `.vs` directory in the project root to force Visual Studio to re-index the project.
#### Git Line Ending (CRLF) Issues
**Cause**: Git may be automatically converting line endings, which can break shell scripts and other assets.
**Solution**: The verification script checks for this. Use the `-FixIssues` flag or run `git config --global core.autocrlf false` to prevent this behavior.
#### File Path Length Limit on Windows
**Cause**: By default, Windows has a 260-character path limit, which can be exceeded by nested dependencies.
**Solution**: The verification script checks for this. Use the `-FixIssues` flag or run `git config --global core.longpaths true` to enable long path support.

224
docs/public/build/platform-compatibility.md Normal file
View File

@@ -0,0 +1,224 @@
# Platform Compatibility & CI/CD Fixes
**Last Updated**: October 9, 2025
---
## Platform-Specific Notes
### Windows
**Build System:**
- Supported compilers: clang-cl (preferred), MSVC 2022 17.4+
- Uses vcpkg for: SDL2, yaml-cpp (fast packages only)
- Uses FetchContent for: gRPC v1.75.1, protobuf, abseil, zlib (better caching)
**Why FetchContent for gRPC?**
- vcpkg gRPC v1.71.0 has no pre-built binaries (builds from source: 45-90 min)
- FetchContent uses v1.75.1 (latest stable with modern compiler support)
- BoringSSL ASM disabled on Windows (avoids NASM build conflicts with clang-cl)
- Better caching in CI/CD (separate cache keys for vcpkg vs FetchContent)
- First build: ~10-15 min, subsequent: <1 min (cached)
- zlib bundled with gRPC (avoids vcpkg conflicts)
**Compiler Flags (both clang-cl and MSVC):**
- `/bigobj` - Support large object files (required for gRPC)
- `/permissive-` - Standards conformance mode
- `/wd4267 /wd4244` - Suppress harmless conversion warnings
- `/constexpr:depth2048` - Deep template instantiation (MSVC 2019+)
**Macro Definitions:**
- `WIN32_LEAN_AND_MEAN` - Reduce Windows header pollution
- `NOMINMAX` - Prevent min/max macro conflicts
- `NOGDI` - Prevent GDI macro conflicts (DWORD, etc.)
- `__PRFCHWINTRIN_H` - Work around Clang 20 `_m_prefetchw` linkage clash
**Build Times:**
- First build (no cache): ~10-15 minutes
- Incremental build (cached): ~3-5 minutes
- CI/CD with full caching: ~5-8 minutes
**CI/CD Configuration:**
- Compiler matrix: clang-cl + MSVC
- 3-tier caching: vcpkg packages, FetchContent deps, sccache objects
- Binary caching via GitHub Actions for vcpkg
- Parallel builds: 4 jobs
### macOS
**Build System:**
- Uses vcpkg for: SDL2, yaml-cpp, zlib
- Uses FetchContent for: gRPC, abseil, protobuf
**Supported Architectures:**
- x64 (Intel Macs) - macOS 13+
- ARM64 (Apple Silicon) - macOS 14+
**Build Times:**
- First build: ~20-30 minutes
- Subsequent builds: ~3-5 minutes
### Linux
**Build System:**
- Uses system packages (apt) for most dependencies
- Does NOT use vcpkg
- Uses FetchContent for: gRPC, abseil, protobuf (when not in system)
**Required System Packages:**
```bash
build-essential ninja-build pkg-config libglew-dev libxext-dev
libwavpack-dev libabsl-dev libboost-all-dev libpng-dev python3-dev
libpython3-dev libasound2-dev libpulse-dev libaudio-dev libx11-dev
libxrandr-dev libxcursor-dev libxinerama-dev libxi-dev libxss-dev
libxxf86vm-dev libxkbcommon-dev libwayland-dev libdecor-0-dev
libgtk-3-dev libdbus-1-dev gcc-12 g++-12 clang-15
```
**Build Times:**
- First build: ~15-20 minutes
- Subsequent builds: ~2-4 minutes
---
## Cross-Platform Code Validation
The following subsystems run unchanged across Windows, macOS, and Linux:
- Audio backend (`src/app/emu/audio`) uses SDL2 only; no platform branches.
- Input backend/manager (`src/app/emu/input`) runs on SDL2 abstractions.
- Debug tools (`src/app/emu/debug`) avoid OS-specific headers.
- Emulator UI (`src/app/emu/ui`) is pure ImGui + SDL2.
---
## Common Build Issues & Solutions
### Windows: "use of undefined type 'PromiseLike'"
**Cause:** Old gRPC version (< v1.67.1)
**Fix:** Clear build cache and reconfigure
```powershell
rm -r build/_deps/grpc-*
cmake -B build -G "Visual Studio 17 2022" -A x64
```
### macOS: "absl not found"
**Cause:** vcpkg not finding abseil packages
**Fix:** Use FetchContent (default) - abseil is fetched automatically by gRPC
```bash
cmake -B build
```
### Linux: CMake configuration fails
**Cause:** Missing system dependencies
**Fix:** Install required packages
```bash
sudo apt-get update
sudo apt-get install -y [see package list above]
```
### Windows: "DWORD syntax error"
**Cause:** Windows macros conflicting with protobuf enums
**Fix:** Ensure `NOGDI` is defined (now automatic in grpc.cmake)
---
## CI/CD Validation Checklist
Before merging platform-specific changes:
- Confirm the vcpkg baseline matches `vcpkg.json`.
- Do not reintroduce the Windows x86 build (cpp-httplib incompatibility).
- Keep Windows macro guards (`NOGDI`, `NOMINMAX`, `WIN32_LEAN_AND_MEAN`) in place.
- Build against gRPC 1.67.1 with the MSVC workaround flags.
- Leave shared code paths on SDL2/ImGui abstractions.
- Re-run the full matrix if caches or presets change.
### CI/CD Performance Roadmap
- **Dependency caching**: Cache vcpkg installs on Windows plus Homebrew/apt
archives to trim 5-10 minutes per job; track cache keys via OS + lockfiles.
- **Compiler caching**: Enable `ccache`/`sccache` across the matrix using the
`hendrikmuhs/ccache-action` with 500 MB per-run limits for 3-5 minute wins.
- **Conditional work**: Add a path-filter job that skips emulator builds or
full test runs when only docs or CLI code change; fall back to full matrix on
shared components.
- **Reusable workflows**: Centralize setup steps (checking out submodules,
restoring caches, configuring presets) to reduce duplication between `ci.yml`
and `release.yml`.
- **Release optimizations**: Use lean presets without test targets, run platform
builds in parallel, and reuse cached artifacts from CI when hashes match.
---
## Testing Strategy
### Automated (CI)
- Ubuntu 22.04 (GCC-12, Clang-15)
- macOS 13/14 (x64 and ARM64)
- Windows Server 2022 (x64)
- Core tests: `AsarWrapperTest`, `SnesTileTest`, others tagged `STABLE`
- Tooling: clang-format, clang-tidy, cppcheck
- Sanitizers: Linux AddressSanitizer job
### Manual Testing
After successful CI build:
- Windows: verify audio backend, keyboard input, APU debugger UI.
- Linux: verify input polling and audio output.
- macOS: spot-check rendering, input, audio.
---
## Quick Reference
### Build Command (All Platforms)
```bash
cmake -B build
cmake --build build --parallel
cmake --preset [mac-dbg|lin-dbg|win-dbg]
cmake --build --preset [mac-dbg|lin-dbg|win-dbg]
```
### Enable Features
All features (JSON, gRPC, AI) are **always enabled** by default.
No need to specify `-DZ3ED_AI=ON` or `-DYAZE_WITH_GRPC=ON`.
### Windows Troubleshooting
```powershell
# Verify environment
.\scripts\verify-build-environment.ps1
# Use vcpkg for faster builds
.\scripts\setup-vcpkg-windows.ps1
cmake -B build -DCMAKE_TOOLCHAIN_FILE="vcpkg/scripts/buildsystems/vcpkg.cmake"
```
---
## Filesystem Abstraction
To ensure robust and consistent behavior across platforms, YAZE has standardized its filesystem operations:
- **`std::filesystem`**: All new and refactored code uses the C++17 `std::filesystem` library for path manipulation, directory iteration, and file operations. This eliminates platform-specific bugs related to path separators (`/` vs `\`).
- **`PlatformPaths` Utility**: A dedicated utility class, `yaze::util::PlatformPaths`, provides platform-aware API for retrieving standard directory locations:
- **Application Data**: `%APPDATA%` on Windows, `~/Library/Application Support` on macOS, XDG Base Directory on Linux
- **Configuration Files**: Semantically clear API for config file locations
- **Home and Temporary Directories**: Safely resolves user-specific and temporary folders
This removes legacy platform-specific APIs (like `dirent.h` or Win32 directory functions) for cleaner, more maintainable file handling.
---
## Native File Dialog Support
YAZE features native file dialogs on all platforms:
- **macOS**: Cocoa-based file selection with proper sandboxing support
- **Windows**: Windows Explorer integration with COM APIs (`IFileOpenDialog`/`IFileSaveDialog`)
- **Linux**: GTK3 dialogs that match system appearance
- **Fallback**: Cross-platform implementation when native dialogs unavailable
---
**Status:** All CI/CD issues resolved. Next push should build successfully on all platforms.

206
docs/public/build/presets.md Normal file
View File

@@ -0,0 +1,206 @@
# Build Presets Guide
This document explains the reorganized CMake preset system for Yaze.
## Design Principles
1. **Short, memorable names** - No more `macos-dev-z3ed-ai`, just `mac-ai`
2. **Warnings off by default** - Add `-v` suffix for verbose (e.g., `mac-dbg-v`)
3. **Clear architecture support** - Explicit ARM64 and x86_64 presets
4. **Platform prefixes** - `mac-`, `win-`, `lin-` for easy identification
## Quick Start
### macOS Development
```bash
# Most common: AI-enabled development
cmake --preset mac-ai
cmake --build --preset mac-ai
# Basic debug build (no AI)
cmake --preset mac-dbg
cmake --build --preset mac-dbg
# Verbose warnings for debugging
cmake --preset mac-dbg-v
cmake --build --preset mac-dbg-v
# Release build
cmake --preset mac-rel
cmake --build --preset mac-rel
```
### Windows Development
```bash
# Debug build (x64)
cmake --preset win-dbg
cmake --build --preset win-dbg
# AI-enabled development
cmake --preset win-ai
cmake --build --preset win-ai
# ARM64 support
cmake --preset win-arm
cmake --build --preset win-arm
```
## All Presets
### macOS Presets
| Preset | Description | Arch | Warnings | Features |
|--------|-------------|------|----------|----------|
| `mac-dbg` | Debug build | ARM64 | Off | Basic |
| `mac-dbg-v` | Debug verbose | ARM64 | On | Basic |
| `mac-rel` | Release | ARM64 | Off | Basic |
| `mac-x64` | Debug x86_64 | x86_64 | Off | Basic |
| `mac-uni` | Universal binary | Both | Off | Basic |
| `mac-dev` | Development | ARM64 | Off | ROM tests |
| `mac-ai` | AI development | ARM64 | Off | z3ed, JSON, gRPC, ROM tests |
| `mac-z3ed` | z3ed CLI | ARM64 | Off | AI agent support |
| `mac-rooms` | Dungeon editor | ARM64 | Off | Minimal build for room editing |
### Windows Presets
| Preset | Description | Arch | Warnings | Features |
|--------|-------------|------|----------|----------|
| `win-dbg` | Debug build | x64 | Off | Basic |
| `win-dbg-v` | Debug verbose | x64 | On | Basic |
| `win-rel` | Release | x64 | Off | Basic |
| `win-arm` | Debug ARM64 | ARM64 | Off | Basic |
| `win-arm-rel` | Release ARM64 | ARM64 | Off | Basic |
| `win-dev` | Development | x64 | Off | ROM tests |
| `win-ai` | AI development | x64 | Off | z3ed, JSON, gRPC, ROM tests |
| `win-z3ed` | z3ed CLI | x64 | Off | AI agent support |
### Linux Presets
| Preset | Description | Compiler | Warnings |
|--------|-------------|----------|----------|
| `lin-dbg` | Debug | GCC | Off |
| `lin-clang` | Debug | Clang | Off |
### Special Purpose
| Preset | Description |
|--------|-------------|
| `ci` | Continuous integration (no ROM tests) |
| `asan` | AddressSanitizer build |
| `coverage` | Code coverage build |
## Warning Control
By default, all presets suppress compiler warnings with `-w` for a cleaner build experience.
### To Enable Verbose Warnings:
1. Use a preset with `-v` suffix (e.g., `mac-dbg-v`, `win-dbg-v`)
2. Or set `YAZE_SUPPRESS_WARNINGS=OFF` manually:
```bash
cmake --preset mac-dbg -DYAZE_SUPPRESS_WARNINGS=OFF
```
## Architecture Support
### macOS
- **ARM64 (Apple Silicon)**: `mac-dbg`, `mac-rel`, `mac-ai`, etc.
- **x86_64 (Intel)**: `mac-x64`
- **Universal Binary**: `mac-uni` (both ARM64 + x86_64)
### Windows
- **x64**: `win-dbg`, `win-rel`, `win-ai`, etc.
- **ARM64**: `win-arm`, `win-arm-rel`
## Build Directories
Most presets use `build/` directory. Exceptions:
- `mac-rooms`: Uses `build_rooms/` to avoid conflicts
## Feature Flags
Common CMake options you can override:
```bash
# Enable/disable components
-DYAZE_BUILD_TESTS=ON/OFF
-DYAZE_BUILD_Z3ED=ON/OFF
-DYAZE_BUILD_EMU=ON/OFF
-DYAZE_BUILD_APP=ON/OFF
# AI features
-DZ3ED_AI=ON/OFF
-DYAZE_WITH_JSON=ON/OFF
-DYAZE_WITH_GRPC=ON/OFF
# Testing
-DYAZE_ENABLE_ROM_TESTS=ON/OFF
-DYAZE_TEST_ROM_PATH=/path/to/zelda3.sfc
# Build optimization
-DYAZE_MINIMAL_BUILD=ON/OFF
```
## Examples
### Development with AI features and verbose warnings
```bash
cmake --preset mac-dbg-v -DZ3ED_AI=ON -DYAZE_WITH_GRPC=ON
cmake --build --preset mac-dbg-v
```
### Release build for distribution (macOS Universal)
```bash
cmake --preset mac-uni
cmake --build --preset mac-uni
cpack --preset mac-uni
```
### Quick minimal build for testing
```bash
cmake --preset mac-dbg -DYAZE_MINIMAL_BUILD=ON
cmake --build --preset mac-dbg -j12
```
## Updating compile_commands.json
After configuring with a new preset, copy the compile commands for IDE support:
```bash
cp build/compile_commands.json .
```
This ensures clangd and other LSP servers can find headers and understand build flags.
## Migration from Old Presets
Old preset names have been simplified:
| Old Name | New Name |
|----------|----------|
| `macos-dev-z3ed-ai` | `mac-ai` |
| `macos-debug` | `mac-dbg` |
| `macos-release` | `mac-rel` |
| `macos-debug-universal` | `mac-uni` |
| `macos-dungeon-dev` | `mac-rooms` |
| `windows-debug` | `win-dbg` |
| `windows-release` | `win-rel` |
| `windows-arm64-debug` | `win-arm` |
| `linux-debug` | `lin-dbg` |
| `linux-clang` | `lin-clang` |
## Troubleshooting
### Warnings are still showing
- Make sure you're using a preset without `-v` suffix
- Check `cmake` output for `✓ Warnings suppressed` message
- Reconfigure and rebuild: `rm -rf build && cmake --preset mac-dbg`
### clangd can't find nlohmann/json
- Run `cmake --preset <your-preset>` to regenerate compile_commands.json
- Copy to root: `cp build/compile_commands.json .`
- Restart your IDE or LSP server
### Build fails with missing dependencies
- Ensure submodules are initialized: `git submodule update --init --recursive`
- For AI features, make sure you have OpenSSL: `brew install openssl` (macOS)

80
docs/public/build/quick-reference.md Normal file
View File

@@ -0,0 +1,80 @@
# Build & Test Quick Reference
Use this document as the single source of truth for configuring, building, and testing YAZE across
platforms. Other guides (README, CLAUDE.md, GEMINI.md, etc.) should link here instead of duplicating
steps.
## 1. Environment Prep
- Clone with submodules: `git clone --recursive https://github.com/scawful/yaze.git`
- Run the verifier once per machine:
- macOS/Linux: `./scripts/verify-build-environment.sh --fix`
- Windows PowerShell: `.\scripts\verify-build-environment.ps1 -FixIssues`
## 2. Build Presets
Use `cmake --preset <name>` followed by `cmake --build --preset <name> [--target …]`.
| Preset | Platform(s) | Notes |
|-------------|-------------|-------|
| `mac-dbg`, `lin-dbg`, `win-dbg` | macOS/Linux/Windows | Standard debug builds, tests on by default. |
| `mac-ai`, `lin-ai`, `win-ai` | macOS/Linux/Windows | Enables gRPC, agent UI, `z3ed`, and AI runtime. |
| `mac-rel`, `lin-rel`, `win-rel` | macOS/Linux/Windows | Optimized release builds. |
| `mac-dev`, `lin-dev`, `win-dev` | macOS/Linux/Windows | Development builds with ROM-dependent tests enabled. |
| `mac-uni` | macOS | Universal binary (ARM64 + x86_64) for distribution. |
| `ci-*` presets | Platform-specific | Mirrors CI matrix; see `CMakePresets.json`. |
**Verbose builds**: add `-v` suffix (e.g., `mac-dbg-v`, `lin-dbg-v`, `win-dbg-v`) to turn off compiler warning suppression.
## 3. AI/Assistant Build Policy
- Human developers typically use `build` or `build_test` directories.
- AI assistants **must use dedicated directories** (`build_ai`, `build_agent`, etc.) to avoid
clobbering user builds.
- When enabling AI features, prefer the `*-ai` presets and target only the binaries you need
(`yaze`, `z3ed`, `yaze_test`, …).
- Windows helpers: use `scripts/agents/windows-smoke-build.ps1` for quick builds and `scripts/agents/run-tests.sh` (or its PowerShell equivalent) for test runs so preset + generator settings stay consistent.
## 4. Common Commands
```bash
# Debug GUI build (macOS)
cmake --preset mac-dbg
cmake --build --preset mac-dbg --target yaze
# Debug GUI build (Linux)
cmake --preset lin-dbg
cmake --build --preset lin-dbg --target yaze
# Debug GUI build (Windows)
cmake --preset win-dbg
cmake --build --preset win-dbg --target yaze
# AI-enabled build with gRPC (macOS)
cmake --preset mac-ai
cmake --build --preset mac-ai --target yaze z3ed
# AI-enabled build with gRPC (Linux)
cmake --preset lin-ai
cmake --build --preset lin-ai --target yaze z3ed
# AI-enabled build with gRPC (Windows)
cmake --preset win-ai
cmake --build --preset win-ai --target yaze z3ed
```
## 5. Testing
- Build target: `cmake --build --preset <preset> --target yaze_test`
- Run all tests: `./build/bin/yaze_test`
- Filtered runs:
- `./build/bin/yaze_test --unit`
- `./build/bin/yaze_test --integration`
- `./build/bin/yaze_test --e2e --show-gui`
- `./build/bin/yaze_test --rom-dependent --rom-path path/to/zelda3.sfc`
- Preset-based ctest: `ctest --preset dev`
Environment variables:
- `YAZE_TEST_ROM_PATH` – default ROM for ROM-dependent tests.
- `YAZE_SKIP_ROM_TESTS`, `YAZE_ENABLE_UI_TESTS` – gate expensive suites.
## 6. Troubleshooting & References
- Detailed troubleshooting: `docs/public/build/troubleshooting.md`
- Platform compatibility: `docs/public/build/platform-compatibility.md`
- Internal agents must follow coordination protocol in
`docs/internal/agents/coordination-board.md` before running builds/tests.

472
docs/public/build/troubleshooting.md Normal file
View File

@@ -0,0 +1,472 @@
# YAZE Build Troubleshooting Guide
**Last Updated**: October 2025
**Related Docs**: BUILD-GUIDE.md, ci-cd/CI-SETUP.md
## Table of Contents
- [gRPC ARM64 Issues](#grpc-arm64-issues)
- [Windows Build Issues](#windows-build-issues)
- [macOS Issues](#macos-issues)
- [Linux Issues](#linux-issues)
- [Common Build Errors](#common-build-errors)
---
## gRPC ARM64 Issues
### Status: Known Issue with Workarounds
The ARM64 macOS build has persistent issues with Abseil's random number generation targets when building gRPC from source. This issue has been ongoing through multiple attempts to fix.
### The Problem
**Error**:
```
clang++: error: unsupported option '-msse4.1' for target 'arm64-apple-darwin25.0.0'
```
**Target**: `absl_random_internal_randen_hwaes_impl`
**Root Cause**: Abseil's random number generation targets are being built with x86-specific compiler flags (`-msse4.1`, `-maes`, `-msse4.2`) on ARM64 macOS.
### Working Configuration
**gRPC Version**: v1.67.1 (tested and stable)
**Protobuf Version**: 3.28.1 (bundled with gRPC)
**Abseil Version**: 20240116.0 (bundled with gRPC)
### Solution Approaches Tried
#### ❌ Failed Attempts
1. **CMake flag configuration** - Abseil variables being overridden by gRPC
2. **Global CMAKE_CXX_FLAGS** - Flags set at target level, not global
3. **Pre-configuration Abseil settings** - gRPC overrides them
4. **Different gRPC versions** - v1.62.0, v1.68.0, v1.60.0, v1.58.0 all have issues
#### ✅ Working Approach: Target Property Manipulation
The working solution involves manipulating target properties after gRPC is configured:
```cmake
# In cmake/grpc.cmake (from working commit 6db7ba4782)
if(APPLE AND CMAKE_OSX_ARCHITECTURES STREQUAL "arm64")
# List of Abseil targets with x86-specific flags
set(_absl_targets_with_x86_flags
absl_random_internal_randen_hwaes_impl
absl_random_internal_randen_hwaes
absl_crc_internal_cpu_detect
)
foreach(_absl_target IN LISTS _absl_targets_with_x86_flags)
if(TARGET ${_absl_target})
get_target_property(_absl_opts ${_absl_target} COMPILE_OPTIONS)
if(_absl_opts)
# Remove SSE flags: -maes, -msse4.1, -msse2, -Xarch_x86_64
list(FILTER _absl_opts EXCLUDE REGEX "^-m(aes|sse)")
list(FILTER _absl_opts EXCLUDE REGEX "^-Xarch_x86_64")
set_property(TARGET ${_absl_target} PROPERTY COMPILE_OPTIONS ${_absl_opts})
endif()
endif()
endforeach()
endif()
```
### Current Workaround
**Option 1**: Use the bundled Abseil with target property manipulation (as above)
**Option 2**: Disable gRPC for ARM64 development
```cmake
if(APPLE AND CMAKE_OSX_ARCHITECTURES STREQUAL "arm64")
set(YAZE_WITH_GRPC OFF CACHE BOOL "" FORCE)
message(STATUS "ARM64: Disabling gRPC due to build issues")
endif()
```
**Option 3**: Use pre-built vcpkg packages (Windows-style approach for macOS)
```bash
brew install grpc protobuf abseil
# Then use find_package instead of FetchContent
```
### Environment Configuration
**Homebrew LLVM Configuration**:
- **Toolchain**: `cmake/llvm-brew.toolchain.cmake`
- **SDK**: `/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk`
- **C++ Standard Library**: Homebrew's libc++ (not system libstdc++)
### Build Commands for Testing
```bash
# Clean build
rm -rf build/_deps/grpc-build
# Test configuration
cmake --preset mac-dbg
# Test build
cmake --build --preset mac-dbg --target protoc
```
### Success Criteria
The build succeeds when:
```bash
cmake --build --preset mac-dbg --target protoc
# Returns exit code 0 (no SSE flag errors)
```
### Files to Monitor
**Critical Files**:
- `cmake/grpc.cmake` - Main gRPC configuration
- `build/_deps/grpc-build/third_party/abseil-cpp/` - Abseil build output
- `build/_deps/grpc-build/third_party/abseil-cpp/absl/random/CMakeFiles/` - Random target build files
**Log Files**:
- CMake configuration output (look for Abseil configuration messages)
- Build output (look for compiler flag errors)
- `build/_deps/grpc-build/CMakeCache.txt` - Check if ARM64 flags are set
### Additional Resources
- **gRPC ARM64 Issues**: https://github.com/grpc/grpc/issues (search for ARM64, macOS, Abseil)
- **Abseil Random Documentation**: https://abseil.io/docs/cpp/guides/random
- **CMake FetchContent**: https://cmake.org/cmake/help/latest/module/FetchContent.html
---
## Windows Build Issues
### MSVC Compatibility with gRPC
**Problem**: gRPC v1.75.1 has MSVC compilation errors in UPB (micro protobuf) code
**Error**:
```
error C2099: initializer is not a constant
```
**Solution**: Use gRPC v1.67.1 (MSVC-compatible, tested) or use vcpkg pre-built packages
### vcpkg Integration (Recommended)
#### Setup vcpkg
```powershell
# Install vcpkg if not already installed
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg
.\bootstrap-vcpkg.bat
# Install packages
.\vcpkg install grpc:x64-windows protobuf:x64-windows sdl2:x64-windows yaml-cpp:x64-windows
```
#### Configure CMake to Use vcpkg
**Option 1**: Set environment variable
```powershell
$env:CMAKE_TOOLCHAIN_FILE = "C:\path\to\vcpkg\scripts\buildsystems\vcpkg.cmake"
cmake --preset win-dbg
```
**Option 2**: Use CMake preset with toolchain
```json
// CMakePresets.json (already configured)
{
"name": "win-dbg",
"cacheVariables": {
"CMAKE_TOOLCHAIN_FILE": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake"
}
}
```
#### Expected Build Times
| Method | Time | Version | Status |
|--------|------|---------|--------|
| **vcpkg** (recommended) | ~5-10 min | gRPC 1.71.0 | ✅ Pre-compiled binaries |
| **FetchContent** (fallback) | ~30-40 min | gRPC 1.67.1 | ✅ MSVC-compatible |
| **FetchContent** (old) | ~45+ min | gRPC 1.75.1 | ❌ UPB compilation errors |
### Long Path Issues
Windows has a default path length limit of 260 characters, which can cause issues with deep dependency trees.
**Solution**:
```powershell
# Enable long paths for Git
git config --global core.longpaths true
# Enable long paths system-wide (requires admin)
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" `
-Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force
```
### Missing Visual Studio Components
**Error**: "Could not find Visual C++ compiler"
**Solution**: Install "Desktop development with C++" workload via Visual Studio Installer
```powershell
# Verify Visual Studio installation
.\scripts\verify-build-environment.ps1
```
### Package Detection Issues
**Problem**: `find_package(gRPC CONFIG)` not finding vcpkg-installed packages
**Causes**:
1. Case sensitivity: vcpkg uses lowercase `grpc` config
2. Namespace mismatch: vcpkg provides `gRPC::grpc++` target
3. Missing packages in `vcpkg.json`
**Solution**: Enhanced detection in `cmake/grpc_windows.cmake`:
```cmake
# Try both case variations
find_package(gRPC CONFIG QUIET)
if(NOT gRPC_FOUND)
find_package(grpc CONFIG QUIET) # Try lowercase
if(grpc_FOUND)
set(gRPC_FOUND TRUE)
endif()
endif()
# Create aliases for non-namespaced targets
if(TARGET gRPC::grpc++)
add_library(grpc++ ALIAS gRPC::grpc++)
add_library(grpc++_reflection ALIAS gRPC::grpc++_reflection)
endif()
```
---
## macOS Issues
### Homebrew SDL2 Not Found
**Problem**: SDL.h headers not found even with Homebrew SDL2 installed
**Solution**: Add Homebrew include path explicitly
```cmake
# In cmake/dependencies/sdl2.cmake
if(APPLE)
include_directories(/opt/homebrew/opt/sdl2/include/SDL2) # Apple Silicon
include_directories(/usr/local/opt/sdl2/include/SDL2) # Intel
endif()
```
### Code Signing Issues
**Problem**: "yaze.app is damaged and can't be opened"
**Solution**: Sign the application bundle
```bash
codesign --force --deep --sign - build/bin/yaze.app
```
### Multiple Xcode Versions
**Problem**: CMake using wrong SDK or compiler version
**Solution**: Select Xcode version explicitly
```bash
sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
```
---
## Linux Issues
### Missing Dependencies
**Error**: Headers not found for various libraries
**Solution**: Install development packages
**Ubuntu/Debian**:
```bash
sudo apt-get update
sudo apt-get install -y \
build-essential cmake ninja-build pkg-config \
libglew-dev libxext-dev libwavpack-dev libboost-all-dev \
libpng-dev python3-dev \
libasound2-dev libpulse-dev \
libx11-dev libxrandr-dev libxcursor-dev libxinerama-dev libxi-dev \
libxss-dev libxxf86vm-dev libxkbcommon-dev libwayland-dev libdecor-0-dev \
libgtk-3-dev libdbus-1-dev git
```
**Fedora/RHEL**:
```bash
sudo dnf install -y \
gcc-c++ cmake ninja-build pkg-config \
glew-devel libXext-devel wavpack-devel boost-devel \
libpng-devel python3-devel \
alsa-lib-devel pulseaudio-libs-devel \
libX11-devel libXrandr-devel libXcursor-devel libXinerama-devel libXi-devel \
libXScrnSaver-devel libXxf86vm-devel libxkbcommon-devel wayland-devel \
gtk3-devel dbus-devel git
```
### GCC Version Too Old
**Error**: C++23 features not supported
**Solution**: Install newer GCC or use Clang
```bash
# Install GCC 13
sudo apt-get install -y gcc-13 g++-13
# Configure CMake to use GCC 13
cmake --preset lin-dbg \
-DCMAKE_C_COMPILER=gcc-13 \
-DCMAKE_CXX_COMPILER=g++-13
```
---
## Common Build Errors
### "Target not found" Errors
**Error**: `CMake Error: Cannot specify link libraries for target "X" which is not built by this project`
**Causes**:
1. Target aliasing issues
2. Dependency order problems
3. Missing `find_package()` calls
**Solutions**:
1. Check `cmake/dependencies.cmake` for proper target exports
2. Ensure dependencies are included before they're used
3. Verify target names match (e.g., `grpc++` vs `gRPC::grpc++`)
### Protobuf Version Mismatch
**Error**: "Protobuf C++ gencode is built with an incompatible version"
**Cause**: System protoc version doesn't match bundled protobuf runtime
**Solution**: Use bundled protoc
```cmake
set(_gRPC_PROTOBUF_PROTOC_EXECUTABLE $<TARGET_FILE:protoc>)
```
### compile_commands.json Not Generated
**Problem**: IntelliSense/clangd not working
**Solution**: Ensure preset uses Ninja Multi-Config generator
```bash
cmake --preset mac-dbg # Uses Ninja Multi-Config
# compile_commands.json will be at build/compile_commands.json
```
### ImGui ID Collisions
**Error**: "Dear ImGui: Duplicate ID"
**Solution**: Add `PushID/PopID` scopes around widgets
```cpp
ImGui::PushID("unique_identifier");
// ... widgets here ...
ImGui::PopID();
```
### ASAR Library Build Errors
**Status**: Known issue with stubbed implementation
**Current State**: ASAR methods return `UnimplementedError`
**Workaround**: Assembly patching features are disabled until ASAR CMakeLists.txt macro errors are fixed
---
## Debugging Tips
### Enable Verbose Build Output
```bash
# Verbose CMake configuration
cmake --preset mac-dbg -- -LAH
# Verbose build
cmake --build --preset mac-dbg --verbose
# Very verbose build
cmake --build --preset mac-dbg -- -v VERBOSE=1
```
### Check Dependency Detection
```bash
# See what CMake found
cmake --preset mac-dbg 2>&1 | grep -E "(Found|Using|Detecting)"
# Check cache for specific variables
cmake -LA build/ | grep -i grpc
```
### Isolate Build Issues
```bash
# Build specific targets to isolate issues
cmake --build build --target yaze_canvas # Just canvas library
cmake --build build --target yaze_gfx # Just graphics library
cmake --build build --target protoc # Just protobuf compiler
```
### Clean Builds
```bash
# Clean build directory (fast)
cmake --build build --target clean
# Remove build artifacts but keep dependencies (medium)
rm -rf build/bin build/lib
# Nuclear option - full rebuild (slow, 30+ minutes)
rm -rf build/
cmake --preset mac-dbg
```
---
## Getting Help
1. **Check existing documentation**:
- BUILD-GUIDE.md - General build instructions
- CLAUDE.md - Project overview
- CI/CD logs - .github/workflows/ci.yml
2. **Search git history** for working configurations:
```bash
git log --all --grep="grpc" --oneline
git show <commit-hash>:cmake/grpc.cmake
```
3. **Enable debug logging**:
```bash
YAZE_LOG_LEVEL=DEBUG ./build/bin/yaze 2>&1 | tee debug.log
```
4. **Create a minimal reproduction**:
- Isolate the failing component
- Create a minimal CMakeLists.txt
- Test with minimal dependencies
5. **File an issue** with:
- Platform and OS version
- CMake preset used
- Full error output
- `cmake -LA build/` output
- Relevant CMakeCache.txt entries

181
docs/public/developer/api-reference.md Normal file
View File

@@ -0,0 +1,181 @@
# API Reference
This document provides a reference for the yaze C and C++ APIs, intended for developers creating extensions or contributing to the core application.
## C API (`incl/yaze.h`)
The C API provides a stable, language-agnostic interface for interacting with yaze's core functionalities.
### Core Library Functions
```c
/**
* @brief Initializes the yaze library. Must be called before any other API function.
* @return YAZE_OK on success, or an error code on failure.
*/
yaze_status yaze_library_init(void);
/**
* @brief Shuts down the yaze library and releases all resources.
*/
void yaze_library_shutdown(void);
/**
* @brief Gets the current version of the yaze library as a string.
* @return A constant string representing the version (e.g., "0.3.2").
*/
const char* yaze_get_version_string(void);
```
### ROM Operations
```c
/**
* @brief Loads a Zelda 3 ROM from a file.
* @param filename The path to the ROM file.
* @return A pointer to a zelda3_rom object, or NULL on failure.
*/
zelda3_rom* yaze_load_rom(const char* filename);
/**
* @brief Unloads a ROM and frees associated memory.
* @param rom A pointer to the zelda3_rom object to unload.
*/
void yaze_unload_rom(zelda3_rom* rom);
/**
* @brief Saves a ROM to a file.
* @param rom A pointer to the ROM to save.
* @param filename The path to save the file to.
* @return YAZE_OK on success.
*/
yaze_status yaze_save_rom(zelda3_rom* rom, const char* filename);
```
## C++ API
The C++ API offers a more powerful, object-oriented interface. The primary entry point for many operations is the `yaze::core::AsarWrapper` class.
### AsarWrapper (`src/core/asar_wrapper.h`)
This class provides a complete, cross-platform interface for applying assembly patches, extracting symbols, and validating assembly code using the Asar library.
#### CLI Examples (`z3ed`)
While the `AsarWrapper` can be used programmatically, the `z3ed` CLI is the most common way to interact with it.
```bash
# Apply an assembly patch to a ROM file.
z3ed asar my_patch.asm --rom=zelda3.sfc
# For more complex operations, use the AI agent.
z3ed agent chat --rom zelda3.sfc
```
> **Prompt:** "Apply the patch `mosaic_change.asm` to my ROM."
#### C++ API Example
```cpp
#include "core/asar_wrapper.h"
#include <vector>
#include <string>
// Assume rom_data is a std::vector<uint8_t> holding the ROM content.
yaze::core::AsarWrapper wrapper;
wrapper.Initialize();
// Apply a patch to the ROM data in memory.
auto result = wrapper.ApplyPatch("patch.asm", rom_data);
if (result.ok() && result->success) {
// On success, print the symbols generated by the patch.
for (const auto& symbol : result->symbols) {
std::cout << symbol.name << " @ $" << std::hex << symbol.address << std::endl;
}
}
```
#### Class Definition
```cpp
namespace yaze::core {
class AsarWrapper {
public:
/** @brief Initializes the Asar library. */
absl::Status Initialize();
/** @brief Shuts down the Asar library. */
void Shutdown();
/**
* @brief Applies an assembly patch to ROM data.
* @param patch_path Path to the main .asm file.
* @param rom_data A vector of bytes representing the ROM data.
* @param include_paths Optional paths for Asar to search for included files.
* @return A StatusOr containing the patch result, including success status and symbols.
*/
absl::StatusOr<AsarPatchResult> ApplyPatch(
const std::string& patch_path,
std::vector<uint8_t>& rom_data,
const std::vector<std::string>& include_paths = {});
/**
* @brief Extracts symbols from an assembly file without patching.
* @param asm_path Path to the .asm file.
* @return A StatusOr containing a vector of extracted symbols.
*/
absl::StatusOr<std::vector<AsarSymbol>> ExtractSymbols(
const std::string& asm_path,
const std::vector<std::string>& include_paths = {});
/**
* @brief Validates the syntax of an assembly file.
* @param asm_path Path to the .asm file.
* @return An OK status if syntax is valid, or an error status if not.
*/
absl::Status ValidateAssembly(const std::string& asm_path);
};
} // namespace yaze::core
```
## Data Structures
### `snes_color`
Represents a 15-bit SNES color, composed of 5 bits for each red, green, and blue component.
```c
typedef struct snes_color {
uint16_t raw; //!< Raw 15-bit BGR color value (0bbbbbgggggrrrrr).
uint8_t red; //!< Red component (0-31).
uint8_t green; //!< Green component (0-31).
uint8_t blue; //!< Blue component (0-31).
} snes_color;
```
### `zelda3_message`
Represents an in-game text message.
```c
typedef struct zelda3_message {
uint16_t id; //!< The message ID (0-65535).
uint32_t rom_address; //!< The address of the message data in the ROM.
uint16_t length; //!< The length of the raw message data in bytes.
uint8_t* raw_data; //!< A pointer to the raw, compressed message data.
char* parsed_text; //!< The decoded, human-readable UTF-8 text.
bool is_compressed; //!< Flag indicating if the message data is compressed.
}
zelda3_message;
```
## Error Handling
The C API uses an enum `yaze_status` for error handling, while the C++ API uses `absl::Status` and `absl::StatusOr`.
### C API Error Pattern
```c
yaze_status status = yaze_library_init();
if (status != YAZE_OK) {
fprintf(stderr, "Failed to initialize YAZE: %s\n", yaze_status_to_string(status));
return 1;
}
// ... operations ...
yaze_library_shutdown();
```

230
docs/public/developer/architecture.md Normal file
View File

@@ -0,0 +1,230 @@
# E2 - Development Guide
This guide summarizes the architecture and implementation standards used across the editor codebase.
## Editor Status (October 2025)
| Editor | State | Notes |
|-------------------|--------------|-------|
| Overworld | Stable | Full feature set; continue regression testing after palette fixes. |
| Message | Stable | Re-test rendering after recent palette work. |
| Emulator | Stable | UI and core subsystems aligned with production builds. |
| Palette | Stable | Serves as the source of truth for palette helpers. |
| Assembly | Stable | No outstanding refactors. |
| Dungeon | Experimental | Requires thorough manual coverage before release. |
| Graphics | Experimental | Large rendering changes in flight; validate texture pipeline. |
| Sprite | Experimental | UI patterns still migrating to the new card system. |
### Screen Editor Notes
- **Title screen**: Vanilla ROM tilemap parsing remains broken. Implement a DMA
parser and confirm the welcome screen renders before enabling painting.
- **Overworld map**: Mode 7 tiling, palette switching, and custom map import/export are in place. The next milestone is faster tile painting.
- **Dungeon map**: Rendering is wired up; tile painting and ROM write-back are still pending.
## 1. Core Architectural Patterns
These patterns, established during the Overworld Editor refactoring, should be applied to all new and existing editor components.
### Pattern 1: Modular Systems
**Principle**: Decompose large, monolithic editor classes into smaller, single-responsibility modules.
- **Rendering**: All drawing logic should be extracted into dedicated `*Renderer` classes (e.g., `OverworldEntityRenderer`). The main editor class should delegate drawing calls, not implement them.
- **UI Panels**: Complex UI panels should be managed by their own classes (e.g., `MapPropertiesSystem`), which then communicate with the parent editor via callbacks.
- **Interaction**: Canvas interaction logic (mouse handling, editing modes) should be separated from the main editor class to simplify state management.
**Benefit**: Smaller, focused modules are easier to test, debug, and maintain. The main editor class becomes a coordinator, which is a much cleaner architecture.
### Pattern 2: Callback-Based Communication
**Principle**: Use `std::function` callbacks for child-to-parent communication to avoid circular dependencies.
- **Implementation**: A parent editor provides its child components with callbacks (typically via a `SetCallbacks` method) during initialization. The child component invokes these callbacks to notify the parent of events or to request actions (like a refresh).
- **Example**: `MapPropertiesSystem` receives a `RefreshCallback` from `OverworldEditor`. When a property is changed in the UI, it calls the function, allowing the `OverworldEditor` to execute the refresh logic without the `MapPropertiesSystem` needing to know anything about the editor itself.
### Pattern 3: Centralized Progressive Loading via `gfx::Arena`
**Principle**: All expensive asset loading operations must be performed asynchronously to prevent UI freezes. The `gfx::Arena` singleton provides a centralized, priority-based system for this.
- **How it Works**:
1. **Queue**: Instead of loading a texture directly, queue it with the arena: `gfx::Arena::Get().QueueDeferredTexture(bitmap, priority);`
2. **Prioritize**: Assign a numerical priority. Lower numbers are higher priority. Use a high priority for assets the user is currently viewing and a low priority for assets that can be loaded in the background.
3. **Process**: In the main `Update()` loop of an editor, process a small batch of textures each frame: `auto batch = gfx::Arena::Get().GetNextDeferredTextureBatch(4, 2);` (e.g., 4 high-priority, 2 low-priority).
- **Benefit**: This provides a globally consistent, non-blocking loading mechanism that is available to all editors and ensures the UI remains responsive.
## 2. UI & Theming System
To ensure a consistent and polished look and feel, all new UI components must adhere to the established theme and helper function system.
### 2.1. The Theme System (`AgentUITheme`)
- **Principle**: **Never use hardcoded colors (`ImVec4`)**. All UI colors must be derived from the central theme.
- **Implementation**: The `AgentUITheme` system (`src/app/editor/agent/agent_ui_theme.h`) provides a struct of semantic color names (e.g., `panel_bg_color`, `status_success`, `provider_ollama`). These colors are automatically derived from the application's current `ThemeManager`.
- **Usage**: Fetch the theme at the beginning of a draw call and use the semantic colors:
```cpp
const auto& theme = AgentUI::GetTheme();
ImGui::PushStyleColor(ImGuiCol_ChildBg, theme.panel_bg_color);
```
### 2.2. Reusable UI Helper Functions
- **Principle**: Encapsulate common UI patterns into helper functions to reduce boilerplate and ensure consistency.
- **Available Helpers** (in `AgentUI` and `gui` namespaces):
- **Panels**: `AgentUI::PushPanelStyle()` / `PopPanelStyle()`
- **Headers**: `AgentUI::RenderSectionHeader(icon, label, color)`
- **Indicators**: `AgentUI::RenderStatusIndicator()` (status dot), `AgentUI::StatusBadge()` (colored text badge).
- **Buttons**: `AgentUI::StyledButton()`, `AgentUI::IconButton()`.
- **Layout**: `AgentUI::VerticalSpacing()`, `AgentUI::HorizontalSpacing()`.
- **Benefit**: Creates a consistent visual language and makes the UI code far more readable and maintainable.
### 2.3. Toolbar Implementation (`CompactToolbar`)
- **Stretching**: To prevent ImGui from stretching the last item in a toolbar, do not use `ImGui::BeginGroup()`. Instead, manage layout with `ImGui::SameLine()` and end the toolbar with `ImGui::NewLine()`.
- **Separators**: Use `ImGui::SeparatorEx(ImGuiSeparatorFlags_Vertical)` for vertical separators that do not affect item layout.
- **Property Inputs**: Use the `toolbar.AddProperty()` method for consistent spacing and sizing of input fields within the toolbar.
## 3. Key System Implementations & Gotchas
### 3.1. Graphics Refresh Logic
- **Immediate vs. Deferred**: When a visual property changes, the texture must be updated on the GPU immediately. Use `Renderer::Get().RenderBitmap()` for an immediate, blocking update. `UpdateBitmap()` is deferred and should not be used for changes the user expects to see instantly.
- **Call Order is Critical**: When a property affecting graphics is changed, the correct sequence of operations is crucial:
1. Update the property in the data model.
2. Call the relevant `Load*()` method (e.g., `map.LoadAreaGraphics()`) to load the new data from the ROM into memory.
3. Force a redraw/re-render of the bitmap.
### 3.2. Multi-Area Map Configuration
- **Use the Helper**: When changing a map's area size (e.g., from `Small` to `Large`), you **must** use the `zelda3::Overworld::ConfigureMultiAreaMap()` method. Do not set the `area_size` property directly.
- **Why**: This method correctly handles the complex logic of assigning parent IDs to all sibling maps and updating the necessary ROM data for persistence. Failure to use it will result in inconsistent state and refresh bugs.
### 3.3. Version-Specific Feature Gating
- **Principle**: The UI must adapt to the features supported by the loaded ROM. Do not show UI for features that are not available.
- **Implementation**: Check the ROM's `asm_version` byte before rendering a UI component. If the feature is not supported, display a helpful message (e.g., "This feature requires ZSCustomOverworld v3+") instead of the UI.
### 3.4. Entity Visibility for Visual Testing
- **Standard**: All overworld entity markers (entrances, exits, items, sprites) should be rendered with a high-contrast color and an alpha of `0.85f` to ensure they are clearly visible against any background.
- **Entrances**: Bright yellow-gold
- **Exits**: Cyan-white
- **Items**: Bright red
- **Sprites**: Bright magenta
## 4. Clang Tooling Configuration
The repository ships curated `.clangd` and `.clang-tidy` files that mirror our
Google-style C++23 guidelines while accommodating ROM hacking patterns.
- `.clangd` consumes `build/compile_commands.json`, enumerates `src/`, `incl/`,
`third_party/`, generated directories, and sets feature flags such as
`YAZE_WITH_GRPC`, `YAZE_WITH_JSON`, and `Z3ED_AI` so IntelliSense matches the
active preset.
- `.clang-tidy` enables the `clang-analyzer`, `performance`, `bugprone`,
`readability`, `modernize`, `google`, and `abseil` suites, but relaxes common
ROM hacking pain points (magic numbers, explicit integer sizing, C arrays,
carefully scoped narrowing conversions).
- The `gfx::SnesColor` utilities intentionally return ImVec4 values in 0‑255
space; rely on the helper converters instead of manual scaling to avoid
precision loss.
- Regenerate the compilation database whenever you reconfigure: `cmake --preset
mac-dbg` (or the platform equivalent) and ensure the file lives at
`build/compile_commands.json`.
- Spot-check tooling with `clang-tidy path/to/file.cc -p build --quiet` or a
batch run via presets before sending larger patches.
## 5. Debugging and Testing
### 5.1. Quick Debugging with Startup Flags
To accelerate your debugging workflow, use command-line flags to jump directly to specific editors and open relevant UI cards:
```bash
# Quick dungeon room testing
./yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Room 0"
# Compare multiple rooms
./yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Room 0,Room 1,Room 105"
# Full dungeon workspace
./yaze --rom_file=zelda3.sfc --editor=Dungeon \
--cards="Rooms List,Room Matrix,Object Editor,Palette Editor"
# Enable debug logging
./yaze --debug --log_file=debug.log --rom_file=zelda3.sfc --editor=Dungeon
```
**Available Editors**: Assembly, Dungeon, Graphics, Music, Overworld, Palette, Screen, Sprite, Message, Hex, Agent, Settings
**Dungeon Editor Cards**: Rooms List, Room Matrix, Entrances List, Room Graphics, Object Editor, Palette Editor, Room N (where N is room ID 0-319)
See [debugging-startup-flags.md](debugging-startup-flags.md) for complete documentation.
### 5.2. Testing Strategies
For a comprehensive overview of debugging tools and testing strategies, including how to use the logging framework, command-line test runners, and the GUI automation harness for AI agents, please refer to the [Debugging and Testing Guide](debugging-guide.md).
## 5. Command-Line Flag Standardization
**Decision**: All binaries in the yaze project (`yaze`, `z3ed`, `yaze_test`, etc.) will standardize on the **Abseil Flags** library for command-line argument parsing.
### Rationale
- **Consistency**: Provides a single, consistent flag system and user experience across all tools.
- **Industry Standard**: Abseil is a battle-tested and well-documented library used extensively by Google.
- **Features**: It provides robust features out-of-the-box, including automatic `--help` generation, type safety, validation, and `--flagfile` support.
- **Reduced Maintenance**: Eliminates the need to maintain multiple custom flag parsers, reducing the project's technical debt.
- **Existing Dependency**: Abseil is already included as a dependency via gRPC, so this adds no new third-party code.
### Migration Plan
The project will migrate away from the legacy `yaze::util::Flag` system and manual parsing in phases. All new flags should be implemented using `ABSL_FLAG`.
- **Phase 1 (Complete)**: `z3ed` and `yaze_emu_test` already use Abseil flags.
- **Phase 2 (In Progress)**: The main `yaze` application will be migrated from the custom `util::Flag` system to Abseil flags.
- **Phase 3 (Future)**: The `yaze_test` runner's manual argument parsing will be replaced with Abseil flags.
- **Phase 4 (Cleanup)**: The legacy `util::flag` source files will be removed from the project.
Developers should refer to the Abseil Flags Guide for documentation on defining, declaring, and accessing flags.
---
When working with bitmaps and textures, understand that two memory locations must stay synchronized:
1. **`data_` vector**: C++ std::vector<uint8_t> holding pixel data
2. **`surface_->pixels`**: SDL surface's raw pixel buffer (used for texture creation)
**Critical Rules**:
- Use `set_data()` for bulk data replacement (syncs both vector and surface)
- Use `WriteToPixel()` for single-pixel modifications
- Never assign directly to `mutable_data()` for replacements (only updates vector, not surface)
- Always call `ProcessTextureQueue()` every frame to process pending texture operations
**Example**:
```cpp
// WRONG - only updates vector
bitmap.mutable_data() = new_data;
// CORRECT - updates both vector and SDL surface
bitmap.set_data(new_data);
```
### 3.6. Graphics Sheet Management
Graphics sheets (223 total) are managed centrally by `gfx::Arena`. When modifying a sheet:
1. Modify the sheet: `auto& sheet = Arena::Get().mutable_gfx_sheet(index);`
2. Notify Arena: `Arena::Get().NotifySheetModified(index);`
3. Changes automatically propagate to all editors
Default palettes are applied during ROM loading based on sheet index:
- Sheets 0-112: Dungeon main palettes
- Sheets 113-127: Sprite palettes
- Sheets 128-222: HUD/menu palettes
### Naming Conventions
- Load: Reading data from ROM into memory
- Render: Processing graphics data into bitmaps/textures (CPU pixel operations)
- Draw: Displaying textures/shapes on canvas via ImGui (GPU rendering)
- Update: UI state changes, property updates, input handling

223
docs/public/developer/asm-style-guide.md Normal file
View File

@@ -0,0 +1,223 @@
# Asm Style Guide
65816 Assembly is the assembly language used by the Super Nintendo Entertainment System (SNES) and its Ricoh 5A22 processor. This style guide provides conventions and best practices for writing 65816 assembly code in the context of the yaze project. Following these guidelines will help maintain consistency and readability across the codebase.
This guide is based primarily on the [Oracle of Secrets](https://github.com/scawful/Oracle-of-Secrets) codebase and is meant for the [Asar](https://github.com/RPGHacker/asar) assembler and derives influence from the [Asar 1.9 Manual](https://rpghacker.github.io/asar/asar_19/manual/).
Custom assembly code applied to the game should be included through the `yaze.asm` file found in `assets/asm`. This file can be applied to the ROM by the editor using the Asar library or included into a projects codebase for use with the Asar assembler.
## Table of Contents
- [File Structure](#file-structure)
- [Labels and Symbols](#labels-and-symbols)
- [Comments](#comments)
- [Directives](#directives)
- [Instructions](#instructions)
- [Macros](#macros)
- [Loops and Branching](#loops-and-branching)
- [Data Structures](#data-structures)
- [Code Organization](#code-organization)
- [Custom Code](#custom-code)
## File Structure
- **File Extension**: Use `.asm` as the file extension for 65816 assembly files.
- **Header Comments**: Include a header comment at the beginning of each file describing its purpose and the author.
Example:
```asm
; =========================================================
; Purpose: [Brief description of the file’s functionality]
; Author: [Your Name]
; =========================================================
```
- **Section Headers**: Use clear and consistent section headers to divide code into logical blocks. Each major section (e.g., sprite properties, main logic, subroutines) should start with a delineated header.
Example:
```asm
; =========================================================
; Minecart Sprite Properties
; =========================================================
```
- **Macro Definitions and Includes**: Place macros and include directives at the beginning of the file to keep them organized and easily accessible.
## Labels and Symbols
- **Naming Conventions**:
- **Global Labels**: Use descriptive names in `PascalCase` for global labels (e.g., `Sprite_Minecart_Main`).
- **Local Labels**: Prefix local labels with a dot (`.`) to indicate their limited scope (e.g., `.check_direction`).
- **Constants and Flags**: Use `ALL_CAPS_WITH_UNDERSCORES` for constants and flags (e.g., `!MINECART_SPEED`, `!HARVESTING_FLAG`).
- **Variables**: Use `CamelCase` for variable names to maintain readability (e.g., `LinkInCart`, `SpriteDirection`).
- **Alignment**: Align labels to the left margin for better readability. Indent instructions and comments to separate them from labels.
Example:
```asm
Sprite_Minecart_Main:
{
JSR HandleTileDirections
JSR HandleDynamicSwitchTileDirections
RTS
}
```
## Comments
- **Purpose**: Comments should explain why the code exists and what it is intended to do, especially for complex logic.
- **Placement**:
- Comments can be placed above the code block they describe for longer explanations.
- Inline comments can be used for single lines of code where the purpose might not be immediately clear.
- **Clarity**: Avoid stating the obvious. Focus on explaining the logic rather than restating the code.
Example:
```asm
LDA $22 : SEC : SBC $3F : STA $31 ; Adjust X position for camera movement
```
## Instructions
- **Single Line Instructions**: Combine multiple instructions on a single line using colons (`:`) where appropriate for related operations.
- **Separation**: Use line breaks to separate distinct sections of code logically, improving readability.
- **Optimization**: Always consider the most efficient instruction for the task at hand, especially in performance-critical sections.
Example:
```asm
LDA #$01 : STA !LinkInCart ; Set Link in cart flag
```
## Macros
- **Naming**: Use `PascalCase` for macro names, with the first letter of each word capitalized (e.g., `InitMovement`, `MoveCart`).
- **Parameters**: Clearly define and document parameters within macros to ensure they are used correctly.
- **Reuse**: Encourage the reuse of macros to avoid code duplication and simplify maintenance.
Example:
```asm
%macro HandlePlayerCamera
LDA $22 : SEC : SBC $3F : STA $31
LDA $20 : SEC : SBC $3E : STA $30
JSL Link_HandleMovingAnimation_FullLongEntry
JSL HandleIndoorCameraAndDoors
RTS
endmacro
```
## Loops and Branching
- **Branch Labels**: Use meaningful names for branch labels, prefixed with a dot (`.`) for local branches.
- **Optimization**: Minimize the number of instructions within loops and branches to improve performance.
Example:
```asm
.loop_start
LDA $00 : CMP #$10 : BEQ .end_loop
INC $00
BRA .loop_start
.end_loop
RTS
```
## Data Structures
- **Alignment**: Align data tables and structures clearly, and use comments to describe the purpose and layout of each.
- **Access**: Ensure that data structures are accessed consistently, with clear boundaries between read and write operations.
Example:
```asm
.DirectionTileLookup
{
db $02, $00, $04, $00 ; North
db $00, $00, $03, $01 ; East
db $00, $02, $00, $04 ; South
db $03, $01, $00, $00 ; West
}
```
- **Structs**: Use structs to group related data together, improving readability and maintainability.
Example:
```asm
struct AncillaAdd_HookshotData $099AF8
.speed_y: skip 4
.speed_x: skip 4
.offset_y: skip 8
.offset_x: skip 8
endstruct
AncillaAdd_Hookshot:
; $099AF0
.speed_y
db -64 ; up
db 64 ; down
db 0 ; left
db 0 ; right
; $099AFC
.speed_x
db 0 ; up
db 0 ; down
db -64 ; left
db 64 ; right
; $099B00
.offset_y
dw 4 ; up
dw 20 ; down
dw 8 ; left
dw 8 ; right
; $099B08
.offset_x
dw 0 ; up
dw 0 ; down
dw -4 ; left
dw 11 ; right
```
## Code Organization
- **Logical Grouping**: Organize code into logical sections, with related routines and macros grouped together.
- **Separation of Concerns**: Ensure that each section of code is responsible for a specific task or set of related tasks, avoiding tightly coupled code.
- **Modularity**: Write code in a modular way, making it easier to reuse and maintain.
Example:
```asm
; =========================================================
; Minecart Sprite Logic
; =========================================================
Sprite_Minecart_Main:
{
PHX
JSR HandleMinecartMovement
PLX
REP #$20
LDA !SpriteDirection : STA $00
SEP #$20
RTS
}
```
## Custom Code
- **Integration**: Include custom assembly code in the `yaze.asm` file to ensure it is applied correctly to the ROM. The module should include a define and conditional statement to allow users to disable the module if needed.
Example:
```asm
!YAZE_CUSTOM_MOSAIC = 1
if !YAZE_CUSTOM_MOSAIC != 0
incsrc "mosaic_change.asm"
endif
```

171
docs/public/developer/canvas-system.md Normal file
View File

@@ -0,0 +1,171 @@
# Canvas System Guide
This guide provides a comprehensive overview of the `yaze` canvas system, its architecture, and best practices for integration. It reflects the state of the system after the October 2025 refactoring.
## 1. Architecture
The canvas system was refactored from a monolithic class into a modular, component-based architecture. The main `gui::Canvas` class now acts as a façade, coordinating a set of single-responsibility components and free functions.
### Core Principles
- **Modular Components**: Logic is broken down into smaller, testable units (e.g., state, rendering, interaction, menus).
- **Data-Oriented Design**: Plain-old-data (POD) structs like `CanvasState` and `CanvasConfig` hold state, which is operated on by free functions.
- **Backward Compatibility**: The refactor was designed to be 100% backward compatible. Legacy APIs still function, but new patterns are encouraged.
- **Editor Agnostic**: The core canvas system has no knowledge of `zelda3` specifics, making it reusable for any editor.
### Code Organization
The majority of the canvas code resides in `src/app/gui/canvas/`.
```
src/app/gui/canvas/
├── canvas.h/cc # Main Canvas class (facade)
├── canvas_state.h # POD state structs
├── canvas_config.h # Unified configuration struct
├── canvas_geometry.h/cc # Geometry calculation helpers
├── canvas_rendering.h/cc # Rendering free functions
├── canvas_events.h # Interaction event structs
├── canvas_interaction.h/cc # Interaction event handlers
├── canvas_menu.h/cc # Declarative menu structures
├── canvas_menu_builder.h/cc # Fluent API for building menus
├── canvas_popup.h/cc # PopupRegistry for persistent popups
└── canvas_utils.h/cc # General utility functions
```
## 2. Core Concepts
### Configuration (`CanvasConfig`)
- A single, unified `gui::CanvasConfig` struct (defined in `canvas_config.h`) holds all configuration for a canvas instance.
- This includes display settings (grid, labels), sizing, scaling, and usage mode.
- This replaces duplicated config structs from previous versions.
### State (`CanvasState`)
- A POD struct (`canvas_state.h`) that holds the dynamic state of the canvas, including geometry, zoom, and scroll.
- Editors can inspect this state for custom rendering and logic.
### Coordinate Systems
The canvas operates with three distinct coordinate spaces. Using the correct one is critical to avoid bugs.
1. **Screen Space**: Absolute pixel coordinates on the monitor (from `ImGui::GetIO().MousePos`). **Never use this for canvas logic.**
2. **Canvas/World Space**: Coordinates relative to the canvas's content, accounting for scrolling and panning. Use `Canvas::hover_mouse_pos()` to get this. This is the correct space for entity positioning and high-level calculations.
3. **Tile/Grid Space**: Coordinates in tile units. Use `Canvas::CanvasToTile()` to convert from world space.
A critical fix was made to ensure `Canvas::hover_mouse_pos()` is updated continuously whenever the canvas is hovered, decoupling it from specific actions like painting.
## 3. Interaction System
The canvas supports several interaction modes, managed via the `CanvasUsage` enum.
### Interaction Modes
- `kTilePainting`: For painting tiles onto a tilemap.
- `kTileSelection`: For selecting one or more tiles.
- `kRectangleSelection`: For drag-selecting a rectangular area.
- `kEntityManipulation`: For moving and interacting with entities.
- `kPaletteEditing`: For palette-related work.
- `kDiagnostics`: For performance and debug overlays.
Set the mode using `canvas.SetUsageMode(gui::CanvasUsage::kTilePainting)`. This ensures the context menu and interaction handlers behave correctly.
### Event-Driven Model
Interaction logic is moving towards an event-driven model. Instead of inspecting canvas state directly, editors should handle events returned by interaction functions.
**Example**:
```cpp
RectSelectionEvent event = HandleRectangleSelection(geometry, ...);
if (event.is_complete) {
// Process the selection event
}
```
## 4. Context Menu & Popups
The context menu system is now unified, data-driven, and supports persistent popups.
### Key Features
- **Unified Item Definition**: All menu items use the `gui::CanvasMenuItem` struct.
- **Priority-Based Ordering**: Menu sections are automatically sorted based on the `MenuSectionPriority` enum, ensuring a consistent layout:
1. `kEditorSpecific` (highest priority)
2. `kBitmapOperations`
3. `kCanvasProperties`
4. `kDebug` (lowest priority)
- **Automatic Popup Persistence**: Popups defined declaratively will remain open until explicitly closed by the user (ESC or close button), rather than closing on any click outside.
- **Fluent Builder API**: The `gui::CanvasMenuBuilder` provides a clean, chainable API for constructing complex menus.
### API Patterns
**Add a Simple Menu Item**:
```cpp
canvas.AddContextMenuItem(
gui::CanvasMenuItem("Label", ICON_MD_ICON, []() { /* Action */ })
);
```
**Add a Declarative Popup Item**:
This pattern automatically handles popup registration and persistence.
```cpp
auto item = gui::CanvasMenuItem::WithPopup(
"Properties",
"props_popup_id",
[]() {
// Render popup content here
ImGui::Text("My Properties");
}
);
canvas.AddContextMenuItem(item);
```
**Build a Complex Menu with the Builder**:
```cpp
gui::CanvasMenuBuilder builder;
canvas.editor_menu() = builder
.BeginSection("Editor Actions", gui::MenuSectionPriority::kEditorSpecific)
.AddItem("Cut", ICON_MD_CUT, []() { Cut(); })
.AddPopupItem("Settings", "settings_popup", []() { RenderSettings(); })
.EndSection()
.Build();
```
## 5. Entity System
A generic, Zelda-agnostic entity system allows editors to manage on-canvas objects.
- **Flat Functions**: Entity creation logic is handled by pure functions in `src/app/editor/overworld/operations/entity_operations.h`, such as `InsertEntrance`, `InsertSprite`, etc. These functions are designed for ZScream feature parity.
- **Delegation Pattern**: The `OverworldEditor` delegates to the `MapPropertiesSystem`, which in turn calls these flat functions to modify the ROM state.
- **Mode-Aware Menu**: The "Insert Entity" context submenu is only available when the canvas is in `kEntityManipulation` mode.
**Usage Flow**:
1. Set canvas mode to `kEntityManipulation`.
2. Right-click on the canvas to open the context menu.
3. Select "Insert Entity" and choose the entity type.
4. The appropriate callback is fired, which calls the corresponding `Insert...` function.
5. A popup appears to configure the new entity's properties.
## 6. Integration Guide for Editors
1. **Construct `Canvas`**: Instantiate `gui::Canvas`, providing a unique ID.
2. **Configure**: Set the desired `CanvasUsage` mode via `canvas.SetUsageMode()`. Configure available modes and other options in the `CanvasConfig` struct.
3. **Register Callbacks**: If using interaction modes like tile painting, register callbacks for events like `finish_paint`.
4. **Render Loop**:
- Call `canvas.Begin(size)`.
- Draw your editor-specific content (bitmaps, entities, overlays).
- Call `canvas.End()`. This handles rendering the grid, overlays, and the context menu.
5. **Provide Custom Menus**: Use `canvas.AddContextMenuItem()` or the `CanvasMenuBuilder` to add editor-specific actions to the context menu. Assign the `kEditorSpecific` priority to ensure they appear at the top.
6. **Handle State**: Respond to user interactions by inspecting the `CanvasState` or handling events returned from interaction helpers.
## 7. Debugging
If you encounter issues with the canvas, check the following:
- **Context Menu Doesn't Appear**:
- Is `config.enable_context_menu` true?
- Is the mouse button right-click?
- Is the canvas focused and not being dragged?
- **Popup Doesn't Persist**:
- Are you using the `CanvasMenuItem::WithPopup` pattern?
- Is `canvas.End()` being called every frame to allow the `PopupRegistry` to render?
- **Incorrect Coordinates**:
- Are you using `canvas.hover_mouse_pos()` for world coordinates instead of `ImGui::GetIO().MousePos`?
- Verify that you are correctly converting between world space and tile space.
- **Menu Items in Wrong Order**:
- Have you set the correct `MenuSectionPriority` for your custom menu sections?
## 8. Automation API
The `CanvasAutomationAPI` provides hooks for testing and automation. It allows for programmatic control of tile operations (`SetTileAt`, `SelectRect`), view controls (`ScrollToTile`, `SetZoom`), and entity manipulation. This API is exposed via the `z3ed` CLI and a gRPC service.

151
docs/public/developer/debug-flags.md Normal file
View File

@@ -0,0 +1,151 @@
# YAZE Startup Debugging Flags
This guide explains how to use command-line flags to quickly open specific editors and cards during development for faster debugging workflows.
## Basic Usage
```bash
./yaze [flags]
```
## Available Flags
### `--rom_file`
Load a specific ROM file on startup.
```bash
./yaze --rom_file=/path/to/zelda3.sfc
```
### `--debug`
Enable debug logging with verbose output.
```bash
./yaze --debug --log_file=yaze_debug.log
```
### `--editor`
Open a specific editor on startup. This saves time by skipping manual navigation through the UI.
**Available editors:**
- `Assembly` - Assembly code editor
- `Dungeon` - Dungeon/underworld editor
- `Graphics` - Graphics and tile editor
- `Music` - Music and sound editor
- `Overworld` - Overworld map editor
- `Palette` - Palette editor
- `Screen` - Screen editor
- `Sprite` - Sprite editor
- `Message` - Message/text editor
- `Hex` - Hex/memory editor
- `Agent` - AI agent interface
- `Settings` - Settings editor
**Example:**
```bash
./yaze --rom_file=zelda3.sfc --editor=Dungeon
```
### `--cards`
Open specific cards/panels within an editor. Most useful with the Dungeon editor.
**Dungeon Editor Cards:**
- `Rooms List` - Shows the list of all dungeon rooms
- `Room Matrix` - Shows the dungeon room layout matrix
- `Entrances List` - Shows dungeon entrance configurations
- `Room Graphics` - Shows room graphics settings
- `Object Editor` - Shows the object placement editor
- `Palette Editor` - Shows the palette editor
- `Room N` - Opens a specific room by ID (e.g., `Room 0`, `Room 105`)
**Example:**
```bash
./yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Rooms List,Room 0"
```
## Common Debugging Scenarios
### 1. Quick Dungeon Room Testing
Open a specific dungeon room for testing:
```bash
./yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Room 0,Room Graphics"
```
### 2. Multiple Room Comparison
Compare multiple rooms side-by-side:
```bash
./yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Room 0,Room 1,Room 105"
```
### 3. Full Dungeon Editor Workspace
Open all dungeon editor tools:
```bash
./yaze --rom_file=zelda3.sfc --editor=Dungeon \
--cards="Rooms List,Room Matrix,Room Graphics,Object Editor,Palette Editor"
```
### 4. Debug Mode with Logging
Enable full debug output while working:
```bash
./yaze --rom_file=zelda3.sfc --debug --log_file=debug.log \
--editor=Dungeon --cards="Room 0"
```
### 5. Quick Overworld Editing
Jump straight to overworld editing:
```bash
./yaze --rom_file=zelda3.sfc --editor=Overworld
```
## gRPC Test Harness (Developer Feature)
If compiled with `YAZE_WITH_GRPC=ON`, you can enable automated GUI testing:
```bash
./yaze --enable_test_harness --test_harness_port=50051
```
This allows remote control via gRPC for automated testing and AI agent interaction.
## Combining Flags
All flags can be combined for powerful debugging setups:
```bash
# Full debugging setup for room 105
./yaze \
--rom_file=/path/to/zelda3.sfc \
--debug \
--log_file=room_105_debug.log \
--editor=Dungeon \
--cards="Room 105,Room Graphics,Palette Editor,Object Editor"
```
## Notes
- Card names are case-sensitive and must match exactly
- Use quotes around comma-separated card lists
- Invalid editor or card names will be logged as warnings but won't crash the application
- The `--cards` flag is currently only implemented for the Dungeon editor
- Room IDs range from 0-319 in the vanilla game
## Troubleshooting
**Editor doesn't open:**
- Check spelling (case-sensitive)
- Verify ROM loaded successfully
- Check log output with `--debug`
**Cards don't appear:**
- Ensure editor is set (e.g., `--editor=Dungeon`)
- Check card name spelling
- Some cards require a loaded ROM
**Want to add more card support?**
See `EditorManager::OpenEditorAndCardsFromFlags()` in `src/app/editor/editor_manager.cc`

224
docs/public/developer/debugging-guide.md Normal file
View File

@@ -0,0 +1,224 @@
# E5 - Debugging and Testing Guide
**Last Updated**: October 9, 2025
**Status**: Active
This document provides a comprehensive guide to debugging and testing the `yaze` application. It covers strategies for developers and provides the necessary information for AI agents to interact with, test, and validate the application.
---
## 1. Standardized Logging for Print Debugging
For all print-based debugging, `yaze` uses a structured logging system defined in `util/log.h`. This is the **only** approved method for logging; direct use of `printf` or `std::cout` should be avoided and replaced with the appropriate `LOG_*` macro.
### Log Levels and Usage
- `LOG_DEBUG(category, "message", ...)`: For verbose, development-only information.
- `LOG_INFO(category, "message", ...)`: For general, informational messages.
- `LOG_WARN(category, "message", ...)`: For potential issues that don't break functionality.
- `LOG_ERROR(category, "message", ...)`: For errors that cause a specific operation to fail.
### Log Categories
Categories allow you to filter logs to focus on a specific subsystem. Common categories include:
- `"Main"`
- `"TestManager"`
- `"EditorManager"`
- `"APU"`, `"CPU"`, `"SNES"` (for the emulator)
### Enabling and Configuring Logs via CLI
You can control logging behavior using command-line flags when launching `yaze` or `yaze_test`.
- **Enable Verbose Debug Logging**:
```bash
./build/bin/yaze --debug
```
- **Log to a File**:
```bash
./build/bin/yaze --log_file=yaze_debug.log
```
- **Filter by Category**:
```bash
# Only show logs from the APU and CPU emulator components
./build/bin/yaze_emu --emu_debug_apu=true --emu_debug_cpu=true
```
**Best Practice**: When debugging a specific component, add detailed `LOG_DEBUG` statements with a unique category. Then, run `yaze` with the appropriate flags to isolate the output.
---
## 2. Command-Line Workflows for Testing
The `yaze` ecosystem provides several executables and flags to streamline testing and debugging.
### Launching the GUI for Specific Tasks
- **Load a ROM on Startup**: To immediately test a specific ROM, use the `--rom_file` flag. This bypasses the welcome screen.
```bash
./build/bin/yaze --rom_file /path/to/your/zelda3.sfc
```
- **Enable the GUI Test Harness**: To allow the `z3ed` CLI to automate the GUI, you must start `yaze` with the gRPC server enabled.
```bash
./build/bin/yaze --rom_file zelda3.sfc --enable_test_harness
```
- **Open a Specific Editor and Cards**: To quickly test a specific editor and its components, use the `--editor` and `--cards` flags. This is especially useful for debugging complex UIs like the Dungeon Editor.
```bash
# Open the Dungeon Editor with the Room Matrix and two specific room cards
./build/bin/yaze --rom_file zelda3.sfc --editor=Dungeon --cards="Room Matrix,Room 0,Room 105"
# Available editors: Assembly, Dungeon, Graphics, Music, Overworld, Palette,
# Screen, Sprite, Message, Hex, Agent, Settings
# Dungeon editor cards: Rooms List, Room Matrix, Entrances List, Room Graphics,
# Object Editor, Palette Editor, Room N (where N is room ID)
```
**Quick Examples**:
```bash
# Fast dungeon room testing
./build/bin/yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Room 0"
# Compare multiple rooms side-by-side
./build/bin/yaze --rom_file=zelda3.sfc --editor=Dungeon --cards="Room 0,Room 1,Room 105"
# Full dungeon workspace with all tools
./build/bin/yaze --rom_file=zelda3.sfc --editor=Dungeon \
--cards="Rooms List,Room Matrix,Object Editor,Palette Editor"
# Jump straight to overworld editing
./build/bin/yaze --rom_file=zelda3.sfc --editor=Overworld
```
For a complete reference, see [docs/debugging-startup-flags.md](debugging-startup-flags.md).
### Running Automated C++ Tests
The `yaze_test` executable is used to run the project's suite of unit, integration, and E2E tests.
- **Run All Tests**:
```bash
./build_ai/bin/yaze_test
```
- **Run Specific Categories**:
```bash
# Run only fast, dependency-free unit tests
./build_ai/bin/yaze_test --unit
# Run tests that require a ROM file
./build_ai/bin/yaze_test --rom-dependent --rom-path /path/to/zelda3.sfc
```
- **Run GUI-based E2E Tests**:
```bash
# Run E2E tests and watch the GUI interactions
./build_ai/bin/yaze_test --e2e --show-gui
```
### Inspecting ROMs with `z3ed`
The `z3ed` CLI is a powerful tool for inspecting ROM data without launching the full GUI. This is ideal for quick checks and scripting.
- **Get ROM Information**:
```bash
z3ed rom info --rom zelda3.sfc
```
- **Inspect Dungeon Sprites**:
```bash
z3ed dungeon list-sprites --rom zelda3.sfc --dungeon 2
```
---
## 3. GUI Automation for AI Agents
The primary way for an AI agent to test its changes and interact with `yaze` is through the GUI automation framework. This system consists of the `yaze` gRPC server (Test Harness) and the `z3ed` CLI client.
### Architecture Overview
1. **`yaze` (Server)**: When launched with `--enable_test_harness`, it starts a gRPC server that exposes the UI for automation.
2. **`z3ed` (Client)**: The `z3ed agent test` commands connect to the gRPC server to send commands and receive information.
3. **AI Agent**: The agent generates `z3ed` commands to drive the UI and verify its actions.
### Step-by-Step Workflow for AI
#### Step 1: Launch `yaze` with the Test Harness
The AI must first ensure the `yaze` GUI is running and ready for automation.
```bash
./build/bin/yaze --rom_file zelda3.sfc --enable_test_harness --test_harness_port 50051
```
#### Step 2: Discover UI Elements
Before interacting with the UI, the agent needs to know the stable IDs of the widgets.
```bash
# Discover all widgets in the Dungeon editor window
z3ed agent test discover --window "Dungeon" --grpc localhost:50051
```
This will return a list of widget IDs (e.g., `Dungeon/Canvas/Map`) that can be used in scripts.
**Tip**: You can also launch `yaze` with the `--editor` flag to automatically open a specific editor:
```bash
./build/bin/yaze --rom_file zelda3.sfc --enable_test_harness --editor=Dungeon --cards="Room 0"
```
#### Step 3: Record or Write a Test Script
An agent can either generate a test script from scratch or use a pre-recorded one.
- **Recording a human interaction**:
```bash
z3ed agent test record --suite my_test.jsonl
```
- **A generated script might look like this**:
```json
// my_test.jsonl
{"action": "click", "target": "Dungeon/Toolbar/Open Room"}
{"action": "wait", "duration_ms": 500}
{"action": "type", "target": "Room Selector/Filter", "text": "Room 105"}
{"action": "click", "target": "Room Selector/List/Room 105"}
{"action": "assert_visible", "target": "Room Card 105"}
```
- **Or use startup flags to prepare the environment**:
```bash
# Start yaze with the room already open
./build/bin/yaze --rom_file zelda3.sfc --enable_test_harness \
--editor=Dungeon --cards="Room 105"
# Then your test script just needs to validate the state
{"action": "assert_visible", "target": "Room Card 105"}
{"action": "assert_visible", "target": "Dungeon/Canvas"}
```
#### Step 4: Replay the Test and Verify
The agent executes the script to perform the actions and validate the outcome.
```bash
z3ed agent test replay my_test.jsonl --watch
```
The `--watch` flag streams results back to the CLI in real-time. The agent can parse this output to confirm its actions were successful.
---
## 4. Advanced Debugging Tools
For more complex issues, especially within the emulator, `yaze` provides several advanced debugging windows. These are covered in detail in the [Emulator Development Guide](emulator-development-guide.md).
- **Disassembly Viewer**: A live, interactive view of the 65816 and SPC700 CPU execution.
- **Breakpoint Manager**: Set breakpoints on code execution, memory reads, or memory writes.
- **Memory Viewer**: Inspect WRAM, SRAM, VRAM, and ROM.
- **APU Inspector**: A dedicated debugger for the audio subsystem.
- **Event Viewer**: A timeline of all hardware events (NMI, IRQ, DMA).
These tools are accessible from the **Debug** menu in the main application.

106
docs/public/developer/dependency-architecture.md Normal file
View File

@@ -0,0 +1,106 @@
# Dependency & Build Overview
_Last reviewed: November 2025. All information in this document is derived from the current
`src/CMakeLists.txt` tree and shipped presets._
This guide explains how the major YAZE libraries fit together, which build switches control
them, and when a code change actually forces a full rebuild. It is intentionally concise so you
can treat it as a quick reference while editing.
## Build Switches & Presets
| CMake option | Default | Effect |
| --- | --- | --- |
| `YAZE_BUILD_APP` | `ON` | Build the main GUI editor (`yaze`). Disable when you only need CLI/tests. |
| `YAZE_BUILD_Z3ED` | `ON` | Build the `z3ed` automation tool and supporting agent libraries. |
| `YAZE_BUILD_EMU` | `OFF` | Build the standalone emulator binary. Always enabled inside the GUI build. |
| `YAZE_BUILD_TESTS` | `ON` in `*-dbg` presets | Compiles test helpers plus `yaze_test`. Required for GUI test dashboard. |
| `YAZE_ENABLE_GRPC` | `OFF` | Pulls in gRPC/protobuf for automation and remote control features. |
| `YAZE_MINIMAL_BUILD` | `OFF` | Skips optional editors/assets. Useful for CI smoke builds. |
| `YAZE_BUILD_LIB` | `OFF` | Produces the `yaze_core` INTERFACE target used by external tooling. |
| `YAZE_BUILD_AGENT_UI` | `ON` when `YAZE_BUILD_GUI` is `ON` | Compiles ImGui chat widgets. Disable for lighter GUI builds. |
| `YAZE_ENABLE_REMOTE_AUTOMATION` | `OFF` in `win-*` core presets | Builds gRPC servers/clients plus proto generation. |
| `YAZE_ENABLE_AI_RUNTIME` | `OFF` in `win-*` core presets | Enables Gemini/Ollama transports, proposal planning, and advanced routing code. |
| `YAZE_ENABLE_AGENT_CLI` | `ON` when `YAZE_BUILD_CLI` is `ON` | Compiles the conversational agent stack used by `z3ed`. |
Use the canned presets from `CMakePresets.json` so these options stay consistent across
platforms: `mac-dbg`, `mac-ai`, `lin-dbg`, `win-dbg`, etc. The `*-ai` presets enable both
`YAZE_BUILD_Z3ED` and `YAZE_ENABLE_GRPC` so the CLI and agent features match what ships.
## Library Layers
### 1. Foundation (`src/util`, `incl/`)
- **`yaze_common`**: cross-platform macros, generated headers, and lightweight helpers shared by
every other target.
- **`yaze_util`**: logging, file I/O, BPS patch helpers, and the legacy flag system. Only depends
on `yaze_common` plus optional gRPC/Abseil symbols.
- **Third-party**: SDL2, ImGui, Abseil, yaml-cpp, FTXUI, Asar. They are configured under
`cmake/dependencies/*.cmake` and linked where needed.
Touching headers in this layer effectively invalidates most of the build. Keep common utilities
stable and prefer editor-specific helpers instead of bloating `yaze_util`.
### 2. Graphics & UI (`src/app/gfx`, `src/app/gui`)
- **`yaze_gfx`**: bitmap containers, palette math, deferred texture arena, canvas abstractions.
Depends on SDL2 + `yaze_util`.
- **`yaze_gui`**: shared ImGui widgets, docking layout utilities, and theme plumbing. Depends on
ImGui + `yaze_gfx`.
Changes here rebuild all editors but do not touch the lower-level Zelda 3 logic. Use the graphics
layer for rendering and asset streaming primitives; keep domain logic in the Zelda 3 library.
### 3. Game Domain (`src/zelda3`, `src/app/editor`)
- **`yaze_zelda3`**: map/room/sprite models, parsers, and ROM serialization. It links
`yaze_gfx`, `yaze_util`, and Abseil.
- **`yaze_editor`**: ImGui editors (overworld, dungeon, palette, etc.). Depends on
`yaze_gui`, `yaze_zelda3`, `yaze_gfx`, and optional agent/test hooks.
- **`yaze_emulator`**: CPU, PPU, and APU subsystems plus the debugger UIs (`src/app/emu`). The GUI
app links this to surface emulator panels.
Touching Zelda 3 headers triggers rebuilds of the editor and CLI but leaves renderer-only changes
alone. Touching editor UI code does **not** require rebuilding `yaze_emulator`.
### 4. Tooling & Export Targets
- **`yaze_agent`** (`src/cli/agent`): shared logic behind the CLI and AI workflows. Built whenever
`YAZE_ENABLE_AGENT_CLI` is enabled (automatically true when `YAZE_BUILD_Z3ED=ON`). When both the CLI and the agent UI are disabled, CMake now emits a lightweight stub target so GUI-only builds don't drag in unnecessary dependencies.
- **`z3ed` binary** (`src/cli/z3ed.cmake`): links `yaze_agent`, `yaze_zelda3`, `yaze_gfx`, and
Abseil/FTXUI.
- **`yaze_core_lib`** (`src/core`): static library that exposes project management helpers and the
Asar integration. When `YAZE_BUILD_LIB=ON` it can be consumed by external tools.
- **`yaze_test_support`** (`src/app/test`): harness for the in-editor dashboard and `yaze_test`.
- **`yaze_grpc_support`**: server-only aggregation of gRPC/protobuf code, gated by `YAZE_ENABLE_REMOTE_AUTOMATION`. CLI clients (`cli/service/gui/**`, `cli/service/planning/**`) now live solely in `yaze_agent` so GUI builds can opt out entirely.
### 5. Final Binaries
- **`yaze`**: GUI editor. Links every layer plus `yaze_test_support` when tests are enabled.
- **`yaze_test`**: GoogleTest runner (unit, integration, e2e). Built from `test/CMakeLists.txt`.
- **`z3ed`**: CLI + TUI automation tool. Built when `YAZE_BUILD_Z3ED=ON`.
- **`yaze_emu`**: optional standalone emulator for fast boot regression tests.
## Rebuild Cheatsheet
| Change | Targets Affected |
| --- | --- |
| `src/util/*.h` or `incl/yaze/*.h` | Everything (foundation dependency) |
| `src/app/gfx/**` | `yaze_gfx`, `yaze_gui`, editors, CLI. Emulator core unaffected. |
| `src/zelda3/**` | All editors, CLI, tests. Rebuild does **not** touch renderer-only changes. |
| `src/app/editor/**` | GUI editor + CLI (shared panels). Emulator/test support untouched. |
| `src/app/emu/**` | Emulator panels + GUI app. CLI and Zelda 3 libraries unaffected. |
| `src/cli/**` | `yaze_agent`, `z3ed`. No impact on GUI/editor builds. |
| `src/app/test/**` | `yaze_test_support`, `yaze_test`, GUI app (only when tests enabled). |
Use this table when deciding whether to invalidate remote build caches or to schedule longer CI
runs. Whenever possible, localize changes to the upper layers to avoid rebuilding the entire
stack.
## Tips for Faster Iteration
1. **Leverage presets** – `cmake --build --preset mac-ai --target yaze` automatically enables
precompiled headers and shared dependency trees.
2. **Split work by layer** – renderer bugs usually live in `yaze_gfx`; leave Zelda 3 logic alone
unless you need ROM serialization tweaks.
3. **Turn off unused targets** – set `YAZE_BUILD_Z3ED=OFF` when working purely on GUI features to
shave a few hundred object files.
4. **Test without ROMs** – `docs/public/developer/testing-without-roms.md` documents the mock ROM
harness so you do not need to rebuild assets between iterations.
5. **See also** – for deep dives into refactors or planned changes, read the internal blueprints in
`docs/internal/blueprints/` instead of bloating the public docs.

1334
docs/public/developer/emulator-development-guide.md Normal file
View File

File diff suppressed because it is too large Load Diff

566
docs/public/developer/git-workflow.md Normal file
View File

@@ -0,0 +1,566 @@
# Git Workflow and Branching Strategy
**Last Updated:** October 10, 2025
**Status:** Active
**Current Phase:** Pre-1.0 (Relaxed Rules)
## Warning: Pre-1.0 Workflow (Current)
**TLDR for now:** Since yaze is pre-1.0 and actively evolving, we use a **simplified workflow**:
- **Documentation changes**: Commit directly to `master` or `develop`
- **Small bug fixes**: Can go direct to `develop`, no PR required
- **Solo work**: Push directly when you're the only one working
- Warning: **Breaking changes**: Use feature branches and document in changelog
- Warning: **Major refactors**: Use feature branches for safety (can always revert)
- **Always keep local backups**: copy ROMs/assets before editing; never risk the only copy.
- **Before rebasing/rewriting history**, stash or copy work elsewhere to prevent accidental loss.
**Why relaxed?**
- Small team / solo development
- Pre-1.0 means breaking changes are expected
- Documentation needs to be public quickly
- Overhead of PRs/reviews slows down experimentation
**When to transition to strict workflow:**
- Multiple active contributors
- Stable API (post-1.0)
- Large user base depending on stability
- Critical bugs need rapid hotfixes
---
## Pre-1.0 Release Strategy: Best Effort Releases
For all versions prior to 1.0.0, yaze follows a **"best effort"** release strategy. This prioritizes getting working builds to users quickly, even if not all platforms build successfully on the first try.
### Core Principles
1. **Release Can Proceed with Failed Platforms**: The `release` CI/CD workflow will create a GitHub Release even if one or more platform-specific build jobs (e.g., Windows, Linux, macOS) fail.
2. **Missing Platforms Can Be Added Later**: A failed job for a specific platform can be re-run from the GitHub Actions UI. If it succeeds, the binary artifact will be **automatically added to the existing GitHub Release**.
3. **Transparency is Key**: The release notes will automatically generate a "Platform Availability" report, clearly indicating which platforms succeeded () and which failed (❌), so users know the current status.
### How It Works in Practice
- The `build-and-package` jobs in the `release.yml` workflow have `continue-on-error: true`.
- The final `create-github-release` job has `if: always()` and uses `softprops/action-gh-release@v2`, which intelligently updates an existing release if the tag already exists.
- If a platform build fails, a developer can investigate the issue and simply re-run the failed job. Upon success, the new binary is uploaded and attached to the release that was already created.
This strategy provides flexibility and avoids blocking a release for all users due to a transient issue on a single platform. Once the project reaches v1.0.0, this policy will be retired in favor of a stricter approach where all platforms must pass for a release to proceed.
---
## 📚 Full Workflow Reference (Future/Formal)
The sections below document the **formal Git Flow model** that yaze will adopt post-1.0 or when the team grows. For now, treat this as aspirational best practices.
## Branch Structure
### Main Branches
#### `master`
- **Purpose**: Production-ready release branch
- **Protection**: Protected, requires PR approval
- **Versioning**: Tagged with semantic versions (e.g., `v0.3.2`, `v0.4.0`)
- **Updates**: Only via approved PRs from `develop` or hotfix branches
#### `develop`
- **Purpose**: Main development branch, integration point for all features
- **Protection**: Protected, requires PR approval
- **State**: Should always build and pass tests
- **Updates**: Merges from feature branches, releases merge back after tagging
### Supporting Branches
#### Feature Branches
**Naming Convention:** `feature/<short-description>`
**Examples:**
- `feature/overworld-editor-improvements`
- `feature/dungeon-room-painter`
- `feature/add-sprite-animations`
**Rules:**
- Branch from: `develop`
- Merge back to: `develop`
- Lifetime: Delete after merge
- Naming: Use kebab-case, be descriptive but concise
**Workflow:**
```bash
# Create feature branch
git checkout develop
git pull origin develop
git checkout -b feature/my-feature
# Work on feature
git add .
git commit -m "feat: add new feature"
# Keep up to date with develop
git fetch origin
git rebase origin/develop
# Push and create PR
git push -u origin feature/my-feature
```
#### Bugfix Branches
**Naming Convention:** `bugfix/<issue-number>-<short-description>`
**Examples:**
- `bugfix/234-canvas-scroll-regression`
- `bugfix/fix-dungeon-crash`
**Rules:**
- Branch from: `develop`
- Merge back to: `develop`
- Lifetime: Delete after merge
- Reference issue number when applicable
#### Hotfix Branches
**Naming Convention:** `hotfix/<version>-<description>`
**Examples:**
- `hotfix/v0.3.3-memory-leak`
- `hotfix/v0.3.2-crash-on-startup`
**Rules:**
- Branch from: `master`
- Merge to: BOTH `master` AND `develop`
- Creates new patch version
- Used for critical production bugs only
**Workflow:**
```bash
# Create hotfix from master
git checkout master
git pull origin master
git checkout -b hotfix/v0.3.3-critical-fix
# Fix the issue
git add .
git commit -m "fix: critical production bug"
# Merge to master
git checkout master
git merge --no-ff hotfix/v0.3.3-critical-fix
git tag -a v0.3.3 -m "Hotfix: critical bug"
git push origin master --tags
# Merge to develop
git checkout develop
git merge --no-ff hotfix/v0.3.3-critical-fix
git push origin develop
# Delete hotfix branch
git branch -d hotfix/v0.3.3-critical-fix
```
#### Release Branches
**Naming Convention:** `release/<version>`
**Examples:**
- `release/v0.4.0`
- `release/v0.3.2`
**Rules:**
- Branch from: `develop`
- Merge to: `master` AND `develop`
- Used for release preparation (docs, version bumps, final testing)
- Only bugfixes allowed, no new features
**Workflow:**
```bash
# Create release branch
git checkout develop
git pull origin develop
git checkout -b release/v0.4.0
# Prepare release (update version, docs, changelog)
# ... make changes ...
git commit -m "chore: prepare v0.4.0 release"
# Merge to master and tag
git checkout master
git merge --no-ff release/v0.4.0
git tag -a v0.4.0 -m "Release v0.4.0"
git push origin master --tags
# Merge back to develop
git checkout develop
git merge --no-ff release/v0.4.0
git push origin develop
# Delete release branch
git branch -d release/v0.4.0
```
#### Experimental Branches
**Naming Convention:** `experiment/<description>`
**Examples:**
- `experiment/vulkan-renderer`
- `experiment/wasm-build`
## Git Safety Crash Course
- Run `git status` often and avoid staging ROMs or build artifacts; add ignore rules when necessary.
- Never force-push shared branches (`develop`, `master`). PRs and feature branches are safer places
for rewrites.
- Keep backups of any tools that mutate large files (scripts, automation) so you can revert quickly.
- Before deleting branches that touched ROMs/assets, confirm those files were merged and backed up.
**Rules:**
- Branch from: `develop` or `master`
- May never merge (prototypes, research)
- Document findings in docs/experiments/
- Delete when concluded or merge insights into features
## Commit Message Conventions
Follow **Conventional Commits** specification:
### Format
```
<type>(<scope>): <subject>
<body>
<footer>
```
### Types
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation only
- `style`: Code style (formatting, semicolons, etc.)
- `refactor`: Code refactoring
- `perf`: Performance improvements
- `test`: Adding or updating tests
- `build`: Build system changes (CMake, dependencies)
- `ci`: CI/CD configuration changes
- `chore`: Maintenance tasks
### Scopes (optional)
- `overworld`: Overworld editor
- `dungeon`: Dungeon editor
- `graphics`: Graphics editor
- `emulator`: Emulator core
- `canvas`: Canvas system
- `gui`: GUI/ImGui components
### Examples
```bash
# Good commit messages
feat(overworld): add tile16 quick-select palette
fix(canvas): resolve scroll regression after refactoring
docs: update build instructions for SDL3
refactor(emulator): extract APU timing to cycle-accurate model
perf(dungeon): optimize room rendering with batched draw calls
# With body and footer
feat(overworld): add multi-tile selection tool
Allows users to select and copy/paste rectangular regions
of tiles in the overworld editor. Supports undo/redo.
Closes #123
```
## Pull Request Guidelines
### PR Title
Follow commit message convention:
```
feat(overworld): add new feature
fix(dungeon): resolve crash
```
### PR Description Template
```markdown
## Description
Brief description of changes
## Type of Change
- [ ] Bug fix (non-breaking change)
- [ ] New feature (non-breaking change)
- [ ] Breaking change (fix or feature that breaks existing functionality)
- [ ] Documentation update
## Testing
- [ ] All tests pass
- [ ] Added new tests for new features
- [ ] Manual testing completed
## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review completed
- [ ] Documentation updated
- [ ] No new warnings
- [ ] Dependent changes merged
```
### PR Review Process
1. **Author**: Create PR, fill out template, request reviewers
2. **CI**: Automatic build and test on all platforms
3. **Reviewers**: Code review, suggest changes
4. **Author**: Address feedback, push updates
5. **Approval**: At least 1 approval required for merge
6. **Merge**: Squash or merge commit (case-by-case)
## Version Numbering
Follow **Semantic Versioning (SemVer)**: `MAJOR.MINOR.PATCH`
### MAJOR (e.g., 1.0.0)
- Breaking API changes
- Major architectural changes
- First stable release
### MINOR (e.g., 0.4.0)
- New features (backward compatible)
- Significant improvements
- Major dependency updates (SDL3)
### PATCH (e.g., 0.3.2)
- Bug fixes
- Minor improvements
- Documentation updates
### Pre-release Tags
- `v0.4.0-alpha.1` - Early testing
- `v0.4.0-beta.1` - Feature complete
- `v0.4.0-rc.1` - Release candidate
## Release Process
### For Minor/Major Releases (0.x.0, x.0.0)
1. **Create release branch**
```bash
git checkout -b release/v0.4.0
```
2. **Update version numbers**
- `CMakeLists.txt`
- `../reference/changelog.md`
- `README.md`
3. **Update documentation**
- Review all docs for accuracy
- Update migration guides if breaking changes
- Finalize changelog
4. **Create release commit**
```bash
git commit -m "chore: prepare v0.4.0 release"
```
5. **Merge and tag**
```bash
git checkout master
git merge --no-ff release/v0.4.0
git tag -a v0.4.0 -m "Release v0.4.0"
git push origin master --tags
```
6. **Merge back to develop**
```bash
git checkout develop
git merge --no-ff release/v0.4.0
git push origin develop
```
7. **Create GitHub Release**
- Draft release notes
- Attach build artifacts (CI generates these)
- Publish release
### For Patch Releases (0.3.x)
1. **Collect fixes on develop**
- Merge all bugfix PRs to develop
- Ensure tests pass
2. **Create release branch**
```bash
git checkout -b release/v0.3.2
```
3. **Follow steps 2-7 above**
## Long-Running Feature Branches
For large features (e.g., v0.4.0 modernization), use a **feature branch with sub-branches**:
```
develop
└── feature/v0.4.0-modernization (long-running)
├── feature/v0.4.0-sdl3-core
├── feature/v0.4.0-sdl3-graphics
└── feature/v0.4.0-sdl3-audio
```
**Rules:**
- Long-running branch stays alive during development
- Sub-branches merge to long-running branch
- Long-running branch periodically rebases on `develop`
- Final merge to `develop` when complete
## Tagging Strategy
### Release Tags
- Format: `v<MAJOR>.<MINOR>.<PATCH>[-prerelease]`
- Examples: `v0.3.2`, `v0.4.0-rc.1`, `v1.0.0`
- Annotated tags with release notes
### Internal Milestones
- Format: `milestone/<name>`
- Examples: `milestone/canvas-refactor-complete`
- Used for tracking major internal achievements
## Best Practices
### DO
- Keep commits atomic and focused
- Write descriptive commit messages
- Rebase feature branches on develop regularly
- Run tests before pushing
- Update documentation with code changes
- Delete branches after merging
### DON'T ❌
- Commit directly to master or develop
- Force push to shared branches
- Mix unrelated changes in one commit
- Merge without PR review
- Leave stale branches
## Quick Reference
```bash
# Start new feature
git checkout develop
git pull
git checkout -b feature/my-feature
# Update feature branch with latest develop
git fetch origin
git rebase origin/develop
# Finish feature
git push -u origin feature/my-feature
# Create PR on GitHub → Merge → Delete branch
# Start hotfix
git checkout master
git pull
git checkout -b hotfix/v0.3.3-fix
# ... fix, commit, merge to master and develop ...
# Create release
git checkout develop
git pull
git checkout -b release/v0.4.0
# ... prepare, merge to master, tag, merge back to develop ...
```
## Emergency Procedures
### If master is broken
1. Create hotfix branch immediately
2. Fix critical issue
3. Fast-track PR review
4. Hotfix deploy ASAP
### If develop is broken
1. Identify breaking commit
2. Revert if needed
3. Fix in new branch
4. Merge fix with priority
### If release needs to be rolled back
1. Tag current state as `v0.x.y-broken`
2. Revert master to previous tag
3. Create hotfix branch
4. Fix and release as patch version
---
## Current Simplified Workflow (Pre-1.0)
### Daily Development Pattern
```bash
# For documentation or small changes
git checkout master # or develop, your choice
git pull
# ... make changes ...
git add docs/
git commit -m "docs: update workflow guide"
git push origin master
# For experimental features
git checkout -b feature/my-experiment
# ... experiment ...
git push -u origin feature/my-experiment
# If it works: merge to develop
# If it doesn't: delete branch, no harm done
```
### When to Use Branches (Pre-1.0)
**Use a branch for:**
- Large refactors that might break things
- Experimenting with new ideas
- Features that take multiple days
- SDL3 migration or other big changes
**Don't bother with branches for:**
- Documentation updates
- Small bug fixes
- Typo corrections
- README updates
- Adding comments or tests
### Current Branch Usage
For now, treat `master` and `develop` interchangeably:
- `master`: Latest stable-ish code + docs
- `develop`: Optional staging area for integration
When you want docs public, just push to `master`. The GitHub Pages / docs site will update automatically.
### Commit Message (Simplified)
Still try to follow the convention, but don't stress:
```bash
# Good enough
git commit -m "docs: reorganize documentation structure"
git commit -m "fix: dungeon editor crash on load"
git commit -m "feat: add overworld sprite editor"
# Also fine for now
git commit -m "update docs"
git commit -m "fix crash"
```
### Releases (Pre-1.0)
Just tag and push:
```bash
# When you're ready for v0.3.2
git tag -a v0.3.2 -m "Release v0.3.2"
git push origin v0.3.2
# GitHub Actions builds it automatically
```
No need for release branches or complex merging until you have multiple contributors.
---
**References:**
- [Git Flow](https://nvie.com/posts/a-successful-git-branching-model/)
- [Conventional Commits](https://www.conventionalcommits.org/)
- [Semantic Versioning](https://semver.org/)

1332
docs/public/developer/gui-consistency-guide.md Normal file
View File

File diff suppressed because it is too large Load Diff

123
docs/public/developer/networking.md Normal file
View File

@@ -0,0 +1,123 @@
# B5 - Architecture and Networking
This document provides a comprehensive overview of the yaze application's architecture, focusing on its service-oriented design, gRPC integration, and real-time collaboration features. For build/preset instructions when enabling gRPC/automation presets, refer to the [Build & Test Quick Reference](../build/quick-reference.md).
## 1. High-Level Architecture
The yaze ecosystem is split into two main components: the **YAZE GUI Application** and the **`z3ed` CLI Tool**.
```
┌─────────────────────────────────────────────────────────────────────────┐
│ YAZE GUI Application │
│ (Runs on local machine) │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ UnifiedGRPCServer (Port 50051) │ │
│ │ ════════════════════════════════════════════════════════ │ │
│ │ Hosts 3 gRPC Services on a SINGLE PORT: │ │
│ │ │ │
│ │ 1. ImGuiTestHarness Service │ │
│ │ • GUI automation (click, type, wait, assert) │ │
│ │ │ │
│ │ 2. RomService │ │
│ │ • Read/write ROM bytes │ │
│ │ • Proposal system for collaborative editing │ │
│ │ │ │
│ │ 3. CanvasAutomation Service │ │
│ │ • High-level canvas operations (tile ops, selection) │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ ↑ │
│ │ gRPC Connection │
└────────────────────────────────────┼──────────────────────────────────────┘
│
↓
┌────────────────────────────────────┼──────────────────────────────────────┐
│ z3ed CLI Tool │
│ (Command-line interface) │
├─────────────────────────────────────────────────────────────────────────┤
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ CLI Services (Business Logic - NOT gRPC servers) │ │
│ │ ══════════════════════════════════════════════════ │ │
│ │ - AI Services (Gemini, Ollama) │ │
│ │ - Agent Services (Chat, Tool Dispatcher) │ │
│ │ - Network Clients (gRPC and WebSocket clients) │ │
│ └──────────────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────┘
```
## 2. Service Taxonomy
It's important to distinguish between the two types of "services" in the yaze project.
### APP Services (gRPC Servers)
- **Location**: `src/app/core/service/`, `src/app/net/`
- **Runs In**: YAZE GUI application
- **Purpose**: Expose application functionality to remote clients (like the `z3ed` CLI).
- **Type**: gRPC **SERVER** implementations.
### CLI Services (Business Logic)
- **Location**: `src/cli/service/`
- **Runs In**: `z3ed` CLI tool
- **Purpose**: Implement the business logic for the CLI commands.
- **Type**: These are helper classes, **NOT** gRPC servers. They may include gRPC **CLIENTS** to connect to the APP Services.
## 3. gRPC Services
yaze exposes its core functionality through a `UnifiedGRPCServer` that hosts multiple services on a single port (typically 50051).
### ImGuiTestHarness Service
- **Proto**: `imgui_test_harness.proto`
- **Purpose**: GUI automation.
- **Features**: Click, type, screenshots, widget discovery.
### RomService
- **Proto**: `rom_service.proto`
- **Purpose**: Low-level ROM manipulation.
- **Features**: Read/write bytes, proposal system for collaborative editing, snapshots for version management.
### CanvasAutomation Service
- **Proto**: `canvas_automation.proto`
- **Purpose**: High-level, abstracted control over canvas-based editors.
- **Features**: Tile operations (get/set), selection management, view control (pan/zoom).
## 4. Real-Time Collaboration
Real-time collaboration is enabled through a WebSocket-based protocol managed by the `yaze-server`, a separate Node.js application.
### Architecture
```
┌─────────────┐ WebSocket ┌──────────────┐
│ yaze app │◄────────────────────────────►│ yaze-server │
│ (GUI) │ │ (Node.js) │
└─────────────┘ └──────────────┘
▲ ▲
│ gRPC │ WebSocket
└─────────────┐ │
┌──────▼──────┐ │
│ z3ed CLI │◄──────────────────────┘
└─────────────┘
```
### Core Components
- **ROM Version Manager**: Protects the ROM from corruption with snapshots and safe points.
- **Proposal Approval Manager**: Manages a collaborative voting system for applying changes.
- **Collaboration Panel**: A UI within the YAZE editor for managing collaboration.
### WebSocket Protocol
The WebSocket protocol handles real-time messaging for:
- Hosting and joining sessions.
- Broadcasting ROM diffs (`rom_sync`).
- Sharing and voting on proposals (`proposal_share`, `proposal_vote`).
- Sharing snapshots.
## 5. Data Flow Example: AI Agent Edits a Tile
1. **User** runs `z3ed agent --prompt "Paint grass at 10,10"`.
2. The **GeminiAIService** (a CLI Service) parses the prompt and returns a tool call to `overworld-set-tile`.
3. The **ToolDispatcher** (a CLI Service) routes this to the appropriate handler.
4. The **Tile16ProposalGenerator** (a CLI Service) creates a proposal.
5. The **GuiAutomationClient** (a CLI Service acting as a gRPC client) calls the `CanvasAutomation.SetTile()` RPC method on the YAZE application's `UnifiedGRPCServer`.
6. The **CanvasAutomationService** in the YAZE app receives the request and uses the `CanvasAutomationAPI` to paint the tile.
7. If collaboration is active, the change is submitted through the **ProposalApprovalManager**, which communicates with the `yaze-server` via WebSocket to manage voting.
8. Once approved, the change is applied to the ROM and synced with all collaborators.

122
docs/public/developer/overworld-entity-system.md Normal file
View File

@@ -0,0 +1,122 @@
# Overworld Entity System
This document provides a technical overview of the overworld entity system, including critical bug fixes that enable its functionality and the ongoing plan to refactor it for modularity and ZScream feature parity.
## 1. System Overview
The overworld entity system manages all interactive objects on the overworld map, such as entrances, exits, items, and sprites. The system is undergoing a refactor to move from a monolithic architecture within the `Overworld` class to a modular design where each entity's save/load logic is handled in dedicated files.
**Key Goals of the Refactor**:
- **Modularity**: Isolate entity logic into testable, self-contained units.
- **ZScream Parity**: Achieve feature compatibility with ZScream's entity management, including support for expanded ROM formats.
- **Maintainability**: Simplify the `Overworld` class by delegating I/O responsibilities.
## 2. Core Components & Bug Fixes
Several critical bugs were fixed to make the entity system functional. Understanding these fixes is key to understanding the system's design.
### 2.1. Entity Interaction and Hover Detection
**File**: `src/app/editor/overworld/overworld_entity_renderer.cc`
- **Problem**: Exit entities were not responding to mouse interactions because the hover state was being improperly reset.
- **Fix**: The hover state (`hovered_entity_`) is now reset only once at the beginning of the entity rendering cycle, specifically in `DrawExits()`, which is the first rendering function called. Subsequent functions (`DrawEntrances()`, `DrawItems()`, etc.) can set the hover state without it being cleared, preserving the correct hover priority (last-drawn entity wins).
```cpp
// In DrawExits(), which is called first:
hovered_entity_ = nullptr; // Reset hover state at the start of the cycle.
// In DrawEntrances() and other subsequent renderers:
// The reset is removed to allow hover state to persist.
```
### 2.2. Entity Property Popup Save/Cancel Logic
**File**: `src/app/editor/overworld/entity.cc`
- **Problem**: The "Done" and "Cancel" buttons in entity property popups had inverted logic, causing changes to be saved on "Cancel" and discarded on "Done".
- **Fix**: The `set_done` flag, which controls the popup's return value, is now correctly managed. The "Done" and "Delete" buttons set `set_done = true` to signal a save action, while the "Cancel" button does not, correctly discarding changes.
```cpp
// Corrected logic for the "Done" button in popups
if (ImGui::Button(ICON_MD_DONE)) {
set_done = true; // Save changes
ImGui::CloseCurrentPopup();
}
// Corrected logic for the "Cancel" button
if (ImGui::Button(ICON_MD_CANCEL)) {
// Discard changes (do not set set_done)
ImGui::CloseCurrentPopup();
}
```
### 2.3. Exit Entity Coordinate System
**File**: `src/zelda3/overworld/overworld_exit.h`
- **Problem**: Saving a vanilla ROM would corrupt exit positions, causing them to load at (0,0). This was because the `OverworldExit` class used `uint8_t` for player coordinates, truncating 16-bit values.
- **Fix**: The coordinate-related members of `OverworldExit` were changed to `uint16_t` to match the full 0-4088 coordinate range, achieving parity with ZScream's data structures.
```cpp
// In OverworldExit class definition:
class OverworldExit : public GameEntity {
public:
// ...
uint16_t y_player_; // Changed from uint8_t
uint16_t x_player_; // Changed from uint8_t
uint16_t y_camera_; // Changed from uint8_t
uint16_t x_camera_; // Changed from uint8_t
// ...
};
```
### 2.4. Coordinate Synchronization on Drag
**File**: `src/zelda3/overworld/overworld_exit.h`
- **Problem**: When dragging an exit, the visual position (`x_`, `y_`) would update, but the underlying data used for saving (`x_player_`, `y_player_`) would not, leading to a data desync and incorrect saves.
- **Fix**: The `UpdateMapProperties` method now explicitly syncs the base entity coordinates to the player coordinates before recalculating scroll and camera values. This ensures that drag operations correctly persist.
```cpp
// In OverworldExit::UpdateMapProperties()
void UpdateMapProperties(uint16_t map_id) override {
// Sync player position from the base entity coordinates updated by the drag system.
x_player_ = static_cast<uint16_t>(x_);
y_player_ = static_cast<uint16_t>(y_);
// Proceed with auto-calculation using the now-correct player coordinates.
// ...
}
```
## 3. Entity I/O Refactoring Plan
The next phase of development is to extract all entity save and load logic from the monolithic `overworld.cc` into dedicated files.
### 3.1. File Structure
New files will be created to handle I/O for each entity type:
- `src/zelda3/overworld/overworld_entrance.cc`
- `src/zelda3/overworld/overworld_exit.cc`
- `src/zelda3/overworld/overworld_item.cc`
- `src/zelda3/overworld/overworld_transport.cc` (for new transport/whirlpool support)
### 3.2. Core Functions
Each new file will implement a standard set of flat functions:
- `LoadAll...()`: Reads all entities of a given type from the ROM.
- `SaveAll...()`: Writes all entities of a given type to the ROM.
- Helper functions for coordinate calculation and data manipulation, mirroring ZScream's logic.
### 3.3. ZScream Parity Goals
The refactor aims to implement key ZScream features:
- **Expanded ROM Support**: Correctly read/write from vanilla or expanded ROM addresses for entrances and items.
- **Pointer Deduplication**: When saving items, reuse pointers for identical item lists on different maps to conserve space.
- **Automatic Coordinate Calculation**: For exits and transports, automatically calculate camera and scroll values based on player position, matching the `UpdateMapStuff` logic in ZScream.
- **Transport Entity**: Add full support for transport entities (whirlpools, birds).
### 3.4. `Overworld` Class Role
After the refactor, the `Overworld` class will act as a coordinator, delegating all entity I/O to the new, modular functions. Its responsibility will be to hold the entity vectors and orchestrate the calls to the `LoadAll...` and `SaveAll...` functions.

353
docs/public/developer/palette-system-overview.md Normal file
View File

@@ -0,0 +1,353 @@
# SNES Palette System Overview
## Understanding SNES Color and Palette Organization
### Core Concepts
#### 1. SNES Color Format (15-bit BGR555)
- **Storage**: 2 bytes per color (16 bits total, 15 bits used)
- **Format**: `0BBB BBGG GGGR RRRR`
- Bits 0-4: Red (5 bits, 0-31)
- Bits 5-9: Green (5 bits, 0-31)
- Bits 10-14: Blue (5 bits, 0-31)
- Bit 15: Unused (always 0)
- **Range**: Each channel has 32 levels (0-31)
- **Total Colors**: 32,768 possible colors (2^15)
#### 2. Palette Groups in Zelda 3
Zelda 3 organizes palettes into logical groups for different game areas and entities:
```cpp
struct PaletteGroupMap {
PaletteGroup overworld_main; // Main overworld graphics (35 colors each)
PaletteGroup overworld_aux; // Auxiliary overworld (21 colors each)
PaletteGroup overworld_animated; // Animated colors (7 colors each)
PaletteGroup hud; // HUD graphics (32 colors each)
PaletteGroup global_sprites; // Sprite palettes (60 colors each)
PaletteGroup armors; // Armor colors (15 colors each)
PaletteGroup swords; // Sword colors (3 colors each)
PaletteGroup shields; // Shield colors (4 colors each)
PaletteGroup sprites_aux1; // Auxiliary sprite palette 1 (7 colors each)
PaletteGroup sprites_aux2; // Auxiliary sprite palette 2 (7 colors each)
PaletteGroup sprites_aux3; // Auxiliary sprite palette 3 (7 colors each)
PaletteGroup dungeon_main; // Dungeon palettes (90 colors each)
PaletteGroup grass; // Grass colors (special handling)
PaletteGroup object_3d; // 3D object palettes (8 colors each)
PaletteGroup overworld_mini_map; // Mini-map palettes (128 colors each)
};
```
#### 3. Color Representations in Code
- **SNES 15-bit (`uint16_t`)**: On-disk format `0bbbbbgggggrrrrr`; store raw ROM
words or write back with `ConvertRgbToSnes`.
- **`gfx::snes_color` struct**: Expands each channel to 0-255 for arithmetic
without floating point; use in converters and palette math.
- **`gfx::SnesColor` class**: High-level wrapper retaining the original SNES
value, a `snes_color`, and an ImVec4. Its `rgb()` accessor purposely returns
0-255 components—run the helper converters (e.g., `ConvertSnesColorToImVec4`)
before handing colors to ImGui widgets that expect 0.0-1.0 floats.
### Dungeon Palette System
#### Structure
- **20 dungeon palettes** in the `dungeon_main` group
- **90 colors per palette** (full SNES palette for BG layers)
- **ROM Location**: `kDungeonMainPalettes` (check `snes_palette.cc` for exact address)
#### Usage
```cpp
// Loading a dungeon palette
auto& dungeon_pal_group = rom->palette_group().dungeon_main;
int num_palettes = dungeon_pal_group.size(); // Should be 20
int palette_id = room.palette; // Room's palette ID (0-19)
// IMPORTANT: Use operator[] not palette() method!
auto palette = dungeon_pal_group[palette_id]; // Returns reference
// NOT: auto palette = dungeon_pal_group.palette(palette_id); // Returns copy!
```
#### Color Distribution (90 colors)
The 90 colors are typically distributed as:
- **BG1 Palette** (Background Layer 1): First 8-16 subpalettes
- **BG2 Palette** (Background Layer 2): Next 8-16 subpalettes
- **Sprite Palettes**: Remaining colors (handled separately)
Each "subpalette" is 16 colors (one SNES palette unit).
### Overworld Palette System
#### Structure
- **Main Overworld**: 35 colors per palette
- **Auxiliary**: 21 colors per palette
- **Animated**: 7 colors per palette (for water, lava effects)
#### 3BPP Graphics and Left/Right Palettes
Overworld graphics use 3BPP (3 bits per pixel) format:
- **8 colors per tile** (2^3 = 8)
- **Left Side**: Uses palette 0-7
- **Right Side**: Uses palette 8-15
When decompressing 3BPP graphics:
```cpp
// Palette assignment for 3BPP overworld tiles
if (tile_position < half_screen_width) {
// Left side of screen
tile_palette_offset = 0; // Use colors 0-7
} else {
// Right side of screen
tile_palette_offset = 8; // Use colors 8-15
}
```
### Common Issues and Solutions
#### Issue 1: Empty Palette
**Symptom**: "Palette size: 0 colors"
**Cause**: Using `palette()` method instead of `operator[]`
**Solution**:
```cpp
// WRONG:
auto palette = group.palette(id); // Returns copy, may be empty
// CORRECT:
auto palette = group[id]; // Returns reference
```
#### Issue 2: Bitmap Corruption
**Symptom**: Graphics render only in top portion of image
**Cause**: Wrong depth parameter in `CreateAndRenderBitmap`
**Solution**:
```cpp
// WRONG:
CreateAndRenderBitmap(0x200, 0x200, 0x200, data, bitmap, palette);
// depth ^^^^ should be 8!
// CORRECT:
CreateAndRenderBitmap(0x200, 0x200, 8, data, bitmap, palette);
// width, height, depth=8 bits
```
### Transparency and Conversion Best Practices
- Preserve ROM palette words exactly as read; hardware enforces transparency on
index 0 so we no longer call `set_transparent(true)` while loading.
- Apply transparency only at render time via `SetPaletteWithTransparent()` for
3BPP sub-palettes or `SetPalette()` for full 256-color assets.
- `SnesColor::rgb()` yields components in 0-255 space; convert to ImGui’s
expected 0.0-1.0 floats with the helper functions instead of manual divides.
- Use the provided conversion helpers (`ConvertSnesToRgb`, `ImVec4ToSnesColor`,
`SnesTo8bppColor`) to prevent rounding mistakes and alpha bugs.
```cpp
ImVec4 rgb_255 = snes_color.rgb();
ImVec4 display = ConvertSnesColorToImVec4(snes_color);
ImGui::ColorButton("color", display);
```
#### Issue 3: ROM Not Loaded in Preview
**Symptom**: "ROM not loaded" error in emulator preview
**Cause**: Initializing before ROM is set
**Solution**:
```cpp
// Initialize emulator preview AFTER ROM is loaded and set
void Load() {
// ... load ROM data ...
// ... set up other components ...
// NOW initialize emulator preview with loaded ROM
object_emulator_preview_.Initialize(rom_);
}
```
### Palette Editor Integration
#### Key Functions for UI
```cpp
// Reading a color from ROM
absl::StatusOr<uint16_t> ReadColorFromRom(uint32_t address, const uint8_t* rom);
// Converting SNES color to RGB
SnesColor color(snes_value); // snes_value is uint16_t
uint8_t r = color.red(); // 0-255 (converted from 0-31)
uint8_t g = color.green(); // 0-255
uint8_t b = color.blue(); // 0-255
// Writing color back to ROM
uint16_t snes_value = color.snes(); // Get 15-bit BGR555 value
rom->WriteByte(address, snes_value & 0xFF); // Low byte
rom->WriteByte(address + 1, (snes_value >> 8) & 0xFF); // High byte
```
#### Palette Widget Requirements
1. **Display**: Show colors in organized grids (16 colors per row for SNES standard)
2. **Selection**: Allow clicking to select a color
3. **Editing**: Provide RGB sliders (0-255) or color picker
4. **Conversion**: Auto-convert RGB (0-255) ↔ SNES (0-31) values
5. **Preview**: Show before/after comparison
6. **Save**: Write modified palette back to ROM
#### Palette UI Helpers
- `InlinePaletteSelector` renders a lightweight selection strip (no editing)
ideal for 8- or 16-color sub-palettes.
- `InlinePaletteEditor` supplies the full editing experience with ImGui color
pickers, context menus, and optional live preview toggles.
- `PopupPaletteEditor` fits in context menus or modals; it caps at 64 colors to
keep popups manageable.
- Legacy helpers such as `DisplayPalette()` remain for backward compatibility
but inherit the 32-color limit—prefer the new helpers for new UI.
-### Metadata-Driven Palette Application
`gfx::BitmapMetadata` tracks the source BPP, palette format, type string, and
expected color count. Set it immediately after creating a bitmap so later code
can make the right choice automatically:
```cpp
bitmap.metadata() = BitmapMetadata{/*source_bpp=*/3,
/*palette_format=*/1, // 0=full, 1=sub-palette
/*source_type=*/"graphics_sheet",
/*palette_colors=*/8};
bitmap.ApplyPaletteByMetadata(palette);
```
- `palette_format == 0` routes to `SetPalette()` and preserves every color
(Mode 7, HUD assets, etc.).
- `palette_format == 1` routes to `SetPaletteWithTransparent()` and injects the
transparent color 0 for 3BPP workflows.
- Validation hooks help catch mismatched palette sizes before they hit SDL.
### Graphics Manager Integration
#### Sheet Palette Assignment
```cpp
// Assigning palette to graphics sheet
if (sheet_id > 115) {
// Sprite sheets use sprite palette
graphics_sheet.SetPaletteWithTransparent(
rom.palette_group().global_sprites[0], 0);
} else {
// Dungeon sheets use dungeon palette
graphics_sheet.SetPaletteWithTransparent(
rom.palette_group().dungeon_main[0], 0);
}
```
### Texture Synchronization and Regression Notes
- Call `bitmap.UpdateSurfacePixels()` after mutating `bitmap.mutable_data()` to
copy rendered bytes into the SDL surface before queuing texture creation or
updates.
- `Bitmap::ApplyStoredPalette()` now rebuilds an `SDL_Color` array sized to the
actual palette instead of forcing 256 entries—this fixes regressions where
8- or 16-color palettes were padded with opaque black.
- When updating SDL palette data yourself, mirror that pattern:
```cpp
std::vector<SDL_Color> colors(palette.size());
for (size_t i = 0; i < palette.size(); ++i) {
const auto& c = palette[i];
const ImVec4 rgb = c.rgb(); // 0-255 components
colors[i] = SDL_Color{static_cast<Uint8>(rgb.x),
static_cast<Uint8>(rgb.y),
static_cast<Uint8>(rgb.z),
c.is_transparent() ? 0 : 255};
}
SDL_SetPaletteColors(surface->format->palette, colors.data(), 0,
static_cast<int>(colors.size()));
```
### Best Practices
1. **Always use `operator[]` for palette access** - returns reference, not copy
2. **Validate palette IDs** before accessing:
```cpp
if (palette_id >= 0 && palette_id < group.size()) {
auto palette = group[palette_id];
}
```
3. **Use correct depth parameter** when creating bitmaps (usually 8 for indexed color)
4. **Initialize ROM-dependent components** only after ROM is fully loaded
5. **Cache palettes** when repeatedly accessing the same palette
6. **Update textures** after changing palettes (textures don't auto-update)
### User Workflow Tips
- Choose the widget that matches the task: selectors for choosing colors,
editors for full control, popups for contextual tweaks.
- The live preview toggle trades responsiveness for performance; disable it
while batch-editing large (64+ color) palettes.
- Right-click any swatch in the editor to copy the color as SNES hex, RGB
tuples, or HTML hex—useful when coordinating with external art tools.
- Remember hardware rules: palette index 0 is always transparent and will not
display even if the stored value is non-zero.
- Keep ROM backups when performing large palette sweeps; palette groups are
shared across screens so a change can have multiple downstream effects.
### ROM Addresses (for reference)
```cpp
// From snes_palette.cc
constexpr uint32_t kOverworldPaletteMain = 0xDE6C8;
constexpr uint32_t kOverworldPaletteAux = 0xDE86C;
constexpr uint32_t kOverworldPaletteAnimated = 0xDE604;
constexpr uint32_t kHudPalettes = 0xDD218;
constexpr uint32_t kGlobalSpritesLW = 0xDD308;
constexpr uint32_t kArmorPalettes = 0xDD630;
constexpr uint32_t kSwordPalettes = 0xDD630;
constexpr uint32_t kShieldPalettes = 0xDD648;
constexpr uint32_t kSpritesPalettesAux1 = 0xDD39E;
constexpr uint32_t kSpritesPalettesAux2 = 0xDD446;
constexpr uint32_t kSpritesPalettesAux3 = 0xDD4E0;
constexpr uint32_t kDungeonMainPalettes = 0xDD734;
constexpr uint32_t kHardcodedGrassLW = 0x5FEA9;
constexpr uint32_t kTriforcePalette = 0xF4CD0;
constexpr uint32_t kOverworldMiniMapPalettes = 0x55B27;
```
## Graphics Sheet Palette Application
### Default Palette Assignment
Graphics sheets receive default palettes during ROM loading based on their index:
```cpp
// In LoadAllGraphicsData() - rom.cc
if (i < 113) {
// Sheets 0-112: Overworld/Dungeon graphics
graphics_sheets[i].SetPalette(rom.palette_group().dungeon_main[0]);
} else if (i < 128) {
// Sheets 113-127: Sprite graphics
graphics_sheets[i].SetPalette(rom.palette_group().sprites_aux1[0]);
} else {
// Sheets 128-222: Auxiliary/HUD graphics
graphics_sheets[i].SetPalette(rom.palette_group().hud.palette(0));
}
```
This ensures graphics are visible immediately after loading rather than appearing white.
### Palette Update Workflow
When changing a palette in any editor:
1. Apply the palette: `bitmap.SetPalette(new_palette)`
2. Notify Arena: `gfx::Arena::Get().NotifySheetModified(sheet_index)`
3. Changes propagate to all editors automatically
### Common Pitfalls
**Wrong Palette Access**:
```cpp
// WRONG - Returns copy, may be empty
auto palette = group.palette(id);
// CORRECT - Returns reference
auto palette = group[id];
```
**Missing Surface Update**:
```cpp
// WRONG - Only updates vector, not SDL surface
bitmap.mutable_data() = new_data;
// CORRECT - Updates both vector and surface
bitmap.set_data(new_data);
```

153
docs/public/developer/testing-guide.md Normal file
View File

@@ -0,0 +1,153 @@
# A1 - Testing Guide
This guide provides a comprehensive overview of the testing framework for the yaze project, including the test organization, execution methods, and the end-to-end GUI automation system.
## 1. Test Organization
The test suite is organized into a clear directory structure that separates tests by their purpose and dependencies. This is the primary way to understand the nature of a test.
```
test/
├── unit/ # Unit tests for individual components
│ ├── core/ # Core functionality (asar, hex utils)
│ ├── cli/ # Command-line interface tests
│ ├── emu/ # Emulator component tests
│ ├── gfx/ # Graphics system (tiles, palettes)
│ ├── gui/ # GUI widget tests
│ ├── rom/ # ROM data structure tests
│ └── zelda3/ # Game-specific logic tests
├── integration/ # Tests for interactions between components
│ ├── ai/ # AI agent and vision tests
│ ├── editor/ # Editor integration tests
│ └── zelda3/ # Game-specific integration tests (ROM-dependent)
├── e2e/ # End-to-end user workflow tests (GUI-driven)
│ ├── rom_dependent/ # E2E tests requiring a ROM
│ └── zscustomoverworld/ # ZSCustomOverworld upgrade E2E tests
├── benchmarks/ # Performance benchmarks
├── mocks/ # Mock objects for isolating tests
└── assets/ # Test assets (patches, data)
```
## 2. Test Categories
Based on the directory structure, tests fall into the following categories:
### Unit Tests (`unit/`)
- **Purpose**: To test individual classes or functions in isolation.
- **Characteristics**:
- Fast, self-contained, and reliable.
- No external dependencies (e.g., ROM files, running GUI).
- Form the core of the CI/CD validation pipeline.
### Integration Tests (`integration/`)
- **Purpose**: To verify that different components of the application work together correctly.
- **Characteristics**:
- May require a real ROM file (especially those in `integration/zelda3/`). These are considered "ROM-dependent".
- Test interactions between modules, such as the `asar` wrapper and the `Rom` class, or AI services with the GUI controller.
- Slower than unit tests but crucial for catching bugs at module boundaries.
### End-to-End (E2E) Tests (`e2e/`)
- **Purpose**: To simulate a full user workflow from start to finish.
- **Characteristics**:
- Driven by the **ImGui Test Engine**.
- Almost always require a running GUI and often a real ROM.
- The slowest but most comprehensive tests, validating the user experience.
- Includes smoke tests, canvas interactions, and complex workflows like ZSCustomOverworld upgrades.
### Benchmarks (`benchmarks/`)
- **Purpose**: To measure and track the performance of critical code paths, particularly in the graphics system.
- **Characteristics**:
- Not focused on correctness but on speed and efficiency.
- Run manually or in specialized CI jobs to prevent performance regressions.
## 3. Running Tests
> 💡 Need a refresher on presets/commands? See the [Build & Test Quick Reference](../build/quick-reference.md)
> for the canonical `cmake`, `ctest`, and helper script usage before running the commands below.
### Using the Enhanced Test Runner (`yaze_test`)
The most flexible way to run tests is by using the `yaze_test` executable directly. It provides flags to filter tests by category, which is ideal for development and AI agent workflows.
```bash
# First, build the test executable
cmake --build build_ai --target yaze_test
# Run all tests
./build_ai/bin/yaze_test
# Run only unit tests
./build_ai/bin/yaze_test --unit
# Run only integration tests
./build_ai/bin/yaze_test --integration
# Run E2E tests (requires a GUI)
./build_ai/bin/yaze_test --e2e --show-gui
# Run ROM-dependent tests with a specific ROM
./build_ai/bin/yaze_test --rom-dependent --rom-path /path/to/zelda3.sfc
# Run tests matching a specific pattern (e.g., all Asar tests)
./build_ai/bin/yaze_test "*Asar*"
# Get a full list of options
./build_ai/bin/yaze_test --help
```
### Using CTest and CMake Presets
For CI/CD or a more traditional workflow, you can use `ctest` with CMake presets.
```bash
# Configure a development build (enables ROM-dependent tests)
cmake --preset mac-dev -DYAZE_TEST_ROM_PATH=/path/to/your/zelda3.sfc
# Build the tests
cmake --build --preset mac-dev --target yaze_test
# Run stable tests (fast, primarily unit tests)
ctest --preset dev
# Run all tests, including ROM-dependent and E2E
ctest --preset all
```
## 4. Writing Tests
When adding new tests, place them in the appropriate directory based on their purpose and dependencies.
- **New class `MyClass`?** Add `test/unit/my_class_test.cc`.
- **Testing `MyClass` with a real ROM?** Add `test/integration/my_class_rom_test.cc`.
- **Testing a full UI workflow involving `MyClass`?** Add `test/e2e/my_class_workflow_test.cc`.
## 5. E2E GUI Testing Framework
The E2E framework uses `ImGuiTestEngine` to automate UI interactions.
### Architecture
- **`test/yaze_test.cc`**: The main test runner that can initialize a GUI for E2E tests.
- **`test/e2e/`**: Contains all E2E test files, such as:
- `framework_smoke_test.cc`: Basic infrastructure verification.
- `canvas_selection_test.cc`: Canvas interaction tests.
- `dungeon_editor_tests.cc`: UI tests for the dungeon editor.
- **`test/test_utils.h`**: Provides high-level helper functions for common actions like loading a ROM (`LoadRomInTest`) or opening an editor (`OpenEditorInTest`).
### Running GUI Tests
To run E2E tests and see the GUI interactions, use the `--show-gui` flag.
```bash
# Run all E2E tests with the GUI visible
./build_ai/bin/yaze_test --e2e --show-gui
# Run a specific E2E test by name
./build_ai/bin/yaze_test --show-gui --gtest_filter="*DungeonEditorSmokeTest"
```
### Widget Discovery and AI Integration
The GUI testing framework is designed for AI agent automation. All major UI elements are registered with stable IDs, allowing an agent to "discover" and interact with them programmatically via the `z3ed` CLI.
Refer to the `z3ed` agent guide for details on using commands like `z3ed gui discover`, `z3ed gui click`, and `z3ed agent test replay`.

372
docs/public/developer/testing-quick-start.md Normal file
View File

@@ -0,0 +1,372 @@
# Testing Quick Start - Before You Push
**Target Audience**: Developers contributing to yaze
**Goal**: Ensure your changes pass tests before pushing to remote
## The 5-Minute Pre-Push Checklist
Before pushing changes to the repository, run these commands to catch issues early:
### 1. Build Tests (30 seconds)
```bash
# Build the test executable
cmake --build build --target yaze_test
```
### 2. Run Fast Tests (<2 minutes)
```bash
# Run unit tests only (fastest)
./build/bin/yaze_test --unit
# Or run all stable tests (unit + non-ROM integration)
./build/bin/yaze_test
```
### 3. Platform-Specific Quick Check
**macOS**:
```bash
scripts/agents/smoke-build.sh mac-dbg yaze
```
**Linux**:
```bash
scripts/agents/smoke-build.sh lin-dbg yaze
```
**Windows (PowerShell)**:
```powershell
pwsh -File scripts/agents/windows-smoke-build.ps1 -Preset win-dbg -Target yaze
```
### 4. Check for Format Issues (optional but recommended)
```bash
# Check if code is formatted correctly
cmake --build build --target format-check
# Auto-fix formatting issues
cmake --build build --target format
```
## When to Run Full Test Suite
Run the **complete test suite** before pushing if:
- You modified core systems (ROM, graphics, editor base classes)
- You changed CMake configuration or build system
- You're preparing a pull request
- CI previously failed on your branch
### Full Test Suite Commands
```bash
# Run all tests (may take 5+ minutes)
./build/bin/yaze_test
# Include ROM-dependent tests (requires zelda3.sfc)
./build/bin/yaze_test --rom-dependent --rom-path /path/to/zelda3.sfc
# Run E2E GUI tests (headless)
./build/bin/yaze_test --e2e
# Run E2E with visible GUI (for debugging)
./build/bin/yaze_test --e2e --show-gui
```
## Common Test Failures and Fixes
### 1. Compilation Errors
**Symptom**: `cmake --build build --target yaze_test` fails
**Fix**:
```bash
# Clean and reconfigure
rm -rf build
cmake --preset mac-dbg # or lin-dbg, win-dbg
cmake --build build --target yaze_test
```
### 2. Unit Test Failures
**Symptom**: `./build/bin/yaze_test --unit` shows failures
**Fix**:
- Read the error message carefully
- Check if you broke contracts in modified code
- Verify test expectations match your changes
- Update tests if behavior change was intentional
### 3. ROM-Dependent Test Failures
**Symptom**: Tests fail with "ROM file not found"
**Fix**:
```bash
# Set environment variable
export YAZE_TEST_ROM_PATH=/path/to/zelda3.sfc
# Or pass directly to test runner
./build/bin/yaze_test --rom-path /path/to/zelda3.sfc
```
### 4. E2E/GUI Test Failures
**Symptom**: E2E tests fail or hang
**Fix**:
- Check if SDL is initialized properly
- Run with `--show-gui` to see what's happening visually
- Verify ImGui Test Engine is enabled in build
- Check test logs for specific assertion failures
### 5. Platform-Specific Failures
**Symptom**: Tests pass locally but fail in CI
**Solution**:
1. Check which platform failed in CI logs
2. If Windows: ensure you're using the `win-*` preset
3. If Linux: check for case-sensitive path issues
4. If macOS: verify you're testing on compatible macOS version
## Test Categories Explained
| Category | What It Tests | When to Run | Duration |
|----------|---------------|-------------|----------|
| **Unit** | Individual functions/classes | Before every commit | <10s |
| **Integration** | Component interactions | Before every push | <30s |
| **E2E** | Full user workflows | Before PRs | 1-5min |
| **ROM-Dependent** | ROM data loading/saving | Before ROM changes | Variable |
## Recommended Workflows
### For Small Changes (typos, docs, minor fixes)
```bash
# Just build to verify no compile errors
cmake --build build --target yaze
```
### For Code Changes (new features, bug fixes)
```bash
# Build and run unit tests
cmake --build build --target yaze_test
./build/bin/yaze_test --unit
# If tests pass, push
git push
```
### For Core System Changes (ROM, graphics, editors)
```bash
# Run full test suite
cmake --build build --target yaze_test
./build/bin/yaze_test
# If all tests pass, push
git push
```
### For Pull Requests
```bash
# Run everything including ROM tests and E2E
./build/bin/yaze_test --rom-dependent --rom-path zelda3.sfc
./build/bin/yaze_test --e2e
# Check code formatting
cmake --build build --target format-check
# If all pass, create PR
git push origin feature-branch
```
## IDE Integration
### Visual Studio Code
Add this to `.vscode/tasks.json`:
```json
{
"version": "2.0.0",
"tasks": [
{
"label": "Build Tests",
"type": "shell",
"command": "cmake --build build --target yaze_test",
"group": "build"
},
{
"label": "Run Unit Tests",
"type": "shell",
"command": "./build/bin/yaze_test --unit",
"group": "test",
"dependsOn": "Build Tests"
},
{
"label": "Run All Tests",
"type": "shell",
"command": "./build/bin/yaze_test",
"group": "test",
"dependsOn": "Build Tests"
}
]
}
```
Then use `Cmd/Ctrl+Shift+B` to build tests or `Cmd/Ctrl+Shift+P` → "Run Test Task" to run them.
### CLion / Visual Studio
Both IDEs auto-detect CTest and provide built-in test runners:
- **CLion**: Tests appear in "Test Explorer" panel
- **Visual Studio**: Use "Test Explorer" window
Configure test presets in `CMakePresets.json` (already configured in this project).
## Environment Variables
Customize test behavior with these environment variables:
```bash
# Path to test ROM file
export YAZE_TEST_ROM_PATH=/path/to/zelda3.sfc
# Skip ROM-dependent tests entirely
export YAZE_SKIP_ROM_TESTS=1
# Enable UI tests (E2E)
export YAZE_ENABLE_UI_TESTS=1
# Verbose test output
export YAZE_TEST_VERBOSE=1
```
## Getting Test Output
### Verbose Test Output
```bash
# Show all test output (even passing tests)
./build/bin/yaze_test --gtest_output=verbose
# Show only failed test output
./build/bin/yaze_test --gtest_output=on_failure
```
### Specific Test Patterns
```bash
# Run only tests matching pattern
./build/bin/yaze_test --gtest_filter="*AsarWrapper*"
# Run tests in specific suite
./build/bin/yaze_test --gtest_filter="RomTest.*"
# Exclude specific tests
./build/bin/yaze_test --gtest_filter="-*SlowTest*"
```
### Repeat Tests for Flakiness
```bash
# Run tests 10 times to catch flakiness
./build/bin/yaze_test --gtest_repeat=10
# Stop on first failure
./build/bin/yaze_test --gtest_repeat=10 --gtest_break_on_failure
```
## CI/CD Testing
After pushing, CI will run tests on all platforms (Linux, macOS, Windows):
1. **Check CI status**: Look for green checkmark in GitHub
2. **If CI fails**: Click "Details" to see which platform/test failed
3. **Fix and push again**: CI re-runs automatically
**Pro tip**: Use remote workflow triggers to test in CI before pushing:
```bash
# Trigger CI remotely (requires gh CLI)
scripts/agents/run-gh-workflow.sh ci.yml -f enable_http_api_tests=true
```
See [GH Actions Remote Guide](../../internal/agents/gh-actions-remote.md) for setup.
## Advanced Topics
### Running Tests with CTest
```bash
# Run all stable tests via ctest
ctest --preset dev
# Run specific test suite
ctest -L unit
# Run with verbose output
ctest --preset dev --output-on-failure
# Run tests in parallel
ctest --preset dev -j8
```
### Debugging Failed Tests
```bash
# Run test under debugger (macOS/Linux)
lldb ./build/bin/yaze_test -- --gtest_filter="*FailingTest*"
# Run test under debugger (Windows)
devenv /debugexe ./build/bin/yaze_test.exe --gtest_filter="*FailingTest*"
```
### Writing New Tests
See [Testing Guide](testing-guide.md) for comprehensive guide on writing tests.
Quick template:
```cpp
#include <gtest/gtest.h>
#include "my_class.h"
namespace yaze {
namespace test {
TEST(MyClassTest, BasicFunctionality) {
MyClass obj;
EXPECT_TRUE(obj.DoSomething());
}
} // namespace test
} // namespace yaze
```
Add your test file to `test/CMakeLists.txt` in the appropriate suite.
## Help and Resources
- **Detailed Testing Guide**: [docs/public/developer/testing-guide.md](testing-guide.md)
- **Build Commands**: [docs/public/build/quick-reference.md](../build/quick-reference.md)
- **Testing Infrastructure**: [docs/internal/testing/README.md](../../internal/testing/README.md)
- **Troubleshooting**: [docs/public/build/troubleshooting.md](../build/troubleshooting.md)
## Questions?
1. Check [Testing Guide](testing-guide.md) for detailed explanations
2. Search existing issues: https://github.com/scawful/yaze/issues
3. Ask in discussions: https://github.com/scawful/yaze/discussions
---
**Remember**: Running tests before pushing saves time for everyone. A few minutes of local testing prevents hours of CI debugging.

287
docs/public/developer/testing-without-roms.md Normal file
View File

@@ -0,0 +1,287 @@
# Testing z3ed Without ROM Files
**Last Updated:** October 10, 2025
**Status:** Active
## Overview
The `z3ed` AI agent now supports **mock ROM mode** for testing without requiring actual ROM files. This is essential for:
- **CI/CD pipelines** - No ROM files can be committed to GitHub
- **Development testing** - Quick iterations without ROM dependencies
- **Contributors** - Test the agent without needing to provide ROMs
- **Automated testing** - Consistent, reproducible test environments
## How Mock ROM Mode Works
Mock ROM mode creates a minimal but valid ROM structure with:
- Proper SNES header (LoROM mapping, 1MB size)
- Zelda3 embedded labels (rooms, sprites, entrances, items, music, etc.)
- Resource label manager initialization
- No actual ROM data (tiles, graphics, maps remain empty)
This allows the AI agent to:
- Answer questions about room names, sprite IDs, entrance numbers
- Lookup labels and constants
- Test function calling and tool dispatch
- Validate agent logic without game data
## Usage
### Command Line Flag
Add `--mock-rom` to any `z3ed agent` command:
```bash
# Simple chat with mock ROM
z3ed agent simple-chat "What is room 5?" --mock-rom
# Test conversation with mock ROM
z3ed agent test-conversation --mock-rom
# AI provider testing
z3ed agent simple-chat "List all dungeons" --mock-rom --ai_provider=ollama
```
### Test Suite
The `agent_test_suite.sh` script now defaults to mock ROM mode:
```bash
# Run tests with mock ROM (default)
./scripts/agent_test_suite.sh ollama
# Or with Gemini
./scripts/agent_test_suite.sh gemini
# Override the Ollama model (CI uses qwen2.5-coder:0.5b)
OLLAMA_MODEL=qwen2.5-coder:0.5b ./scripts/agent_test_suite.sh ollama
```
To use a real ROM instead, edit the script:
```bash
USE_MOCK_ROM=false # At the top of agent_test_suite.sh
```
## What Works with Mock ROM
### Fully Supported
**Label Queries:**
- "What is room 5?" → "Tower of Hera - Moldorm Boss"
- "What sprites are in the game?" → Lists all 256 sprite names
- "What is entrance 0?" → "Link's House Main"
- "List all items" → Bow, Boomerang, Hookshot, etc.
**Resource Lookups:**
- Room names (296 rooms)
- Entrance names (133 entrances)
- Sprite names (256 sprites)
- Overlord names (14 overlords)
- Overworld map names (160 maps)
- Item names
- Music track names
- Graphics sheet names
**AI Testing:**
- Function calling / tool dispatch
- Natural language understanding
- Error handling
- Tool output parsing
- Multi-turn conversations
### Limited Support
**Queries Requiring Data:**
- "What tiles are used in room 5?" → No tile data in mock ROM
- "Show me the palette for map 0" → No palette data
- "What's at coordinate X,Y?" → No map data
- "Export graphics from dungeon 1" → No graphics data
These queries will either return empty results or errors indicating no ROM data is available.
### Not Supported
**Operations That Modify ROM:**
- Editing tiles
- Changing palettes
- Modifying sprites
- Patching ROM data
## Testing Strategy
### For Agent Logic
Use **mock ROM** for testing:
- Function calling mechanisms
- Tool dispatch and routing
- Natural language understanding
- Error handling
- Label resolution
- Resource lookups
### For ROM Operations
Use **real ROM** for testing:
- Tile editing
- Graphics manipulation
- Palette modifications
- Data extraction
- ROM patching
## CI/CD Integration
### GitHub Actions Example
```yaml
name: Test z3ed Agent
on: [push, pull_request]
jobs:
test-agent:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Dependencies
run: |
# Install ollama if testing local models
curl -fsSL https://ollama.ai/install.sh | sh
ollama pull qwen2.5-coder
- name: Build z3ed
run: |
cmake -B build_test
cmake --build build_test --parallel
- name: Run Agent Tests (Mock ROM)
run: |
./scripts/agent_test_suite.sh ollama
env:
# Or use Gemini with API key
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
```
## Embedded Labels Reference
Mock ROM includes all these labels from `zelda3::Zelda3Labels`:
| Resource Type | Count | Example |
|--------------|-------|---------|
| Rooms | 296 | "Sewer - Throne Room" |
| Entrances | 133 | "Link's House Main" |
| Sprites | 256 | "Moldorm (Boss)" |
| Overlords | 14 | "Overlord - Agahnim's Barrier" |
| Overworld Maps | 160 | "Light World - Hyrule Castle" |
| Items | 64+ | "Bow", "Boomerang", "Hookshot" |
| Music Tracks | 64+ | "Title Theme", "Overworld", "Dark World" |
| Graphics Sheets | 128+ | "Link Sprites", "Enemy Pack 1" |
See `src/zelda3/zelda3_labels.h` for the complete list.
## Troubleshooting
### "No ROM loaded" error
Make sure you're using the `--mock-rom` flag:
```bash
# Wrong
z3ed agent simple-chat "test"
# Correct
z3ed agent simple-chat "test" --mock-rom
```
### Mock ROM fails to initialize
Check the error message. Common issues:
- Build system didn't include `mock_rom.cc`
- Missing `zelda3_labels.cc` in build
- Linker errors with resource labels
### Agent returns empty/wrong results
Remember: Mock ROM has **labels only**, no actual game data.
Queries like "What tiles are in room 5?" won't work because there's no tile data.
Use queries about labels and IDs instead: "What is the name of room 5?"
## Development
### Adding New Labels
To add new label types to mock ROM:
1. **Add to `zelda3_labels.h`:**
```cpp
static const std::vector<std::string>& GetNewResourceNames();
```
2. **Implement in `zelda3_labels.cc`:**
```cpp
const std::vector<std::string>& Zelda3Labels::GetNewResourceNames() {
static std::vector<std::string> names = {"Item1", "Item2", ...};
return names;
}
```
3. **Add to `ToResourceLabels()`:**
```cpp
const auto& new_resources = GetNewResourceNames();
for (size_t i = 0; i < new_resources.size(); ++i) {
labels["new_resource"][std::to_string(i)] = new_resources[i];
}
```
4. **Rebuild:**
```bash
cmake --build build --parallel
```
### Testing Mock ROM Directly
```cpp
#include "cli/handlers/mock_rom.h"
Rom rom;
auto status = InitializeMockRom(rom);
if (status.ok()) {
// ROM is ready with all labels
auto* label_mgr = rom.resource_label();
std::string room_name = label_mgr->GetLabel("room", "5");
// room_name == "Tower of Hera - Moldorm Boss"
}
```
## Best Practices
### DO
- Use mock ROM for CI/CD and automated tests
- Use mock ROM for agent logic development
- Use mock ROM when contributing (no ROM files needed)
- Test with real ROM before releasing features
- Document which features require real ROM data
### DON'T ❌
- Commit ROM files to Git (legal issues)
- Assume mock ROM has actual game data
- Use mock ROM for testing data extraction
- Skip real ROM testing entirely
## Related Documentation
- [z3ed CLI Guide](../usage/z3ed-cli.md) - Main agent and CLI documentation
- [Testing Guide](testing-guide.md) - General testing strategy
- [API Reference](api-reference.md) - ROM API documentation
---
**Implementation Status:** Complete
**Since Version:** v0.3.3
**Files:**
- `src/cli/handlers/mock_rom.h`
- `src/cli/handlers/mock_rom.cc`
- `src/cli/flags.cc` (--mock-rom flag)
- `scripts/agent_test_suite.sh` (updated)

362
docs/public/developer/tile16-palette-system.md Normal file
View File

@@ -0,0 +1,362 @@
# Tile16 Editor Palette System
**Status: Work in Progress** - Documentation for the ongoing Tile16 Editor palette system redesign, implementation, and refactoring improvements.
## Executive Summary
The tile16 editor palette system is undergoing a complete redesign to resolve critical crashes and ensure proper color alignment between the tile16 editor and overworld display. Significant progress has been made addressing fundamental architectural issues with palette mapping, implementing crash fixes, and creating an improved three-column layout. However, palette handling for the tile8 source canvas and palette button functionality remain works in progress.
## Problem Analysis
### Critical Issues Identified
1. **SIGBUS Crash in SnesColor::rgb()**
- **Root Cause**: `SetPaletteWithTransparent()` method used incorrect `index * 7` calculation
- **Impact**: Crashes when selecting palette buttons 4-7 due to out-of-bounds memory access
- **Location**: `bitmap.cc:371` - `start_index = index * 7`
2. **Fundamental Architecture Misunderstanding**
- **Root Cause**: Attempting to use `SetPaletteWithTransparent()` instead of `SetPalette()`
- **Impact**: Broke the 256-color palette system that the overworld relies on
- **Reality**: Overworld system uses complete 256-color palettes with `SetPalette()`
3. **Color Mapping Misunderstanding**
- **Root Cause**: Confusion about how color mapping works in the graphics system
- **Impact**: Incorrect palette handling in tile16 editor
- **Reality**: Pixel data already contains correct color indices for the 256-color palette
## Solution Architecture
### Core Design Principles
1. **Use Same Palette System as Overworld**: Use `SetPalette()` with complete 256-color palette
2. **Direct Color Mapping**: The pixel data in graphics already contains correct color indices that map to the 256-color palette
3. **Proper Bounds Checking**: Prevent out-of-bounds memory access in palette operations
4. **Consistent Architecture**: Match overworld editor's palette handling exactly
### 256-Color Overworld Palette Structure
Based on analysis of `SetColorsPalette()` in `overworld_map.cc`:
```
Row 0-1: HUD palette (32 colors) - Slots 0-31
Row 2-6: MAIN palette (35 colors) - Slots 32-87 (rows 2-6, cols 1-7)
Row 7: ANIMATED palette (7 colors) - Slots 112-118 (row 7, cols 1-7)
Row 8: Sprite AUX1 + AUX3 (14 colors) - Slots 128-141
Row 9-12: Global sprites (60 colors) - Slots 144-203
Row 13: Sprite palette 1 (7 colors) - Slots 208-214
Row 14: Sprite palette 2 (7 colors) - Slots 224-230
Row 15: Armor palettes (15 colors) - Slots 240-254
Right side (cols 9-15):
Row 2-4: AUX1 palette (21 colors) - Slots 136-151
Row 5-7: AUX2 palette (21 colors) - Slots 152-167
```
### Sheet-to-Palette Mapping
```
Sheet 0,3,4: → AUX1 region (slots 136-151)
Sheet 5,6: → AUX2 region (slots 152-167)
Sheet 1,2: → MAIN region (slots 32-87)
Sheet 7: → ANIMATED region (slots 112-118)
```
### Palette Button Mapping
| Button | Sheet 0,3,4 (AUX1) | Sheet 1,2 (MAIN) | Sheet 5,6 (AUX2) | Sheet 7 (ANIMATED) |
|--------|---------------------|-------------------|-------------------|---------------------|
| 0 | 136 | 32 | 152 | 112 |
| 1 | 138 | 39 | 154 | 113 |
| 2 | 140 | 46 | 156 | 114 |
| 3 | 142 | 53 | 158 | 115 |
| 4 | 144 | 60 | 160 | 116 |
| 5 | 146 | 67 | 162 | 117 |
| 6 | 148 | 74 | 164 | 118 |
| 7 | 150 | 81 | 166 | 119 |
## Implementation Details
### 1. Fixed SetPaletteWithTransparent Method
**File**: `src/app/gfx/bitmap.cc`
**Before**:
```cpp
auto start_index = index * 7; // WRONG: Creates invalid indices for buttons 4-7
```
**After**:
```cpp
// Extract 8-color sub-palette starting at the specified index
// Always start with transparent color (index 0)
colors.push_back(ImVec4(0, 0, 0, 0));
// Extract up to 7 colors from the palette starting at index
for (size_t i = 0; i < 7 && (index + i) < palette.size(); ++i) {
auto &pal_color = palette[index + i];
colors.push_back(pal_color.rgb());
}
```
### 2. Corrected Tile16 Editor Palette System
**File**: `src/app/editor/overworld/tile16_editor.cc`
**Before**:
```cpp
// WRONG: Trying to extract 8-color sub-palettes
current_gfx_individual_[i].SetPaletteWithTransparent(display_palette, base_palette_slot, 8);
```
**After**:
```cpp
// CORRECT: Use complete 256-color palette (same as overworld system)
// The pixel data already contains correct color indices for the 256-color palette
current_gfx_individual_[i].SetPalette(display_palette);
```
### 3. Palette Coordination Flow
```
Overworld System:
ProcessGraphicsBuffer() → adds 0x88 offset to sheets 0,3,4,5
BuildTiles16Gfx() → combines with palette info
current_gfx_bmp_ → contains properly offset pixel data
Tile16 Editor:
Initialize() → receives current_gfx_bmp_ with correct data
LoadTile8() → extracts tiles with existing pixel values
SetPalette() → applies complete 256-color palette
Display → correct colors shown automatically
```
## UI/UX Refactoring
### New Three-Column Layout
The tile16 editor layout was completely restructured into a unified 3-column table for better space utilization:
**Column 1: Tile16 Blockset** (35% width)
- Complete 512-tile blockset display
- Integrated zoom slider (0.5x - 4.0x)
- Zoom in/out buttons for quick adjustments
- Click to select tiles for editing
- Full vertical scrolling support
**Column 2: Tile8 Source Tileset** (35% width)
- All 8x8 source tiles
- Palette group selector (OW Main, OW Aux, etc.)
- Integrated zoom slider (1.0x - 8.0x)
- Click to select tiles for painting
- Full vertical scrolling support
**Column 3: Editor & Controls** (30% width)
- Tile16 editor canvas with dynamic zoom (2.0x - 8.0x)
- Canvas size adjusts automatically with zoom level
- All controls in compact vertical layout:
- Tile8 preview and ID
- Flip controls (X, Y, Priority)
- Palette selector (8 buttons)
- Action buttons (Clear, Copy, Paste)
- Save/Discard changes
- Undo button
- Advanced controls (collapsible)
- Debug information (collapsible)
**Benefits**:
- Better space utilization - all three main components visible simultaneously
- Improved workflow - blockset, source tiles, and editor all in one view
- Resizable columns allow users to adjust layout to preference
### Canvas Context Menu Fixes
**Problem**: The "Advanced Canvas Properties" and "Scaling Controls" popup dialogs were not appearing when selected from the context menu.
**Solution**:
- Changed popup windows from modal popups to regular windows
- Added boolean flags `show_advanced_properties_` and `show_scaling_controls_` to track window state
- Windows now appear as floating windows instead of modal dialogs
- Added public accessor methods:
- `OpenAdvancedProperties()`
- `OpenScalingControls()`
- `CloseAdvancedProperties()`
- `CloseScalingControls()`
**Files Modified**:
- `src/app/gui/canvas.cc` - Updated popup implementation
- `src/app/gui/canvas.h` - Added boolean flags and accessor methods
### Dynamic Zoom Controls
Each canvas now has independent zoom control with real-time feedback:
**Tile16 Blockset Zoom**:
- Range: 0.5x to 4.0x
- Slider control with +/- buttons
- Properly scales mouse coordinate calculations
**Tile8 Source Zoom**:
- Range: 1.0x to 8.0x
- Essential for viewing small pixel details
- Correctly accounts for zoom in tile selection
**Tile16 Editor Zoom**:
- Range: 2.0x to 8.0x
- Canvas size dynamically adjusts with zoom
- Grid scales proportionally
- Mouse coordinates properly transformed
**Implementation Details**:
```cpp
// Mouse coordinate calculations account for dynamic zoom
int tile_x = static_cast<int>(mouse_pos.x / (8 * tile8_zoom));
// Grid drawing scaled with zoom
tile16_edit_canvas_.DrawGrid(8.0F * tile16_zoom / 4.0F);
// Canvas display size calculated dynamically
float canvas_display_size = 16 * tile16_zoom + 4;
```
## Testing Protocol
### Crash Prevention Testing
1. **Load ROM** and open tile16 editor
2. **Test all palette buttons 0-7** - should not crash
3. **Verify color display** - should match overworld appearance
4. **Test different graphics sheets** - should use appropriate palette regions
### Color Alignment Testing
1. **Select tile16 in overworld** - note colors displayed
2. **Open tile16 editor** - load same tile
3. **Test palette buttons 0-7** - colors should match overworld
4. **Verify sheet-specific behavior** - different sheets should show different colors
### UI/UX Testing
1. **Canvas Popups**: Right-click on each canvas → verify "Advanced Properties" and "Scaling Controls" open correctly
2. **Zoom Controls**: Test each zoom slider and button for all three canvases
3. **Tile Selection**: Verify clicking works at various zoom levels for blockset, tile8 source, and tile16 editor
4. **Layout Responsiveness**: Resize window and columns to verify proper behavior
5. **Workflow**: Test complete tile editing workflow from selection to save
## Error Handling
### Bounds Checking
- **Palette Index Validation**: Ensure palette indices don't exceed palette size
- **Sheet Index Validation**: Ensure sheet indices are within valid range (0-7)
- **Surface Validation**: Ensure SDL surface exists before palette operations
### Fallback Mechanisms
- **Default Palette**: Use MAIN region if sheet detection fails
- **Safe Indices**: Clamp palette indices to valid ranges
- **Error Logging**: Comprehensive logging for debugging
## Debug Information Display
The debug panel (collapsible by default) shows:
1. **Current State**: Tile16 ID, Tile8 ID, selected palette button
2. **Mapping Info**: Sheet index, actual palette slot
3. **Reference Table**: Complete button-to-slot mapping for all sheets
4. **Color Preview**: Visual display of actual colors being used
## Known Issues and Ongoing Work
### Completed Items
- **No Crashes**: Fixed SIGBUS errors - palette buttons 0-7 work without crashing
- **Three-Column Layout**: Unified layout with blockset, tile8 source, and editor
- **Dynamic Zoom Controls**: Independent zoom for all three canvases
- **Canvas Popup Fixes**: Advanced properties and scaling controls now working
- **Stable Memory**: No memory leaks or corruption
- **Code Architecture**: Proper bounds checking and error handling
### Active Issues Warning:
**1. Tile8 Source Canvas Palette Issues**
- **Problem**: The tile8 source canvas (displaying current area graphics) does not show correct colors
- **Impact**: Source tiles appear with incorrect palette, making it difficult to preview how tiles will look
- **Root Cause**: Area graphics not receiving proper palette application
- **Status**: Under investigation - may be related to graphics buffer processing
**2. Palette Button Functionality**
- **Problem**: Palette buttons (0-7) do not properly update the displayed colors
- **Impact**: Cannot switch between different palette groups as intended
- **Expected Behavior**: Clicking palette buttons should change the active palette for tile8 graphics
- **Actual Behavior**: Button clicks do not trigger palette updates correctly
- **Status**: Needs implementation of proper palette switching logic
**3. Color Alignment Between Canvases**
- **Problem**: Colors in tile8 source canvas don't match tile16 editor canvas
- **Impact**: Inconsistent visual feedback during tile editing
- **Related To**: Issues #1 and #2 above
- **Status**: Blocked by palette button functionality
### Current Status Summary
| Component | Status | Notes |
|-----------|--------|-------|
| Crash Prevention | Complete | No SIGBUS errors |
| Three-Column Layout | Complete | Fully functional |
| Zoom Controls | Complete | All canvases working |
| Tile16 Editor Palette | Complete | Shows correct colors |
| Tile8 Source Palette | Warning: In Progress | Incorrect colors displayed |
| Palette Button Updates | Warning: In Progress | Not updating palettes |
| Sheet-Aware Logic | Warning: Partial | Foundation in place, needs fixes |
| Overall Color System | Warning: In Progress | Ongoing development |
### Future Enhancements
1. **Zoom Presets**: Add preset buttons (1x, 2x, 4x) for quick zoom levels
2. **Zoom Sync**: Option to sync zoom levels across related canvases
3. **Canvas Layout Persistence**: Save user's preferred column widths and zoom levels
4. **Keyboard Shortcuts**: Add +/- keys for zoom control
5. **Mouse Wheel Zoom**: Ctrl+Wheel to zoom canvases
6. **Performance Optimization**: Cache palette calculations for frequently used combinations
7. **Extended Debug Tools**: Add pixel-level color comparison tools
8. **Palette Preview**: Real-time preview of palette changes before applying
9. **Batch Operations**: Apply palette changes to multiple tiles simultaneously
## Maintenance Notes
1. **Palette Structure Changes**: Update `GetActualPaletteSlot()` if overworld palette structure changes
2. **Sheet Detection**: Verify `GetSheetIndexForTile8()` logic if graphics organization changes
3. **ProcessGraphicsBuffer**: Monitor for changes in pixel data offset logic
4. **Static Zoom Variables**: User preferences preserved across sessions
---
## Next Steps
### Immediate Priorities
1. **Fix Tile8 Source Canvas Palette Application**
- Investigate graphics buffer processing for area graphics
- Ensure consistent palette application across all graphics sources
- Test with different area graphics combinations
2. **Implement Palette Button Update Logic**
- Add proper event handling for palette button clicks
- Trigger palette refresh for tile8 source canvas
- Update visual feedback to show active palette
3. **Verify Color Consistency**
- Ensure tile8 source colors match tile16 editor
- Test sheet-specific palette regions
- Validate color mapping for all graphics sheets
### Investigation Areas
- Review `current_gfx_individual_` palette application
- Check palette group management for tile8 source
- Verify graphics buffer offset logic for area graphics
- Test palette switching across different overworld areas
---
*Development Status: January 2025*
*Status: Partial implementation - UI complete, palette system in progress*
*Not yet ready for production use - active development ongoing*

49
docs/public/examples/README.md Normal file
View File

@@ -0,0 +1,49 @@
# Examples & Recipes
Short, task-focused snippets for everyday YAZE workflows. These examples supplement the primary
guides (Getting Started, z3ed CLI, Dungeon/Overworld editors) and should remain concise. When in
doubt, link back to the relevant guide instead of duplicating long explanations.
## 1. Launching Common Editors
```bash
# Open YAZE directly in the Dungeon editor with room cards preset
./build/bin/yaze --rom_file=zelda3.sfc \
--editor=Dungeon \
--cards="Rooms List,Room Graphics,Object Editor"
# Jump to an Overworld map from the CLI/TUI companion
./build/bin/z3ed overworld describe-map --map 0x80 --rom zelda3.sfc
```
## 2. AI/Automation Recipes
```bash
# Generate an AI plan to reposition an entrance, but do not apply yet
./build/bin/z3ed agent plan \
--rom zelda3.sfc \
--prompt "Move the desert palace entrance 2 tiles north" \
--sandbox
# Resume the plan and apply it once reviewed
./build/bin/z3ed agent accept --proposal-id <ID> --rom zelda3.sfc --sandbox
```
## 3. Building & Testing Snippets
```bash
# Debug build with tests
cmake --preset mac-dbg
cmake --build --preset mac-dbg --target yaze yaze_test
./build/bin/yaze_test --unit
# AI-focused build in a dedicated directory (recommended for assistants)
cmake --preset mac-ai -B build_ai
cmake --build build_ai --target yaze z3ed
```
## 4. Quick Verification
- Run `./scripts/verify-build-environment.sh --fix` (or the PowerShell variant on Windows) whenever
pulling major build changes.
- See the [Build & Test Quick Reference](../build/quick-reference.md) for the canonical list of
commands and testing recipes.
Want to contribute another recipe? Add it here with a short description and reference the relevant
guide so the examples stay focused.

53
docs/public/index.md Normal file
View File

@@ -0,0 +1,53 @@
/**
@mainpage YAZE Documentation
@tableofcontents
*/
# YAZE Documentation
YAZE documentation now focuses on concise, Doxygen-friendly sections. Use the categories
below for human-readable guides and reference material. Internal planning, AI agent playbooks,
and research notes were moved to `docs/internal/` so the public docs stay focused.
## Overview
- [Getting Started](overview/getting-started.md)
## Build & Tooling
- [Build Quick Reference](build/quick-reference.md)
- [Build From Source](build/build-from-source.md)
- [Platform Compatibility](build/platform-compatibility.md)
- [CMake Presets](build/presets.md)
- [Build Troubleshooting](build/troubleshooting.md)
## Usage Guides
- [Dungeon Editor](usage/dungeon-editor.md)
- [Overworld Loading](usage/overworld-loading.md)
- [z3ed CLI](usage/z3ed-cli.md)
- [Examples & Recipes](examples/)
## Developer Guides
- [Architecture Overview](developer/architecture.md)
- [Dependency Architecture](developer/dependency-architecture.md)
- [Git Workflow](developer/git-workflow.md)
- [Networking Overview](developer/networking.md)
- [Testing Guide](developer/testing-guide.md)
- [Testing Without ROMs](developer/testing-without-roms.md)
- [Debugging Guide](developer/debugging-guide.md)
- [Debug Flags](developer/debug-flags.md)
- [Assembler Style Guide](developer/asm-style-guide.md)
- [API Reference](developer/api-reference.md)
- [Emulator Development Guide](developer/emulator-development-guide.md)
- [Canvas System](developer/canvas-system.md)
- [Palette System Overview](developer/palette-system-overview.md)
- [Tile16 Palette System](developer/tile16-palette-system.md)
- [Overworld Entity System](developer/overworld-entity-system.md)
- [GUI Consistency Guide](developer/gui-consistency-guide.md)
## Reference
- [ROM Reference](reference/rom-reference.md)
- [Changelog](reference/changelog.md)
---
Need editor playbooks, refactors, or AI workflows? Head over to [`docs/internal/`](../internal/README.md)
for internal documentation that stays out of the public Doxygen site.

68
docs/public/overview/getting-started.md Normal file
View File

@@ -0,0 +1,68 @@
# Getting Started
This software allows you to modify "The Legend of Zelda: A Link to the Past" (US or JP) ROMs. It is built for compatibility with ZScream projects and designed to be cross-platform.
## Quick Start
1. **Download** the latest release for your platform from the [releases page](https://github.com/scawful/yaze/releases).
2. **Load ROM** via `File > Open ROM`.
3. **Select an Editor** from the main toolbar (e.g., Overworld, Dungeon, Graphics).
4. **Make Changes** and save your project.
> Building from source or enabling AI tooling? Use the
> [Build & Test Quick Reference](../build/quick-reference.md) for the canonical commands and presets.
## General Tips
- **Experiment Flags**: Enable or disable new features in `File > Options > Experiment Flags`.
- **Backup Files**: Enabled by default. Each save creates a timestamped backup of your ROM.
- **Extensions**: Load custom tools via the `Extensions` menu (C library and Python module support is planned).
## Feature Status
| Feature | State | Notes |
|---|---|---|
| Overworld Editor | Stable | Supports vanilla and ZSCustomOverworld v2/v3 projects. |
| Dungeon Editor | Experimental | Requires extensive manual testing before production use. |
| Tile16 Editor | Experimental | Palette and tile workflows are still being tuned. |
| Palette Editor | Stable | Reference implementation for palette utilities. |
| Graphics Editor | Experimental | Rendering pipeline under active refactor. |
| Sprite Editor | Experimental | Card/UI patterns mid-migration. |
| Message Editor | Stable | Re-test after recent palette fixes. |
| Hex Editor | Stable | Direct ROM editing utility. |
| Asar Patching | Stable | Wraps the bundled Asar assembler. |
## Command-Line Interface (`z3ed`)
`z3ed` provides scripted access to the same ROM editors.
### AI Agent Chat
Chat with an AI to perform edits using natural language.
```bash
# Start an interactive chat session with the AI agent
z3ed agent chat --rom zelda3.sfc
```
> **Prompt:** "What sprites are in dungeon 2?"
### Resource Inspection
Directly query ROM data.
```bash
# List all sprites in the Eastern Palace (dungeon 2)
z3ed dungeon list-sprites --rom zelda3.sfc --dungeon 2
# Get information about a specific overworld map area
z3ed overworld describe-map --rom zelda3.sfc --map 80
```
### Patching
Apply assembly patches using the integrated Asar assembler.
```bash
# Apply an assembly patch to the ROM
z3ed asar patch.asm --rom zelda3.sfc
```
## Extending Functionality
YAZE exports a C API that is still evolving. Treat it as experimental and expect breaking changes while the plugin system is built out.

414
docs/public/reference/changelog.md Normal file
View File

@@ -0,0 +1,414 @@
# Changelog
## 0.3.2 (October 2025)
### AI Agent Infrastructure
**z3ed CLI Agent System**:
- **Conversational Agent Service**: Full chat integration with learned knowledge, TODO management, and context injection
- **Emulator Debugging Service**: 20/24 gRPC debugging methods for AI-driven emulator debugging
- Breakpoint management (execute, read, write, access)
- Step execution (single-step, run to breakpoint)
- Memory inspection (read/write WRAM and hardware registers)
- CPU state capture (full 65816 registers + flags)
- Performance metrics (FPS, cycles, audio queue)
- **Command Registry**: Unified command architecture eliminating duplication across CLI/agent systems
- **Learned Knowledge Service**: Persistent preferences, ROM patterns, project context, and conversation memory
- **TODO Manager**: Task tracking with dependencies, execution plan generation, and priority-based scheduling
- **Advanced Router**: Response synthesis and enhancement with data type inference
- **Agent Pretraining**: ROM structure knowledge injection and tool usage examples
**Impact Metrics**:
- Debugging Time: 3+ hours → 15 minutes (92% faster)
- Code Iterations: 15+ rebuilds → 1-2 tool calls (93% fewer)
- AI Autonomy: 30% → 85% (2.8x better)
- Session Continuity: None → Full memory (∞% better)
**Documentation**: 2,000+ lines of comprehensive guides and real-world examples
### CI/CD & Release Improvements
**Release Workflow Fixes**:
- Fixed build matrix artifact upload issues (platform-specific uploads for Windows/Linux/macOS)
- Corrected macOS universal binary merge process with proper artifact paths
- Enhanced release to only include final artifacts (no intermediate build slices)
- Improved build diagnostics and error reporting across all platforms
**CI/CD Pipeline Enhancements**:
- Added manual workflow trigger with configurable build types and options
- Implemented vcpkg caching for faster Windows builds
- Enhanced Windows diagnostics (vcpkg status, Visual Studio info, disk space monitoring)
- Added Windows-specific build failure analysis (linker errors, missing dependencies)
- Conditional artifact uploads for CI builds with configurable retention
- Comprehensive job summaries with platform-specific information
### Rendering Pipeline Fixes
**Graphics Editor White Sheets Fixed**:
- Graphics sheets now receive appropriate default palettes during ROM loading
- Sheets 0-112: Dungeon main palettes, Sheets 113-127: Sprite palettes, Sheets 128-222: HUD/menu palettes
- Eliminated white/blank graphics on initial load
**Message Editor Preview Updates**:
- Fixed static message preview issue where changes weren't visible
- Corrected `mutable_data()` usage to `set_data()` for proper SDL surface synchronization
- Message preview now updates in real-time when selecting or editing messages
**Cross-Editor Graphics Synchronization**:
- Added `Arena::NotifySheetModified()` for centralized texture management
- Graphics changes in one editor now propagate to all other editors
- Replaced raw `printf()` calls with structured `LOG_*` macros throughout graphics pipeline
### Card-Based UI System
**EditorCardManager**:
- Centralized card registration and visibility management
- Context-sensitive card controls in main menu bar
- Category-based keyboard shortcuts (Ctrl+Shift+D for Dungeon, Ctrl+Shift+B for browser)
- Card browser for visual card management
**Editor Integration**:
- All major editors (Dungeon, Graphics, Screen, Sprite, Overworld, Assembly, Message, Emulator) now use card system
- Cards can be closed with X button, proper docking behavior across all editors
- Cards hidden by default to prevent crashes on ROM load
### Tile16 Editor & Graphics System
**Palette System Enhancements**:
- Comprehensive palette coordination with overworld palette system
- Sheet-based palette mapping (Sheets 0,3-6: AUX; Sheets 1-2: MAIN; Sheet 7: ANIMATED)
- Enhanced scrollable UI layout with right-click tile picking
- Save/discard workflow preventing ROM changes until explicit user action
**Performance & Stability**:
- Fixed segmentation faults caused by tile cache `std::move()` operations invalidating Bitmap surface pointers
- Disabled problematic tile cache, implemented direct SDL texture updates
- Added comprehensive bounds checking to prevent palette crashes
- Implemented surface/texture pooling and performance profiling
### Windows Platform Stability
**Build System Fixes**:
- Increased Windows stack size from 1MB to 8MB (matches macOS/Linux defaults)
- Fixed linker errors in development utilities (`extract_vanilla_values`, `rom_patch_utility`)
- Implemented Windows COM-based file dialog fallback for minimal builds
- Consistent cross-platform behavior and stack resources
### Emulator: Audio System Infrastructure
**Audio Backend Abstraction:**
- **IAudioBackend Interface**: Clean abstraction layer for audio implementations, enabling easy migration between SDL2, SDL3, and custom backends
- **SDL2AudioBackend**: Complete implementation with volume control, status queries, and smart buffer management (2-6 frames)
- **AudioBackendFactory**: Factory pattern for creating backends with minimal coupling
- **Benefits**: Future-proof audio system, easy to add platform-native backends (CoreAudio, WASAPI, PulseAudio)
**APU Debugging System:**
- **ApuHandshakeTracker**: Monitors CPU-SPC700 communication in real-time
- **Phase Tracking**: Tracks handshake progression (RESET → IPL_BOOT → WAITING_BBAA → HANDSHAKE_CC → TRANSFER_ACTIVE → RUNNING)
- **Port Activity Monitor**: Records last 1000 port write events with PC addresses
- **Visual Debugger UI**: Real-time phase display, port activity log, transfer progress bars, force handshake testing
- **Integration**: Connected to both CPU (Snes::WriteBBus) and SPC700 (Apu::Write) port operations
**Music Editor Integration:**
- **Live Playback**: `PlaySong(int song_id)` triggers songs via $7E012C memory write
- **Volume Control**: `SetVolume(float)` controls backend volume at abstraction layer
- **Playback Controls**: Stop/pause/resume functionality ready for UI integration
**Documentation:**
- Created comprehensive audio system guides covering IPL ROM protocol, handshake debugging, and testing procedures
### Emulator: Critical Performance Fixes
**Console Logging Performance Killer Fixed:**
- **Issue**: Console logging code was executing on EVERY instruction even when disabled, causing severe performance degradation (< 1 FPS)
- **Impact**: ~1,791,000 console writes per second with mutex locks and buffer flushes
- **Fix**: Removed 73 lines of console output from CPU instruction execution hot path
- **Result**: Emulator now runs at full 60 FPS
**Instruction Logging Default Changed:**
- **Changed**: `kLogInstructions` flag default from `true` to `false`
- **Reason**: Even without console spam, logging every instruction to DisassemblyViewer caused significant slowdown
- **Impact**: No logging overhead unless explicitly enabled by user
**Instruction Log Unbounded Growth Fixed:**
- **Issue**: Legacy `instruction_log_` vector growing to 60+ million entries after 10 minutes, consuming 6GB+ RAM
- **Fix**: Added automatic trimming to 10,000 most recent instructions
- **Result**: Memory usage stays bounded at ~50MB
**Audio Buffer Allocation Bug Fixed:**
- **Issue**: Audio buffer allocated as single `int16_t` instead of array, causing immediate buffer overflow
- **Fix**: Properly allocate as array using `new int16_t[size]` with custom deleter
- **Result**: Audio system can now queue samples without corruption
### Emulator: UI Organization & Input System
**New UI Architecture:**
- **Created `src/app/emu/ui/` directory** for separation of concerns
- **EmulatorUI Layer**: Separated all ImGui rendering code from emulator logic
- **Input Abstraction**: `IInputBackend` interface with SDL2 implementation for future SDL3 migration
- **InputHandler**: Continuous polling system using `SDL_GetKeyboardState()` instead of event-based ImGui keys
**Keyboard Input Fixed:**
- **Issue**: Event-based `ImGui::IsKeyPressed()` only fires once per press, doesn't work for held buttons
- **Fix**: New `InputHandler` uses continuous SDL keyboard state polling every frame
- **Result**: Proper game controls with held button detection
**DisassemblyViewer Enhancement:**
- **Sparse Address Map**: Mesen-style storage of unique addresses only, not every execution
- **Execution Counter**: Increments on re-execution for hotspot analysis
- **Performance**: Tracks millions of instructions with ~5MB RAM vs 6GB+ with old system
- **Always Active**: No need for toggle flag, efficiently active by default
**Feature Flags Cleanup:**
- Removed deprecated `kLogInstructions` flag entirely
- DisassemblyViewer now always active with zero performance cost
### Debugger: Breakpoint & Watchpoint Systems
**BreakpointManager:**
- **CRUD Operations**: Add/Remove/Enable/Disable breakpoints with unique IDs
- **Breakpoint Types**: Execute, Read, Write, Access, and Conditional breakpoints
- **Dual CPU Support**: Separate tracking for 65816 CPU and SPC700
- **Hit Counting**: Tracks how many times each breakpoint is triggered
- **CPU Integration**: Connected to CPU execution via callback system
**WatchpointManager:**
- **Memory Access Tracking**: Monitor reads/writes to memory ranges
- **Range-Based**: Watch single addresses or memory regions ($7E0000-$7E00FF)
- **Access History**: Deque-based storage of last 1000 memory accesses
- **Break-on-Access**: Optional execution pause when watchpoint triggered
- **Export**: CSV export of access history for analysis
**CPU Debugger UI Enhancements:**
- **Integrated Controls**: Play/Pause/Step/Reset buttons directly in debugger window
- **Breakpoint UI**: Address input (hex), add/remove buttons, enable/disable checkboxes, hit count display
- **Live Disassembly**: DisassemblyViewer showing real-time execution
- **Register Display**: Real-time CPU state (A, X, Y, D, SP, PC, PB, DB, flags)
### Build System Simplifications
**Eliminated Conditional Compilation:**
- **Before**: Optional flags for JSON (`YAZE_WITH_JSON`), gRPC (`YAZE_WITH_GRPC`), AI (`Z3ED_AI`)
- **After**: All features always enabled, no configuration required
- **Benefits**: Simpler development, easier onboarding, fewer ifdef-related bugs, consistent builds across all platforms
- **Build Command**: Just `cmake -B build && cmake --build build` - no flags needed!
**DisassemblyViewer Performance Limits:**
- Max 10,000 instructions stored (prevents memory bloat)
- Auto-trim to 8,000 when limit reached (keeps hottest code paths)
- Toggle recording on/off for performance testing
- Clear button to free memory
### Build System: Windows Platform Improvements
**gRPC v1.67.1 Upgrade:**
- **Issue**: v1.62.0 had template instantiation errors on MSVC
- **Fix**: Upgraded to v1.67.1 with MSVC template fixes and better C++17/20 compatibility
- **Result**: Builds successfully on Visual Studio 2022
**MSVC-Specific Compiler Flags:**
- `/bigobj` - Allow large object files (gRPC generates many)
- `/permissive-` - Standards conformance mode
- `/wd4267 /wd4244` - Suppress harmless conversion warnings
- `/constexpr:depth2048` - Handle deep template instantiations
**Cross-Platform Validation:**
- All new audio and input code uses cross-platform SDL2 APIs
- No platform-specific code in audio backend or input abstraction
- Ready for SDL3 migration with minimal changes
### GUI & UX Modernization
- **Theme System**: Implemented comprehensive theme system (`AgentUITheme`) centralizing all UI colors
- **UI Helper Library**: Created 30+ reusable UI helper functions reducing boilerplate code by over 50%
- **Visual Polish**: Enhanced UI panels with theme-aware colors, status badges, connection indicators
### Overworld Editor Refactoring
- **Modular Architecture**: Refactored 3,400-line `OverworldEditor` into smaller focused modules
- **Progressive Loading**: Implemented priority-based progressive loading in `gfx::Arena` to prevent UI freezes
- **Critical Graphics Fixes**: Resolved bugs with graphics refresh, multi-quadrant map updates, and feature visibility
- **Multi-Area Map Configuration**: Robust `ConfigureMultiAreaMap()` handling all area sizes
### Build System & Stability
- **Build Fixes**: Resolved 7 critical build errors including linker issues and filesystem crashes
- **C API Separation**: Decoupled C API library from main application for improved modularity
### Future Optimizations (Planned)
**Graphics System:**
- Lazy loading of graphics sheets (load on-demand rather than all at once)
- Heap-based allocation for large data structures instead of stack
- Streaming/chunked loading for large ROM assets
- Consider if all 223 sheets need to be in memory simultaneously
**Build System:**
- Further reduce CI build times
- Enhanced dependency caching strategies
- Improved vcpkg integration reliability
### Technical Notes
**Breaking Changes:**
- None - this is a patch release focused on stability and fixes
**Deprecations:**
- None
**Migration Guide:**
- No migration required - this release is fully backward compatible with 0.3.1
## 0.3.1 (September 2025)
### Major Features
- **Complete Tile16 Editor Overhaul**: Professional-grade tile editing with modern UI and advanced capabilities
- **Advanced Palette Management**: Full access to all SNES palette groups with configurable normalization
- **Comprehensive Undo/Redo System**: 50-state history with intelligent time-based throttling
- **ZSCustomOverworld v3 Full Support**: Complete implementation of ZScream Save.cs functionality with complex transition calculations
- **ZEML System Removal**: Converted overworld editor from markup to pure ImGui for better performance and maintainability
- **OverworldEditorManager**: New management system to handle complex v3 overworld features
### Tile16 Editor Enhancements
- **Modern UI Layout**: Fully resizable 3-column interface (Tile8 Source, Editor, Preview & Controls)
- **Multi-Palette Group Support**: Access to Overworld Main/Aux1/Aux2, Dungeon Main, Global Sprites, Armors, and Swords palettes
- **Advanced Transform Operations**: Flip horizontal/vertical, rotate 90°, fill with tile8, clear operations
- **Professional Workflow**: Copy/paste, 4-slot scratch space, live preview with auto-commit
- **Pixel Normalization Settings**: Configurable pixel value masks (0x01-0xFF) for handling corrupted graphics sheets
### ZSCustomOverworld v3 Implementation
- **SaveLargeMapsExpanded()**: Complex neighbor-aware transition calculations for all area sizes (Small, Large, Wide, Tall)
- **Interactive Overlay System**: Full `SaveMapOverlays()` with ASM code generation for revealing holes and changing map elements
- **SaveCustomOverworldASM()**: Complete custom overworld ASM application with feature toggles and data tables
- **Expanded Memory Support**: Automatic detection and use of v3 expanded memory locations (0x140xxx)
- **Area-Specific Features**: Background colors, main palettes, mosaic transitions, GFX groups, subscreen overlays, animated tiles
- **Transition Logic**: Sophisticated camera transition calculations based on neighboring area types and quadrants
- **Version Compatibility**: Maintains vanilla/v2 compatibility while adding full v3+ feature support
### Technical Improvements
- **SNES Data Accuracy**: Proper 4-bit palette index handling with configurable normalization
- **Bitmap Pipeline Fixes**: Corrected tile16 extraction using `GetTilemapData()` with manual fallback
- **Real-time Updates**: Immediate visual feedback for all editing operations
- **Memory Safety**: Enhanced bounds checking and error handling throughout
- **ASM Version Detection**: Automatic detection of custom overworld ASM version for feature availability
- **Conditional Save Logic**: Different save paths for vanilla, v2, and v3+ ROMs
### User Interface
- **Keyboard Shortcuts**: Comprehensive shortcuts for all operations (H/V/R for transforms, Q/E for palette cycling, 1-8 for direct palette selection)
- **Visual Feedback**: Hover preview restoration, current palette highlighting, texture status indicators
- **Compact Controls**: Streamlined property panel with essential tools easily accessible
- **Settings Dialog**: Advanced palette normalization controls with real-time application
- **Pure ImGui Layout**: Removed ZEML markup system in favor of native ImGui tabs and tables for better performance
- **v3 Settings Panel**: Dedicated UI for ZSCustomOverworld v3 features with ASM version detection and feature toggles
### Bug Fixes
- **Tile16 Bitmap Display**: Fixed blank/white tile issue caused by unnormalized pixel values
- **Hover Preview**: Restored tile8 preview when hovering over tile16 canvas
- **Canvas Scaling**: Corrected coordinate scaling for 8x magnification factor
- **Palette Corruption**: Fixed high-bit contamination in graphics sheets
- **UI Layout**: Proper column sizing and resizing behavior
- **Linux CI/CD Build**: Fixed undefined reference errors for `ShowSaveFileDialog` method
- **ZSCustomOverworld v3**: Fixed complex area transition calculations and neighbor-aware tilemap adjustments
- **ZEML Performance**: Eliminated markup parsing overhead by converting to native ImGui components
### ZScream Compatibility Improvements
- **Complete Save.cs Implementation**: All major methods from ZScream's Save.cs now implemented in YAZE
- **Area Size Support**: Full support for Small, Large, Wide, and Tall area types with proper transitions
- **Interactive Overlays**: Complete overlay save system matching ZScream's functionality
- **Custom ASM Integration**: Proper handling of ZSCustomOverworld ASM versions 1-3+
- **Memory Layout**: Correct usage of expanded vs vanilla memory locations based on ROM type
## 0.3.0 (September 2025)
### Major Features
- **Complete Theme Management System**: 5+ built-in themes with custom theme creation and editing
- **Multi-Session Workspace**: Work with multiple ROMs simultaneously in enhanced docked interface
- **Enhanced Welcome Screen**: Themed interface with quick access to all editors and features
- **Asar 65816 Assembler Integration**: Complete cross-platform ROM patching with assembly code
- **ZSCustomOverworld v3**: Full integration with enhanced overworld editing capabilities
- **Advanced Message Editing**: Enhanced text editing interface with improved parsing and real-time preview
- **GUI Docking System**: Improved docking and workspace management for better user workflow
- **Symbol Extraction**: Extract symbol names and opcodes from assembly files
- **Modernized Build System**: Upgraded to CMake 3.16+ with target-based configuration
### User Interface & Theming
- **Built-in Themes**: Classic YAZE, YAZE Tre, Cyberpunk, Sunset, Forest, and Midnight themes
- **Theme Editor**: Complete custom theme creation with save-to-file functionality
- **Animated Background Grid**: Optional moving grid with color breathing effects
- **Theme Import/Export**: Share custom themes with the community
- **Responsive UI**: All UI elements properly adapt to selected themes
### Enhancements
- **Enhanced CLI Tools**: Improved z3ed with modern command line interface and TUI
- **CMakePresets**: Added development workflow presets for better productivity
- **Cross-Platform CI/CD**: Multi-platform automated builds and testing with lenient code quality checks
- **Professional Packaging**: NSIS, DMG, and DEB/RPM installers
- **ROM-Dependent Testing**: Separated testing infrastructure for CI compatibility with 46+ core tests
- **Comprehensive Documentation**: Updated guides, help menus, and API documentation
### Technical Improvements
- **Modern C++23**: Latest language features for performance and safety
- **Memory Safety**: Enhanced memory management with RAII and smart pointers
- **Error Handling**: Improved error handling using absl::Status throughout
- **Cross-Platform**: Consistent experience across Windows, macOS, and Linux
- **Performance**: Optimized rendering and data processing
### Bug Fixes
- **Graphics Arena Crash**: Fixed double-free error during Arena singleton destruction
- **SNES Tile Format**: Corrected tile unpacking algorithm based on SnesLab documentation
- **Palette System**: Fixed color conversion functions (ImVec4 float to uint8_t conversion)
- **CI/CD**: Fixed missing cstring include for Ubuntu compilation
- **ROM Loading**: Fixed file path issues in tests
## 0.2.2 (December 2024)
### Core Features
- DungeonMap editing improvements
- ZSCustomOverworld support
- Cross platform file handling
## 0.2.1 (August 2024)
- Improved MessageEditor parsing
- Added integration test window
- Bitmap bug fixes
## 0.2.0 (July 2024)
- iOS app support
- Graphics Sheet Browser
- Project Files
## 0.1.0 (May 2024)
- Bitmap bug fixes
- Error handling improvements
## 0.0.9 (April 2024)
- Documentation updates
- Entrance tile types
- Emulator subsystem overhaul
## 0.0.8 (February 2024)
- Hyrule Magic Compression
- Dungeon Room Entrances
- PNG Export
## 0.0.7 (January 2024)
- OverworldEntities
- Entrances
- Exits
- Items
- Sprites
## 0.0.6 (November 2023)
- ScreenEditor DungeonMap
- Tile16 Editor
- Canvas updates
## 0.0.5 (November 2023)
- DungeonEditor
- DungeonObjectRenderer
## 0.0.4 (November 2023)
- Tile16Editor
- GfxGroupEditor
- Add GfxGroups functions to Rom
- Add Tile16Editor and GfxGroupEditor to OverworldEditor
## 0.0.3 (October 2023)
- Emulator subsystem
- SNES PPU and PPURegisters

261
docs/public/reference/rom-reference.md Normal file
View File

@@ -0,0 +1,261 @@
# A Link to the Past ROM Reference
## Graphics System
### Graphics Sheets
The ROM contains 223 graphics sheets, each 128x32 pixels (16 tiles × 4 tiles, 8×8 tile size):
**Sheet Categories**:
- Sheets 0-112: Overworld/Dungeon graphics (compressed 3BPP)
- Sheets 113-114: Uncompressed 2BPP graphics
- Sheets 115-126: Uncompressed 3BPP graphics
- Sheets 127-217: Additional dungeon/sprite graphics
- Sheets 218-222: Menu/HUD graphics (2BPP)
**Graphics Format**:
- 3BPP (3 bits per pixel): 8 colors per tile, commonly used for backgrounds
- 2BPP (2 bits per pixel): 4 colors per tile, used for fonts and simple graphics
### Palette System
#### Color Format
SNES uses 15-bit BGR555 color format:
- 2 bytes per color
- Format: `0BBB BBGG GGGR RRRR`
- Each channel: 0-31 (5 bits)
- Total possible colors: 32,768
**Conversion Formulas:**
```cpp
// From BGR555 to RGB888 (0-255 range)
uint8_t red = (snes_color & 0x1F) * 8;
uint8_t green = ((snes_color >> 5) & 0x1F) * 8;
uint8_t blue = ((snes_color >> 10) & 0x1F) * 8;
// From RGB888 to BGR555
uint16_t snes_color = ((r / 8) & 0x1F) |
(((g / 8) & 0x1F) << 5) |
(((b / 8) & 0x1F) << 10);
```
#### 16-Color Row Structure
The SNES organizes palettes in **rows of 16 colors**:
- Each row starts with a **transparent color** at indices 0, 16, 32, 48, 64, etc.
- Palettes must respect these boundaries for proper display
- Multiple sub-palettes can share a row if they're 8 colors or less
#### Palette Groups
**Dungeon Palettes** (0xDD734):
- 20 palettes × 90 colors each
- Structure: 5 full rows (0-79) + 10 colors (80-89)
- Transparent at: indices 0, 16, 32, 48, 64
- Distributed as: BG1 colors, BG2 colors, sprite colors
- Applied per-room via palette ID
**Overworld Palettes**:
- Main: 35 colors per palette (0xDE6C8)
- Structure: 2 full rows (0-15, 16-31) + 3 colors (32-34)
- Transparent at: 0, 16
- Auxiliary: 21 colors per palette (0xDE86C)
- Structure: 1 full row (0-15) + 5 colors (16-20)
- Transparent at: 0
- Animated: 7 colors per palette (0xDE604)
- Overlay palette (no transparent)
**Sprite Palettes**:
- Global sprites: 2 palettes of 60 colors each
- Light World: 0xDD218
- Dark World: 0xDD290
- Structure: 4 rows per set (0-15, 16-31, 32-47, 48-59)
- Transparent at: 0, 16, 32, 48
- Auxiliary 1: 12 palettes × 7 colors (0xDD39E)
- Auxiliary 2: 11 palettes × 7 colors (0xDD446)
- Auxiliary 3: 24 palettes × 7 colors (0xDD4E0)
- Note: Aux palettes store 7 colors; transparent added at runtime
**Other Palettes**:
- HUD: 2 palettes × 32 colors (0xDD218)
- Structure: 2 full rows (0-15, 16-31)
- Transparent at: 0, 16
- Armors: 5 palettes × 15 colors (0xDD630)
- 15 colors + transparent at runtime = full 16-color row
- Swords: 4 palettes × 3 colors (overlay, no transparent)
- Shields: 3 palettes × 4 colors (overlay, no transparent)
- Grass: 3 individual hardcoded colors (LW, DW, Special)
- 3D Objects: 2 palettes × 8 colors (Triforce, Crystal)
- Overworld Mini Map: 2 palettes × 128 colors
- Structure: 8 full rows (0-127)
- Transparent at: 0, 16, 32, 48, 64, 80, 96, 112
#### Palette Application to Graphics
**8-Color Sub-Palettes:**
- Index 0: Transparent (not rendered)
- Indices 1-7: Visible colors
- Each tile/sprite references a specific 8-color sub-palette
**Graphics Format:**
- 2BPP: 4 colors per tile (uses indices 0-3)
- 3BPP: 8 colors per tile (uses indices 0-7)
- 4BPP: 16 colors per tile (uses indices 0-15)
**Rendering Process:**
1. Load compressed graphics sheet from ROM
2. Decompress and convert to indexed pixels (values 0-7 for 3BPP)
3. Select appropriate palette group for room/area
4. Map pixel indices to actual colors from palette
5. Upload to GPU texture
## Dungeon System
### Room Data Structure
**Room Objects** (3 bytes each):
```
Byte 1: YYYXXXXX
YYY = Y coordinate (0-31)
XXXXX = X coordinate (0-31) + layer flag (bit 5)
Byte 2: SSSSSSSS
Size byte (interpretation varies by object type)
Byte 3: IIIIIIII
Object ID or ID component (0-255)
```
**Object Patterns**:
- 0x34: 1x1 solid block
- 0x00-0x08: Rightward 2x2 expansion
- 0x60-0x68: Downward 2x2 expansion
- 0x09-0x14: Diagonal acute (/)
- 0x15-0x20: Diagonal grave (\)
- 0x33, 0x70-0x71: 4x4 blocks
### Tile16 Format
Each Tile16 is composed of four 8x8 tiles:
```
[Top-Left] [Top-Right]
[Bot-Left] [Bot-Right]
```
Stored as 8 bytes (2 bytes per 8x8 tile):
```cpp
// 16-bit tile info: vhopppcccccccccc
// v = vertical flip
// h = horizontal flip
// o = priority
// ppp = palette index (0-7)
// cccccccccc = tile ID (0-1023)
```
### Blocksets
Each dungeon room uses a blockset defining which graphics sheets are loaded:
- 16 sheets per blockset
- First 8: Background graphics
- Last 8: Sprite graphics
**Blockset Pointer**: 0x0FFC40
## Message System
### Text Data Locations
- Text Block 1: 0xE0000 - 0xE7FFF (32KB)
- Text Block 2: 0x75F40 - 0x773FF (5.3KB)
- Dictionary Pointers: 0x74703
### Character Encoding
- 0x00-0x66: Standard characters (A-Z, a-z, 0-9, punctuation)
- 0x67-0x80: Text commands (line breaks, colors, window controls)
- 0x88-0xE8: Dictionary references (compressed common words)
- 0x7F: Message terminator
### Text Commands
- 0x6A: Player name insertion
- 0x6B: Window border (+ 1 argument byte)
- 0x73: Scroll text vertically
- 0x74-0x76: Line 1, 2, 3 markers
- 0x77: Text color (+ 1 argument byte)
- 0x7E: Wait for key press
### Font Graphics
- Location: 0x70000
- Format: 2BPP
- Size: 8KB (0x2000 bytes)
- Organized as 8x8 tiles for characters
## Overworld System
### Map Structure
- 3 worlds: Light, Dark, Special
- Each world: 8×8 grid of 512×512 pixel maps
- Total maps: 192 (64 per world)
### Area Sizes (ZSCustomOverworld v3+)
- Small: 512×512 (1 map)
- Wide: 1024×512 (2 maps horizontal)
- Tall: 512×1024 (2 maps vertical)
- Large: 1024×1024 (4 maps in 2×2 grid)
### Tile Format
- Map uses Tile16 (16×16 pixel tiles)
- Each Tile16 composed of four 8x8 tiles
- 32×32 Tile16s per 512×512 map
## Compression
### LC-LZ2 Algorithm
Both graphics and map data use a variant of LZ compression:
**Commands**:
- 0x00-0x1F: Direct copy (copy N bytes verbatim)
- 0x20-0x3F: Byte fill (repeat 1 byte N times)
- 0x40-0x5F: Word fill (repeat 2 bytes N times)
- 0x60-0x7F: Incremental fill (0x05, 0x06, 0x07...)
- 0x80-0xFF: LZ copy (copy from earlier in buffer)
**Extended Headers** (for lengths > 31):
- 0xE0-0xFF: Extended command headers
- Allows compression of up to 1024 bytes
**Modes**:
- Mode 1 (Nintendo): Graphics decompression (byte order variation)
- Mode 2 (Nintendo): Overworld decompression (byte order variation)
## Memory Map
### ROM Banks (LoROM)
- Bank 0x00-0x7F: ROM data
- Bank 0x80-0xFF: Mirror of 0x00-0x7F
### Important ROM Locations
```
Graphics:
0x080000+ Link graphics (uncompressed)
0x0F8000+ Dungeon object subtype tables
0x0FB373 Object subtype 1 table
0x0FFC40 Blockset pointers
Palettes:
0x0DD218 HUD palettes
0x0DD308 Global sprite palettes
0x0DD734 Dungeon main palettes
0x0DE6C8 Overworld main palettes
0x0DE604 Animated palettes
Text:
0x070000 Font graphics
0x0E0000 Main text data
0x075F40 Extended text data
Tile Data:
0x0F8000 Tile16 data (4032 tiles)
```
---
**Last Updated**: October 13, 2025

116
docs/public/usage/dungeon-editor.md Normal file
View File

@@ -0,0 +1,116 @@
# F2: Dungeon Editor v2 Guide
**Scope**: DungeonEditorV2 (card-based UI), DungeonEditorSystem, dungeon canvases
**Related**: [Architecture Overview](../developer/architecture.md), [Canvas System](../developer/canvas-system.md)
---
## 1. Overview
The Dungeon Editor ships with the multi-card workspace introduced in the 0.3.x releases.
Self-contained room buffers keep graphics, objects, and palettes isolated so you can switch between
rooms without invalidating the entire renderer.
### Key Features
- 512×512 canvas per room with pan/zoom, grid, and collision overlays.
- Layer-specific visualization (BG1/BG2 toggles, colored object outlines, slot labels).
- Modular cards for rooms, objects, palettes, entrances, and toolsets.
- Undo/Redo shared across cards via `DungeonEditorSystem`.
- Tight overworld integration: double-click an entrance to open the linked dungeon room.
---
## 2. Architecture Snapshot
```
DungeonEditorV2 (UI)
├─ Cards & docking
├─ Canvas presenter
└─ Menu + toolbar actions
DungeonEditorSystem (Backend)
├─ Room/session state
├─ Undo/Redo stack
├─ Sprite/entrance/item helpers
└─ Persistence + ROM writes
Room Model (Data)
├─ bg1_buffer_, bg2_buffer_
├─ tile_objects_, door data, metadata
└─ Palette + blockset caches
```
### Room Rendering Pipeline
1. **Load** – `DungeonRoomLoader` reads the room header, blockset pointers, and door/entrance
metadata, producing a `Room` instance with immutable layout info.
2. **Decode** – The requested blockset is converted into `current_gfx16_` bitmaps; objects are parsed
into `tile_objects_` grouped by layer and palette slot.
3. **Draw** – `DungeonCanvasViewer` builds BG1/BG2 bitmaps, then overlays each object layer via
`ObjectDrawer`. Palette state comes from the room’s 90-color dungeon palette.
4. **Queue** – The finished bitmaps are pushed into the graphics `Arena`, which uploads a bounded
number of textures per frame so UI latency stays flat.
5. **Present** – When textures become available, the canvas displays the layers, draws interaction
widgets (selection rectangles, door gizmos, entity labels), and applies zoom/grid settings.
Changing tiles, palettes, or objects invalidates the affected room cache so steps 2–5 rerun only for
that room.
---
## 3. Editing Workflow
### Opening Rooms
1. Launch `yaze` with a ROM (`./build/bin/yaze --rom_file=zelda3.sfc`).
2. Use the **Room Matrix** or **Rooms List** card to choose a room. The toolbar “+” button also opens
the selector.
3. Pin multiple rooms by opening them in separate cards; each card maintains its own canvas state.
### Working with Cards
| Card | Purpose |
|------|---------|
| **Room Graphics** | Primary canvas, BG toggles, collision/grid switches. |
| **Object Editor** | Filter by type/layer, edit coordinates, duplicate/delete objects. |
| **Palette Editor** | Adjust per-room palette slots and preview results immediately. |
| **Entrances List** | Jump between overworld entrances and their mapped rooms. |
| **Room Matrix** | Visual grid of all rooms grouped per dungeon for quick navigation. |
Cards can be docked, detached, or saved as workspace presets; use the sidebar to store favorite
layouts (e.g., Room Graphics + Object Editor + Palette).
### Canvas Interactions
- Left-click to select an object; Shift-click to add to the selection.
- Drag handles to move objects or use the property grid for precise coordinates.
- Right-click to open the context menu, which includes quick inserts for common objects and a “jump
to entrance” helper.
- Hold Space to pan, use mouse wheel (or trackpad pinch) to zoom. The status footer shows current
zoom and cursor coordinates.
- Enable **Object Labels** from the toolbar to show layer-colored labels (e.g., `L1 Chest 0x23`).
### Saving & Undo
- The editor queues every change through `DungeonEditorSystem`. Use `Cmd/Ctrl+Z` and `Cmd/Ctrl+Shift+Z`
to undo/redo across cards.
- Saving writes back the room buffers, door metadata, and palettes for the active session. Keep
backups enabled (`File → Options → Experiment Flags`) for safety.
---
## 4. Tips & Troubleshooting
- **Layer sanity**: If objects appear on the wrong layer, check the BG toggles in Room Graphics and
the layer filter in Object Editor—they operate independently.
- **Palette issues**: Palettes are per room. After editing, ensure `Palette Editor` writes the new
values before switching rooms; the status footer confirms pending writes.
- **Door alignment**: Use the entrance/door inspector popup (right-click a door marker) to verify
leads-to IDs without leaving the canvas.
- **Performance**: Large ROMs with many rooms can accumulate textures. If the editor feels sluggish,
close unused room cards; each card releases its textures when closed.
---
## 5. Related Docs
- [Developer Architecture Overview](../developer/architecture.md) – patterns shared across editors.
- [Canvas System Guide](../developer/canvas-system.md) – detailed explanation of canvas usage,
context menus, and popups.
- [Debugging Guide](../developer/debugging-guide.md) – startup flags and logging tips (e.g.,
`--editor=Dungeon --cards="Room 0"` for focused debugging).

554
docs/public/usage/overworld-loading.md Normal file
View File

@@ -0,0 +1,554 @@
# Overworld Loading Guide
This document provides a comprehensive guide to understanding how overworld loading works in both ZScream (C#) and yaze (C++), including the differences between vanilla ROMs and ZSCustomOverworld v2/v3 ROMs.
## Table of Contents
1. [Overview](#overview)
2. [ROM Types and Versions](#rom-types-and-versions)
3. [Overworld Map Structure](#overworld-map-structure)
4. [Loading Process](#loading-process)
5. [ZScream Implementation](#zscream-implementation)
6. [Yaze Implementation](#yaze-implementation)
7. [Key Differences](#key-differences)
8. [Common Issues and Solutions](#common-issues-and-solutions)
## Overview
Both ZScream and yaze are Zelda 3 ROM editors that support editing overworld maps. They handle three main types of ROMs:
- **Vanilla ROMs**: Original Zelda 3 ROMs without modifications
- **ZSCustomOverworld v2**: ROMs with expanded overworld features
- **ZSCustomOverworld v3**: ROMs with additional features like overlays and custom background colors
## ROM Types and Versions
### Version Detection
Both editors detect the ROM version using the same constant:
```cpp
// Address: 0x140145
constexpr int OverworldCustomASMHasBeenApplied = 0x140145;
// Version values:
// 0xFF = Vanilla ROM
// 0x02 = ZSCustomOverworld v2
// 0x03 = ZSCustomOverworld v3
```
### Feature Support by Version
| Feature | Vanilla | v2 | v3 |
|---------|---------|----|----|
| Basic Overworld Maps | | | |
| Area Size Enum | ❌ | ❌ | |
| Main Palette | ❌ | | |
| Custom Background Colors | ❌ | | |
| Subscreen Overlays | | | |
| Animated GFX | ❌ | ❌ | |
| Custom Tile Graphics | ❌ | ❌ | |
| Vanilla Overlays | | | |
**Note:** Subscreen overlays are visual effects (fog, rain, backgrounds, etc.) that are shared between vanilla ROMs and ZSCustomOverworld. ZSCustomOverworld v2+ expands on this by adding support for custom overlay configurations and additional overlay types.
## Overworld Map Structure
### Core Properties
Each overworld map contains the following core properties:
```cpp
class OverworldMap {
// Basic properties
uint8_t index_; // Map index (0-159)
uint8_t parent_; // Parent map ID
uint8_t world_; // World type (0=LW, 1=DW, 2=SW)
uint8_t game_state_; // Game state (0=Beginning, 1=Zelda, 2=Agahnim)
// Graphics and palettes
uint8_t area_graphics_; // Area graphics ID
uint8_t area_palette_; // Area palette ID
uint8_t main_palette_; // Main palette ID (v2+)
std::array<uint8_t, 3> sprite_graphics_; // Sprite graphics IDs
std::array<uint8_t, 3> sprite_palette_; // Sprite palette IDs
// Map properties
uint16_t message_id_; // Message ID
bool mosaic_; // Mosaic effect enabled
bool large_map_; // Is large map (vanilla)
AreaSizeEnum area_size_; // Area size (v3)
// Custom features (v2/v3)
uint16_t area_specific_bg_color_; // Custom background color
uint16_t subscreen_overlay_; // Subscreen overlay ID (references special area maps)
uint8_t animated_gfx_; // Animated graphics ID
std::array<uint8_t, 8> custom_gfx_ids_; // Custom tile graphics
// Overlay support (vanilla and custom)
uint16_t vanilla_overlay_id_; // Vanilla overlay ID
bool has_vanilla_overlay_; // Has vanilla overlay data
std::vector<uint8_t> vanilla_overlay_data_; // Raw overlay data
};
```
## Overlays and Special Area Maps
### Understanding Overlays
Overlays in Zelda 3 are **visual effects** that are displayed over or behind the main overworld map. They include effects like fog, rain, canopy, backgrounds, and other atmospheric elements. Overlays are collections of tile positions and tile IDs that specify where to place specific graphics on the map.
### Special Area Maps (0x80-0x9F)
The special area maps (0x80-0x9F) contain the actual tile data for overlays. These maps store the graphics that overlays reference and use to create visual effects:
- **0x80-0x8F**: Various special area maps containing overlay graphics
- **0x90-0x9F**: Additional special area maps including more overlay graphics
### Overlay ID Mappings
Overlay IDs directly correspond to special area map indices. Common overlay mappings:
| Overlay ID | Special Area Map | Description |
|------------|------------------|-------------|
| 0x0093 | 0x93 | Triforce Room Curtain |
| 0x0094 | 0x94 | Under the Bridge |
| 0x0095 | 0x95 | Sky Background (LW Death Mountain) |
| 0x0096 | 0x96 | Pyramid Background |
| 0x0097 | 0x97 | First Fog Overlay (Master Sword Area) |
| 0x009C | 0x9C | Lava Background (DW Death Mountain) |
| 0x009D | 0x9D | Second Fog Overlay (Lost Woods/Skull Woods) |
| 0x009E | 0x9E | Tree Canopy (Forest) |
| 0x009F | 0x9F | Rain Effect (Misery Mire) |
### Drawing Order
Overlays are drawn in a specific order based on their type:
- **Background Overlays** (0x95, 0x96, 0x9C): Drawn behind the main map tiles
- **Foreground Overlays** (0x9D, 0x97, 0x93, 0x94, 0x9E, 0x9F): Drawn on top of the main map tiles with transparency
### Vanilla Overlay Loading
In vanilla ROMs, overlays are loaded by parsing SNES assembly-like commands that specify tile positions and IDs:
```cpp
absl::Status LoadVanillaOverlay() {
uint8_t asm_version = (*rom_)[OverworldCustomASMHasBeenApplied];
// Only load vanilla overlays for vanilla ROMs
if (asm_version != 0xFF) {
has_vanilla_overlay_ = false;
return absl::OkStatus();
}
// Load overlay pointer for this map
int address = (kOverlayPointersBank << 16) +
((*rom_)[kOverlayPointers + (index_ * 2) + 1] << 8) +
(*rom_)[kOverlayPointers + (index_ * 2)];
// Parse overlay commands:
// LDA #$xxxx - Load tile ID into accumulator
// LDX #$xxxx - Load position into X register
// STA $xxxx - Store tile at position
// STA $xxxx,x - Store tile at position + X
// INC A - Increment accumulator (for sequential tiles)
// JMP $xxxx - Jump to another overlay routine
// END (0x60) - End of overlay data
return absl::OkStatus();
}
```
### Special Area Graphics Loading
Special area maps require special handling for graphics loading:
```cpp
void LoadAreaInfo() {
if (parent_ >= kSpecialWorldMapIdStart) {
// Special World (SW) areas
if (asm_version >= 3 && asm_version != 0xFF) {
// Use expanded sprite tables for v3
sprite_graphics_[0] = (*rom_)[kOverworldSpecialSpriteGfxGroupExpandedTemp +
parent_ - kSpecialWorldMapIdStart];
} else {
// Use original sprite tables for v2/vanilla
sprite_graphics_[0] = (*rom_)[kOverworldSpecialGfxGroup +
parent_ - kSpecialWorldMapIdStart];
}
// Handle special cases for specific maps
if (index_ == 0x88 || index_ == 0x93) {
area_graphics_ = 0x51;
area_palette_ = 0x00;
} else if (index_ == 0x95) {
// Make this the same GFX as LW death mountain areas
area_graphics_ = (*rom_)[kAreaGfxIdPtr + 0x03];
area_palette_ = (*rom_)[kOverworldMapPaletteIds + 0x03];
} else if (index_ == 0x96) {
// Make this the same GFX as pyramid areas
area_graphics_ = (*rom_)[kAreaGfxIdPtr + 0x5B];
area_palette_ = (*rom_)[kOverworldMapPaletteIds + 0x5B];
} else if (index_ == 0x9C) {
// Make this the same GFX as DW death mountain areas
area_graphics_ = (*rom_)[kAreaGfxIdPtr + 0x43];
area_palette_ = (*rom_)[kOverworldMapPaletteIds + 0x43];
}
}
}
```
## Loading Process
### 1. Version Detection
Both editors first detect the ROM version:
```cpp
uint8_t asm_version = rom[OverworldCustomASMHasBeenApplied];
```
### 2. Map Initialization
For each of the 160 overworld maps (0x00-0x9F):
```cpp
// ZScream
var map = new OverworldMap(index, overworld);
// Yaze
OverworldMap map(index, rom);
```
### 3. Property Loading
The loading process varies by ROM version:
#### Vanilla ROMs (asm_version == 0xFF)
```cpp
void LoadAreaInfo() {
// Load from vanilla tables
message_id_ = rom[kOverworldMessageIds + index_ * 2];
area_graphics_ = rom[kOverworldMapGfx + index_];
area_palette_ = rom[kOverworldMapPaletteIds + index_];
// Determine large map status
large_map_ = (rom[kOverworldMapSize + index_] != 0);
// Load vanilla overlay
LoadVanillaOverlay();
}
```
#### ZSCustomOverworld v2/v3
```cpp
void LoadAreaInfo() {
// Use expanded tables for v3
if (asm_version >= 3) {
message_id_ = rom[kOverworldMessagesExpanded + index_ * 2];
area_size_ = static_cast<AreaSizeEnum>(rom[kOverworldScreenSize + index_]);
} else {
message_id_ = rom[kOverworldMessageIds + index_ * 2];
area_size_ = large_map_ ? LargeArea : SmallArea;
}
// Load custom overworld data
LoadCustomOverworldData();
}
```
### 4. Custom Data Loading
For ZSCustomOverworld ROMs:
```cpp
void LoadCustomOverworldData() {
// Load main palette
main_palette_ = rom[OverworldCustomMainPaletteArray + index_];
// Load custom background color
if (rom[OverworldCustomAreaSpecificBGEnabled] != 0) {
area_specific_bg_color_ = rom[OverworldCustomAreaSpecificBGPalette + index_ * 2];
}
// Load v3 features
if (asm_version >= 3) {
subscreen_overlay_ = rom[OverworldCustomSubscreenOverlayArray + index_ * 2];
animated_gfx_ = rom[OverworldCustomAnimatedGFXArray + index_];
// Load custom tile graphics (8 sheets)
for (int i = 0; i < 8; i++) {
custom_gfx_ids_[i] = rom[OverworldCustomTileGFXGroupArray + index_ * 8 + i];
}
}
}
```
## ZScream Implementation
### OverworldMap Constructor
```csharp
public OverworldMap(byte index, Overworld overworld) {
Index = index;
this.overworld = overworld;
// Load area info
LoadAreaInfo();
// Load custom data if available
if (ROM.DATA[Constants.OverworldCustomASMHasBeenApplied] != 0xFF) {
LoadCustomOverworldData();
}
// Build graphics and palette
BuildMap();
}
```
### Key Methods
- `LoadAreaInfo()`: Loads basic map properties from ROM
- `LoadCustomOverworldData()`: Loads ZSCustomOverworld features
- `LoadPalette()`: Loads and processes palette data
- `BuildMap()`: Constructs the final map bitmap
**Note**: ZScream is the original C# implementation that yaze is designed to be compatible with.
## Yaze Implementation
### OverworldMap Constructor
```cpp
OverworldMap::OverworldMap(int index, Rom* rom) : index_(index), rom_(rom) {
LoadAreaInfo();
LoadCustomOverworldData();
SetupCustomTileset(asm_version);
}
```
### Key Methods
- `LoadAreaInfo()`: Loads basic map properties
- `LoadCustomOverworldData()`: Loads ZSCustomOverworld features
- `LoadVanillaOverlay()`: Loads vanilla overlay data
- `LoadPalette()`: Loads and processes palette data
- `BuildTileset()`: Constructs graphics tileset
- `BuildBitmap()`: Creates the final map bitmap
### Mode 7 Tileset Conversion
Mode 7 graphics live at PC `0x0C4000` as 0x4000 bytes of tiled 8×8 pixel data.
Yaze mirrors ZScream’s tiled-to-linear conversion so SDL can consume it:
```cpp
std::array<uint8_t, 0x4000> mode7_raw = rom_->ReadRange(kMode7Tiles, 0x4000);
int pos = 0;
for (int sy = 0; sy < 16 * 1024; sy += 1024) {
for (int sx = 0; sx < 16 * 8; sx += 8) {
for (int y = 0; y < 8 * 128; y += 128) {
for (int x = 0; x < 8; ++x) {
tileset_[x + sx + y + sy] = mode7_raw[pos++];
}
}
}
}
```
The result is a contiguous 128×128 tileset used by both Light and Dark world
maps.
### Interleaved Tilemap Layout
The 64×64 tilemap (4 096 bytes) is interleaved across four 0x400-byte banks
plus a Dark World override. Copying logic mirrors the original IDK/Zarby docs:
```cpp
auto load_quadrant = [&](uint8_t* dest, const uint8_t* left,
const uint8_t* right) {
for (int count = 0, col = 0; count < 0x800; ++count, ++col) {
*dest++ = (col < 32 ? left : right)[count & 0x3FF];
if (col == 63) col = -1; // wrap every 64 tiles
}
};
load_quadrant(lw_map_, p1, p2); // top half
load_quadrant(lw_map_ + 0x800, p3, p4); // bottom half
```
The Dark World map reuses Light World data except for the final quadrant stored
at `+0x1000`.
### Palette Addresses
- Light World palette: `0x055B27` (128 colors)
- Dark World palette: `0x055C27` (128 colors)
- Conversion uses the shared helper discussed in [Palette System Overview](../developer/palette-system-overview.md).
### Custom Map Import/Export
The editor ships binary import/export to accelerate iteration:
```cpp
absl::Status OverworldMap::LoadCustomMap(std::string_view path);
absl::Status OverworldMap::SaveCustomMap(std::string_view path, bool dark_world);
```
- Load expects a raw 4 096-byte tilemap; it replaces the active Light/Dark world
buffer and triggers a redraw.
- Save writes either the Light World tilemap or the Dark World override,
allowing collaboration with external tooling.
### Current Status
**ZSCustomOverworld v2/v3 Support**: Fully implemented and tested
**Vanilla ROM Support**: Complete compatibility maintained
**Overlay System**: Both vanilla and custom overlays supported
**Map Properties System**: Integrated with UI components
**Graphics Loading**: Optimized with caching and performance monitoring
## Key Differences
### 1. Language and Architecture
| Aspect | ZScream | Yaze |
|--------|---------|------|
| Language | C# | C++ |
| Memory Management | Garbage Collected | Manual (RAII) |
| Graphics | System.Drawing | Custom OpenGL |
| UI Framework | WinForms | ImGui |
### 2. Data Structures
**ZScream:**
```csharp
public class OverworldMap {
public byte Index { get; set; }
public AreaSizeEnum AreaSize { get; set; }
public Bitmap GFXBitmap { get; set; }
// ... other properties
}
```
**Yaze:**
```cpp
class OverworldMap {
uint8_t index_;
AreaSizeEnum area_size_;
std::vector<uint8_t> bitmap_data_;
// ... other member variables
};
```
### 3. Error Handling
**ZScream:** Uses exceptions and try-catch blocks
**Yaze:** Uses `absl::Status` return values and `RETURN_IF_ERROR` macros
### 4. Graphics Processing
**ZScream:** Uses .NET's `Bitmap` class and GDI+
**Yaze:** Uses custom `gfx::Bitmap` class with OpenGL textures
## Common Issues and Solutions
### 1. Version Detection Issues
**Problem:** ROM not recognized as ZSCustomOverworld
**Solution:** Check that `OverworldCustomASMHasBeenApplied` is set correctly
### 2. Palette Loading Errors
**Problem:** Maps appear with wrong colors
**Solution:** Verify palette group addresses and 0xFF fallback handling
### 3. Graphics Not Loading
**Problem:** Blank textures or missing graphics
**Solution:** Check graphics buffer bounds and ProcessGraphicsBuffer implementation
### 4. Overlay Issues
**Problem:** Vanilla overlays not displaying
**Solution:**
- Verify overlay pointer addresses and SNES-to-PC conversion
- Ensure special area maps (0x80-0x9F) are properly loaded with correct graphics
- Check that overlay ID mappings are correct (e.g., 0x009D → map 0x9D)
- Verify that overlay preview shows the actual bitmap of the referenced special area map
**Problem:** Overlay preview showing incorrect information
**Solution:** Ensure overlay preview correctly maps overlay IDs to special area map indices and displays the appropriate bitmap from the special area maps (0x80-0x9F)
### 5. Large Map Problems
**Problem:** Large maps not rendering correctly
**Solution:** Check parent-child relationships and large map detection logic
### 6. Special Area Graphics Issues
**Problem:** Special area maps (0x80-0x9F) showing blank or incorrect graphics
**Solution:**
- Verify special area graphics loading in `LoadAreaInfo()`
- Check that special cases for maps like 0x88, 0x93, 0x95, 0x96, 0x9C are handled correctly
- Ensure proper sprite graphics table selection for v2 vs v3 ROMs
- Verify that special area maps use the correct graphics from referenced LW/DW maps
## Best Practices
### 1. Version-Specific Code
Always check the ASM version before accessing version-specific features:
```cpp
uint8_t asm_version = (*rom_)[OverworldCustomASMHasBeenApplied];
if (asm_version >= 3) {
// v3 features
} else if (asm_version == 0xFF) {
// Vanilla features
}
```
### 2. Error Handling
Use proper error handling for ROM operations:
```cpp
absl::Status LoadPalette() {
RETURN_IF_ERROR(LoadPaletteData());
RETURN_IF_ERROR(ProcessPalette());
return absl::OkStatus();
}
```
### 3. Memory Management
Be careful with memory management in C++:
```cpp
// Good: RAII and smart pointers
std::vector<uint8_t> data;
std::unique_ptr<OverworldMap> map;
// Bad: Raw pointers without cleanup
uint8_t* raw_data = new uint8_t[size];
OverworldMap* map = new OverworldMap();
```
### 4. Thread Safety
Both editors use threading for performance:
```cpp
// Yaze: Use std::async for parallel processing
auto future = std::async(std::launch::async, [this](int map_index) {
RefreshChildMap(map_index);
}, map_index);
```
## Conclusion
Understanding the differences between ZScream and yaze implementations is crucial for maintaining compatibility and adding new features. Both editors follow similar patterns but use different approaches due to their respective languages and architectures.
The key is to maintain the same ROM data structure understanding while adapting to each editor's specific implementation patterns.

107
docs/public/usage/z3ed-cli.md Normal file
View File

@@ -0,0 +1,107 @@
# z3ed CLI Guide
_Last reviewed: November 2025. `z3ed` ships alongside the main editor in every `*-ai` preset and
runs on Windows, macOS, and Linux._
`z3ed` exposes the same ROM-editing capabilities as the GUI but in a scriptable form. Use it to
apply patches, inspect resources, run batch conversions, or drive the AI-assisted workflows that
feed the in-editor proposals.
## 1. Building & Configuration
```bash
# Enable the agent/CLI toolchain
cmake --preset mac-ai
cmake --build --preset mac-ai --target z3ed
# Run the text UI (FTXUI)
./build/bin/z3ed --tui
```
The AI features require at least one provider:
- **Ollama (local)** – install via `brew install ollama`, run `ollama serve`, then set
`OLLAMA_MODEL=qwen2.5-coder:0.5b` (the lightweight default used in CI) or any other supported
model. Pass `--ai_model "$OLLAMA_MODEL"` on the CLI to override per-run.
- **Gemini (cloud)** – export `GEMINI_API_KEY` before launching `z3ed`.
If no provider is configured the CLI still works, but agent subcommands will fall back to manual
plans.
## 2. Everyday Commands
| Task | Example |
| --- | --- |
| Apply an Asar patch | `z3ed asar patch.asm --rom zelda3.sfc` |
| Export all sprites from a dungeon | `z3ed dungeon list-sprites --dungeon 2 --rom zelda3.sfc --format json` |
| Inspect an overworld map | `z3ed overworld describe-map --map 80 --rom zelda3.sfc` |
| Dump palette data | `z3ed palette export --rom zelda3.sfc --output palettes.json` |
| Validate ROM headers | `z3ed rom info --rom zelda3.sfc` |
Pass `--help` after any command to see its flags. Most resource commands follow the
`<noun> <verb>` convention (`overworld set-tile`, `dungeon import-room`, etc.).
## 3. Agent & Proposal Workflow
### 3.1 Interactive Chat
```bash
z3ed agent chat --rom zelda3.sfc --theme overworld
```
- Maintains conversation history on disk so you can pause/resume.
- Supports tool-calling: the agent invokes subcommands (e.g., `overworld describe-map`) and
returns structured diffs.
### 3.2 Plans & Batches
```bash
# Generate a proposal but do not apply it
z3ed agent plan --prompt "Move the eastern palace entrance 3 tiles east" --rom zelda3.sfc
# List pending plans
z3ed agent list
# Apply a plan after review
z3ed agent accept --proposal-id <id> --rom zelda3.sfc
```
Plans store the command transcript, diffs, and metadata inside
`$XDG_DATA_HOME/yaze/proposals/` (or `%APPDATA%\yaze\proposals\`). Review them before applying to
non-sandbox ROMs.
### 3.3 Non-interactive Scripts
```bash
# Run prompts from a file
z3ed agent simple-chat --file scripts/queries.txt --rom zelda3.sfc --stdout
# Feed stdin (useful in CI)
cat <<'PROMPTS' | z3ed agent simple-chat --rom zelda3.sfc --stdout
Describe tile 0x3A in map 0x80.
Suggest palette swaps for dungeon 2.
PROMPTS
```
## 4. Automation Tips
1. **Sandbox first** – point the agent at a copy of your ROM (`--sandbox` flag) so you can review
patches safely.
2. **Log everything** – `--log-file agent.log` captures the provider transcript for auditing.
3. **Structure output** – most list/describe commands support `--format json` or `--format yaml`
for downstream tooling.
4. **Combine with `yaze_test`** – run `./build_ai/bin/yaze_test --unit` after batch patches to
confirm nothing regressed.
5. **Use TUI filters** – in `--tui`, press `:` to open the command palette, type part of a command,
hit Enter, and the tool auto-fills the available flags.
## 5. Troubleshooting
| Symptom | Fix |
| --- | --- |
| `agent chat` hangs after a prompt | Ensure `ollama serve` or the Gemini API key is configured. |
| `libgrpc` or `absl` missing | Re-run the `*-ai` preset; plain debug presets do not pull the agent stack. |
| CLI cannot find the ROM | Use absolute paths or set `YAZE_DEFAULT_ROM=/path/to/zelda3.sfc`. |
| Tool reports "command not found" | Run `z3ed --help` to refresh the command index; stale binaries from older builds lack new verbs. |
| Proposal diffs are empty | Provide `--rom` plus either `--sandbox` or `--workspace` so the agent knows where to stage files. |
## 6. Related Documentation
- `docs/public/developer/testing-without-roms.md` – ROM-less fixtures for CI.
- `docs/public/developer/debugging-guide.md` – logging and instrumentation tips shared between the
GUI and CLI.
- `docs/internal/agents/` – deep dives into the agent architecture and refactor plans (internal
audience only).