backend-infra-engineer: Post v0.3.9-hotfix7 snapshot (build cleanup)

This commit is contained in:
scawful
2025-12-22 00:20:49 +00:00
parent 2934c82b75
commit 5c4cd57ff8
1259 changed files with 239160 additions and 43801 deletions

View File

@@ -1,89 +1,124 @@
# Build Instructions
# Build from Source
yaze uses a modern CMake build system with presets for easy configuration. This guide explains the
environment checks, dependencies, and platform-specific considerations. For concise per-platform
commands, always start with the [Build & Test Quick Reference](quick-reference.md).
YAZE uses a modern CMake build system with presets for easy configuration. This guide covers environment setup, dependencies, and platform-specific considerations.
> **Quick Start:** For concise build commands, see the [Build and Test Quick Reference](quick-reference.md).
---
## 1. Environment Verification
**Before your first build**, run the verification script to ensure your environment is configured correctly.
Before your first build, run the verification script to ensure your environment is configured correctly.
### Windows (PowerShell)
```powershell
.\scripts\verify-build-environment.ps1
# With automatic fixes
.\scripts\verify-build-environment.ps1 -FixIssues
```
> Tip: After verification, run `.\scripts\setup-vcpkg-windows.ps1` to bootstrap vcpkg, ensure `clang-cl`/Ninja are installed, and cache the `x64-windows` triplet.
### macOS & Linux (Bash)
> **Tip:** After verification, run `.\scripts\setup-vcpkg-windows.ps1` to bootstrap vcpkg, install clang-cl/Ninja, and cache the x64-windows triplet.
### macOS and Linux
```bash
./scripts/verify-build-environment.sh
# With automatic fixes
./scripts\verify-build-environment.sh --fix
./scripts/verify-build-environment.sh --fix
```
The script checks for required tools like CMake, a C++23 compiler, and platform-specific dependencies.
The script checks for CMake, a C++23 compiler, and platform-specific dependencies.
---
## 2. Using Presets
- Pick the preset that matches your platform/workflow (debug: `mac-dbg` / `lin-dbg` / `win-dbg`,
AI-enabled: `mac-ai` / `win-ai`, release: `*-rel`, etc.).
- Configure with `cmake --preset <name>` and build with `cmake --build --preset <name> [--target …]`.
- Add `-v` to a preset name (e.g., `mac-dbg-v`) to surface compiler warnings.
- Need a full matrix? See the [CMake Presets Guide](presets.md) for every preset and the quick
reference for ready-to-run command snippets.
Select a preset that matches your platform and workflow:
## Feature Toggles & Windows Profiles
| Workflow | Presets |
|----------|---------|
| Debug builds | `mac-dbg`, `lin-dbg`, `win-dbg` |
| AI-enabled builds | `mac-ai`, `lin-ai`, `win-ai` |
| Release builds | `mac-rel`, `lin-rel`, `win-rel` |
| Development (ROM tests) | `mac-dev`, `lin-dev`, `win-dev` |
**Build Commands:**
```bash
cmake --preset <name> # Configure
cmake --build --preset <name> --target yaze # Build
```
Add `-v` suffix (e.g., `mac-dbg-v`) to enable verbose compiler warnings.
See the [CMake Presets Guide](presets.md) for the complete preset reference.
---
## Feature Toggles
### Windows Presets
| Preset | Purpose |
| --- | --- |
| `win-dbg`, `win-rel`, `ci-windows` | Core builds without agent UI, gRPC, or AI runtimes. Fastest option for MSVC/clang-cl. |
| `win-ai`, `win-vs-ai` | Full agent stack for local development (UI panels + remote automation + AI runtime). |
| `ci-windows-ai` | Nightly/weekly CI preset that exercises the entire automation stack on Windows. |
|--------|---------|
| `win-dbg`, `win-rel` | Core builds without agent UI or AI. Fastest option. |
| `win-ai`, `win-vs-ai` | Full agent stack (UI + automation + AI runtime) |
| `ci-windows-ai` | CI preset for the complete automation stack |
### Agent Feature Flags
### CMake Feature Flags
| Option | Default | Effect |
| --- | --- | --- |
| `YAZE_BUILD_AGENT_UI` | `ON` when `YAZE_BUILD_GUI=ON` | Builds the ImGui widgets used by the chat/agent panels. |
| `YAZE_ENABLE_REMOTE_AUTOMATION` | `ON` for `*-ai` presets | Adds gRPC/protobuf services plus GUI automation clients. |
| `YAZE_ENABLE_AI_RUNTIME` | `ON` for `*-ai` presets | Enables Gemini/Ollama transports, proposal planning, and advanced routing logic. |
| `YAZE_ENABLE_AGENT_CLI` | `ON` when `YAZE_BUILD_CLI=ON` | Compiles the conversational agent stack consumed by `z3ed`. |
| Option | Default | Description |
|--------|---------|-------------|
| `YAZE_BUILD_AGENT_UI` | ON with GUI | ImGui chat/agent panels |
| `YAZE_ENABLE_REMOTE_AUTOMATION` | ON for `*-ai` | gRPC services and automation |
| `YAZE_ENABLE_AI_RUNTIME` | ON for `*-ai` | Gemini/Ollama AI providers |
| `YAZE_ENABLE_AGENT_CLI` | ON with CLI | z3ed agent commands |
Combine these switches to match your workflow: keep everything `OFF` for lightweight GUI hacking or turn them `ON` for automation-heavy work with sketchybar/yabai/skhd, tmux, or remote runners.
Keep features `OFF` for lightweight GUI development, or enable them for automation workflows.
---
## 3. Dependencies
- **Required**: CMake 3.16+, C++23 Compiler (GCC 13+, Clang 16+, MSVC 2019+), Git.
- **Bundled**: All other dependencies (SDL2, ImGui, Asar, nlohmann/json, cpp-httplib, GoogleTest, etc.) live under the `ext/` directory or are managed by CMake's `FetchContent`. No external package manager is required for a basic build.
- **Optional**:
- **gRPC**: For GUI test automation. Can be enabled with `-DYAZE_WITH_GRPC=ON`.
- **vcpkg (Windows)**: Can be used for faster gRPC builds on Windows (optional).
### Required
- **CMake** 3.16 or later
- **C++23 Compiler**: GCC 13+, Clang 16+, or MSVC 2022 17.4+
- **Git**
### Bundled (No Installation Required)
All core dependencies are included in the `ext/` directory or fetched automatically:
- SDL2, ImGui, Asar, nlohmann/json, cpp-httplib, GoogleTest
### Optional
- **gRPC**: Required for GUI automation and AI features. Enable with `-DYAZE_ENABLE_GRPC=ON`.
- **vcpkg (Windows)**: Speeds up gRPC builds on Windows.
---
## 4. Platform Setup
### macOS
```bash
# Install Xcode Command Line Tools
xcode-select --install
# Recommended: Install build tools via Homebrew
# Install build tools via Homebrew
brew install cmake pkg-config
# For sandboxed/offline builds: Install dependencies to avoid network fetch
# Optional: For sandboxed/offline builds
brew install yaml-cpp googletest
```
**Note**: When building in sandboxed/offline environments (e.g., via Claude Code or restricted networks), install `yaml-cpp` and `googletest` via Homebrew to avoid GitHub fetch failures. The build system automatically detects Homebrew installations and uses them as fallback:
- yaml-cpp: `/opt/homebrew/opt/yaml-cpp`, `/usr/local/opt/yaml-cpp`
- googletest: `/opt/homebrew/opt/googletest`, `/usr/local/opt/googletest`
> **Note:** In sandboxed environments, install yaml-cpp and googletest via Homebrew to avoid network fetch failures. The build system auto-detects Homebrew installations at `/opt/homebrew/opt/` (Apple Silicon) or `/usr/local/opt/` (Intel).
### Linux (Ubuntu/Debian)
```bash
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build pkg-config \
@@ -91,15 +126,21 @@ sudo apt-get install -y build-essential cmake ninja-build pkg-config \
```
### Windows
1. **Install Visual Studio 2022** with the “Desktop development with C++” workload (requires MSVC + MSBuild).
2. **Install Ninja** (recommended): `choco install ninja` or enable the “CMake tools for Windows” optional component.
3. Run the verifier: `.\scripts\verify-build-environment.ps1 -FixIssues` this checks Visual Studio workloads, Ninja, clang-cl, Git settings, and vcpkg cache.
4. Bootstrap vcpkg once: `.\scripts\setup-vcpkg-windows.ps1` (prefetches SDL2, yaml-cpp, etc.).
5. Use the `win-*` presets (Ninja) or `win-vs-*` presets (Visual Studio generator) as needed. For AI/gRPC features, prefer `win-ai` / `win-vs-ai`.
6. For quick validation, run the PowerShell helper:
1. **Install Visual Studio 2022** with "Desktop development with C++" workload
2. **Install Ninja** (recommended): `choco install ninja` or via Visual Studio Installer
3. **Run the verifier:**
```powershell
pwsh -File scripts/agents/windows-smoke-build.ps1 -Preset win-ai -Target z3ed
.\scripts\verify-build-environment.ps1 -FixIssues
```
4. **Bootstrap vcpkg:**
```powershell
.\scripts\setup-vcpkg-windows.ps1
```
5. **Build:**
- Use `win-*` presets with Ninja generator
- Use `win-vs-*` presets for Visual Studio IDE
- For AI features, use `win-ai` or `win-vs-ai`
## 5. Testing

View File

@@ -0,0 +1,71 @@
# YAZE Installation Options (Distribution Guide)
Status: Draft
Audience: Users/distributors who want alternatives to direct GitHub release binaries.
## Overview
YAZE is distributed primarily via GitHub release binaries. This guide summarizes current install paths and outlines packaging-friendly options per platform. Use the table to pick what is available today vs. what would require packaging work.
## Platform Matrix
| Platform | Status | Recommended Path | Notes |
|----------|--------|------------------|-------|
| macOS (Intel/Apple) | Available | GitHub release tarball; custom Homebrew tap (see below) | Prefer Apple silicon builds; Intel works under Rosetta. |
| Windows (x64) | Available | GitHub release zip; vcpkg-from-source (community) | No official winget/choco package yet. |
| Linux (x86_64) | Available | GitHub release AppImage (if provided) or build from source | Test on Ubuntu/Debian/Fedora; Wayland users may need XWayland. |
| Web (WASM) | Preview | Hosted demo or local `npm http-server` of `build-wasm` artifact | Requires modern browser; no install. |
## macOS
### 1) Release binary (recommended)
1. Download the macOS tarball from GitHub releases.
2. `tar -xf yaze-<version>-macos.tar.gz && cd yaze-<version>-macos`
3. Run `./yaze.app/Contents/MacOS/yaze` (GUI) or `./bin/z3ed` (CLI).
### 2) Homebrew (custom tap)
- If you publish a tap: `brew tap <your/tap>` then `brew install yaze`.
- Sample formula inputs:
- URL: GitHub release tarball.
- Dependencies: `cmake`, `ninja`, `pkg-config`, `sdl2`, `glew`, `glm`, `ftxui`, `abseil`, `protobuf`, `gtest`.
- For development builds: `cmake --preset mac-dbg` then `cmake --build --preset mac-dbg`.
## Windows
### 1) Release zip (recommended)
1. Download the Windows zip from GitHub releases.
2. Extract to a writable directory.
3. Run `yaze.exe` (GUI) or `z3ed.exe` (CLI) from the `bin` folder.
### 2) vcpkg-from-source (DIY)
If you prefer source builds with vcpkg dependencies:
1. Install vcpkg and integrate: `vcpkg integrate install`.
2. Configure: `cmake --preset win-dbg -DCMAKE_TOOLCHAIN_FILE=%VCPKG_ROOT%/scripts/buildsystems/vcpkg.cmake`.
3. Build: `cmake --build --preset win-dbg`.
4. Run tests (optional): `ctest --test-dir build`.
## Linux
### 1) Release AppImage (if available)
- `chmod +x yaze-<version>-linux.AppImage && ./yaze-<version>-linux.AppImage`
- If graphics fail under Wayland, try `XWAYLAND_FORCE=1 ./yaze-<version>-linux.AppImage`.
### 2) Build from source
Prereqs: `cmake`, `ninja-build`, `pkg-config`, `libsdl2-dev`, `libglew-dev`, `libglm-dev`, `protobuf-compiler`, `libprotobuf-dev`, `libabsl-dev`, `libftxui-dev` (or build from source), `zlib1g-dev`.
```
cmake --preset lin-dbg
cmake --build --preset lin-dbg
ctest --test-dir build -L stable # optional
```
## Web (WASM Preview)
- Use the published web build (if provided) or self-host the `build-wasm` output:
```
cd build-wasm && npx http-server .
```
- Open the local URL in a modern browser; no installation required.
## Packaging Notes
- Prefer static/runtime-complete bundles for end users (AppImage on Linux, app bundle on macOS, zip on Windows).
- When creating packages (Homebrew/Chocolatey/winget), pin the release URL and checksum and align dependencies to the CMake presets (`mac-*/lin-*/win-*`).
- Keep CLI and GUI in the same archive to avoid mismatched versions; CLI entry is `z3ed`, GUI entry is `yaze`.
## Quick Links
- Build quick reference: `docs/public/build/quick-reference.md`
- CMake presets: `CMakePresets.json`
- Tests (optional after build): `ctest --test-dir build -L stable`

View File

@@ -1,6 +1,6 @@
# Platform Compatibility & CI/CD Fixes
# Platform Compatibility & CI/CD
**Last Updated**: October 9, 2025
**Last Updated**: November 27, 2025
---

View File

@@ -114,8 +114,7 @@ By default, all presets suppress compiler warnings with `-w` for a cleaner build
## Build Directories
Most presets use `build/` directory. Exceptions:
- `mac-rooms`: Uses `build_rooms/` to avoid conflicts
Most presets use `build/`. WASM presets use `build-wasm/`. Use `CMakeUserPresets.json` for custom directories.
## Feature Flags

View File

@@ -1,149 +1,185 @@
# Build & Test Quick Reference
# Build and Test Quick Reference
Use this document as the single source of truth for configuring, building, and testing YAZE across
platforms. Other guides (README, CLAUDE.md, GEMINI.md, etc.) should link here instead of duplicating
steps.
This document is the single source of truth for configuring, building, and testing YAZE on all platforms. Other documentation files link here rather than duplicating these instructions.
## 1. Environment Prep
- Clone with submodules: `git clone --recursive https://github.com/scawful/yaze.git`
- Run the verifier once per machine:
- macOS/Linux: `./scripts/verify-build-environment.sh --fix`
- Windows PowerShell: `.\scripts\verify-build-environment.ps1 -FixIssues`
---
## 1. Environment Setup
### Clone the Repository
```bash
git clone --recursive https://github.com/scawful/yaze.git
cd yaze
```
### Verify Your Environment
Run the verification script once per machine to check dependencies and fix common issues:
**macOS / Linux:**
```bash
./scripts/verify-build-environment.sh --fix
```
**Windows (PowerShell):**
```powershell
.\scripts\verify-build-environment.ps1 -FixIssues
```
---
## 2. Build Presets
Use `cmake --preset <name>` followed by `cmake --build --preset <name> [--target …]`.
| Preset | Platform(s) | Notes |
|-------------|-------------|-------|
| `mac-dbg`, `lin-dbg`, `win-dbg` | macOS/Linux/Windows | Standard debug builds, tests on by default. |
| `mac-ai`, `lin-ai`, `win-ai` | macOS/Linux/Windows | Enables gRPC, agent UI, `z3ed`, and AI runtime. |
| `mac-rel`, `lin-rel`, `win-rel` | macOS/Linux/Windows | Optimized release builds. |
| `mac-dev`, `lin-dev`, `win-dev` | macOS/Linux/Windows | Development builds with ROM-dependent tests enabled. |
| `mac-uni` | macOS | Universal binary (ARM64 + x86_64) for distribution. |
| `ci-*` presets | Platform-specific | Mirrors CI matrix; see `CMakePresets.json`. |
YAZE uses CMake presets for consistent builds. Configure with `cmake --preset <name>`, then build with `cmake --build --preset <name>`.
**Verbose builds**: add `-v` suffix (e.g., `mac-dbg-v`, `lin-dbg-v`, `win-dbg-v`) to turn off compiler warning suppression.
### Available Presets
## 3. AI/Assistant Build Policy
- Human developers typically use `build` or `build_test` directories.
- AI assistants **must use dedicated directories** (`build_ai`, `build_agent`, etc.) to avoid
clobbering user builds.
- When enabling AI features, prefer the `*-ai` presets and target only the binaries you need
(`yaze`, `z3ed`, `yaze_test`, …).
- Windows helpers: use `scripts/agents/windows-smoke-build.ps1` for quick builds and `scripts/agents/run-tests.sh` (or its PowerShell equivalent) for test runs so preset + generator settings stay consistent.
| Preset | Platform | Description |
|--------|----------|-------------|
| `mac-dbg`, `lin-dbg`, `win-dbg` | macOS / Linux / Windows | Standard debug builds with tests enabled |
| `mac-ai`, `lin-ai`, `win-ai` | macOS / Linux / Windows | Full AI stack: gRPC, agent UI, z3ed CLI, AI runtime |
| `mac-rel`, `lin-rel`, `win-rel` | macOS / Linux / Windows | Optimized release builds |
| `mac-dev`, `lin-dev`, `win-dev` | macOS / Linux / Windows | Development builds with ROM-dependent tests |
| `mac-uni` | macOS | Universal binary (ARM64 + x86_64) for distribution |
| `mac-test`, `lin-test`, `win-test` | All | Optimized builds for fast test iteration |
| `ci-*` | Platform-specific | CI/CD configurations (see CMakePresets.json) |
## 4. Common Commands
**Tip:** Add `-v` suffix (e.g., `mac-dbg-v`) to enable verbose compiler warnings.
---
## 3. Build Directory Policy
| Build Type | Default Directory |
|------------|-------------------|
| Native (desktop/CLI) | `build/` |
| WASM | `build-wasm/` |
If you need per-user or per-agent isolation, create a local `CMakeUserPresets.json` that points `binaryDir` to a custom path.
Example:
```bash
cp CMakeUserPresets.json.example CMakeUserPresets.json
export YAZE_BUILD_ROOT="$HOME/.cache/yaze"
cmake --preset dev-local
cmake --build --preset dev-local --target yaze
```
For AI-enabled builds, use the `*-ai` presets and specify only the targets you need:
```bash
cmake --build --preset mac-ai --target yaze z3ed
```
**Windows Helper Scripts:**
- Quick builds: `scripts/agents/windows-smoke-build.ps1`
- Test runs: `scripts/agents/run-tests.sh` (or PowerShell equivalent)
---
## 4. Common Build Commands
### Standard Debug Build
**macOS:**
```bash
# Debug GUI build (macOS)
cmake --preset mac-dbg
cmake --build --preset mac-dbg --target yaze
```
# Debug GUI build (Linux)
**Linux:**
```bash
cmake --preset lin-dbg
cmake --build --preset lin-dbg --target yaze
```
# Debug GUI build (Windows)
**Windows:**
```bash
cmake --preset win-dbg
cmake --build --preset win-dbg --target yaze
```
# AI-enabled build with gRPC (macOS)
### AI-Enabled Build (with gRPC and z3ed CLI)
**macOS:**
```bash
cmake --preset mac-ai
cmake --build --preset mac-ai --target yaze z3ed
```
# AI-enabled build with gRPC (Linux)
**Linux:**
```bash
cmake --preset lin-ai
cmake --build --preset lin-ai --target yaze z3ed
```
# AI-enabled build with gRPC (Windows)
**Windows:**
```bash
cmake --preset win-ai
cmake --build --preset win-ai --target yaze z3ed
```
---
## 5. Testing
### Default Tests (Always Available)
YAZE uses CTest with GoogleTest. Tests are organized by category using labels.
Default test suites run automatically with debug/dev presets. Include stable unit/integration tests and GUI smoke tests:
### Quick Start
```bash
# Build stable test suite (always included in debug presets)
cmake --build --preset mac-dbg --target yaze_test_stable
# Run stable tests (fast, no ROM required)
ctest --test-dir build -L stable -j4
# Run with ctest (recommended approach)
ctest --preset mac-dbg -L stable # Stable tests only
ctest --preset mac-dbg -L gui # GUI smoke tests
ctest --test-dir build -L "stable|gui" # Both stable + GUI
# Run all enabled tests
ctest --test-dir build --output-on-failure
# Run tests matching a pattern
ctest --test-dir build -R "Dungeon"
```
### Optional: ROM-Dependent Tests
### Test Categories
For tests requiring Zelda3 ROM file (ASAR ROM tests, complete edit workflows, ZSCustomOverworld upgrades):
| Category | Command | Description |
|----------|---------|-------------|
| Stable | `ctest --test-dir build -L stable` | Core unit tests, always available |
| GUI | `ctest --test-dir build -L gui` | GUI smoke tests |
| ROM-dependent | `ctest --test-dir build -L rom_dependent` | Requires a Zelda 3 ROM |
| Experimental | `ctest --test-dir build -L experimental` | AI/experimental features |
### Enabling ROM-Dependent Tests
```bash
# Configure with ROM path
cmake --preset mac-dbg -DYAZE_ENABLE_ROM_TESTS=ON -DYAZE_TEST_ROM_PATH=~/zelda3.sfc
cmake --preset mac-dev -DYAZE_TEST_ROM_PATH=/path/to/zelda3.sfc
# Build ROM test suite
cmake --build --preset mac-dbg --target yaze_test_rom_dependent
# Run ROM tests
# Build and run
cmake --build --preset mac-dev --target yaze_test
ctest --test-dir build -L rom_dependent
```
### Optional: Experimental AI Tests
For AI-powered feature tests (requires `YAZE_ENABLE_AI_RUNTIME=ON`):
```bash
# Use AI-enabled preset
cmake --preset mac-ai
# Build experimental test suite
cmake --build --preset mac-ai --target yaze_test_experimental
# Run AI tests
ctest --test-dir build -L experimental
```
### Test Commands Reference
```bash
# Stable tests only (recommended for quick iteration)
ctest --test-dir build -L stable -j4
# All enabled tests (respects preset configuration)
ctest --test-dir build --output-on-failure
# GUI smoke tests
ctest --test-dir build -L gui
# Headless GUI tests (CI mode)
ctest --test-dir build -L headless_gui
# Tests matching pattern
ctest --test-dir build -R "Dungeon"
# Verbose output
ctest --test-dir build --verbose
```
### Test Organization by Preset
### Test Coverage by Preset
| Preset | Stable | GUI | ROM-Dep | Experimental |
|--------|--------|-----|---------|--------------|
| `mac-dbg`, `lin-dbg`, `win-dbg` | Yes | Yes | No | No |
| `mac-ai`, `lin-ai`, `win-ai` | Yes | Yes | No | Yes |
| `mac-dev`, `lin-dev`, `win-dev` | Yes | Yes | Yes | No |
| `mac-rel`, `lin-rel`, `win-rel` | No | No | No | No |
|--------|:------:|:---:|:-------:|:------------:|
| `*-dbg` | Yes | Yes | No | No |
| `*-ai` | Yes | Yes | No | Yes |
| `*-dev` | Yes | Yes | Yes | No |
| `*-rel` | No | No | No | No |
### Environment Variables
- `YAZE_TEST_ROM_PATH` - Set ROM path for ROM-dependent tests (or use `-DYAZE_TEST_ROM_PATH=...` in CMake)
- `YAZE_SKIP_ROM_TESTS` - Skip ROM tests if set (useful for CI without ROM)
- `YAZE_ENABLE_UI_TESTS` - Enable GUI tests (default if display available)
| Variable | Purpose |
|----------|---------|
| `YAZE_TEST_ROM_PATH` | Path to ROM for ROM-dependent tests |
| `YAZE_SKIP_ROM_TESTS` | Skip ROM tests (useful for CI) |
| `YAZE_ENABLE_UI_TESTS` | Enable GUI tests (auto-detected if display available) |
## 6. Troubleshooting & References
- Detailed troubleshooting: `docs/public/build/troubleshooting.md`
- Platform compatibility: `docs/public/build/platform-compatibility.md`
- Internal agents must follow coordination protocol in
`docs/internal/agents/coordination-board.md` before running builds/tests.
---
## 6. Further Reading
- **[Build Troubleshooting](troubleshooting.md)** - Solutions for common build issues
- **[Platform Compatibility](platform-compatibility.md)** - Platform-specific notes and CI/CD details
- **[CMake Presets Guide](presets.md)** - Complete preset reference
- **[Testing Guide](../developer/testing-guide.md)** - Comprehensive testing documentation