diff --git a/README.md b/README.md index 29476e6e..2cdc179f 100644 --- a/README.md +++ b/README.md @@ -60,26 +60,26 @@ All bundled third-party code (SDL, ImGui, ImGui Test Engine, Asar, nlohmann/json - **`./build/bin/yaze`** – full GUI editor with multi-session dockspace, theming, and ROM patching. - **Web App (Preview)** – browser-based editor at your deployed instance; see [`docs/public/usage/web-app.md`](docs/public/usage/web-app.md) for details and limitations. - **`./build/bin/z3ed --tui`** – CLI/TUI companion for scripting, AI-assisted edits, and Asar workflows. -- **`./build_ai/bin/yaze_test --unit|--integration|--e2e`** – structured test runner for quick regression checks. +- **`ctest --test-dir build -L unit|integration|e2e`** – structured test runner for quick regression checks. - **`z3ed` + macOS automation** – pair the CLI with sketchybar/yabai/skhd or Emacs/Spacemacs to drive ROM workflows without opening the GUI. Typical commands: ```bash # Launch GUI with a ROM -./build/bin/yaze zelda3.sfc +./build/bin/yaze roms/alttp_vanilla.sfc # Apply a patch via CLI -./build/bin/z3ed asar patch.asm --rom zelda3.sfc +./build/bin/z3ed asar patch.asm --rom roms/alttp_vanilla.sfc # Run focused tests cmake --build --preset mac-ai --target yaze_test -./build_ai/bin/yaze_test --unit +ctest --test-dir build -L unit ``` ## Testing -- `./build_ai/bin/yaze_test --unit` for fast checks; add `--integration` or `--e2e --show-gui` for broader coverage. +- `ctest --test-dir build -L unit` for fast checks; add `-L integration` or `-L e2e --output-on-failure` for broader coverage. - `ctest --preset dev` mirrors CI’s stable set; `ctest --preset all` runs the full matrix. -- Set `YAZE_TEST_ROM_PATH` or pass `--rom-path` when a test needs a real ROM image. +- Set `YAZE_TEST_ROM_VANILLA` or pass `--rom-vanilla` when a test needs a real ROM image (legacy `--rom-path` still works). ## Documentation - Human-readable docs live under `docs/public/` with an entry point at [`docs/public/index.md`](docs/public/index.md). diff --git a/docs/public/build/build-from-source.md b/docs/public/build/build-from-source.md index ab8afca5..4d5fcce2 100644 --- a/docs/public/build/build-from-source.md +++ b/docs/public/build/build-from-source.md @@ -42,8 +42,10 @@ Select a preset that matches your platform and workflow: |----------|---------| | Debug builds | `mac-dbg`, `lin-dbg`, `win-dbg` | | AI-enabled builds | `mac-ai`, `lin-ai`, `win-ai` | +| AI + system gRPC (macOS) | `mac-ai-fast` | | Release builds | `mac-rel`, `lin-rel`, `win-rel` | | Development (ROM tests) | `mac-dev`, `lin-dev`, `win-dev` | +| Fast test builds | `mac-test`, `lin-test`, `win-test` | **Build Commands:** ```bash @@ -57,6 +59,20 @@ See the [CMake Presets Guide](presets.md) for the complete preset reference. --- +## Build Directory Policy + +By default, native builds live in `build/` and WASM builds live in `build-wasm/`. Keep additional build directories outside the repo to avoid size creep: + +```bash +cp CMakeUserPresets.json.example CMakeUserPresets.json +export YAZE_BUILD_ROOT="$HOME/.cache/yaze" +cmake --preset dev-local +``` + +For scripts and agent-driven builds, set `YAZE_BUILD_DIR` to an external path (for example, `$HOME/.cache/yaze/build`). + +--- + ## Feature Toggles ### Windows Presets @@ -117,6 +133,21 @@ brew install yaml-cpp googletest > **Note:** In sandboxed environments, install yaml-cpp and googletest via Homebrew to avoid network fetch failures. The build system auto-detects Homebrew installations at `/opt/homebrew/opt/` (Apple Silicon) or `/usr/local/opt/` (Intel). +#### Toolchain Options (macOS) + +AppleClang (`/usr/bin/clang`) is the default and most reliable. If you want Homebrew LLVM, use the provided toolchain file so libc++ headers and rpaths are consistent: + +```bash +# AppleClang (recommended) +cmake --preset mac-dbg --fresh -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ + +# Homebrew LLVM +brew install llvm +cmake --preset mac-dbg --fresh -DCMAKE_TOOLCHAIN_FILE=cmake/llvm-brew.toolchain.cmake +``` + +`mac-ai-fast` expects Homebrew gRPC/protobuf/abseil (`brew install grpc protobuf abseil`). + ### Linux (Ubuntu/Debian) ```bash @@ -152,13 +183,13 @@ The easiest way to run tests is with `ctest` presets. ```bash # Configure a development build (enables ROM-dependent tests) -cmake --preset mac-dev -DYAZE_TEST_ROM_PATH=/path/to/your/zelda3.sfc +cmake --preset mac-dev -DYAZE_TEST_ROM_VANILLA_PATH="$PWD/roms/alttp_vanilla.sfc" # Build the tests cmake --build --preset mac-dev --target yaze_test # Run stable tests (fast, run in CI) -ctest --preset dev +ctest --preset stable # Run all tests, including ROM-dependent and experimental ctest --preset all @@ -169,8 +200,11 @@ ctest --preset all You can also run tests by invoking the test executable directly or using CTest with labels. ```bash -# Run all tests via the executable -./build/bin/yaze_test +# Run tests via the executables (multi-config paths on macOS/Windows) +./build/bin/Debug/yaze_emu_test --emu_test_rom=roms/alttp_vanilla.sfc +./build/bin/Debug/yaze_test_stable --rom=roms/alttp_vanilla.sfc +./build/bin/Debug/yaze_test_gui --rom=roms/alttp_vanilla.sfc +./build/bin/Debug/yaze_test_benchmark --rom=roms/alttp_vanilla.sfc # Run only stable tests using CTest labels ctest --test-dir build --label-regex "STABLE" @@ -329,10 +363,10 @@ After pulling major changes or switching branches, your build directory can beco ### Common Issues #### "nlohmann/json.hpp: No such file or directory" -**Cause**: You are building code that requires AI features without using an AI-enabled preset, or your Git submodules are not initialized. +**Cause**: You are building code that requires AI features without using an AI-enabled preset, or CMake has not fetched CPM dependencies yet. **Solution**: 1. Use an AI preset like `win-ai` or `mac-ai`. -2. Ensure submodules are present by running `git submodule update --init --recursive`. +2. Reconfigure to fetch dependencies: `cmake --preset ` (or delete the build folder and re-run). #### "Cannot open file 'yaze.exe': Permission denied" **Cause**: A previous instance of `yaze.exe` is still running in the background. diff --git a/docs/public/build/presets.md b/docs/public/build/presets.md index 970bd21b..fdb1aee1 100644 --- a/docs/public/build/presets.md +++ b/docs/public/build/presets.md @@ -49,45 +49,82 @@ cmake --build --preset win-arm ### macOS Presets -| Preset | Description | Arch | Warnings | Features | -|--------|-------------|------|----------|----------| -| `mac-dbg` | Debug build | ARM64 | Off | Basic | -| `mac-dbg-v` | Debug verbose | ARM64 | On | Basic | -| `mac-rel` | Release | ARM64 | Off | Basic | -| `mac-x64` | Debug x86_64 | x86_64 | Off | Basic | -| `mac-uni` | Universal binary | Both | Off | Basic | -| `mac-dev` | Development | ARM64 | Off | ROM tests | -| `mac-ai` | AI development | ARM64 | Off | z3ed, JSON, gRPC, ROM tests | -| `mac-z3ed` | z3ed CLI | ARM64 | Off | AI agent support | -| `mac-rooms` | Dungeon editor | ARM64 | Off | Minimal build for room editing | +| Preset | Description | Arch | Warnings | Notes | +|--------|-------------|------|----------|-------| +| `mac-dbg` | Debug build | Host | Off | Tests enabled | +| `mac-dbg-v` | Debug build (verbose) | Host | On | Extra warnings | +| `mac-rel` | Release build | Host | Off | LTO enabled | +| `mac-dev` | Development build | Host | Off | ROM tests enabled | +| `mac-ai` | AI + gRPC dev | Host | Off | Agent UI + automation | +| `mac-ai-fast` | AI dev (system gRPC) | Host | Off | `brew install grpc protobuf abseil` | +| `mac-uni` | Universal binary | arm64 + x86_64 | Off | Release packaging | +| `mac-sdl3` | SDL3 build | Host | Off | Experimental | +| `mac-test` | Fast test build | Host | Off | RelWithDebInfo | ### Windows Presets -| Preset | Description | Arch | Warnings | Features | -|--------|-------------|------|----------|----------| -| `win-dbg` | Debug build | x64 | Off | Basic | -| `win-dbg-v` | Debug verbose | x64 | On | Basic | -| `win-rel` | Release | x64 | Off | Basic | -| `win-arm` | Debug ARM64 | ARM64 | Off | Basic | -| `win-arm-rel` | Release ARM64 | ARM64 | Off | Basic | -| `win-dev` | Development | x64 | Off | ROM tests | -| `win-ai` | AI development | x64 | Off | z3ed, JSON, gRPC, ROM tests | -| `win-z3ed` | z3ed CLI | x64 | Off | AI agent support | +| Preset | Description | Arch | Generator | Notes | +|--------|-------------|------|-----------|-------| +| `win-dbg` | Debug build | x64 | Ninja | Tests enabled | +| `win-dbg-v` | Debug build (verbose) | x64 | Ninja | Extra warnings | +| `win-rel` | Release build | x64 | Ninja | LTO enabled | +| `win-dev` | Development build | x64 | Ninja | ROM tests enabled | +| `win-ai` | AI + gRPC dev | x64 | Ninja | Agent UI + automation | +| `win-z3ed` | z3ed CLI | x64 | Ninja | CLI stack only | +| `win-arm` | Debug ARM64 | ARM64 | Ninja | | +| `win-arm-rel` | Release ARM64 | ARM64 | Ninja | | +| `win-vs-dbg` | Debug build | x64 | Visual Studio | | +| `win-vs-rel` | Release build | x64 | Visual Studio | | +| `win-vs-ai` | AI + gRPC dev | x64 | Visual Studio | | +| `win-sdl3` | SDL3 build | x64 | Ninja | Experimental | +| `win-test` | Fast test build | x64 | Ninja | RelWithDebInfo | ### Linux Presets -| Preset | Description | Compiler | Warnings | -|--------|-------------|----------|----------| -| `lin-dbg` | Debug | GCC | Off | -| `lin-clang` | Debug | Clang | Off | +| Preset | Description | Compiler | Notes | +|--------|-------------|----------|-------| +| `lin-dbg` | Debug build | GCC/Clang | Tests enabled | +| `lin-dbg-v` | Debug build (verbose) | GCC/Clang | Extra warnings | +| `lin-rel` | Release build | GCC/Clang | LTO enabled | +| `lin-dev` | Development build | GCC/Clang | ROM tests enabled | +| `lin-ai` | AI + gRPC dev | GCC/Clang | Agent UI + automation | +| `lin-sdl3` | SDL3 build | GCC/Clang | Experimental | +| `lin-test` | Fast test build | GCC/Clang | RelWithDebInfo | -### Special Purpose +### CI Presets | Preset | Description | |--------|-------------| -| `ci` | Continuous integration (no ROM tests) | -| `asan` | AddressSanitizer build | -| `coverage` | Code coverage build | +| `ci-linux` | Linux CI build | +| `ci-macos` | macOS CI build | +| `ci-windows` | Windows CI build | +| `ci-windows-ai` | Windows AI/automation CI build | + +### WASM Presets + +| Preset | Description | +|--------|-------------| +| `wasm-debug` | WebAssembly debug build | +| `wasm-release` | WebAssembly release build | +| `wasm-crash-repro` | Minimal repro build | +| `wasm-ai` | WebAssembly AI build | + +### Test Presets + +Run with `ctest --preset `. + +| Preset | Description | +|--------|-------------| +| `all` | All tests (including ROM-dependent) | +| `stable` | Stable tests only | +| `unit` | Unit tests only | +| `integration` | Integration tests only | +| `fast` | macOS fast test preset | +| `fast-win` | Windows fast test preset | +| `fast-lin` | Linux fast test preset | +| `stable-ai` | Stable tests against `ci-windows-ai` | +| `unit-ai` | Unit tests against `ci-windows-ai` | +| `integration-ai` | Integration tests against `ci-windows-ai` | ## Warning Control @@ -104,9 +141,9 @@ By default, all presets suppress compiler warnings with `-w` for a cleaner build ## Architecture Support ### macOS -- **ARM64 (Apple Silicon)**: `mac-dbg`, `mac-rel`, `mac-ai`, etc. -- **x86_64 (Intel)**: `mac-x64` -- **Universal Binary**: `mac-uni` (both ARM64 + x86_64) +- **Apple Silicon (arm64)**: `mac-dbg`, `mac-rel`, `mac-ai`, `mac-dev`, etc. +- **Intel (x86_64)**: Use the same `mac-*` presets (they target the host arch). +- **Universal Binary**: `mac-uni` (arm64 + x86_64) ### Windows - **x64**: `win-dbg`, `win-rel`, `win-ai`, etc. @@ -114,7 +151,19 @@ By default, all presets suppress compiler warnings with `-w` for a cleaner build ## Build Directories -Most presets use `build/`. WASM presets use `build-wasm/`. Use `CMakeUserPresets.json` for custom directories. +Most presets use `build/`. WASM presets use `build-wasm/`. Keep only these two build directories in the repo to avoid bloat. For isolated agent builds, point `binaryDir` to a path outside the repo via `CMakeUserPresets.json`. + +### Shared Dependency Caches + +To avoid re-downloading dependencies after cleaning build dirs, set shared caches via `CMakeUserPresets.json` (see `CMakeUserPresets.json.example`) or environment variables: + +```bash +export CPM_SOURCE_CACHE="$HOME/.cpm-cache" +export VCPKG_DOWNLOADS="$HOME/.cache/vcpkg/downloads" +export VCPKG_BINARY_SOURCES="clear;files,$HOME/.cache/vcpkg/bincache,readwrite" +``` + +For scripts and the agent build tool, you can also set `YAZE_BUILD_DIR` to an external path (e.g., `$HOME/.cache/yaze/build`) to keep the working tree clean. ## Feature Flags @@ -123,28 +172,29 @@ Common CMake options you can override: ```bash # Enable/disable components -DYAZE_BUILD_TESTS=ON/OFF --DYAZE_BUILD_Z3ED=ON/OFF --DYAZE_BUILD_EMU=ON/OFF --DYAZE_BUILD_APP=ON/OFF +-DYAZE_BUILD_AGENT_UI=ON/OFF # AI features --DZ3ED_AI=ON/OFF --DYAZE_WITH_JSON=ON/OFF --DYAZE_WITH_GRPC=ON/OFF +-DYAZE_ENABLE_AI=ON/OFF +-DYAZE_ENABLE_AI_RUNTIME=ON/OFF +-DYAZE_ENABLE_GRPC=ON/OFF +-DYAZE_PREFER_SYSTEM_GRPC=ON/OFF +-DYAZE_ENABLE_JSON=ON/OFF +-DYAZE_ENABLE_REMOTE_AUTOMATION=ON/OFF # Testing -DYAZE_ENABLE_ROM_TESTS=ON/OFF --DYAZE_TEST_ROM_PATH=/path/to/zelda3.sfc - -# Build optimization --DYAZE_MINIMAL_BUILD=ON/OFF +-DYAZE_TEST_ROM_VANILLA_PATH=/path/to/alttp_vanilla.sfc +-DYAZE_TEST_ROM_EXPANDED_PATH=/path/to/oos168.sfc +-DYAZE_USE_SDL3=ON/OFF +-DYAZE_ENABLE_LTO=ON/OFF ``` ## Examples ### Development with AI features and verbose warnings ```bash -cmake --preset mac-dbg-v -DZ3ED_AI=ON -DYAZE_WITH_GRPC=ON +cmake --preset mac-dbg-v -DYAZE_ENABLE_AI=ON -DYAZE_ENABLE_GRPC=ON -DYAZE_BUILD_AGENT_UI=ON cmake --build --preset mac-dbg-v ``` @@ -157,7 +207,7 @@ cpack --preset mac-uni ### Quick minimal build for testing ```bash -cmake --preset mac-dbg -DYAZE_MINIMAL_BUILD=ON +cmake --preset mac-dbg -DYAZE_ENABLE_AI=OFF -DYAZE_ENABLE_GRPC=OFF -DYAZE_BUILD_AGENT_UI=OFF cmake --build --preset mac-dbg -j12 ``` @@ -181,12 +231,10 @@ Old preset names have been simplified: | `macos-debug` | `mac-dbg` | | `macos-release` | `mac-rel` | | `macos-debug-universal` | `mac-uni` | -| `macos-dungeon-dev` | `mac-rooms` | | `windows-debug` | `win-dbg` | | `windows-release` | `win-rel` | | `windows-arm64-debug` | `win-arm` | | `linux-debug` | `lin-dbg` | -| `linux-clang` | `lin-clang` | ## Troubleshooting @@ -201,5 +249,6 @@ Old preset names have been simplified: - Restart your IDE or LSP server ### Build fails with missing dependencies -- Ensure submodules are initialized: `git submodule update --init --recursive` +- Re-run configure to fetch CPM dependencies: `cmake --preset ` +- If offline, set `CPM_SOURCE_CACHE` to a populated cache directory - For AI features, make sure you have OpenSSL: `brew install openssl` (macOS) diff --git a/docs/public/build/quick-reference.md b/docs/public/build/quick-reference.md index 42b96842..c5c98b17 100644 --- a/docs/public/build/quick-reference.md +++ b/docs/public/build/quick-reference.md @@ -27,23 +27,33 @@ Run the verification script once per machine to check dependencies and fix commo .\scripts\verify-build-environment.ps1 -FixIssues ``` +### macOS Toolchain Selection + +AppleClang (`/usr/bin/clang`) is the default and most reliable choice. If Homebrew LLVM is on your PATH and you hit SDK header errors, pick the toolchain explicitly: + +```bash +# AppleClang (recommended) +cmake --preset mac-dbg --fresh -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ + +# Homebrew LLVM (optional) +cmake --preset mac-dbg --fresh -DCMAKE_TOOLCHAIN_FILE=cmake/llvm-brew.toolchain.cmake +``` + --- ## 2. Build Presets YAZE uses CMake presets for consistent builds. Configure with `cmake --preset `, then build with `cmake --build --preset `. -### Available Presets +### Available Presets (High Level) -| Preset | Platform | Description | -|--------|----------|-------------| -| `mac-dbg`, `lin-dbg`, `win-dbg` | macOS / Linux / Windows | Standard debug builds with tests enabled | -| `mac-ai`, `lin-ai`, `win-ai` | macOS / Linux / Windows | Full AI stack: gRPC, agent UI, z3ed CLI, AI runtime | -| `mac-rel`, `lin-rel`, `win-rel` | macOS / Linux / Windows | Optimized release builds | -| `mac-dev`, `lin-dev`, `win-dev` | macOS / Linux / Windows | Development builds with ROM-dependent tests | -| `mac-uni` | macOS | Universal binary (ARM64 + x86_64) for distribution | -| `mac-test`, `lin-test`, `win-test` | All | Optimized builds for fast test iteration | -| `ci-*` | Platform-specific | CI/CD configurations (see CMakePresets.json) | +- **macOS**: `mac-dbg`, `mac-dbg-v`, `mac-rel`, `mac-dev`, `mac-ai`, `mac-ai-fast`, `mac-uni`, `mac-sdl3`, `mac-test` +- **Windows**: `win-dbg`, `win-dbg-v`, `win-rel`, `win-dev`, `win-ai`, `win-z3ed`, `win-arm`, `win-arm-rel`, `win-vs-dbg`, `win-vs-rel`, `win-vs-ai`, `win-sdl3`, `win-test` +- **Linux**: `lin-dbg`, `lin-dbg-v`, `lin-rel`, `lin-dev`, `lin-ai`, `lin-sdl3`, `lin-test` +- **CI**: `ci-linux`, `ci-macos`, `ci-windows`, `ci-windows-ai` +- **WASM**: `wasm-debug`, `wasm-release`, `wasm-crash-repro`, `wasm-ai` + +`mac-ai-fast` prefers the Homebrew gRPC/protobuf stack for faster configure times (`brew install grpc protobuf abseil`). **Tip:** Add `-v` suffix (e.g., `mac-dbg-v`) to enable verbose compiler warnings. @@ -56,7 +66,9 @@ YAZE uses CMake presets for consistent builds. Configure with `cmake --preset #include +#include #include #include #include @@ -39,8 +40,11 @@ namespace fs = std::filesystem; BuildTool::BuildTool(const BuildConfig& config) : config_(config) { // Ensure build directory is set with a default value - if (config_.build_directory.empty()) { - config_.build_directory = "build_ai"; + const char* env_build_dir = std::getenv("YAZE_BUILD_DIR"); + if (env_build_dir != nullptr && env_build_dir[0] != '\0') { + config_.build_directory = env_build_dir; + } else if (config_.build_directory.empty()) { + config_.build_directory = "build"; } // Convert to absolute path if relative @@ -171,9 +175,13 @@ absl::StatusOr BuildTool::RunTests( absl::StrFormat("ROM file not found: %s", rom_path)); } #ifdef _WIN32 - env_setup = absl::StrFormat("set YAZE_TEST_ROM_PATH=\"%s\" && ", rom_path); + env_setup = absl::StrFormat( + "set YAZE_TEST_ROM_VANILLA=\"%s\" && set YAZE_TEST_ROM_PATH=\"%s\" && ", + rom_path, rom_path); #else - env_setup = absl::StrFormat("YAZE_TEST_ROM_PATH=\"%s\" ", rom_path); + env_setup = absl::StrFormat( + "YAZE_TEST_ROM_VANILLA=\"%s\" YAZE_TEST_ROM_PATH=\"%s\" ", + rom_path, rom_path); #endif } @@ -919,4 +927,4 @@ absl::Status BuildStatusCommandHandler::Execute( } // namespace tools } // namespace agent } // namespace cli -} // namespace yaze \ No newline at end of file +} // namespace yaze diff --git a/src/cli/service/agent/tools/build_tool.h b/src/cli/service/agent/tools/build_tool.h index c488c201..c9c031b3 100644 --- a/src/cli/service/agent/tools/build_tool.h +++ b/src/cli/service/agent/tools/build_tool.h @@ -33,26 +33,26 @@ class BuildTool { public: // Build operation results struct BuildResult { - bool success; + bool success = false; std::string output; std::string error_output; - int exit_code; - std::chrono::seconds duration; + int exit_code = 0; + std::chrono::seconds duration{0}; std::string command_executed; }; // Build status information struct BuildStatus { - bool is_running; + bool is_running = false; std::string current_operation; std::string last_result_summary; - std::chrono::system_clock::time_point start_time; - int progress_percent; // -1 if unknown + std::chrono::system_clock::time_point start_time{}; + int progress_percent = 0; // -1 if unknown }; // Build configuration struct BuildConfig { - std::string build_directory = "build_ai"; // Default AI agent build dir + std::string build_directory = "build"; // Override via YAZE_BUILD_DIR std::chrono::seconds timeout = std::chrono::seconds(600); // 10 min default bool capture_output = true; bool verbose = false; @@ -265,4 +265,4 @@ class BuildStatusCommandHandler : public resources::CommandHandler { } // namespace cli } // namespace yaze -#endif // YAZE_SRC_CLI_SERVICE_AGENT_TOOLS_BUILD_TOOL_H_ \ No newline at end of file +#endif // YAZE_SRC_CLI_SERVICE_AGENT_TOOLS_BUILD_TOOL_H_