backend-infra-engineer: Release v0.3.3 snapshot

This commit is contained in:
scawful
2025-11-21 21:35:50 -05:00
parent 3d71417f62
commit 476dd1cd1c
818 changed files with 65706 additions and 35514 deletions

View File

@@ -0,0 +1,264 @@
# YAZE Build Guide
**Status**: CI/CD Overhaul Complete ✅
**Last Updated**: October 2025
**Platforms**: macOS (ARM64/Intel), Linux, Windows
## Quick Start
### macOS (Apple Silicon)
```bash
# Basic debug build
cmake --preset mac-dbg && cmake --build --preset mac-dbg
# With AI features (z3ed agent, gRPC, JSON)
cmake --preset mac-ai && cmake --build --preset mac-ai
# Release build
cmake --preset mac-rel && cmake --build --preset mac-rel
```
### Linux
```bash
# Debug build
cmake --preset lin-dbg && cmake --build --preset lin-dbg
# With AI features
cmake --preset lin-ai && cmake --build --preset lin-ai
```
### Windows (Visual Studio)
```bash
# Debug build
cmake --preset win-dbg && cmake --build --preset win-dbg
# With AI features
cmake --preset win-ai && cmake --build --preset win-ai
```
## Build System Overview
### CMake Presets
The project uses a streamlined preset system with short, memorable names:
| Preset | Platform | Features | Build Dir |
|--------|----------|----------|-----------|
| `mac-dbg`, `lin-dbg`, `win-dbg` | All | Basic debug builds | `build/` |
| `mac-ai`, `lin-ai`, `win-ai` | All | AI features (z3ed, gRPC, JSON) | `build_ai/` |
| `mac-rel`, `lin-rel`, `win-rel` | All | Release builds | `build/` |
| `mac-dev`, `win-dev` | Desktop | Development with ROM tests | `build/` |
| `mac-uni` | macOS | Universal binary (ARM64+x86_64) | `build/` |
Add `-v` suffix (e.g., `mac-dbg-v`) for verbose compiler warnings.
### Build Configuration
- **C++ Standard**: C++23 (required)
- **Generator**: Ninja Multi-Config (all platforms)
- **Dependencies**: Bundled via Git submodules or CMake FetchContent
- **Optional Features**:
- gRPC: Enable with `-DYAZE_WITH_GRPC=ON` (for GUI automation)
- AI Agent: Enable with `-DZ3ED_AI=ON` (requires JSON and gRPC)
- ROM Tests: Enable with `-DYAZE_ENABLE_ROM_TESTS=ON -DYAZE_TEST_ROM_PATH=/path/to/zelda3.sfc`
## CI/CD Build Fixes (October 2025)
### Issues Resolved
#### 1. CMake Integration ✅
**Problem**: Generator mismatch between `CMakePresets.json` and VSCode settings
**Fixes**:
- Updated `.vscode/settings.json` to use Ninja Multi-Config
- Fixed compile_commands.json path to `build/compile_commands.json`
- Created proper `.vscode/tasks.json` with preset-based tasks
- Updated `scripts/dev-setup.sh` for future setups
#### 2. gRPC Dependency ✅
**Problem**: CPM downloading but not building gRPC targets
**Fixes**:
- Fixed target aliasing for non-namespaced targets (grpc++ → grpc::grpc++)
- Exported `ABSL_TARGETS` for project-wide use
- Added `target_add_protobuf()` function for protobuf code generation
- Fixed protobuf generation paths and working directory
#### 3. Protobuf Code Generation ✅
**Problem**: `.pb.h` and `.grpc.pb.h` files weren't being generated
**Fixes**:
- Changed all `YAZE_WITH_GRPC``YAZE_ENABLE_GRPC` (compile definition vs CMake variable)
- Fixed variable scoping using `CACHE INTERNAL` for functions
- Set up proper include paths for generated files
- All proto files now generate successfully:
- `rom_service.proto`
- `canvas_automation.proto`
- `imgui_test_harness.proto`
- `emulator_service.proto`
#### 4. SDL2 Configuration ✅
**Problem**: SDL.h headers not found
**Fixes**:
- Changed all `SDL_TARGETS``YAZE_SDL2_TARGETS`
- Fixed variable export using `PARENT_SCOPE`
- Added Homebrew SDL2 include path (`/opt/homebrew/opt/sdl2/include/SDL2`)
- Fixed all library targets to link SDL2 properly
#### 5. ImGui Configuration ✅
**Problem**: Conflicting ImGui versions (bundled vs CPM download)
**Fixes**:
- Used bundled ImGui from `ext/imgui/` instead of downloading
- Created proper ImGui static library target
- Added `imgui_stdlib.cpp` for std::string support
- Exported with `PARENT_SCOPE`
#### 6. nlohmann_json Configuration ✅
**Problem**: JSON headers not found
**Fixes**:
- Created `cmake/dependencies/json.cmake`
- Set up bundled `ext/json/`
- Added include directories to all targets that need JSON
#### 7. GTest and GMock ✅
**Problem**: GMock was disabled but test targets required it
**Fixes**:
- Changed `BUILD_GMOCK OFF``BUILD_GMOCK ON` in testing.cmake
- Added verification for both gtest and gmock targets
- Linked all four testing libraries: gtest, gtest_main, gmock, gmock_main
- Built ImGuiTestEngine from bundled source for GUI test automation
### Build Statistics
**Main Application**:
- Compilation Units: 310 targets
- Executable: `build/bin/Debug/yaze.app/Contents/MacOS/yaze` (macOS)
- Size: 120MB (ARM64 Mach-O)
- Status: ✅ Successfully built
**Test Suites**:
- `yaze_test_stable`: 126MB - Unit + Integration tests for CI/CD
- `yaze_test_gui`: 123MB - GUI automation tests
- `yaze_test_experimental`: 121MB - Experimental features
- `yaze_test_benchmark`: 121MB - Performance benchmarks
- Status: ✅ All test executables built successfully
## Test Execution
### Build Tests
```bash
# Build tests
cmake --build build --target yaze_test
# Run all tests
./build/bin/yaze_test
# Run specific categories
./build/bin/yaze_test --unit # Unit tests only
./build/bin/yaze_test --integration # Integration tests
./build/bin/yaze_test --e2e --show-gui # End-to-end GUI tests
# Run with ROM-dependent tests
./build/bin/yaze_test --rom-dependent --rom-path zelda3.sfc
# Run specific test by name
./build/bin/yaze_test "*Asar*"
```
### Using CTest
```bash
# Run all stable tests
ctest --preset stable --output-on-failure
# Run all tests
ctest --preset all --output-on-failure
# Run unit tests only
ctest --preset unit
# Run integration tests only
ctest --preset integration
```
## Platform-Specific Notes
### macOS
- Supports both Apple Silicon (ARM64) and Intel (x86_64)
- Use `mac-uni` preset for universal binaries
- Bundled Abseil used by default to avoid deployment target mismatches
- Requires Xcode Command Line Tools
**ARM64 Considerations**:
- gRPC v1.67.1 is the tested stable version
- Abseil SSE flags are handled automatically
- See docs/BUILD-TROUBLESHOOTING.md for gRPC ARM64 issues
### Windows
- Requires Visual Studio 2022 with "Desktop development with C++" workload
- Run `scripts\verify-build-environment.ps1` before building
- gRPC builds take 15-20 minutes first time (use vcpkg for faster builds)
- Watch for path length limits: Enable long paths with `git config --global core.longpaths true`
**vcpkg Integration**:
- Optional: Use `-DYAZE_USE_VCPKG_GRPC=ON` for pre-built packages
- Faster builds (~5-10 min vs 30-40 min)
- See docs/BUILD-TROUBLESHOOTING.md for vcpkg setup
### Linux
- Requires GCC 13+ or Clang 16+
- Install dependencies: `libgtk-3-dev`, `libdbus-1-dev`, `pkg-config`
- See `.github/workflows/ci.yml` for complete dependency list
## Build Verification
After a successful build, verify:
- ✅ CMake configuration completes successfully
-`compile_commands.json` generated (62,066 lines, 10,344 source files indexed)
- ✅ Main executable links successfully
- ✅ All test executables build successfully
- ✅ IntelliSense working with full codebase indexing
## Troubleshooting
For platform-specific issues, dependency problems, and error resolution, see:
- **docs/BUILD-TROUBLESHOOTING.md** - Comprehensive troubleshooting guide
- **docs/ci-cd/LOCAL-CI-TESTING.md** - Local testing strategies
## Files Modified (CI/CD Overhaul)
### Core Build System (9 files)
1. `cmake/dependencies/grpc.cmake` - gRPC setup, protobuf generation
2. `cmake/dependencies/sdl2.cmake` - SDL2 configuration
3. `cmake/dependencies/imgui.cmake` - ImGui + ImGuiTestEngine
4. `cmake/dependencies/json.cmake` - nlohmann_json setup
5. `cmake/dependencies/testing.cmake` - GTest + GMock
6. `cmake/dependencies.cmake` - Dependency coordination
7. `src/yaze_pch.h` - Removed Abseil includes
8. `CMakeLists.txt` - Top-level configuration
9. `CMakePresets.json` - Preset definitions
### VSCode/CMake Integration (4 files)
10. `.vscode/settings.json` - CMake integration
11. `.vscode/c_cpp_properties.json` - Compile commands path
12. `.vscode/tasks.json` - Build tasks
13. `scripts/dev-setup.sh` - VSCode config generation
### Library Configuration (6 files)
14. `src/app/gfx/gfx_library.cmake` - SDL2 variable names
15. `src/app/net/net_library.cmake` - JSON includes
16. `src/app/app.cmake` - SDL2 targets for macOS
17. `src/app/gui/gui_library.cmake` - SDL2 targets
18. `src/app/emu/emu_library.cmake` - SDL2 targets
19. `src/app/service/grpc_support.cmake` - SDL2 targets
**Total: 26 files modified/created**
## See Also
- **CLAUDE.md** - Project overview and development guidelines
- **docs/BUILD-TROUBLESHOOTING.md** - Platform-specific troubleshooting
- **docs/ci-cd/CI-SETUP.md** - CI/CD pipeline configuration
- **docs/testing/TEST-GUIDE.md** - Testing strategies and execution

View File

@@ -0,0 +1,416 @@
# YAZE Build Guide
## Quick Start
### Prerequisites
- **CMake 3.16+**
- **C++20 compatible compiler** (GCC 12+, Clang 14+, MSVC 19.30+)
- **Ninja** (recommended) or Make
- **Git** (for submodules)
### 3-Command Build
```bash
# 1. Clone and setup
git clone --recursive https://github.com/scawful/yaze.git
cd yaze
# 2. Configure
cmake --preset dev
# 3. Build
cmake --build build
```
That's it! The build system will automatically:
- Download and build all dependencies using CPM.cmake
- Configure the project with optimal settings
- Build the main `yaze` executable and libraries
## Platform-Specific Setup
### Linux (Ubuntu 22.04+)
```bash
# Install dependencies
sudo apt update
sudo apt install -y build-essential ninja-build pkg-config ccache \
libsdl2-dev libyaml-cpp-dev libgtk-3-dev libglew-dev
# Build
cmake --preset dev
cmake --build build
```
### macOS (14+)
```bash
# Install dependencies
brew install cmake ninja pkg-config ccache sdl2 yaml-cpp
# Build
cmake --preset dev
cmake --build build
```
### Windows (10/11)
```powershell
# Install dependencies via vcpkg
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
.\bootstrap-vcpkg.bat
.\vcpkg integrate install
# Install packages
.\vcpkg install sdl2 yaml-cpp
# Build
cmake --preset dev
cmake --build build
```
## Build Presets
YAZE provides several CMake presets for different use cases:
| Preset | Description | Use Case |
|--------|-------------|----------|
| `dev` | Full development build | Local development |
| `ci` | CI build | Continuous integration |
| `release` | Optimized release | Production builds |
| `minimal` | Minimal build | CI without gRPC/AI |
| `coverage` | Debug with coverage | Code coverage analysis |
| `sanitizer` | Debug with sanitizers | Memory debugging |
| `verbose` | Verbose warnings | Development debugging |
### Examples
```bash
# Development build (default)
cmake --preset dev
cmake --build build
# Release build
cmake --preset release
cmake --build build
# Minimal build (no gRPC/AI)
cmake --preset minimal
cmake --build build
# Coverage build
cmake --preset coverage
cmake --build build
```
## Feature Flags
YAZE supports several build-time feature flags:
| Flag | Default | Description |
|------|---------|-------------|
| `YAZE_BUILD_GUI` | ON | Build GUI application |
| `YAZE_BUILD_CLI` | ON | Build CLI tools (z3ed) |
| `YAZE_BUILD_EMU` | ON | Build emulator components |
| `YAZE_BUILD_LIB` | ON | Build static library |
| `YAZE_BUILD_TESTS` | ON | Build test suite |
| `YAZE_ENABLE_GRPC` | ON | Enable gRPC agent support |
| `YAZE_ENABLE_JSON` | ON | Enable JSON support |
| `YAZE_ENABLE_AI` | ON | Enable AI agent features |
| `YAZE_ENABLE_LTO` | OFF | Enable link-time optimization |
| `YAZE_ENABLE_SANITIZERS` | OFF | Enable AddressSanitizer/UBSanitizer |
| `YAZE_ENABLE_COVERAGE` | OFF | Enable code coverage |
| `YAZE_MINIMAL_BUILD` | OFF | Minimal build for CI |
### Custom Configuration
```bash
# Custom build with specific features
cmake -B build -G Ninja \
-DYAZE_ENABLE_GRPC=OFF \
-DYAZE_ENABLE_AI=OFF \
-DYAZE_ENABLE_LTO=ON \
-DCMAKE_BUILD_TYPE=Release
cmake --build build
```
## Testing
### Run All Tests
```bash
# Build with tests
cmake --preset dev
cmake --build build
# Run all tests
cd build
ctest --output-on-failure
```
### Run Specific Test Suites
```bash
# Stable tests only
ctest -L stable
# Unit tests only
ctest -L unit
# Integration tests only
ctest -L integration
# Experimental tests (requires ROM)
ctest -L experimental
```
### Test with ROM
```bash
# Set ROM path
export YAZE_TEST_ROM_PATH=/path/to/zelda3.sfc
# Run ROM-dependent tests
ctest -L experimental
```
## Code Quality
### Formatting
```bash
# Format code
cmake --build build --target yaze-format
# Check formatting
cmake --build build --target yaze-format-check
```
### Static Analysis
```bash
# Run clang-tidy
find src -name "*.cc" | xargs clang-tidy --header-filter='src/.*\.(h|hpp)$'
# Run cppcheck
cppcheck --enable=warning,style,performance src/
```
## Packaging
### Create Packages
```bash
# Build release
cmake --preset release
cmake --build build
# Create packages
cd build
cpack
```
### Platform-Specific Packages
| Platform | Package Types | Command |
|----------|---------------|---------|
| Linux | DEB, TGZ | `cpack -G DEB -G TGZ` |
| macOS | DMG | `cpack -G DragNDrop` |
| Windows | NSIS, ZIP | `cpack -G NSIS -G ZIP` |
## Troubleshooting
### Common Issues
#### 1. CMake Not Found
```bash
# Ubuntu/Debian
sudo apt install cmake
# macOS
brew install cmake
# Windows
# Download from https://cmake.org/download/
```
#### 2. Compiler Not Found
```bash
# Ubuntu/Debian
sudo apt install build-essential
# macOS
xcode-select --install
# Windows
# Install Visual Studio Build Tools
```
#### 3. Dependencies Not Found
```bash
# Clear CPM cache and rebuild
rm -rf ~/.cpm-cache
rm -rf build
cmake --preset dev
cmake --build build
```
#### 4. Build Failures
```bash
# Clean build
rm -rf build
cmake --preset dev
cmake --build build --verbose
# Check logs
cmake --build build 2>&1 | tee build.log
```
#### 5. gRPC Build Issues
```bash
# Use minimal build (no gRPC)
cmake --preset minimal
cmake --build build
# Or disable gRPC explicitly
cmake -B build -DYAZE_ENABLE_GRPC=OFF
cmake --build build
```
### Debug Build
```bash
# Debug build with verbose output
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Debug \
-DYAZE_VERBOSE_BUILD=ON
cmake --build build --verbose
```
### Memory Debugging
```bash
# AddressSanitizer build
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Debug \
-DYAZE_ENABLE_SANITIZERS=ON
cmake --build build
# Run with sanitizer
ASAN_OPTIONS=detect_leaks=1:abort_on_error=1 ./build/bin/yaze
```
## Performance Optimization
### Release Build
```bash
# Optimized release build
cmake --preset release
cmake --build build
```
### Link-Time Optimization
```bash
# LTO build
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DYAZE_ENABLE_LTO=ON
cmake --build build
```
### Unity Builds
```bash
# Unity build (faster compilation)
cmake -B build -G Ninja \
-DYAZE_UNITY_BUILD=ON
cmake --build build
```
## CI/CD
### Local CI Testing
```bash
# Test CI build locally
cmake --preset ci
cmake --build build
# Run CI tests
cd build
ctest -L stable
```
### GitHub Actions
The project includes comprehensive GitHub Actions workflows:
- **CI Pipeline**: Builds and tests on Linux, macOS, Windows
- **Code Quality**: Formatting, linting, static analysis
- **Security**: CodeQL, dependency scanning
- **Release**: Automated packaging and release creation
## Advanced Configuration
### Custom Toolchain
```bash
# Use specific compiler
cmake -B build -G Ninja \
-DCMAKE_C_COMPILER=gcc-12 \
-DCMAKE_CXX_COMPILER=g++-12
cmake --build build
```
### Cross-Compilation
```bash
# Cross-compile for different architecture
cmake -B build -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/linux-gcc.cmake
cmake --build build
```
### Custom Dependencies
```bash
# Use system packages instead of CPM
cmake -B build -G Ninja \
-DYAZE_USE_SYSTEM_DEPS=ON
cmake --build build
```
## Getting Help
- **Issues**: [GitHub Issues](https://github.com/scawful/yaze/issues)
- **Discussions**: [GitHub Discussions](https://github.com/scawful/yaze/discussions)
- **Documentation**: [docs/](docs/)
- **CI Status**: [GitHub Actions](https://github.com/scawful/yaze/actions)
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: `cmake --build build --target yaze-format-check`
5. Submit a pull request
For more details, see [CONTRIBUTING.md](CONTRIBUTING.md).