backend-infra-engineer: Release v0.3.2 snapshot
This commit is contained in:
scawful
@@ -1,55 +1,224 @@
|
||||
# Platform Compatibility Improvements
|
||||
# Platform Compatibility & CI/CD Fixes
|
||||
|
||||
Recent improvements to ensure YAZE works reliably across all supported platforms.
|
||||
**Last Updated**: October 9, 2025
|
||||
|
||||
---
|
||||
|
||||
## Platform-Specific Notes
|
||||
|
||||
### Windows
|
||||
|
||||
**Build System:**
|
||||
- Supported compilers: clang-cl (preferred), MSVC 2022 17.4+
|
||||
- Uses vcpkg for: SDL2, yaml-cpp (fast packages only)
|
||||
- Uses FetchContent for: gRPC v1.75.1, protobuf, abseil, zlib (better caching)
|
||||
|
||||
**Why FetchContent for gRPC?**
|
||||
- vcpkg gRPC v1.71.0 has no pre-built binaries (builds from source: 45-90 min)
|
||||
- FetchContent uses v1.75.1 with Windows compatibility fixes
|
||||
- BoringSSL ASM disabled on Windows (avoids NASM build conflicts with clang-cl)
|
||||
- Better caching in CI/CD (separate cache keys for vcpkg vs FetchContent)
|
||||
- First build: ~10-15 min, subsequent: <1 min (cached)
|
||||
- zlib bundled with gRPC (avoids vcpkg conflicts)
|
||||
|
||||
**Compiler Flags (both clang-cl and MSVC):**
|
||||
- `/bigobj` - Support large object files (required for gRPC)
|
||||
- `/permissive-` - Standards conformance mode
|
||||
- `/wd4267 /wd4244` - Suppress harmless conversion warnings
|
||||
- `/constexpr:depth2048` - Deep template instantiation (MSVC 2019+)
|
||||
|
||||
**Macro Definitions:**
|
||||
- `WIN32_LEAN_AND_MEAN` - Reduce Windows header pollution
|
||||
- `NOMINMAX` - Prevent min/max macro conflicts
|
||||
- `NOGDI` - Prevent GDI macro conflicts (DWORD, etc.)
|
||||
- `__PRFCHWINTRIN_H` - Work around Clang 20 `_m_prefetchw` linkage clash
|
||||
|
||||
**Build Times:**
|
||||
- First build (no cache): ~10-15 minutes
|
||||
- Incremental build (cached): ~3-5 minutes
|
||||
- CI/CD with full caching: ~5-8 minutes
|
||||
|
||||
**CI/CD Configuration:**
|
||||
- Compiler matrix: clang-cl + MSVC
|
||||
- 3-tier caching: vcpkg packages, FetchContent deps, sccache objects
|
||||
- Binary caching via GitHub Actions for vcpkg
|
||||
- Parallel builds: 4 jobs
|
||||
|
||||
### macOS
|
||||
|
||||
**Build System:**
|
||||
- Uses vcpkg for: SDL2, yaml-cpp, zlib
|
||||
- Uses FetchContent for: gRPC, abseil, protobuf
|
||||
|
||||
**Supported Architectures:**
|
||||
- x64 (Intel Macs) - macOS 13+
|
||||
- ARM64 (Apple Silicon) - macOS 14+
|
||||
|
||||
**Build Times:**
|
||||
- First build: ~20-30 minutes
|
||||
- Subsequent builds: ~3-5 minutes
|
||||
|
||||
### Linux
|
||||
|
||||
**Build System:**
|
||||
- Uses system packages (apt) for most dependencies
|
||||
- Does NOT use vcpkg
|
||||
- Uses FetchContent for: gRPC, abseil, protobuf (when not in system)
|
||||
|
||||
**Required System Packages:**
|
||||
```bash
|
||||
build-essential ninja-build pkg-config libglew-dev libxext-dev
|
||||
libwavpack-dev libabsl-dev libboost-all-dev libpng-dev python3-dev
|
||||
libpython3-dev libasound2-dev libpulse-dev libaudio-dev libx11-dev
|
||||
libxrandr-dev libxcursor-dev libxinerama-dev libxi-dev libxss-dev
|
||||
libxxf86vm-dev libxkbcommon-dev libwayland-dev libdecor-0-dev
|
||||
libgtk-3-dev libdbus-1-dev gcc-12 g++-12 clang-15
|
||||
```
|
||||
|
||||
**Build Times:**
|
||||
- First build: ~15-20 minutes
|
||||
- Subsequent builds: ~2-4 minutes
|
||||
|
||||
---
|
||||
|
||||
## Cross-Platform Code Validation
|
||||
|
||||
The following subsystems run unchanged across Windows, macOS, and Linux:
|
||||
|
||||
- Audio backend (`src/app/emu/audio`) uses SDL2 only; no platform branches.
|
||||
- Input backend/manager (`src/app/emu/input`) runs on SDL2 abstractions.
|
||||
- Debug tools (`src/app/emu/debug`) avoid OS-specific headers.
|
||||
- Emulator UI (`src/app/emu/ui`) is pure ImGui + SDL2.
|
||||
|
||||
---
|
||||
|
||||
## Common Build Issues & Solutions
|
||||
|
||||
### Windows: "use of undefined type 'PromiseLike'"
|
||||
**Cause:** Old gRPC version (< v1.67.1)
|
||||
**Fix:** Clear build cache and reconfigure
|
||||
```powershell
|
||||
rm -r build/_deps/grpc-*
|
||||
cmake -B build -G "Visual Studio 17 2022" -A x64
|
||||
```
|
||||
|
||||
### macOS: "absl not found"
|
||||
**Cause:** vcpkg not finding abseil packages
|
||||
**Fix:** Use FetchContent (default) - abseil is fetched automatically by gRPC
|
||||
```bash
|
||||
cmake -B build
|
||||
```
|
||||
|
||||
### Linux: CMake configuration fails
|
||||
**Cause:** Missing system dependencies
|
||||
**Fix:** Install required packages
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y [see package list above]
|
||||
```
|
||||
|
||||
### Windows: "DWORD syntax error"
|
||||
**Cause:** Windows macros conflicting with protobuf enums
|
||||
**Fix:** Ensure `NOGDI` is defined (now automatic in grpc.cmake)
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Validation Checklist
|
||||
|
||||
Before merging platform-specific changes:
|
||||
|
||||
- Confirm the vcpkg baseline matches `vcpkg.json`.
|
||||
- Do not reintroduce the Windows x86 build (cpp-httplib incompatibility).
|
||||
- Keep Windows macro guards (`NOGDI`, `NOMINMAX`, `WIN32_LEAN_AND_MEAN`) in place.
|
||||
- Build against gRPC 1.67.1 with the MSVC workaround flags.
|
||||
- Leave shared code paths on SDL2/ImGui abstractions.
|
||||
- Re-run the full matrix if caches or presets change.
|
||||
|
||||
### CI/CD Performance Roadmap
|
||||
|
||||
- **Dependency caching**: Cache vcpkg installs on Windows plus Homebrew/apt
|
||||
archives to trim 5-10 minutes per job; track cache keys via OS + lockfiles.
|
||||
- **Compiler caching**: Enable `ccache`/`sccache` across the matrix using the
|
||||
`hendrikmuhs/ccache-action` with 500 MB per-run limits for 3-5 minute wins.
|
||||
- **Conditional work**: Add a path-filter job that skips emulator builds or
|
||||
full test runs when only docs or CLI code change; fall back to full matrix on
|
||||
shared components.
|
||||
- **Reusable workflows**: Centralize setup steps (checking out submodules,
|
||||
restoring caches, configuring presets) to reduce duplication between `ci.yml`
|
||||
and `release.yml`.
|
||||
- **Release optimizations**: Use lean presets without test targets, run platform
|
||||
builds in parallel, and reuse cached artifacts from CI when hashes match.
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Automated (CI)
|
||||
- Ubuntu 22.04 (GCC-12, Clang-15)
|
||||
- macOS 13/14 (x64 and ARM64)
|
||||
- Windows Server 2022 (x64)
|
||||
- Core tests: `AsarWrapperTest`, `SnesTileTest`, others tagged `STABLE`
|
||||
- Tooling: clang-format, clang-tidy, cppcheck
|
||||
- Sanitizers: Linux AddressSanitizer job
|
||||
|
||||
### Manual Testing
|
||||
After successful CI build:
|
||||
- Windows: verify audio backend, keyboard input, APU debugger UI.
|
||||
- Linux: verify input polling and audio output.
|
||||
- macOS: spot-check rendering, input, audio.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Build Command (All Platforms)
|
||||
```bash
|
||||
cmake -B build
|
||||
cmake --build build --parallel
|
||||
|
||||
cmake --preset [mac-dbg|lin-dbg|win-dbg]
|
||||
cmake --build --preset [mac-dbg|lin-dbg|win-dbg]
|
||||
```
|
||||
|
||||
### Enable Features
|
||||
All features (JSON, gRPC, AI) are **always enabled** by default.
|
||||
No need to specify `-DZ3ED_AI=ON` or `-DYAZE_WITH_GRPC=ON`.
|
||||
|
||||
### Windows Troubleshooting
|
||||
```powershell
|
||||
# Verify environment
|
||||
.\scripts\verify-build-environment.ps1
|
||||
|
||||
# Use vcpkg for faster builds
|
||||
.\scripts\setup-vcpkg-windows.ps1
|
||||
cmake -B build -DCMAKE_TOOLCHAIN_FILE="vcpkg/scripts/buildsystems/vcpkg.cmake"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Filesystem Abstraction
|
||||
|
||||
To ensure robust and consistent behavior across platforms, YAZE has standardized its filesystem operations:
|
||||
|
||||
- **`std::filesystem`**: All new and refactored code uses the C++17 `std::filesystem` library for path manipulation, directory iteration, and file operations. This eliminates platform-specific bugs related to path separators (`/` vs `\`).
|
||||
|
||||
- **`PlatformPaths` Utility**: A dedicated utility class, `yaze::util::PlatformPaths`, provides platform-aware API for retrieving standard directory locations:
|
||||
- **Application Data**: `%APPDATA%` on Windows, `~/Library/Application Support` on macOS, XDG Base Directory on Linux
|
||||
- **Configuration Files**: Semantically clear API for config file locations
|
||||
- **Home and Temporary Directories**: Safely resolves user-specific and temporary folders
|
||||
|
||||
This removes legacy platform-specific APIs (like `dirent.h` or Win32 directory functions) for cleaner, more maintainable file handling.
|
||||
|
||||
---
|
||||
|
||||
## Native File Dialog Support
|
||||
|
||||
YAZE now features native file dialogs on all platforms:
|
||||
YAZE features native file dialogs on all platforms:
|
||||
- **macOS**: Cocoa-based file selection with proper sandboxing support
|
||||
- **Windows**: Windows Explorer integration with COM APIs
|
||||
- **Windows**: Windows Explorer integration with COM APIs (`IFileOpenDialog`/`IFileSaveDialog`)
|
||||
- **Linux**: GTK3 dialogs that match system appearance
|
||||
- **Fallback**: Bespoke implementation when native dialogs unavailable
|
||||
- **Fallback**: Cross-platform implementation when native dialogs unavailable
|
||||
|
||||
## Cross-Platform Build Reliability
|
||||
---
|
||||
|
||||
Enhanced build system ensures consistent compilation:
|
||||
- **Windows**: Resolved MSVC compatibility issues and dependency conflicts
|
||||
- **Linux**: Fixed standard library compatibility for older distributions
|
||||
- **macOS**: Proper support for both Intel and Apple Silicon architectures
|
||||
- **All Platforms**: Bundled dependencies eliminate external package requirements
|
||||
|
||||
## Build Configuration Options
|
||||
|
||||
YAZE supports different build configurations for various use cases:
|
||||
|
||||
### Full Build (Development)
|
||||
Includes all features: emulator, CLI tools, UI testing framework, and optional libraries.
|
||||
|
||||
### Minimal Build
|
||||
Streamlined build excluding complex components, optimized for automated testing and CI environments.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
The build system automatically detects platform capabilities and adjusts feature sets accordingly:
|
||||
|
||||
- **File Dialogs**: Uses native platform dialogs when available, with cross-platform fallbacks
|
||||
- **Dependencies**: Bundles all required libraries to eliminate external package requirements
|
||||
- **Testing**: Separates ROM-dependent tests from unit tests for CI compatibility
|
||||
- **Architecture**: Supports both Intel and Apple Silicon on macOS without conflicts
|
||||
|
||||
## Platform-Specific Adaptations
|
||||
|
||||
### Windows
|
||||
- Complete COM-based file dialog implementation
|
||||
- MSVC compatibility improvements for modern C++ features
|
||||
- Resource file handling for proper application integration
|
||||
|
||||
### macOS
|
||||
- Cocoa-based native file dialogs with sandboxing support
|
||||
- Universal binary support for Intel and Apple Silicon
|
||||
- Proper bundle configuration for macOS applications
|
||||
|
||||
### Linux
|
||||
- GTK3 integration for native file dialogs
|
||||
- Package manager integration for system dependencies
|
||||
- Support for multiple compiler toolchains (GCC, Clang)
|
||||
**Status:** All CI/CD issues resolved. Next push should build successfully on all platforms.
|
||||
|
||||
Reference in New Issue
Block a user