yaze/docs/B5-stability-improvements.md

Stability, Testability & Release Workflow Improvements

This document outlines recent improvements to YAZE's stability, testability, and release workflows, along with recommendations for future optimizations.

Recent Improvements (v0.3.2)

Windows Platform Stability

Stack Size Optimization

Problem: Windows default stack size (1MB) was insufficient for EditorManager::LoadAssets() which loads 223 graphics sheets and initializes multiple editors.

Solution: Increased stack size to 8MB to match Unix-like systems.

Implementation:

# src/app/app.cmake
if(MSVC)
  target_link_options(yaze PRIVATE /STACK:8388608)  # 8MB stack
elseif(MINGW)
  target_link_options(yaze PRIVATE -Wl,--stack,8388608)
endif()

Impact:

• ✅ Eliminated stack overflow crashes during ROM loading
• ✅ Consistent behavior across all platforms
• ✅ Handles deep call chains from editor initialization

Development Utility Isolation

Problem: Development-only utilities (extract_vanilla_values, rom_patch_utility) were being built in CI/release workflows, causing linker errors.

Solution: Isolated development utilities from CI/release builds using environment detection.

Implementation:

# test/CMakeLists.txt
if(NOT YAZE_MINIMAL_BUILD AND YAZE_ENABLE_ROM_TESTS AND NOT DEFINED ENV{GITHUB_ACTIONS})
  add_executable(extract_vanilla_values ...)
  target_link_libraries(extract_vanilla_values yaze_core ...)
endif()

Impact:

• ✅ Clean release builds without development artifacts
• ✅ Proper library linkage for development tools
• ✅ Faster CI/CD pipelines

Graphics System Stability

Segmentation Fault Resolution

Problem: Tile cache system using std::move() operations invalidated Bitmap surface pointers, causing crashes.

Solution: Disabled move semantics in tile cache and implemented direct SDL texture updates.

Technical Details:

• Root cause: std::move() operations on Bitmap objects invalidated internal SDL_Surface pointers
• Fix: Disabled tile cache and use direct texture updates
• Optimization: Maintained surface/texture pooling while ensuring pointer stability

Impact:

• ✅ Eliminated all segmentation faults in tile16 editor
• ✅ Stable graphics rendering pipeline
• ✅ Reliable texture management

Comprehensive Bounds Checking

Problem: Out-of-bounds access to tile and palette data caused crashes and corruption.

Solution: Added extensive bounds checking throughout graphics pipeline.

Areas Covered:

• Tile16 canvas access
• Palette group selection
• Graphics sheet indexing
• Texture coordinate calculations

Impact:

• ✅ Prevents crashes from invalid data
• ✅ Better error reporting
• ✅ Safer memory access patterns

Build System Improvements

Modern Windows Workflow

Previous Approach: Generate Visual Studio solution files using Python script.

New Approach: Use Visual Studio's native CMake support.

Benefits:

• No project generation step required
• CMakeLists.txt is the single source of truth
• Changes reflect immediately without regeneration
• Better IntelliSense and debugging experience
• Cross-platform consistency

Migration:

# Old workflow
python scripts/generate-vs-projects.py
# Open YAZE.sln

# New workflow  
# File → Open → Folder → yaze
# Visual Studio detects CMakeLists.txt automatically

Enhanced CI/CD Reliability

Improvements:

• Automatic vcpkg fallback mechanisms
• Proper development utility isolation
• Consistent test execution across platforms
• Better error reporting and debugging

Recommended Optimizations

High Priority

1. Lazy Graphics Loading

Current: All 223 graphics sheets loaded on ROM open.

Proposed: Load graphics sheets on-demand when editors access them.

Benefits:

• Faster ROM loading (3-5x improvement expected)
• Reduced memory footprint
• Better startup performance
• Eliminates stack pressure

Implementation Strategy:

class LazyGraphicsLoader {
  std::array<std::optional<gfx::Bitmap>, kNumGfxSheets> sheets_;
  
  gfx::Bitmap& GetSheet(int index) {
    if (!sheets_[index]) {
      sheets_[index] = LoadGraphicsSheet(rom_, index);
    }
    return *sheets_[index];
  }
};

Impact:

• 🔵 Complexity: Medium
• 🟢 Performance Gain: High
• 🟢 Risk: Low (backward compatible)

2. Heap-Based Large Allocations

Current: Large arrays and vectors allocated on stack during asset loading.

Proposed: Move large data structures to heap allocation.

Benefits:

• Reduces stack pressure
• More flexible memory management
• Better for Windows default stack constraints
• Safer for deep call chains

Areas to Convert:

• Graphics sheet arrays in LoadAllGraphicsData()
• Editor initialization data structures
• Temporary buffers in compression/decompression

Impact:

• 🟢 Complexity: Low
• 🟢 Performance Gain: Medium
• 🟢 Risk: Very Low

3. Streaming ROM Assets

Current: Load entire ROM and all assets into memory.

Proposed: Stream assets from ROM file as needed.

Benefits:

• Minimal memory footprint
• Instant ROM opening
• Better for large ROM hacks
• More scalable architecture

Challenges:

• Requires architecture refactoring
• Need efficient caching strategy
• Must maintain edit performance

Impact:

• 🔴 Complexity: High
• 🟢 Performance Gain: Very High
• 🟡 Risk: Medium (requires testing)

Medium Priority

4. Enhanced Test Isolation

Current: Some tests share global state through Arena singleton.

Proposed: Better test isolation with mock singletons.

Benefits:

• More reliable test execution
• Parallel test execution possible
• Better test independence
• Easier debugging

Implementation:

class TestArena : public Arena {
  // Test-specific implementation
};

TEST_F(GraphicsTest, TestCase) {
  TestArena arena;
  Arena::SetInstance(&arena);  // Override singleton
  // Run test
}

Impact:

• 🟡 Complexity: Medium
• 🟢 Performance Gain: Medium (parallel tests)
• 🟢 Risk: Low

5. Dependency Caching Optimization

Current: CI builds re-download and build some dependencies.

Proposed: Enhanced caching strategies for vcpkg and build artifacts.

Benefits:

• Faster CI builds (2-3x improvement)
• Reduced CI costs
• More reliable builds (less network dependency)
• Better developer experience

Implementation:

# GitHub Actions
- uses: actions/cache@v4
  with:
    path: |
      ~/.ccache
      ~/vcpkg_cache
      build/_deps
    key: ${{ runner.os }}-${{ hashFiles('**/CMakeLists.txt') }}

Impact:

• 🟢 Complexity: Low
• 🟢 Performance Gain: High (CI only)
• 🟢 Risk: Very Low

6. Memory Pool for Graphics

Current: Individual allocation for each Bitmap and texture.

Proposed: Memory pool for graphics objects.

Benefits:

• Reduced allocation overhead
• Better cache locality
• Predictable memory usage
• Faster allocation/deallocation

Areas to Apply:

• Bitmap objects
• SDL surfaces and textures
• Tile data structures

Impact:

• 🟡 Complexity: Medium
• 🟡 Performance Gain: Medium
• 🟡 Risk: Medium (requires careful design)

Low Priority

7. Build Time Optimization

Current: Full rebuild takes 10-15 minutes.

Proposed: Optimize compilation units and dependencies.

Strategies:

• Use forward declarations more extensively
• Split large compilation units
• Optimize template instantiations
• Better use of precompiled headers

Impact:

• 🟡 Complexity: Medium
• 🟢 Performance Gain: Medium (developer experience)
• 🟢 Risk: Low

8. Release Workflow Simplification

Current: Three separate release workflows (simplified, standard, complex).

Proposed: Single unified workflow with conditional features.

Benefits:

• Easier maintenance
• Consistent behavior
• Better documentation
• Clearer mental model

Implementation:

jobs:
  release:
    strategy:
      matrix:
        include:
          - profile: minimal  # Quick releases
          - profile: standard  # Normal releases
          - profile: maximum   # Production releases

Impact:

• 🟢 Complexity: Low
• 🟢 Performance Gain: None (maintenance benefit)
• 🟢 Risk: Very Low

Testing Improvements

Current State

• ✅ Comprehensive unit test coverage (46+ tests)
• ✅ Integration tests for major components
• ✅ ROM-dependent tests properly isolated
• ✅ CI-safe test configuration
• ✅ Platform-specific test handling

Recommendations

1. Visual Regression Testing

Goal: Catch graphics rendering regressions automatically.

Approach:

• Capture screenshots of editor states
• Compare against baseline images
• Flag visual differences for review

Tools: ImGui Test Engine (already integrated)

2. Performance Benchmarks

Goal: Track performance regressions in CI.

Metrics:

• ROM load time
• Graphics sheet decompression
• Editor initialization
• Memory usage

Implementation: Google Benchmark (already a dependency)

3. Fuzz Testing

Goal: Find edge cases and crashes through random input.

Areas:

• ROM parsing
• Compression/decompression
• Palette handling
• Tile data processing

Tools: LibFuzzer or AFL

Metrics & Monitoring

Current Measurements

• Build time: ~10-15 minutes (full rebuild)
• ROM load time: ~2-3 seconds
• Memory usage: ~500MB-1GB typical
• Test execution: ~30 seconds (CI), ~2 minutes (full)

Target Improvements

• Build time: <5 minutes (incremental), <10 minutes (full)
• ROM load time: <1 second (with lazy loading)
• Memory usage: <300MB (with streaming)
• Test execution: <15 seconds (CI), <1 minute (full)

Action Items

Immediate (v0.3.2)

• Fix Windows stack overflow
• Isolate development utilities
• Fix graphics segfaults
• Update build documentation
• Complete tile16 palette display fixes

Short Term (v0.3.3)

• Implement lazy graphics loading
• Move large allocations to heap
• Enhanced CI caching
• Performance benchmarks

Medium Term (v0.4.0)

• Streaming ROM assets
• Memory pool for graphics
• Visual regression tests
• Enhanced test isolation

Long Term (v0.5.0+)

• Fuzz testing integration
• Build time optimization
• Release workflow unification
• Advanced memory profiling

Conclusion

The v0.3.2 release focuses on stability and reliability improvements, particularly for the Windows platform. The recommended optimizations provide a clear roadmap for future performance and maintainability improvements while maintaining backward compatibility and code quality.

For questions or suggestions, please open an issue or discussion on the GitHub repository.