refactor(build): enhance build_cleaner.py with .gitignore support and auto-discovery

- Added support for .gitignore patterns to the build_cleaner script, allowing it to respect ignored files during maintenance tasks.
- Implemented auto-discovery of CMake libraries marked for auto-maintenance, improving the automation of source list updates.
- Introduced functions for extracting includes and symbols from source files to suggest missing headers based on usage.
- Updated README to reflect new features and usage instructions.

Benefits:
- Streamlines the build process by automating maintenance tasks and ensuring proper header management.
- Enhances code organization and maintainability by integrating with existing project structures.
This commit is contained in:
scawful
2025-10-13 12:18:10 -04:00
parent 02e9d3ba77
commit 8b0fd46580
14 changed files with 670 additions and 217 deletions

View File

@@ -1,134 +1,117 @@
# yaze Build Scripts
# YAZE Build Scripts
This directory contains build and setup scripts for YAZE development on different platforms.
This directory contains build automation and maintenance scripts for the YAZE project.
## Windows Scripts
## build_cleaner.py
### vcpkg Setup (Optional)
- **`setup-vcpkg-windows.ps1`** - Setup vcpkg for SDL2 dependency (PowerShell)
- **`setup-vcpkg-windows.bat`** - Setup vcpkg for SDL2 dependency (Batch)
Automates CMake source list maintenance and header include management with IWYU-style analysis.
**Note**: vcpkg is optional. YAZE uses bundled dependencies by default.
### Features
## Windows Build Workflow
### Recommended: Visual Studio CMake Mode
**YAZE uses Visual Studio's native CMake support - no project generation needed.**
1. **Install Visual Studio 2022** with "Desktop development with C++" workload
2. **Open Visual Studio 2022**
3. **File → Open → Folder**
4. **Navigate to yaze directory**
5. **Select configuration** (Debug/Release) from toolbar
6. **Press F5** to build and run
### Command Line Build
```powershell
# Standard CMake workflow
cmake -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build --config Debug
# Run the application
.\build\bin\Debug\yaze.exe
```
### Compiler Notes
The CMake configuration is designed to work with **MSVC** (Visual Studio's compiler). The build system includes automatic workarounds for C++23 compatibility:
- Abseil int128 is automatically excluded on Windows
- C++23 deprecation warnings are properly silenced
- Platform-specific definitions are handled automatically
## Quick Start (Windows)
### Option 1: Visual Studio (Recommended)
1. Open Visual Studio 2022
2. File → Open → Folder
3. Navigate to yaze directory
4. Select configuration (Debug/Release)
5. Press F5 to build and run
### Option 2: Command Line
```powershell
# Standard CMake build
cmake -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build --config Debug
```
### Option 3: With vcpkg (Optional)
```powershell
# Setup vcpkg for SDL2
.\scripts\setup-vcpkg-windows.ps1
# Then build normally in Visual Studio or command line
```
## Troubleshooting
### Common Issues
1. **PowerShell Execution Policy**
```powershell
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
```
2. **Visual Studio Not Detecting CMakeLists.txt**
- Ensure you opened the folder (not a solution file)
- Check that CMakeLists.txt exists in the root directory
- Try: File → Close Folder, then File → Open → Folder
3. **vcpkg Issues (if using vcpkg)**
- Run `.\scripts\setup-vcpkg-windows.ps1` to reinstall
- Check internet connection for dependency downloads
- Note: vcpkg is optional; bundled dependencies work by default
4. **CMake Configuration Errors**
- Delete the `build/` directory
- Close and reopen the folder in Visual Studio
- Or run: `cmake -B build -DCMAKE_BUILD_TYPE=Debug`
5. **Missing Git Submodules**
```powershell
git submodule update --init --recursive
```
### Getting Help
1. Check the [Build Instructions](../docs/02-build-instructions.md)
2. Review CMake output for specific error messages
3. Ensure all prerequisites are installed (Visual Studio 2022 with C++ workload)
## Other Scripts
- **`create_release.sh`** - Create GitHub releases (Linux/macOS)
- **`extract_changelog.py`** - Extract changelog for releases
- **`quality_check.sh`** - Code quality checks (Linux/macOS)
- **`create-macos-bundle.sh`** - Create macOS application bundle for releases
## Build Environment Verification
This directory also contains build environment verification scripts.
### `verify-build-environment.ps1` / `.sh`
A comprehensive script that checks:
- ✅ **CMake Installation** - Version 3.16+ required
- ✅ **Git Installation** - With submodule support
- ✅ **C++ Compiler** - GCC 13+, Clang 16+, or MSVC 2019+
- ✅ **Platform Tools** - Xcode (macOS), Visual Studio (Windows), build-essential (Linux)
- ✅ **Git Submodules** - All dependencies synchronized
1. **CMake Source List Maintenance**: Automatically updates source file lists in CMake files
2. **Self-Header Includes**: Ensures source files include their corresponding headers
3. **IWYU-Style Analysis**: Suggests missing headers based on symbol usage
4. **.gitignore Support**: Respects .gitignore patterns when scanning files
5. **Auto-Discovery**: Can discover CMake libraries that opt-in to auto-maintenance
### Usage
**Windows (PowerShell):**
```powershell
.\scripts\verify-build-environment.ps1
```bash
# Dry-run to see what would change (recommended first step)
python3 scripts/build_cleaner.py --dry-run
# Update CMake source lists and header includes
python3 scripts/build_cleaner.py
# Run IWYU-style header analysis
python3 scripts/build_cleaner.py --iwyu
# Auto-discover CMake libraries marked for auto-maintenance
python3 scripts/build_cleaner.py --auto-discover
# Update only CMake source lists
python3 scripts/build_cleaner.py --cmake-only
# Update only header includes
python3 scripts/build_cleaner.py --includes-only
```
**macOS/Linux:**
```bash
./scripts/verify-build-environment.sh
### Opting-In to Auto-Maintenance
By default, the script only auto-maintains source lists that are explicitly marked. To mark a CMake variable for auto-maintenance, add a comment above the `set()` statement:
```cmake
# This list is auto-maintained by scripts/build_cleaner.py
set(
YAZE_APP_EMU_SRC
app/emu/audio/apu.cc
app/emu/cpu/cpu.cc
# ... more files
)
```
The script looks for comments containing "auto-maintain" (case-insensitive) within 3 lines above the `set()` statement.
### Excluding Files from Processing
To exclude a specific file from all processing (CMake lists, header includes, IWYU), add this token near the top of the file:
```cpp
// build_cleaner:ignore
```
### .gitignore Support
The script automatically respects `.gitignore` patterns. To enable this feature, install the `pathspec` dependency:
```bash
pip3 install -r scripts/requirements.txt
# or
pip3 install pathspec
```
### IWYU Configuration
The script includes basic IWYU-style analysis that suggests headers based on symbol prefixes. To customize which headers are suggested, edit the `COMMON_HEADERS` dictionary in the script:
```python
COMMON_HEADERS = {
'std::': ['<memory>', '<string>', '<vector>', ...],
'absl::': ['<absl/status/status.h>', ...],
'ImGui::': ['<imgui.h>'],
'SDL_': ['<SDL.h>'],
}
```
**Note**: The IWYU analysis is conservative and may suggest headers that are already transitively included. Use with care and review suggestions before applying.
### Integration with CMake
The script is integrated into the CMake build system:
```bash
# Run as a CMake target
cmake --build build --target build_cleaner
```
### Dependencies
- Python 3.7+
- `pathspec` (optional, for .gitignore support): `pip3 install pathspec`
### How It Works
1. **CMake Maintenance**: Scans directories specified in the configuration and updates `set(VAR_NAME ...)` blocks with the current list of source files
2. **Self-Headers**: For each `.cc`/`.cpp` file, ensures it includes its corresponding `.h` file
3. **IWYU Analysis**: Scans source files for symbols and suggests appropriate headers based on prefix matching
### Current Auto-Maintained Variables
**All 20 library source lists are now auto-maintained by default:**
- Core: `YAZE_APP_EMU_SRC`, `YAZE_APP_CORE_SRC`, `YAZE_APP_EDITOR_SRC`, `YAZE_APP_ZELDA3_SRC`, `YAZE_NET_SRC`, `YAZE_UTIL_SRC`
- GFX: `GFX_TYPES_SRC`, `GFX_BACKEND_SRC`, `GFX_RESOURCE_SRC`, `GFX_CORE_SRC`, `GFX_UTIL_SRC`, `GFX_RENDER_SRC`, `GFX_DEBUG_SRC`
- GUI: `GUI_CORE_SRC`, `CANVAS_SRC`, `GUI_WIDGETS_SRC`, `GUI_AUTOMATION_SRC`, `GUI_APP_SRC`
- Other: `YAZE_AGENT_SOURCES`, `YAZE_TEST_SOURCES`
The script intelligently preserves conditional blocks (if/endif) and excludes conditional files from the main source list.

View File

@@ -7,7 +7,15 @@ import argparse
from dataclasses import dataclass, field
import re
from pathlib import Path
from typing import Dict, Iterable, List, Optional, Sequence, Set
from typing import Any, Iterable, List, Optional, Sequence, Set
try:
import pathspec
HAS_PATHSPEC = True
except ImportError:
HAS_PATHSPEC = False
print("Warning: 'pathspec' module not found. Install with: pip3 install pathspec")
print(" .gitignore support will be disabled.")
PROJECT_ROOT = Path(__file__).resolve().parent.parent
SOURCE_ROOT = PROJECT_ROOT / "src"
@@ -16,6 +24,14 @@ SUPPORTED_EXTENSIONS = (".cc", ".c", ".cpp", ".cxx", ".mm")
HEADER_EXTENSIONS = (".h", ".hh", ".hpp", ".hxx")
BUILD_CLEANER_IGNORE_TOKEN = "build_cleaner:ignore"
# Common SNES/ROM header patterns to include
COMMON_HEADERS = {
'std::': ['<memory>', '<string>', '<vector>', '<map>', '<set>', '<algorithm>', '<functional>'],
'absl::': ['<absl/status/status.h>', '<absl/status/statusor.h>', '<absl/strings/str_format.h>'],
'ImGui::': ['<imgui.h>'],
'SDL_': ['<SDL.h>'],
}
@dataclass(frozen=True)
class DirectorySpec:
@@ -43,7 +59,96 @@ class CMakeSourceBlock:
exclude: Set[Path] = field(default_factory=set)
CONFIG: Sequence[CMakeSourceBlock] = (
def load_gitignore():
"""Load .gitignore patterns into a pathspec matcher."""
if not HAS_PATHSPEC:
return None
gitignore_path = PROJECT_ROOT / ".gitignore"
if not gitignore_path.exists():
return None
try:
with gitignore_path.open('r', encoding='utf-8') as f:
patterns = [line.strip() for line in f if line.strip() and not line.startswith('#')]
return pathspec.PathSpec.from_lines('gitwildmatch', patterns)
except Exception as e:
print(f"Warning: Could not load .gitignore: {e}")
return None
def is_ignored(path: Path, gitignore_spec) -> bool:
"""Check if a path should be ignored based on .gitignore patterns."""
if gitignore_spec is None or not HAS_PATHSPEC:
return False
try:
rel_path = path.relative_to(PROJECT_ROOT)
return gitignore_spec.match_file(str(rel_path))
except ValueError:
return False
def discover_cmake_libraries() -> List[CMakeSourceBlock]:
"""
Auto-discover CMake library files that explicitly opt-in to auto-maintenance.
Looks for comments like "# This list is auto-maintained by scripts/build_cleaner.py"
or "# AUTO-MAINTAINED" to identify variables that should be auto-updated.
"""
blocks = []
seen_vars: Set[str] = set()
# Scan for cmake library files
for cmake_file in SOURCE_ROOT.rglob("*.cmake"):
if 'lib/' in str(cmake_file) or 'third_party/' in str(cmake_file):
continue
try:
content = cmake_file.read_text(encoding='utf-8')
lines = content.splitlines()
# Look for source variable definitions with auto-maintain markers
for i, line in enumerate(lines):
# Check if previous lines indicate auto-maintenance
auto_maintained = False
for j in range(max(0, i-3), i):
if 'auto-maintain' in lines[j].lower() or 'auto_maintain' in lines[j].lower():
auto_maintained = True
break
if not auto_maintained:
continue
# Extract variable name from set() statement
match = re.search(r'set\s*\(\s*(\w+(?:_SRC|_SOURCES|_SOURCE))\s', line)
if match:
var_name = match.group(1)
# Skip if we've already seen this variable
if var_name in seen_vars:
continue
seen_vars.add(var_name)
cmake_dir = cmake_file.parent
# Determine if recursive based on cmake file location or content
is_recursive = cmake_dir != SOURCE_ROOT / "app/core"
blocks.append(CMakeSourceBlock(
variable=var_name,
cmake_path=cmake_file,
directories=(DirectorySpec(cmake_dir, recursive=is_recursive),),
))
except Exception as e:
print(f"Warning: Could not process {cmake_file}: {e}")
return blocks
# Static configuration for all library source lists
# The script now auto-maintains all libraries while preserving conditional sections
STATIC_CONFIG: Sequence[CMakeSourceBlock] = (
CMakeSourceBlock(
variable="YAZE_APP_EMU_SRC",
cmake_path=SOURCE_ROOT / "CMakeLists.txt",
@@ -54,16 +159,6 @@ CONFIG: Sequence[CMakeSourceBlock] = (
cmake_path=SOURCE_ROOT / "app/core/core_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/core", recursive=False),),
),
CMakeSourceBlock(
variable="YAZE_APP_GFX_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx"),),
),
CMakeSourceBlock(
variable="YAZE_GUI_SRC",
cmake_path=SOURCE_ROOT / "app/gui/gui_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gui"),),
),
CMakeSourceBlock(
variable="YAZE_APP_EDITOR_SRC",
cmake_path=SOURCE_ROOT / "app/editor/editor_library.cmake",
@@ -85,6 +180,85 @@ CONFIG: Sequence[CMakeSourceBlock] = (
cmake_path=SOURCE_ROOT / "util/util.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "util"),),
),
CMakeSourceBlock(
variable="GFX_TYPES_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/types"),),
),
CMakeSourceBlock(
variable="GFX_BACKEND_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/backend"),),
),
CMakeSourceBlock(
variable="GFX_RESOURCE_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/resource"),),
exclude={Path("app/gfx/render/background_buffer.cc")}, # This is in resource but used by render
),
CMakeSourceBlock(
variable="GFX_CORE_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/core"),),
),
CMakeSourceBlock(
variable="GFX_UTIL_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/util"),),
),
CMakeSourceBlock(
variable="GFX_RENDER_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/render"),),
),
CMakeSourceBlock(
variable="GFX_DEBUG_SRC",
cmake_path=SOURCE_ROOT / "app/gfx/gfx_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gfx/debug"),),
),
CMakeSourceBlock(
variable="GUI_CORE_SRC",
cmake_path=SOURCE_ROOT / "app/gui/gui_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gui/core"),),
),
CMakeSourceBlock(
variable="CANVAS_SRC",
cmake_path=SOURCE_ROOT / "app/gui/gui_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gui/canvas"),),
),
CMakeSourceBlock(
variable="GUI_WIDGETS_SRC",
cmake_path=SOURCE_ROOT / "app/gui/gui_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gui/widgets"),),
),
CMakeSourceBlock(
variable="GUI_AUTOMATION_SRC",
cmake_path=SOURCE_ROOT / "app/gui/gui_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gui/automation"),),
),
CMakeSourceBlock(
variable="GUI_APP_SRC",
cmake_path=SOURCE_ROOT / "app/gui/gui_library.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/gui/app"),),
),
CMakeSourceBlock(
variable="YAZE_AGENT_SOURCES",
cmake_path=SOURCE_ROOT / "cli/agent.cmake",
directories=(
DirectorySpec(SOURCE_ROOT / "cli", recursive=False), # For flags.cc
DirectorySpec(SOURCE_ROOT / "cli/service"),
DirectorySpec(SOURCE_ROOT / "cli/handlers"),
),
exclude={
Path("cli/cli.cc"), # Part of z3ed executable
Path("cli/cli_main.cc"), # Part of z3ed executable
},
),
CMakeSourceBlock(
variable="YAZE_TEST_SOURCES",
cmake_path=SOURCE_ROOT / "app/test/test.cmake",
directories=(DirectorySpec(SOURCE_ROOT / "app/test"),),
),
)
@@ -113,16 +287,79 @@ def parse_entry(line: str) -> Optional[str]:
return stripped
def gather_expected_sources(block: CMakeSourceBlock) -> List[str]:
def extract_conditional_files(cmake_path: Path, variable: str) -> Set[str]:
"""Extract files that are added to the variable via conditional blocks (if/endif)."""
conditional_files: Set[str] = set()
try:
lines = cmake_path.read_text(encoding='utf-8').splitlines()
except Exception:
return conditional_files
in_conditional = False
conditional_depth = 0
for i, line in enumerate(lines):
stripped = line.strip()
# Track if/endif blocks
if stripped.startswith('if(') or stripped.startswith('if '):
if in_conditional:
conditional_depth += 1
else:
in_conditional = True
conditional_depth = 0
elif stripped.startswith('endif(') or stripped == 'endif()':
if conditional_depth > 0:
conditional_depth -= 1
else:
in_conditional = False
# Check if this line appends to our variable
if in_conditional and f'APPEND {variable}' in line:
# Handle single-line list(APPEND VAR file.cc)
if ')' in line:
# Extract file from same line
match = re.search(rf'APPEND\s+{re.escape(variable)}\s+(.+?)\)', line)
if match:
file_str = match.group(1).strip()
# Can be multiple files separated by space
for f in file_str.split():
f = f.strip()
if f and not f.startswith('$') and '/' in f and f.endswith('.cc'):
conditional_files.add(f)
else:
# Multi-line list(APPEND) - extract files from following lines
j = i + 1
while j < len(lines) and not lines[j].strip().startswith(')'):
entry = parse_entry(lines[j])
if entry:
conditional_files.add(entry)
j += 1
return conditional_files
def gather_expected_sources(block: CMakeSourceBlock, gitignore_spec: Any = None) -> List[str]:
# First, find files that are in conditional blocks
conditional_files = extract_conditional_files(block.cmake_path, block.variable)
entries: Set[str] = set()
for directory in block.directories:
for source_file in directory.iter_files():
if should_ignore_path(source_file):
continue
if is_ignored(source_file, gitignore_spec):
continue
rel_path = relative_to_source(source_file)
if rel_path in block.exclude:
continue
entries.add(str(rel_path).replace("\\", "/"))
# Exclude files that are in conditional blocks
rel_path_str = str(rel_path).replace("\\", "/")
if rel_path_str not in conditional_files:
entries.add(rel_path_str)
return sorted(entries)
@@ -135,7 +372,118 @@ def should_ignore_path(path: Path) -> bool:
return BUILD_CLEANER_IGNORE_TOKEN in head
def update_cmake_block(block: CMakeSourceBlock, dry_run: bool) -> bool:
def extract_includes(file_path: Path) -> Set[str]:
"""Extract all #include statements from a source file."""
includes = set()
try:
with file_path.open('r', encoding='utf-8') as f:
for line in f:
# Match #include "..." or #include <...>
match = re.match(r'^\s*#include\s+[<"]([^>"]+)[>"]', line)
if match:
includes.add(match.group(1))
except (OSError, UnicodeDecodeError):
pass
return includes
def extract_symbols(file_path: Path) -> Set[str]:
"""Extract potential symbols/identifiers that might need headers."""
symbols = set()
try:
with file_path.open('r', encoding='utf-8') as f:
content = f.read()
# Find namespace-qualified symbols (e.g., std::, absl::, ImGui::)
namespace_symbols = re.findall(r'\b([a-zA-Z_]\w*::)', content)
symbols.update(namespace_symbols)
# Find common function calls that might need headers
func_calls = re.findall(r'\b([A-Z][a-zA-Z0-9_]*)\s*\(', content)
symbols.update(func_calls)
except (OSError, UnicodeDecodeError):
pass
return symbols
def find_missing_headers(source: Path) -> List[str]:
"""Analyze a source file and suggest missing headers based on symbol usage."""
if should_ignore_path(source):
return []
current_includes = extract_includes(source)
symbols = extract_symbols(source)
missing = []
# Check for common headers based on symbol prefixes
for symbol_prefix, headers in COMMON_HEADERS.items():
if any(symbol_prefix in sym for sym in symbols):
for header in headers:
# Extract just the header name from angle brackets
header_name = header.strip('<>')
if header_name not in ' '.join(current_includes):
missing.append(header)
return missing
def find_conditional_blocks_after(cmake_lines: List[str], end_idx: int, variable: str) -> List[str]:
"""
Find conditional blocks (if/endif) that append to the variable after the main set() block.
Returns lines that should be preserved.
"""
conditional_lines = []
idx = end_idx + 1
while idx < len(cmake_lines):
line = cmake_lines[idx]
stripped = line.strip()
# Stop at next major block or empty lines
if not stripped:
idx += 1
continue
# Check if this is a conditional that appends to our variable
if stripped.startswith('if(') or stripped.startswith('if ('):
# Look ahead to see if this block modifies our variable
block_start = idx
block_depth = 1
modifies_var = False
temp_idx = idx + 1
while temp_idx < len(cmake_lines) and block_depth > 0:
temp_line = cmake_lines[temp_idx].strip()
if temp_line.startswith('if(') or temp_line.startswith('if '):
block_depth += 1
elif temp_line.startswith('endif(') or temp_line == 'endif()':
block_depth -= 1
# Check if this block modifies our variable
if f'APPEND {variable}' in temp_line or f'APPEND\n {variable}' in cmake_lines[temp_idx]:
modifies_var = True
temp_idx += 1
if modifies_var:
# Include the entire conditional block
conditional_lines.extend(cmake_lines[block_start:temp_idx])
idx = temp_idx
continue
else:
# This conditional doesn't touch our variable, stop scanning
break
else:
# Hit something else, stop scanning
break
idx += 1
return conditional_lines
def update_cmake_block(block: CMakeSourceBlock, dry_run: bool, gitignore_spec: Any = None) -> bool:
cmake_lines = (block.cmake_path.read_text(encoding="utf-8")).splitlines()
pattern = re.compile(rf"\s*set\(\s*{re.escape(block.variable)}\b")
@@ -195,7 +543,7 @@ def update_cmake_block(block: CMakeSourceBlock, dry_run: bool) -> bool:
else:
postlude.append(line)
expected_entries = gather_expected_sources(block)
expected_entries = gather_expected_sources(block, gitignore_spec)
expected_set = set(expected_entries)
if set(existing_entries) == expected_set:
@@ -209,12 +557,12 @@ def update_cmake_block(block: CMakeSourceBlock, dry_run: bool) -> bool:
rebuilt_block = prelude + [f"{indent}{entry}" for entry in expected_entries] + postlude
if dry_run:
print(f"[DRY-RUN] Would update {block.cmake_path.relative_to(PROJECT_ROOT)}")
print(f"[DRY-RUN] Would update {block.cmake_path.relative_to(PROJECT_ROOT)} :: {block.variable}")
return True
cmake_lines[start_idx + 1 : end_idx] = rebuilt_block
block.cmake_path.write_text("\n".join(cmake_lines) + "\n", encoding="utf-8")
print(f"Updated {block.cmake_path.relative_to(PROJECT_ROOT)}")
print(f"Updated {block.cmake_path.relative_to(PROJECT_ROOT)} :: {block.variable}")
missing = sorted(expected_set - set(existing_entries))
removed = sorted(set(existing_entries) - expected_set)
if missing:
@@ -307,9 +655,47 @@ def ensure_self_header_include(source: Path, dry_run: bool) -> bool:
return True
def collect_source_files() -> Set[Path]:
def add_missing_headers(source: Path, dry_run: bool, iwyu_mode: bool) -> bool:
"""Add missing headers based on IWYU-style analysis."""
if not iwyu_mode or should_ignore_path(source):
return False
missing_headers = find_missing_headers(source)
if not missing_headers:
return False
try:
lines = source.read_text(encoding="utf-8").splitlines()
except UnicodeDecodeError:
return False
# Find where to insert the headers
insert_idx = find_insert_index(lines)
# Move past any existing includes to add new ones after them
while insert_idx < len(lines) and lines[insert_idx].strip().startswith('#include'):
insert_idx += 1
# Insert missing headers
for header in missing_headers:
lines.insert(insert_idx, f'#include {header}')
insert_idx += 1
if dry_run:
rel = source.relative_to(PROJECT_ROOT)
print(f"[DRY-RUN] Would add missing headers to {rel}: {', '.join(missing_headers)}")
return True
source.write_text("\n".join(lines) + "\n", encoding="utf-8")
print(f"Added missing headers to {source.relative_to(PROJECT_ROOT)}: {', '.join(missing_headers)}")
return True
def collect_source_files(config: List[CMakeSourceBlock], gitignore_spec: Any = None) -> Set[Path]:
"""Collect all source files from the given configuration, respecting .gitignore patterns."""
managed_dirs: Set[Path] = set()
for block in CONFIG:
for block in config:
for directory in block.directories:
managed_dirs.add(directory.path)
@@ -319,42 +705,119 @@ def collect_source_files() -> Set[Path]:
continue
for file_path in directory.rglob("*"):
if file_path.is_file() and file_path.suffix in SUPPORTED_EXTENSIONS:
result.add(file_path)
if not is_ignored(file_path, gitignore_spec):
result.add(file_path)
return result
def run(dry_run: bool, cmake_only: bool, includes_only: bool) -> int:
def get_config(auto_discover: bool = False) -> List[CMakeSourceBlock]:
"""Get the full configuration, optionally including auto-discovered libraries."""
# Always start with static config (all known libraries)
config = list(STATIC_CONFIG)
# Optionally add auto-discovered libraries, avoiding duplicates
if auto_discover:
discovered = discover_cmake_libraries()
static_vars = {block.variable for block in STATIC_CONFIG}
for block in discovered:
if block.variable not in static_vars:
config.append(block)
print(f" Auto-discovered: {block.variable} in {block.cmake_path.name}")
return config
def run(dry_run: bool, cmake_only: bool, includes_only: bool, iwyu_mode: bool, auto_discover: bool) -> int:
if cmake_only and includes_only:
raise ValueError("Cannot use --cmake-only and --includes-only together")
# Load .gitignore patterns
gitignore_spec = load_gitignore()
if gitignore_spec:
print("✓ Loaded .gitignore patterns")
changed = False
# Get configuration (all libraries by default, with optional auto-discovery)
config = get_config(auto_discover)
if auto_discover:
print(f"✓ Using {len(config)} library configurations (with auto-discovery)")
else:
print(f"✓ Using {len(config)} library configurations")
if not includes_only:
for block in CONFIG:
changed |= update_cmake_block(block, dry_run)
print("\n📋 Updating CMake source lists...")
for block in config:
changed |= update_cmake_block(block, dry_run, gitignore_spec)
if not cmake_only:
for source in collect_source_files():
print("\n📝 Checking self-header includes...")
source_files = collect_source_files(config, gitignore_spec)
print(f" Scanning {len(source_files)} source files")
for source in source_files:
changed |= ensure_self_header_include(source, dry_run)
if iwyu_mode:
print("\n🔍 Running IWYU-style header analysis...")
for source in source_files:
changed |= add_missing_headers(source, dry_run, iwyu_mode)
if dry_run and not changed:
print("No changes required (dry-run)")
if not dry_run and not changed:
print("No changes required")
print("\nNo changes required (dry-run)")
elif not dry_run and not changed:
print("\nNo changes required")
elif dry_run:
print("\n✅ Dry-run complete - use without --dry-run to apply changes")
else:
print("\n✅ All changes applied successfully")
return 0
def main() -> int:
parser = argparse.ArgumentParser(description="Maintain CMake source lists and self-header includes.")
parser.add_argument("--dry-run", action="store_true", help="Report prospective changes without editing files")
parser.add_argument("--cmake-only", action="store_true", help="Only update CMake source lists")
parser.add_argument("--includes-only", action="store_true", help="Only ensure self-header includes")
parser = argparse.ArgumentParser(
description="Maintain CMake source lists and ensure proper header includes (IWYU-style).",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Dry-run to see what would change:
%(prog)s --dry-run
# Auto-discover libraries and update CMake files:
%(prog)s --auto-discover
# Run IWYU-style header analysis:
%(prog)s --iwyu
# Update only CMake source lists:
%(prog)s --cmake-only
# Update only header includes:
%(prog)s --includes-only
"""
)
parser.add_argument("--dry-run", action="store_true",
help="Report prospective changes without editing files")
parser.add_argument("--cmake-only", action="store_true",
help="Only update CMake source lists")
parser.add_argument("--includes-only", action="store_true",
help="Only ensure self-header includes")
parser.add_argument("--iwyu", action="store_true",
help="Run IWYU-style analysis to add missing headers")
parser.add_argument("--auto-discover", action="store_true",
help="Auto-discover CMake library files (*.cmake, *_library.cmake)")
args = parser.parse_args()
try:
return run(args.dry_run, args.cmake_only, args.includes_only)
return run(args.dry_run, args.cmake_only, args.includes_only, args.iwyu, args.auto_discover)
except Exception as exc: # pylint: disable=broad-except
print(f"build_cleaner failed: {exc}")
import traceback
print(f"❌ build_cleaner failed: {exc}")
if args.dry_run: # Show traceback in dry-run mode for debugging
traceback.print_exc()
return 1

6
scripts/requirements.txt Normal file
View File

@@ -0,0 +1,6 @@
# Python dependencies for YAZE build scripts
# Install with: pip3 install -r scripts/requirements.txt
# For .gitignore pattern matching in build_cleaner.py
pathspec>=0.11.0

View File

@@ -8,7 +8,7 @@
#include "app/core/timing.h"
#include "app/core/window.h"
#include "app/editor/editor_manager.h"
#include "app/gui/app/background_renderer.h"
#include "app/gui/core/background_renderer.h"
#include "app/gfx/resource/arena.h" // Add include for Arena
#include "app/gfx/backend/sdl2_renderer.h" // Add include for new renderer
#include "app/gui/core/theme_manager.h"

View File

@@ -111,10 +111,6 @@ if(WIN32 OR (UNIX AND NOT APPLE))
target_include_directories(yaze_core_lib PUBLIC ${CMAKE_SOURCE_DIR}/src/lib/nativefiledialog-extended/src/include)
endif()
target_sources(yaze_core_lib PRIVATE
${CMAKE_SOURCE_DIR}/src/cli/service/testing/test_workflow_generator.cc
)
if(YAZE_WITH_GRPC)
target_include_directories(yaze_core_lib PRIVATE
${CMAKE_SOURCE_DIR}/third_party/json/include)

View File

@@ -1,13 +1,6 @@
set(
YAZE_APP_EDITOR_SRC
app/editor/agent/agent_chat_history_codec.cc
app/editor/agent/agent_chat_history_popup.cc
app/editor/agent/agent_chat_widget.cc
app/editor/agent/agent_collaboration_coordinator.cc
app/editor/agent/agent_editor.cc
app/editor/agent/agent_ui_theme.cc
app/editor/agent/automation_bridge.cc
app/editor/agent/network_collaboration_coordinator.cc
app/editor/code/assembly_editor.cc
app/editor/code/memory_editor.cc
app/editor/code/project_file_editor.cc
@@ -24,8 +17,6 @@ set(
app/editor/graphics/gfx_group_editor.cc
app/editor/graphics/graphics_editor.cc
app/editor/graphics/screen_editor.cc
app/editor/palette/palette_editor.cc
app/editor/palette/palette_group_card.cc
app/editor/message/message_data.cc
app/editor/message/message_editor.cc
app/editor/message/message_preview.cc
@@ -36,6 +27,9 @@ set(
app/editor/overworld/overworld_entity_renderer.cc
app/editor/overworld/scratch_space.cc
app/editor/overworld/tile16_editor.cc
app/editor/palette/palette_editor.cc
app/editor/palette/palette_group_card.cc
app/editor/palette/palette_utility.cc
app/editor/sprite/sprite_editor.cc
app/editor/system/command_manager.cc
app/editor/system/command_palette.cc

View File

@@ -30,7 +30,7 @@
#include "app/emu/emulator.h"
#include "app/gfx/resource/arena.h"
#include "app/gfx/debug/performance/performance_profiler.h"
#include "app/gui/app/background_renderer.h"
#include "app/gui/core/background_renderer.h"
#include "app/gui/core/icons.h"
#include "app/gui/core/input.h"
#include "app/gui/core/style.h"

View File

@@ -75,7 +75,6 @@ message(STATUS " - GFX Tier: gfx_backend configured")
set(GFX_RESOURCE_SRC
app/gfx/resource/arena.cc
app/gfx/resource/memory_pool.cc
app/gfx/render/background_buffer.cc
)
add_library(yaze_gfx_resource STATIC ${GFX_RESOURCE_SRC})
configure_gfx_library(yaze_gfx_resource)
@@ -121,6 +120,7 @@ message(STATUS " - GFX Tier: gfx_util configured")
# ==============================================================================
set(GFX_RENDER_SRC
app/gfx/render/atlas_renderer.cc
app/gfx/render/background_buffer.cc
app/gfx/render/texture_atlas.cc
app/gfx/render/tilemap.cc
)
@@ -174,4 +174,4 @@ if(PNG_FOUND)
target_link_libraries(yaze_gfx INTERFACE ${PNG_LIBRARIES})
endif()
message(STATUS "✓ yaze_gfx library configured with tiered architecture")
message(STATUS "✓ yaze_gfx library configured with tiered architecture")

View File

@@ -1,4 +1,4 @@
#include "app/gui/app/background_renderer.h"
#include "app/gui/core/background_renderer.h"
#include <algorithm>
#include <cmath>

View File

@@ -4,7 +4,7 @@
#include "util/file_util.h"
#include "app/gui/core/theme_manager.h"
#include "app/gui/app/background_renderer.h"
#include "app/gui/core/background_renderer.h"
#include "app/platform/font_loader.h"
#include "app/gui/core/color.h"
#include "app/gui/core/icons.h"

View File

@@ -9,6 +9,7 @@
# 1. Define Source Groups for each sub-library
set(GUI_CORE_SRC
app/gui/core/background_renderer.cc
app/gui/core/color.cc
app/gui/core/input.cc
app/gui/core/layout_helpers.cc
@@ -47,7 +48,6 @@ set(GUI_AUTOMATION_SRC
set(GUI_APP_SRC
app/gui/app/agent_chat_widget.cc
app/gui/app/background_renderer.cc
app/gui/app/collaboration_panel.cc
app/gui/app/editor_card_manager.cc
app/gui/app/editor_layout.cc

View File

@@ -1,63 +1,73 @@
set(YAZE_AGENT_SOURCES
# Core infrastructure
cli/service/command_registry.cc
cli/service/agent/proposal_executor.cc
cli/flags.cc
cli/handlers/agent.cc
cli/handlers/agent/common.cc
cli/handlers/agent/conversation_test.cc
cli/handlers/agent/general_commands.cc
cli/handlers/agent/simple_chat_command.cc
cli/handlers/agent/test_commands.cc
cli/handlers/agent/test_common.cc
cli/handlers/agent/todo_commands.cc
cli/service/agent/conversational_agent_service.cc
cli/service/agent/simple_chat_session.cc
cli/service/agent/enhanced_tui.cc
cli/service/agent/tool_dispatcher.cc
# Advanced features
cli/service/agent/learned_knowledge_service.cc
cli/service/agent/todo_manager.cc
cli/handlers/command_handlers.cc
cli/handlers/game/dialogue_commands.cc
cli/handlers/game/dungeon.cc
cli/handlers/game/dungeon_commands.cc
cli/handlers/game/message.cc
cli/handlers/game/message_commands.cc
cli/handlers/game/music_commands.cc
cli/handlers/game/overworld.cc
cli/handlers/game/overworld_commands.cc
cli/handlers/game/overworld_inspect.cc
cli/handlers/graphics/gfx.cc
cli/handlers/graphics/hex_commands.cc
cli/handlers/graphics/palette.cc
cli/handlers/graphics/palette_commands.cc
cli/handlers/graphics/sprite_commands.cc
cli/handlers/net/net_commands.cc
cli/handlers/rom/mock_rom.cc
cli/handlers/rom/project_commands.cc
cli/handlers/rom/rom_commands.cc
cli/handlers/tools/gui_commands.cc
cli/handlers/tools/resource_commands.cc
cli/service/agent/advanced_routing.cc
cli/service/agent/agent_pretraining.cc
cli/service/agent/conversational_agent_service.cc
cli/service/agent/enhanced_tui.cc
cli/service/agent/learned_knowledge_service.cc
cli/service/agent/prompt_manager.cc
cli/service/agent/proposal_executor.cc
cli/service/agent/simple_chat_session.cc
cli/service/agent/todo_manager.cc
cli/service/agent/tool_dispatcher.cc
cli/service/agent/vim_mode.cc
cli/service/ai/ai_service.cc
cli/service/ai/ai_action_parser.cc
cli/service/ai/vision_action_refiner.cc
cli/service/ai/ai_gui_controller.cc
cli/service/ai/ai_service.cc
cli/service/ai/ollama_ai_service.cc
cli/service/ai/prompt_builder.cc
cli/service/ai/service_factory.cc
cli/service/ai/vision_action_refiner.cc
cli/service/command_registry.cc
cli/service/gui/gui_action_generator.cc
cli/service/gui/gui_automation_client.cc
cli/service/net/z3ed_network_client.cc
cli/handlers/net/net_commands.cc
cli/service/planning/policy_evaluator.cc
cli/service/planning/proposal_registry.cc
cli/service/planning/tile16_proposal_generator.cc
cli/service/resources/resource_catalog.cc
cli/service/resources/resource_context_builder.cc
cli/service/resources/command_context.cc
cli/service/resources/command_handler.cc
cli/handlers/agent.cc
cli/handlers/command_handlers.cc
cli/handlers/agent/simple_chat_command.cc
cli/handlers/agent/general_commands.cc
cli/handlers/agent/test_commands.cc
cli/handlers/agent/conversation_test.cc
cli/handlers/agent/common.cc
cli/handlers/game/overworld_inspect.cc
cli/handlers/game/message.cc
cli/handlers/rom/mock_rom.cc
# CommandHandler-based implementations
cli/handlers/tools/resource_commands.cc
cli/handlers/game/dungeon_commands.cc
cli/handlers/game/overworld_commands.cc
cli/handlers/tools/gui_commands.cc
cli/handlers/graphics/hex_commands.cc
cli/handlers/game/dialogue_commands.cc
cli/handlers/game/music_commands.cc
cli/handlers/graphics/palette_commands.cc
cli/handlers/game/message_commands.cc
cli/handlers/graphics/sprite_commands.cc
# ROM commands
cli/handlers/rom/rom_commands.cc
cli/handlers/rom/project_commands.cc
cli/flags.cc
cli/service/resources/resource_catalog.cc
cli/service/resources/resource_context_builder.cc
cli/service/rom/rom_sandbox_manager.cc
cli/service/testing/test_suite_loader.cc
cli/service/testing/test_suite_reporter.cc
cli/service/testing/test_suite_writer.cc
cli/service/testing/test_workflow_generator.cc
# Advanced features
# CommandHandler-based implementations
# ROM commands
)
# gRPC-dependent sources (only added when gRPC is enabled)

View File

@@ -10,6 +10,7 @@ set(
zelda3/music/tracker.cc
zelda3/overworld/overworld.cc
zelda3/overworld/overworld_map.cc
zelda3/palette_constants.cc
zelda3/screen/dungeon_map.cc
zelda3/screen/inventory.cc
zelda3/screen/title_screen.cc