Files
yaze/docs/z3ed
scawful 0465d07a55 feat: Add GUI automation client and test workflow generator
- Implemented GuiAutomationClient for gRPC communication with the test harness.
- Added methods for various GUI actions: Click, Type, Wait, Assert, and Screenshot.
- Created TestWorkflowGenerator to convert natural language prompts into structured test workflows.
- Enhanced HandleTestCommand to support new command-line arguments for GUI automation.
- Updated CMakeLists.txt to include new source files for GUI automation and workflow generation.
2025-10-02 01:01:19 -04:00
..

z3ed: AI-Powered CLI for YAZE

Status: Active Development
Version: 0.1.0-alpha
Last Updated: October 2, 2025 (IT-01 Complete, E2E Validation Phase)

Overview

z3ed is a command-line interface for YAZE (Yet Another Zelda3 Editor) that enables AI-driven ROM modifications through a proposal-based workflow. It allows AI agents to suggest changes, which are then reviewed and accepted/rejected by human operators via the YAZE GUI.

Documentation Index

🎯 Essential Documents (Start Here)

  1. Next Priorities - 🚀 CURRENT WORK - Detailed Priority 1-3 tasks with implementation guides
  2. Implementation Plan - MASTER TRACKER - Complete task backlog, progress, architecture
  3. CLI Design - 📐 DESIGN DOC - High-level vision, command structure, workflows
  4. IT-01 Quickstart - QUICK REFERENCE - Test harness commands and examples

<EFBFBD> Archive

Historical documentation (design decisions, phase completions, technical notes) moved to archive/ folder for reference.

Architecture

┌─────────────────────────────────────────────────────────┐
│ z3ed CLI                                                │
│  └─ agent subcommand                                    │
│     ├─ run <prompt> [--sandbox]                         │
│     ├─ list                                             │
│     └─ test <prompt>                                    │
└────────────────────┬────────────────────────────────────┘
                     │
┌────────────────────▼────────────────────────────────────┐
│ Services Layer (Singleton Services)                     │
│  ├─ ProposalRegistry                                    │
│  │   ├─ CreateProposal()                                │
│  │   ├─ ListProposals()                                 │
│  │   └─ LoadProposalsFromDiskLocked()                   │
│  ├─ RomSandboxManager                                   │
│  │   ├─ CreateSandbox()                                 │
│  │   └─ FindSandbox()                                   │
│  └─ PolicyEvaluator (Planned)                           │
│      ├─ LoadPolicies()                                  │
│      └─ EvaluateProposal()                              │
└────────────────────┬────────────────────────────────────┘
                     │
┌────────────────────▼────────────────────────────────────┐
│ Filesystem Layer                                        │
│  ├─ /tmp/yaze/proposals/<id>/                          │
│  │   ├─ metadata.json                                   │
│  │   ├─ execution.log                                   │
│  │   └─ diff.txt                                        │
│  └─ /tmp/yaze/sandboxes/<id>/                          │
│      └─ zelda3.sfc (copy)                               │
└────────────────────┬────────────────────────────────────┘
                     │
┌────────────────────▼────────────────────────────────────┐
│ YAZE GUI                                                │
│  └─ ProposalDrawer (400px right panel)                  │
│     ├─ List View (proposals from registry)              │
│     ├─ Detail View (metadata, diff, log)                │
│     └─ AcceptProposal() → ROM merging                   │
└─────────────────────────────────────────────────────────┘

Current Status

Phase 6: Resource Catalogue (COMPLETE)

  • Resource Catalog System: Comprehensive schema for all CLI commands
  • Agent Describe: Machine-readable API catalog export (JSON/YAML)
  • API Documentation: docs/api/z3ed-resources.yaml for AI/LLM consumption

AW-01 & AW-02: Proposal Infrastructure (COMPLETE)

  • ProposalRegistry: Disk persistence with lazy loading
  • RomSandboxManager: Isolated ROM copies for safe testing
  • Cross-Session Tracking: Proposals persist between CLI runs

AW-03: ProposalDrawer GUI (COMPLETE)

  • ProposalDrawer GUI: Split view, proposal list, detail panel
  • ROM Merging: Sandbox-to-main ROM data copy on acceptance
  • Full Lifecycle: Create (CLI) → Review (GUI) → Accept/Reject → Commit

IT-01: ImGuiTestHarness (COMPLETE) 🎉

All 3 Phases Complete: gRPC + TestManager + ImGuiTestEngine
Time Invested: 11 hours total (Phase 1: 4h, Phase 2: 4h, Phase 3: 3h)

  • Phase 1 : gRPC infrastructure with 6 RPC methods
  • Phase 2 : TestManager integration with dynamic test registration
  • Phase 3 : Full ImGuiTestEngine integration (Type/Wait/Assert RPCs)
  • Testing : E2E test script operational (scripts/test_harness_e2e.sh)
  • Documentation : Complete guides (QUICKSTART, PHASE3-COMPLETE)

See: IT-01-QUICKSTART.md for usage examples

IT-02: CLI Agent Test Command (COMPLETE) 🎉

Implementation Complete: Natural language → automated GUI testing
Time Invested: 4 hours (design + implementation + documentation)
Status: Ready for validation

Components:

  • GuiAutomationClient: gRPC wrapper for CLI usage (6 RPC methods)
  • TestWorkflowGenerator: Natural language prompt parser (4 pattern types)
  • z3ed agent test: End-to-end automation command

Supported Prompts:

  1. "Open Overworld editor" → Click + Wait
  2. "Open Dungeon editor and verify it loads" → Click + Wait + Assert
  3. "Type 'zelda3.sfc' in filename input" → Click + Type
  4. "Click Open ROM button" → Single click

Example Usage:

# Start YAZE with test harness
./build-grpc-test/bin/yaze.app/Contents/MacOS/yaze \
  --enable_test_harness \
  --test_harness_port=50052 \
  --rom_file=assets/zelda3.sfc &

# Run automated test
./build-grpc-test/bin/z3ed agent test \
  --prompt "Open Overworld editor"

# Output:
# === GUI Automation Test ===
# Prompt: Open Overworld editor
# ...
# [1/2] Click(button:Overworld) ... ✓ (125ms)
# [2/2] Wait(window_visible:Overworld Editor, 5000ms) ... ✓ (1250ms)
# ✅ Test passed in 1375ms

See: IMPLEMENTATION_PROGRESS_OCT2.md for complete details

📋 Priority 1: End-to-End Workflow Validation (NEXT)

Goal: Test complete proposal lifecycle with real GUI and widgets
Time Estimate: 2-3 hours
Status: Ready to execute - all prerequisites complete

Tasks:

  1. Run E2E test script and validate all RPCs
  2. Test proposal workflow: Create → Review → Accept/Reject
  3. Test GUI automation with real YAZE widgets
  4. Validate CLI agent test command with multiple prompts
  5. Document edge cases and troubleshooting

See: E2E_VALIDATION_GUIDE.md for detailed checklist

📋 Priority 3: Policy Evaluation Framework (AW-04)

Goal: YAML-based constraint system for gating proposal acceptance
Time Estimate: 6-8 hours
Blocking: None (can work in parallel)

Quick Start

Build and Test

# Build z3ed CLI
cmake --build build --target z3ed -j8

# Build YAZE with test harness
cmake -B build-grpc-test -DYAZE_WITH_GRPC=ON
cmake --build build-grpc-test --target yaze -j$(sysctl -n hw.ncpu)

# Run E2E tests
./scripts/test_harness_e2e.sh

Create and Review Proposal

# 1. Create proposal
./build/bin/z3ed agent run "Test proposal" --sandbox

# 2. List proposals
./build/bin/z3ed agent list

# 3. Review in GUI
./build/bin/yaze.app/Contents/MacOS/yaze
# Open: Debug → Agent Proposals

Test Harness Usage

# Start YAZE with test harness
./build-grpc-test/bin/yaze.app/Contents/MacOS/yaze \
  --enable_test_harness \
  --test_harness_port=50052 \
  --rom_file=assets/zelda3.sfc &

# Test individual RPCs (see IT-01-QUICKSTART.md for full reference)
grpcurl -plaintext -import-path src/app/core/proto -proto imgui_test_harness.proto \
  -d '{"message":"test"}' 127.0.0.1:50052 yaze.test.ImGuiTestHarness/Ping

Key Files & Components

Core Services:

  • src/cli/service/proposal_registry.{h,cc} - Proposal tracking and persistence
  • src/cli/service/rom_sandbox_manager.{h,cc} - Isolated ROM copies
  • src/cli/service/resource_catalog.{h,cc} - Machine-readable API specs

GUI Integration:

  • src/app/editor/system/proposal_drawer.{h,cc} - Proposal review panel
  • src/app/core/imgui_test_harness_service.{h,cc} - gRPC test automation server

CLI Handlers:

  • src/cli/handlers/agent.cc - Agent subcommand (run, list, diff, describe)
  • src/cli/handlers/rom.cc - ROM commands (info, validate, diff)

Configuration:

  • docs/api/z3ed-resources.yaml - Generated API catalog for AI/LLM
  • .yaze/policies/agent.yaml - (Planned) Policy rules

Development Guidelines

See docs/B1-contributing.md for general guidelines.

z3ed-Specific:

  • Use singleton pattern for services (Instance() accessor)
  • Return absl::Status or absl::StatusOr<T> for error handling
  • Update NEXT_PRIORITIES_OCT2.md when starting new work
  • Update E6-z3ed-implementation-plan.md task backlog when completing tasks

Questions? Open an issue or see implementation plan for detailed architecture.