yaze/docs/z3ed/TEST_VALIDATION_STATUS_OCT2.md

Test Validation Status - October 2, 2025

Time: 9:30 PM
Status: E2E Tests Running | Menu Interaction Verified | Window Detection Issue Identified

Current Test Results

Working ✅

1. Ping RPC - Health check fully operational
2. Menu Item Clicks - Successfully clicking menu items via gRPC
   • Example: menuitem: Overworld Editor → clicked successfully
   • Example: menuitem: Dungeon Editor → clicked successfully

Issues Identified 🔍

Issue 1: Window Detection After Menu Click

Problem: Menu items are clicked successfully, but subsequent window visibility checks fail

Observed Behavior:

Test 2: Click (Open Overworld Editor)
✓ Clicked menuitem ' Overworld Editor' (1873ms)

Test 3: Wait (Overworld Editor Window)
✗ Condition 'window_visible:Overworld Editor' not met after 5000ms timeout

Root Cause Analysis:

1. Menu items call editor.set_active(true)
2. This sets a flag but doesn't immediately create ImGui window
3. Window creation happens in next frame's Update() call
4. ImGuiTestEngine's WindowInfo() API may not see newly created windows immediately
5. Window title may include ICON_MD prefix: ICON_MD_LAYERS " Overworld Editor"

Potential Solutions:

• A. Use longer wait time (current: 5s)
• B. Check for window with icon prefix: window_visible: Overworld Editor
• C. Use different condition type (element_visible vs window_visible)
• D. Add frame yield between menu click and window check

Issue 2: Screenshot RPC Proto Mismatch

Problem: Screenshot request proto schema doesn't match client usage

Error Message:

message type yaze.test.ScreenshotRequest has no known field named region

Solution: Update proto or skip for now (non-blocking for core functionality)

Next Steps (Priority Order)

1. Debug Window Detection (30 min)

Goal: Understand why windows aren't detected after menu clicks

Tasks:

• Check actual window titles in YAZE (with icons)
• Test with exact window name including icon
• Add diagnostic logging to Wait RPC
• Try element_visible condition instead
• Increase wait timeout to 10s

Test Command:

# Terminal 1: Start YAZE
./build-grpc-test/bin/yaze.app/Contents/MacOS/yaze \
  --enable_test_harness \
  --test_harness_port=50052 \
  --rom_file=assets/zelda3.sfc &

# Terminal 2: Manual test sequence
sleep 5  # Let YAZE fully initialize

# Click menu item
grpcurl -plaintext -import-path src/app/core/proto -proto imgui_test_harness.proto \
  -d '{"target":"menuitem: Overworld Editor","type":"LEFT"}' \
  127.0.0.1:50052 yaze.test.ImGuiTestHarness/Click

# Wait a few frames
sleep 2

# Try different window name variations
grpcurl -plaintext -import-path src/app/core/proto -proto imgui_test_harness.proto \
  -d '{"condition":"window_visible:Overworld Editor","timeout_ms":10000}' \
  127.0.0.1:50052 yaze.test.ImGuiTestHarness/Wait

# Or with icon
grpcurl -plaintext -import-path src/app/core/proto -proto imgui_test_harness.proto \
  -d '{"condition":"window_visible: Overworld Editor","timeout_ms":10000}' \
  127.0.0.1:50052 yaze.test.ImGuiTestHarness/Wait

2. Fix Window Name Matching (1 hour)

Options:

Option A: Strip Icons from Target Names

// In Wait RPC handler
std::string CleanWindowName(const std::string& name) {
  // Strip ICON_MD_ prefixes and leading spaces
  // " Overworld Editor" → "Overworld Editor"
  return absl::StripAsciiWhitespace(name);
}

Option B: Use Partial Name Matching

// Check if window name contains target (case-insensitive)
bool window_found = false;
for (ImGuiWindow* window : ImGui::GetCurrentContext()->Windows) {
  if (absl::StrContains(absl::AsciiStrToLower(window->Name), 
                        absl::AsciiStrToLower(target))) {
    window_found = true;
    break;
  }
}

Option C: Add Frame Yield

// In Click RPC, after successful click:
// Yield control back to ImGui to process one frame
ImGuiTestEngine_Yield(engine);
// Or sleep briefly
std::this_thread::sleep_for(std::chrono::milliseconds(500));

3. Update E2E Test Script (15 min)

Once window detection works, update test script:

# Use working window names
run_test "Wait (Overworld Editor)" "Wait" \
  '{"condition":"window_visible:Overworld Editor","timeout_ms":10000,"poll_interval_ms":100}'

# Add delay between click and wait
echo "Waiting for window to appear..."
sleep 2

4. Document Widget Naming Convention (30 min)

Create guide for test writers:

Widget Naming Patterns:

• Menu items: menuitem:<Name> (with or without icon prefix)
• Buttons: button:<Label>
• Windows: window:<Title> (may include icons)
• Input fields: input:<Label>
• Tabs: tab:<Name>

Best Practices:

• Use partial matching for windows (handles icon prefixes)
• Add 1-2s delay after menu clicks before checking windows
• Use 10s+ timeouts for initial window creation
• Use shorter timeouts (2-5s) for interactions within open windows

Test Coverage Status

RPC Method	Status	Notes
Ping	✅ Working	Health check operational
Click	⚠️ Partial	Menu items work, window detection needs fix
Type	📋 Pending	Depends on window detection fix
Wait	⚠️ Partial	Polling works, condition matching needs fix
Assert	⚠️ Partial	Same as Wait - condition matching issue
Screenshot	🔧 Blocked	Proto mismatch - non-critical

Time Investment Today

• Build system fixes: 1h
• Type conversion debugging: 0.5h
• Test script updates: 0.5h
• Widget investigation: 1h
• Total: 3 hours

Estimated Remaining Work

• Debug window detection: 0.5h
• Fix window matching logic: 1h
• Update tests and validate: 0.5h
• Document findings: 0.5h
• Total: 2.5 hours

Target Completion: October 3, 2025 (tomorrow morning)

Success Criteria for Validation Complete

• All 5 main RPCs working (Ping, Click, Wait, Assert, Type)
• Can open editor windows via menu clicks
• Can detect window visibility after opening
• Can assert on window state
• E2E test script passes all tests except Screenshot
• Documentation updated with real examples
• Widget naming conventions documented

Next Phase After Validation

Once E2E validation complete: → Priority 3: Policy Evaluation Framework (AW-04)

• 6-8 hours estimated
• Can work in parallel with validation improvements

---

Last Updated: October 2, 2025, 9:30 PM
Author: GitHub Copilot (with @scawful)
Status: In progress - window detection debugging needed