- Added new test commands for GUI automation in `test_commands.cc`, including handling test runs, statuses, listings, and results.
- Refactored instances of `app::core::AsarWrapper` to `core::AsarWrapper` across multiple files for consistency.
- Updated CMake configuration to include new test command files.
- Modified integration and unit tests to reflect the changes in AsarWrapper usage.
- Ensured proper error handling and output formatting for test commands.
@@ -27,8 +27,8 @@ This document is the **source of truth** for the z3ed CLI architecture and desig
- **Test Harness Enhancements (IT-05 to IT-09)**: Expanding from basic automation to comprehensive testing platform
- Test introspection APIs for status/results polling
- Widget discovery for AI-driven interactions
- Test recording/replay for regression testing
- Enhanced error reporting with screenshots
- **✅ Test recording/replay for regression testing**
- Enhanced error reporting with screenshots and application-wide diagnostics
- CI/CD integration with standardized test formats
**📋 Planned Next**:
@@ -317,64 +317,23 @@ available_actions = [w.suggested_action for w in widgets.buttons if w.is_enabled
z3ed_client.Click(target="button:Save Changes")
```
#### IT-07: Test Recording & Replay (8-10 hours)
**Problem**: No way to capture manual workflows for regression. Testers repeat same actions every release.
#### IT-07: Test Recording & Replay ✅ COMPLETE
**Outcome**: Recording workflow, replay runner, and JSON script format shipped alongside CLI commands (`z3ed test record start|stop`, `z3ed test replay`). Regression coverage captured in `scripts/test_record_replay_e2e.sh`; documentation updated with quick-start examples. Focus now shifts to error diagnostics and artifact surfacing (IT-08).
**Solution**: Add recording workflow:
-`StartRecording(output_file)` → Begins capturing all RPC calls
-`StopRecording()` → Saves to JSON test script
-`ReplayTest(test_script)` → Executes recorded actions with validation
#### IT-08: Holistic Error Reporting (5-7 hours)
**Problem**: Errors surface differently across the CLI, ImGuiTestHarness, and EditorManager. Failures lack actionable context, slowing down triage and AI agent autonomy.
- **Harness Diagnostics**: Implement the Screenshot RPC, capture widget tree/state, and bundle execution context for every failed run.
- **Structured Error Envelope**: Introduce a shared `ErrorAnnotatedResult` format (status + metadata + hints) adopted by z3ed, harness services, and EditorManager subsystems.
- **Artifact Surfacing**: Persist artifacts under `test-results/<test_id>/`; expose paths in CLI output and in-app overlays.
- **Developer Experience**: Provide HTML + JSON result formats, actionable hints (“Re-run with --follow”, “Open screenshot: …”), and cross-links to recorded sessions for replay.
**Benefits**:
-QA engineers record test scenarios once, replay forever
-Test scripts version controlled alongside code
-Parameterized tests (e.g., test with different ROMs)
-Foundation for test suite management (smoke, regression, nightly)
#### IT-08: Enhanced Error Reporting (3-4 hours)
**Problem**: Test failures lack context. Developer sees "Window not visible" but doesn't know why.
**Solution**: Capture rich context on failure:
- Screenshot (implement stub RPC)
- Widget state dump (full hierarchy with properties)
@@ -17,14 +17,14 @@ The z3ed CLI and AI agent workflow system has completed major infrastructure mil
- **IT-02**: CLI Agent Test - Natural language → automated GUI testing (implementation complete)
**🔄 Active Phase**:
- **Test Harness Enhancements (IT-05 to IT-09)**: Expanding from basic automation to comprehensive testing platform
- **Test Harness Enhancements (IT-05 to IT-09)**: Expanding from basic automation to comprehensive testing platform with a renewed emphasis on system-wide error reporting
**📋 Next Phases**:
- **Priority 1**: Test Introspection API (IT-05) - Enable test status querying and result polling
- **Priority 2**: Widget Discovery API (IT-06) - AI agents enumerate available GUI interactions
- **Priority 3**: Test Recording & Replay (IT-07) - Capture workflows for regression testing
| IT-04 | Complete E2E validation with real YAZE widgets | ImGuiTest Bridge | Test | ✅ Done | IT-02 - All 5 functional tests passing, window detection fixed with yield buffer |
| IT-05 | Add test introspection RPCs (GetTestStatus, ListTests, GetResults) | ImGuiTest Bridge | Code | 📋 Planned | IT-01 - Enable clients to poll test results and query execution state |
| IT-06 | Implement widget discovery API for AI agents | ImGuiTest Bridge | Code | 📋 Planned | IT-01 - DiscoverWidgets RPC to enumerate windows, buttons, inputs |
| IT-07 | Add test recording/replay for regression testing | ImGuiTest Bridge | Code | 📋 Planned | IT-05 - RecordSession/ReplaySession RPCs with JSON test scripts |
| IT-08 | Enhance error reporting with screenshots and state dumps | ImGuiTest Bridge | Code | 📋 Planned | IT-01 - Capture widget state on failure for debugging |
| IT-07 | Add test recording/replay for regression testing | ImGuiTest Bridge | Code | ✅ Done | IT-05 - RecordSession/ReplaySession RPCs with JSON test scripts |
| IT-08 | Enhance error reporting with screenshots and state dumps | ImGuiTest Bridge | Code | <EFBFBD> Active | IT-01 - Capture widget state on failure for debugging |
| IT-09 | Create standardized test suite format for CI integration | ImGuiTest Bridge | Infra | 📋 Planned | IT-07 - JSON/YAML test suite format compatible with CI/CD pipelines |
| VP-01 | Expand CLI unit tests for new commands and sandbox flow. | Verification Pipeline | Test | 📋 Planned | RC/AW tasks |
| VP-02 | Add harness integration tests with replay scripts. | Verification Pipeline | Test | 📋 Planned | IT tasks |
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
scawful