Commit Graph

2647 Commits

Author SHA1 Message Date
scawful
3715ae98eb Enhance agent chat functionality with ROM context support and structured message rendering 2025-10-03 14:46:22 -04:00
scawful
dc6040551e Refine agent roadmap and README documentation for clarity and guidance on tool integration 2025-10-03 14:19:34 -04:00
scawful
b6ba7cb572 Add 'overworld find-tile' command for locating tile instances across maps 2025-10-03 14:12:05 -04:00
scawful
c7a7707d25 Enhance z3ed Agent Roadmap and CLI Reference with New Tool Commands and Chat Features 2025-10-03 13:51:01 -04:00
scawful
ba9f6533a4 Enhance AI Agent Integration and Tool Command Functionality
- Added support for JSON in CMake configuration for AI integrations.
- Implemented new tool commands: resource-list and dungeon-list-sprites.
- Created ToolDispatcher for managing tool command execution.
- Refactored CMake structure to include agent sources and improve build configuration.
- Updated agent roadmap and README documentation to reflect current status and next steps.
2025-10-03 13:34:54 -04:00
scawful
7c2bf8e1c7 Add ToolDispatcher for Enhanced Tool Call Management
- Introduced `ToolDispatcher` class to handle tool calls from the AI agent, allowing for dynamic execution of commands based on user requests.
- Updated `ConversationalAgentService` to integrate tool dispatching, enabling the agent to respond to tool calls and manage execution results.
- Enhanced `AgentResponse` structure to include a list of tool calls, facilitating communication between the AI and the tool dispatcher.
- Modified AI service implementations to parse and include tool calls in responses, improving the agent's interactive capabilities.

This commit significantly enhances the z3ed system's ability to manage and execute tool calls, paving the way for more complex interactions in ROM hacking.
2025-10-03 12:47:15 -04:00
scawful
208b9ade51 Integrate AI Agent Services and Chat Interface
- Added support for AI agent services, including `ConversationalAgentService`, to facilitate user interactions through a chat interface.
- Implemented `ChatTUI` for a terminal-based chat experience, allowing users to send messages and receive responses from the AI agent.
- Updated `EditorManager` to include options for displaying the agent chat widget and performance dashboard.
- Enhanced CMake configurations to include new source files for AI services and chat interface components.

This commit significantly expands the functionality of the z3ed system, paving the way for a more interactive and user-friendly experience in ROM hacking.
2025-10-03 12:39:48 -04:00
scawful
655c5547b2 Implement new agent commands for resource listing and dungeon sprite retrieval
- Added `HandleResourceListCommand` to list resources by type and format (table or JSON).
- Introduced `HandleDungeonListSpritesCommand` to retrieve and display sprites from specified dungeon rooms.
- Updated command handling in `agent.cc` to support new subcommands.
- Enhanced error handling for invalid inputs and missing ROM files.

This commit expands the functionality of the z3ed agent, enabling users to interact with game resources and dungeon sprites more effectively.
2025-10-03 10:09:53 -04:00
scawful
18eff96e61 Add z3ed Agent Roadmap Document
- Introduced a new `AGENT-ROADMAP.md` file outlining the strategic vision and implementation plan for the `z3ed` AI agent.
- Defined the core vision of transitioning to a conversational ROM hacking assistant with key features such as an interactive chat interface, ROM introspection, and contextual awareness.
- Detailed the technical implementation plan, including the development of a `ConversationalAgentService`, read-only tools for the agent, and user-facing TUI/GUI chat interfaces.
- Consolidated immediate priorities, short-term goals, and long-term vision for the agent's development.

This commit establishes a comprehensive roadmap for enhancing the z3ed agent's capabilities, paving the way for future AI-driven features and user interactions.
2025-10-03 10:06:31 -04:00
scawful
90ddc3d50c Refactor CLI Service Structure and Enhance AI Integration
- Restructured CLI service source files to improve organization, moving files into dedicated directories for better maintainability.
- Introduced new AI service components, including `AIService`, `MockAIService`, and `GeminiAIService`, to facilitate natural language command generation.
- Implemented `PolicyEvaluator` and `ProposalRegistry` for enhanced proposal management and policy enforcement in AI workflows.
- Updated CMake configurations to reflect new file paths and ensure proper linking of the restructured components.
- Enhanced test suite with new test workflow generation capabilities, improving the robustness of automated testing.

This commit significantly advances the architecture of the z3ed system, laying the groundwork for more sophisticated AI-driven features and streamlined development processes.
2025-10-03 09:54:27 -04:00
scawful
b89dcca93f Refactor Agent Commands and Enhance Resource Context Management
- Updated the immediate action plan to focus on integrating `Tile16ProposalGenerator` and `ResourceContextBuilder` into agent commands, improving command handling and proposal generation.
- Implemented the `SetTile` method in the `Overworld` class to facilitate tile modifications based on the current world context.
- Enhanced error handling in command execution to ensure robust feedback during ROM operations.
- Created new files for `Tile16ProposalGenerator` and `ResourceContextBuilder`, enabling structured management of tile changes and resource labels for AI prompts.

This commit advances the functionality of the z3ed system, laying the groundwork for more sophisticated AI-driven editing capabilities.
2025-10-03 09:35:49 -04:00
scawful
3473d37be4 Introduce Overworld & Dungeon AI Integration Plan
- Added a comprehensive plan for integrating AI-driven workflows in overworld and dungeon editing, focusing on visual editing and ResourceLabels awareness.
- Established a phased implementation approach, starting with SSL support and basic Tile16 command integration.
- Outlined success metrics for both overworld and dungeon editing, ensuring AI can effectively understand and manipulate game data.
- Created a new document detailing the strategic shift towards specialized AI workflows, enhancing the overall functionality of the z3ed system.

This commit sets the foundation for advanced AI capabilities in ROM editing, paving the way for future enhancements and user-friendly features.
2025-10-03 09:20:37 -04:00
scawful
923f5af068 feat: Add SSL/HTTPS support and enhance test command handling
- Introduced SSL/HTTPS support in `z3ed.cmake` for secure communication, required for Gemini API and future collaborative features.
- Added options for OpenSSL integration and macOS Keychain support.
- Expanded `test_commands.cc` with new command handlers for running, replaying, and managing tests, improving modularity and usability.
- Updated command usage messages for clarity and consistency.

This commit enhances the security and functionality of the CLI, paving the way for more robust features in future iterations.
2025-10-03 09:12:30 -04:00
scawful
758b0c0a35 fix: Refactor test command handling with modular structure and common utilities
- Added new `test_common` module with helper functions for prompting and input handling.
- Introduced `test_common.cc` and `test_common.h` for shared functionality across test commands.
- Updated `z3ed.cmake` to include `test_common` in the build process.
- Refactored `test_commands.cc` to utilize new common utilities for improved readability and maintainability.
- Adjusted paths for source files in the build configuration to ensure proper linking.

This commit lays the groundwork for a more organized and modular approach to handling test commands, facilitating future enhancements and maintenance.
2025-10-03 02:10:18 -04:00
scawful
ba12075ca9 Upgrade gemini model to 2.5-flash 2025-10-03 01:34:11 -04:00
scawful
ead4abbf33 feat: Add PromptBuilder for enhanced command prompting and examples 2025-10-03 01:32:18 -04:00
scawful
d875b45fcd feat(z3ed): Complete Phase 2 - Gemini AI service enhancement
Phase 2 Implementation Summary:
- Enhanced GeminiAIService with production-ready features
- Added GeminiConfig struct for flexible configuration
- Implemented health check system with graceful degradation
- Updated to Gemini v1beta API format
- Added robust JSON parsing with markdown stripping fallbacks
- Switched default model to gemini-1.5-flash (faster, cheaper)
- Enhanced error messages with actionable guidance
- Integrated into service factory with health checks
- Added comprehensive test infrastructure

Files Modified:
- src/cli/service/gemini_ai_service.h (added config struct)
- src/cli/service/gemini_ai_service.cc (rewritten for v1beta)
- src/cli/handlers/agent/general_commands.cc (factory update)
- docs/z3ed/LLM-IMPLEMENTATION-CHECKLIST.md (progress tracking)

Files Created:
- scripts/test_gemini_integration.sh (test suite)
- docs/z3ed/PHASE2-COMPLETE.md (implementation summary)
- docs/z3ed/LLM-PROGRESS-UPDATE.md (overall progress)

Build Status:  SUCCESS (macOS ARM64)
Test Status:  Graceful fallback validated
Pending: Real API key validation

See docs/z3ed/PHASE2-COMPLETE.md for details.
2025-10-03 01:16:39 -04:00
scawful
6cec21f7aa feat: Implement Ollama AI service integration with health checks and command generation 2025-10-03 01:00:28 -04:00
scawful
40a4e43db9 Add LLM integration summary and quickstart script for Ollama
- Created LLM-INTEGRATION-SUMMARY.md detailing the integration plan for Ollama, Gemini, and Claude.
- Updated README.md to reflect the shift in focus towards LLM integration.
- Added quickstart_ollama.sh script to facilitate testing of Ollama integration with z3ed.
2025-10-03 00:51:05 -04:00
scawful
287f04ffc4 feat: Implement CLI test suite commands with YAML support and interactive creation 2025-10-03 00:39:02 -04:00
scawful
fa831f8719 feat: Add test suite handling with loader and reporter integration 2025-10-03 00:22:45 -04:00
scawful
dbabb8bc39 feat: Add replay command handling with argument parsing and output formatting 2025-10-03 00:12:06 -04:00
scawful
107b681342 Implement test suite loading and reporting functionality
- Added `TestSuiteDefinition`, `TestGroupDefinition`, and `TestCaseDefinition` structures to represent test suite configurations.
- Implemented parsing logic for test suite definitions from a file in `test_suite_loader.cc` and `test_suite_loader.h`.
- Introduced `ReplayTestResult` structure in `gui_automation_client.h` to handle results from replaying tests.
- Added `ReplayTest` method in `GuiAutomationClient` to facilitate test replay functionality.
- Created `BuildTextSummary` and `BuildJUnitReport` functions in `test_suite_reporter.cc` for generating test reports.
- Implemented `WriteJUnitReport` function to save JUnit formatted reports to a specified output path.
- Enhanced error handling and logging throughout the new functionalities.
2025-10-03 00:03:38 -04:00
scawful
0447d6f8a1 feat: Implement auto-capture of screenshots and diagnostics on test failures
- Added a new helper function `CaptureHarnessScreenshot` to encapsulate SDL screenshot logic.
- Updated `ImGuiTestHarnessServiceImpl::Screenshot` to utilize the new screenshot helper.
- Enhanced `TestManager::CaptureFailureContext` to automatically capture screenshots and widget state on test failures.
- Introduced new fields in the `GetTestResultsResponse` proto for screenshot path, size, failure context, and widget state.
- Updated CLI and gRPC client to expose new diagnostic fields in test results.
- Ensured that screenshots are saved in a structured directory under the system's temp directory.
- Improved logging for auto-capture events, including success and failure messages.
2025-10-02 23:36:09 -04:00
scawful
c348f7f91f guard JSON serialization for widget state capture for cross platform 2025-10-02 23:16:15 -04:00
scawful
1f16361284 feat: Add widget state capture utilities and include JSON directory for testing 2025-10-02 22:54:02 -04:00
scawful
c202aa9fa4 feat: Enhance widget discovery with telemetry data and improve output formatting 2025-10-02 22:50:47 -04:00
scawful
21074f6445 feat: Enhance widget state capture with fallback diagnostics and update test harness interactions 2025-10-02 21:44:15 -04:00
scawful
4098011647 Refactor code structure for improved readability and maintainability 2025-10-02 21:16:15 -04:00
scawful
4613722d7a Fix windows build main usage 2025-10-02 21:09:06 -04:00
scawful
92d65e399e feat: Implement auto-capture of failure diagnostics and update related documentation 2025-10-02 21:05:31 -04:00
scawful
56c2132b78 fix: Increase stack size for Windows executables and improve test discovery settings 2025-10-02 20:58:44 -04:00
scawful
0fb8ba4202 Update documentation 2025-10-02 20:55:28 -04:00
scawful
e3621d7a1f Add IT-08 Screenshot RPC Completion Report and IT-10 Collaborative Editing Documentation
- Created IT-08-SCREENSHOT-COMPLETION.md detailing the implementation of the Screenshot RPC, including technical summary, testing results, design decisions, and future work.
- Introduced IT-10-COLLABORATIVE-EDITING.md outlining the vision, user stories, architecture, implementation plan, and success metrics for real-time collaborative editing in YAZE.
2025-10-02 20:43:22 -04:00
scawful
5baa1b144d feat: Implement screenshot capture functionality in ImGuiTestHarnessService 2025-10-02 20:22:47 -04:00
scawful
ef631d263a fix: Enhance build environment verification scripts with automatic issue fixes and improved CMake version checks 2025-10-02 19:55:07 -04:00
scawful
cadda0d446 Enhance build environment verification with comprehensive scripts
- Added `verify-build-environment.ps1` for Windows users to check CMake, Git, Visual Studio, and submodules, with options for verbose output, automatic fixes, and cache cleaning.
- Introduced `verify-build-environment.sh` for macOS/Linux users, providing similar functionality with detailed output and automatic issue resolution.
- Updated documentation to include new verification steps and usage instructions for both scripts.
- Improved dependency checks and cache management in both scripts to ensure a smooth build setup.
2025-10-02 19:49:06 -04:00
scawful
0bd9e64c43 fix: Update platform conditionals to include UNIX for asset handling 2025-10-02 19:41:39 -04:00
scawful
5aa4fb5b56 chore: Update subproject commit for imgui 2025-10-02 19:39:39 -04:00
scawful
a78724d71b fix .gitignore on src/lib 2025-10-02 19:39:17 -04:00
scawful
6b13c2ea0a Implement GUI Automation Test Commands and Refactor AsarWrapper Usage
- Added new test commands for GUI automation in `test_commands.cc`, including handling test runs, statuses, listings, and results.
- Refactored instances of `app::core::AsarWrapper` to `core::AsarWrapper` across multiple files for consistency.
- Updated CMake configuration to include new test command files.
- Modified integration and unit tests to reflect the changes in AsarWrapper usage.
- Ensured proper error handling and output formatting for test commands.
2025-10-02 19:33:05 -04:00
scawful
d8f863a9ce feat: Add test recording and replay functionality to ImGuiTestHarness
- Introduced TestRecorder class to capture GUI automation RPCs and export them as JSON test scripts.
- Implemented StartRecording, StopRecording, and ReplayTest RPCs in ImGuiTestHarnessService.
- Updated imgui_test_harness.proto to include new RPCs and message definitions for recording and replay.
- Created TestScriptParser for reading and writing test scripts in JSON format.
- Enhanced DiscoverWidgetsRequest and DiscoveredWidget messages with detailed comments for clarity.
- Added ScopedSuspension mechanism in TestRecorder to manage recording state.
2025-10-02 18:00:58 -04:00
scawful
e0b308a93f Guard glob.h and mark project build as unimplemented on Windows 2025-10-02 18:00:18 -04:00
scawful
22f0e5006b feat: Implement widget discovery feature in GUI automation
- Added `DiscoverWidgets` RPC to the ImGuiTestHarness service for enumerating GUI widgets.
- Introduced `WidgetDiscoveryService` to handle widget collection and filtering based on various criteria.
- Updated `agent gui discover` command to support new options for filtering and output formats.
- Enhanced `GuiAutomationClient` to facilitate widget discovery requests and responses.
- Added necessary protobuf messages for widget discovery in `imgui_test_harness.proto`.
- Updated CLI command handling to include new GUI discovery functionality.
- Improved documentation for the `agent gui discover` command with examples and output formats.
2025-10-02 16:56:15 -04:00
scawful
3944861b38 refactor(agent): Remove unnecessary preprocessor directive for gRPC includes 2025-10-02 16:25:45 -04:00
scawful
4b5e866871 feat(cli): Enhance test command functionality and add new status features
- Refactored the HandleTestCommand to support subcommands: status, list, and results.
- Implemented HandleTestStatusCommand to fetch and display the status of a test by ID.
- Added HandleTestListCommand to list tests with optional filters for category and status.
- Created HandleTestResultsCommand to retrieve detailed results for a specific test execution.
- Introduced new structures for test status details, test summaries, and results.
- Updated GuiAutomationClient to support gRPC calls for fetching test status and results.
- Enhanced the ModernCLI to reflect new command usage for test operations.
- Added JSON and YAML formatting for test results output.
2025-10-02 16:14:09 -04:00
scawful
b3bcd801a0 feat: Add test introspection APIs and harness test management
- Introduced new gRPC service methods: GetTestStatus, ListTests, and GetTestResults for enhanced test introspection.
- Defined corresponding request and response message types in the proto file.
- Implemented test harness execution tracking in TestManager, including methods to register, mark, and retrieve test execution details.
- Enhanced test logging and summary capabilities to support introspection features.
- Updated existing structures to accommodate new test management functionalities.
2025-10-02 15:42:07 -04:00
scawful
3a573c0764 doc: Plan test harness with introspection capabilities (IT-05) 2025-10-02 15:00:34 -04:00
scawful
fdead0e9e5 feat: Enable conditional compilation for Policy Evaluation Framework in yaze target 2025-10-02 14:29:53 -04:00
scawful
510b11d9d7 doc: Policy Evaluation Framework and Remote Control Workflows
- Added Policy Evaluation Framework with core components including PolicyEvaluator service, policy types, severity levels, and GUI integration.
- Created documentation for the Policy Evaluation Framework detailing implementation, configuration, and testing plans.
- Introduced Remote Control Agent Workflows documentation, outlining gRPC interactions for automated editing in YAZE.
- Removed outdated Test Validation Status document and replaced it with updated Widget ID Next Actions documentation.
- Established widget registry integration for improved remote control capabilities and added support for hierarchical widget IDs.
- Enhanced test harness functionality to support widget discovery and interaction through gRPC.
2025-10-02 14:22:17 -04:00