Enhance z3ed Agent Roadmap and CLI Reference with New Tool Commands and Chat Features
This commit is contained in:
@@ -108,13 +108,12 @@ We have made significant progress in laying the foundation for the conversationa
|
||||
- **GUI Chat Widget Stub**: An `AgentChatWidget` is integrated into the main GUI.
|
||||
- **Initial Agent "Tools"**: `resource-list` and `dungeon-list-sprites` commands are implemented.
|
||||
- **Tool Use Foundation**: The `ToolDispatcher` is implemented, and the AI services are aware of the new tool call format.
|
||||
- **Tool Loop Improvements**: Conversational flow now handles multi-step tool calls with default JSON output, allowing results to feed back into the chat without recursion.
|
||||
|
||||
### ⚠️ Current Blocker: Build Configuration
|
||||
We are currently facing a linker error when building the main `yaze` application with gRPC support. The `ToolDispatcher` is unable to find the definitions for the `HandleResourceListCommand` and `HandleDungeonListSpritesCommand` functions.
|
||||
|
||||
**Root Cause**: These handler functions are only compiled as part of the `z3ed` target, not the `yaze` target. The `ToolDispatcher`, which is now included in the `yaze` build, depends on them.
|
||||
### ✅ Build Configuration Issue Resolved
|
||||
The linker error is fixed. Both the CLI and GUI targets now link against `yaze_agent`, so the shared agent handlers (`HandleResourceListCommand`, `HandleDungeonListSpritesCommand`, etc.) compile once and are available to `ToolDispatcher` everywhere.
|
||||
|
||||
### 🚀 Next Steps
|
||||
1. **Resolve the Build Issue**: The immediate next step is to fix the linker error. This will likely involve a thoughtful refactoring of our CMake configuration to better share sources between the `yaze` and `z3ed` targets.
|
||||
2. **Simplify CMake Structure**: As discussed, the current structure of including `.cmake` files from various subdirectories is becoming difficult to manage. We should consider flattening this into a more centralized source list in the main `src/CMakeLists.txt`.
|
||||
3. **Continue with Tool Integration**: Once the build is fixed, we can proceed with integrating the tool execution results back into the conversational loop.
|
||||
1. **Share ROM Context with the Agent**: Inject the active GUI ROM into `ConversationalAgentService` so tool calls work even when `--rom` flags are unavailable.
|
||||
2. **Surface Tool Output in the UI**: Present JSON/table responses in the chat widgets with formatting instead of raw text dumps.
|
||||
3. **Expand Tool Coverage**: Add the next batch of read-only utilities (`dungeon get-info`, `overworld find-tile`) now that the tooling loop is stable.
|
||||
|
||||
@@ -207,6 +207,52 @@ Examples:
|
||||
- `dungeon` - Dungeon editing
|
||||
- `agent` - Agent commands
|
||||
|
||||
#### `agent resource-list` - Enumerate labeled resources for the AI
|
||||
```bash
|
||||
z3ed agent resource-list --type <resource> [--format <table|json>]
|
||||
|
||||
Options:
|
||||
--type <resource> Required label family (dungeon, overworld, sprite, palette, etc.)
|
||||
--format <mode> Output format, defaults to `table`. Use `json` for LLM tooling.
|
||||
|
||||
Examples:
|
||||
# Show dungeon labels in a table
|
||||
z3ed agent resource-list --type dungeon
|
||||
|
||||
# Emit JSON for the conversation agent to consume
|
||||
z3ed agent resource-list --type overworld --format json
|
||||
```
|
||||
|
||||
**Notes**:
|
||||
- When the conversation agent invokes this tool, JSON output is requested automatically.
|
||||
- Labels are loaded from `ResourceContextBuilder`, so the command reflects project-specific metadata.
|
||||
|
||||
#### `agent dungeon-list-sprites` - Inspect sprites in a dungeon room
|
||||
```bash
|
||||
z3ed agent dungeon-list-sprites --room <hex_id> [--format <table|json>]
|
||||
|
||||
Options:
|
||||
--room <hex_id> Dungeon room ID (hexadecimal). Accepts `0x` prefixes or decimal.
|
||||
--format <mode> Output format, defaults to `table`.
|
||||
|
||||
Examples:
|
||||
z3ed agent dungeon-list-sprites --room 0x012
|
||||
z3ed agent dungeon-list-sprites --room 18 --format json
|
||||
```
|
||||
|
||||
**Output**:
|
||||
- Table view prints sprite id/x/y in hex+decimal for quick inspection.
|
||||
- JSON view is tailored for the LLM toolchain and is returned automatically during tool calls.
|
||||
|
||||
#### `agent chat` - Interactive terminal chat (TUI prototype)
|
||||
```bash
|
||||
z3ed agent chat
|
||||
```
|
||||
|
||||
- Opens an FTXUI-based interface with scrolling history and input box.
|
||||
- Uses the shared `ConversationalAgentService`, so the same backend powers the GUI widget.
|
||||
- Useful for manual testing of tool dispatching and new prompting strategies.
|
||||
|
||||
#### `agent test` - Automated GUI testing (IT-02)
|
||||
```bash
|
||||
z3ed agent test --prompt "<test_description>" [--host <hostname>] [--port <port>]
|
||||
|
||||
@@ -40,11 +40,20 @@ z3ed agent plan --prompt "Place a tree at position 10, 10 on map 0"
|
||||
# Execute in sandbox with auto-approval
|
||||
z3ed agent run --prompt "Create a 3x3 water pond at 15, 20" --rom zelda3.sfc --sandbox
|
||||
|
||||
# Chat with the agent in the terminal (FTXUI prototype)
|
||||
z3ed agent chat
|
||||
|
||||
# List all proposals
|
||||
z3ed agent list
|
||||
|
||||
# View proposal details
|
||||
z3ed agent diff --proposal <id>
|
||||
|
||||
# Inspect project metadata for the LLM toolchain
|
||||
z3ed agent resource-list --type dungeon --format json
|
||||
|
||||
# Dump sprite placements for a dungeon room
|
||||
z3ed agent dungeon-list-sprites --room 0x012
|
||||
```
|
||||
|
||||
### GUI Testing Commands
|
||||
@@ -220,6 +229,11 @@ AI agent features require:
|
||||
- ✅ Updated README with clear dependency requirements
|
||||
- ✅ Added Windows compatibility notes
|
||||
|
||||
### Conversational Loop
|
||||
- ✅ Tool dispatcher defaults to JSON when invoked by the agent, keeping outputs machine-readable.
|
||||
- ✅ Conversational service now replays tool results without recursion, improving chat stability.
|
||||
- ✅ Mock AI service issues sample tool calls so the loop can be exercised without a live LLM.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "OpenSSL not found" warning
|
||||
|
||||
Reference in New Issue
Block a user