diff --git a/docs/z3ed/AGENT-ROADMAP.md b/docs/z3ed/AGENT-ROADMAP.md index 0aa28998..c317fc47 100644 --- a/docs/z3ed/AGENT-ROADMAP.md +++ b/docs/z3ed/AGENT-ROADMAP.md @@ -59,34 +59,30 @@ This vision will be realized through a shared interface available in both the `z ### Immediate Priorities 1. **Expand Overworld Tool Coverage**: - - Add read-only commands for tile searches, area summaries, and teleport destinations. - - Guarantee each tool returns both JSON and human-readable summaries for the chat renderers. -2. **Document & Test the New Tooling**: - - Update the main `README.md` and relevant docs to cover the new chat formatting. - - Add regression tests (unit or golden JSON fixtures) for the new Overworld tools. - -3. **Polish the TUI Chat Experience**: + - ✅ Ship read-only tile searches (`overworld find-tile`) with shared formatting for CLI and agent calls. + - Next: add area summaries, teleport destination lookups, and keep JSON/Text parity for all new tools. +2. **Polish the TUI Chat Experience**: - Tighten keyboard shortcuts, scrolling, and copy-to-clipboard behaviour. - Align log file output with on-screen formatting for easier debugging. -4. **Integrate Tool Use with LLM**: +3. **Integrate Tool Use with LLM**: - Modify the `AIService` to support function calling/tool use. - Teach the agent to call the new read-only commands to answer questions. -5. **Land Overworld Tooling**: - - Ship at least two Overworld inspection commands with comprehensive tests. - -6. **Build GUI Chat Widget**: +4. **Document & Test the New Tooling**: + - Update the main `README.md` and relevant docs to cover the new chat formatting. + - Add regression tests (unit or golden JSON fixtures) for the new Overworld tools. +5. **Build GUI Chat Widget**: - Create the ImGui component. - Ensure it shares the same backend service as the TUI. -7. **Full Integration with Proposal System**: +6. **Full Integration with Proposal System**: - Implement the logic for the agent to transition from conversation to proposal generation. -8. **Expand Tool Arsenal**: +7. **Expand Tool Arsenal**: - Continuously add new read-only commands to give the agent more capabilities to inspect the ROM. -9. **Multi-Modal Agent**: +8. **Multi-Modal Agent**: - Explore the possibility of the agent generating and displaying images (e.g., a map of a dungeon room) in the chat. -10. **Advanced Configuration**: +9. **Advanced Configuration**: - Implement environment variables for selecting AI providers and models (e.g., `YAZE_AI_PROVIDER`, `OLLAMA_MODEL`). - Add CLI flags for overriding the provider and model on a per-command basis. -11. **Performance and Cost-Saving**: +10. **Performance and Cost-Saving**: - Implement a response cache to reduce latency and API costs. - Add token usage tracking and reporting. @@ -103,10 +99,13 @@ We have made significant progress in laying the foundation for the conversationa - **Tool Loop Improvements**: Conversational flow now handles multi-step tool calls with default JSON output, allowing results to feed back into the chat without recursion. - **Structured Tool Output Rendering**: Both the TUI and GUI chat widgets now display tables and JSON payloads with friendly formatting, drastically improving readability. - **Overworld Inspection Suite**: Added `overworld describe-map` and `overworld list-warps` commands producing text/JSON summaries for map metadata and warp points, with agent tooling hooks. - -### ✅ Build Configuration Issue Resolved -The linker error is fixed. Both the CLI and GUI targets now link against `yaze_agent`, so the shared agent handlers (`HandleResourceListCommand`, `HandleDungeonListSpritesCommand`, etc.) compile once and are available to `ToolDispatcher` everywhere. +- **Overworld Tile Search Tool**: Added `overworld find-tile` across CLI and agent tooling with shared ROM context handling and regression tests. ### 🚀 Next Steps -1. **Share ROM Context with the Agent**: Inject the active GUI ROM into `ConversationalAgentService` so tool calls work even when `--rom` flags are unavailable. Analyze the `src/app/rom.cc` and `src/app/rom.h` and `src/app/editor/editor_manager.cc` files for guidance on accessing the current project/ROM. -2. **Expand Tool Coverage**: Target Overworld navigation helpers (`overworld find-tile`, `overworld list-warps`, region summaries) and dialogue inspectors. Prioritize commands that unblock common level-design questions and emit concise table/JSON payloads. \ No newline at end of file +1. **Integrate Tool Use with LLM**: + - Modify the `AIService` to support function calling/tool use. + - Teach the agent to call the new read-only commands to answer questions. +2. **Polish the TUI Chat Experience**: + - Tighten keyboard shortcuts, scrolling, and copy-to-clipboard behaviour. + - Align log file output with on-screen formatting for easier debugging. +2. **Expand Tool Coverage**: Target additional Overworld navigation helpers (region summaries, teleport lookups) and dialogue inspectors. Prioritize commands that unblock common level-design questions and emit concise table/JSON payloads. \ No newline at end of file diff --git a/docs/z3ed/README.md b/docs/z3ed/README.md index 0614da04..9ce54af2 100644 --- a/docs/z3ed/README.md +++ b/docs/z3ed/README.md @@ -54,6 +54,9 @@ z3ed agent resource-list --type dungeon --format json # Dump sprite placements for a dungeon room z3ed agent dungeon-list-sprites --room 0x012 + +# Search overworld maps for a tile ID using shared agent tooling +z3ed agent overworld-find-tile --tile 0x02E --map 0x05 ``` ### GUI Testing Commands @@ -201,6 +204,9 @@ z3ed overworld find-tile --tile 0x02E --format json # Narrow search to Light World map 0x05 z3ed overworld find-tile --tile 0x02E --map 0x05 + +# Ask the agent to perform the same lookup (returns JSON by default) +z3ed agent overworld-find-tile --tile 0x02E --map 0x05 ``` ### Label-Aware Dungeon Edit diff --git a/src/cli/handlers/agent.cc b/src/cli/handlers/agent.cc index 55f0bf6e..53fdeedb 100644 --- a/src/cli/handlers/agent.cc +++ b/src/cli/handlers/agent.cc @@ -12,7 +12,7 @@ namespace agent { namespace { constexpr absl::string_view kUsage = - "Usage: agent " + "Usage: agent " "[options]"; } // namespace @@ -65,6 +65,15 @@ absl::Status Agent::Run(const std::vector& arg_vec) { if (subcommand == "dungeon-list-sprites") { return agent::HandleDungeonListSpritesCommand(subcommand_args); } + if (subcommand == "overworld-find-tile") { + return agent::HandleOverworldFindTileCommand(subcommand_args); + } + if (subcommand == "overworld-describe-map") { + return agent::HandleOverworldDescribeMapCommand(subcommand_args); + } + if (subcommand == "overworld-list-warps") { + return agent::HandleOverworldListWarpsCommand(subcommand_args); + } if (subcommand == "chat") { return agent::HandleChatCommand(rom_); } diff --git a/src/cli/handlers/agent/commands.h b/src/cli/handlers/agent/commands.h index e786b036..35cc10fa 100644 --- a/src/cli/handlers/agent/commands.h +++ b/src/cli/handlers/agent/commands.h @@ -31,6 +31,9 @@ absl::Status HandleResourceListCommand( absl::Status HandleDungeonListSpritesCommand( const std::vector& arg_vec, Rom* rom_context = nullptr); +absl::Status HandleOverworldFindTileCommand( + const std::vector& arg_vec, + Rom* rom_context = nullptr); absl::Status HandleOverworldDescribeMapCommand( const std::vector& arg_vec, Rom* rom_context = nullptr); diff --git a/src/cli/handlers/agent/tool_commands.cc b/src/cli/handlers/agent/tool_commands.cc index adbe0969..8ec21e46 100644 --- a/src/cli/handlers/agent/tool_commands.cc +++ b/src/cli/handlers/agent/tool_commands.cc @@ -209,6 +209,174 @@ absl::Status HandleDungeonListSpritesCommand( return absl::OkStatus(); } +absl::Status HandleOverworldFindTileCommand( + const std::vector& arg_vec, Rom* rom_context) { + std::optional tile_value; + std::optional map_value; + std::optional world_value; + std::string format = "json"; + std::optional rom_override; + + for (size_t i = 0; i < arg_vec.size(); ++i) { + const std::string& token = arg_vec[i]; + if (token == "--tile") { + if (i + 1 >= arg_vec.size()) { + return absl::InvalidArgumentError("--tile requires a value."); + } + tile_value = arg_vec[++i]; + } else if (absl::StartsWith(token, "--tile=")) { + tile_value = token.substr(7); + } else if (token == "--map") { + if (i + 1 >= arg_vec.size()) { + return absl::InvalidArgumentError("--map requires a value."); + } + map_value = arg_vec[++i]; + } else if (absl::StartsWith(token, "--map=")) { + map_value = token.substr(6); + } else if (token == "--world") { + if (i + 1 >= arg_vec.size()) { + return absl::InvalidArgumentError("--world requires a value."); + } + world_value = arg_vec[++i]; + } else if (absl::StartsWith(token, "--world=")) { + world_value = token.substr(8); + } else if (token == "--format") { + if (i + 1 >= arg_vec.size()) { + return absl::InvalidArgumentError("--format requires a value."); + } + format = absl::AsciiStrToLower(arg_vec[++i]); + } else if (absl::StartsWith(token, "--format=")) { + format = absl::AsciiStrToLower(token.substr(9)); + } else if (token == "--rom") { + if (i + 1 >= arg_vec.size()) { + return absl::InvalidArgumentError("--rom requires a value."); + } + rom_override = arg_vec[++i]; + } else if (absl::StartsWith(token, "--rom=")) { + rom_override = token.substr(6); + } + } + + if (!tile_value.has_value()) { + return absl::InvalidArgumentError( + "Usage: agent overworld-find-tile --tile [--map ] [--world ] [--format ]"); + } + + ASSIGN_OR_RETURN(int tile_numeric, + overworld::ParseNumeric(*tile_value)); + if (tile_numeric < 0 || tile_numeric > 0xFFFF) { + return absl::InvalidArgumentError( + absl::StrCat("Tile ID must be between 0x0000 and 0xFFFF (got ", + *tile_value, ")")); + } + + std::optional map_filter; + if (map_value.has_value()) { + ASSIGN_OR_RETURN(int parsed_map, + overworld::ParseNumeric(*map_value)); + if (parsed_map < 0 || parsed_map >= zelda3::kNumOverworldMaps) { + return absl::InvalidArgumentError( + absl::StrCat("Map ID out of range: ", *map_value)); + } + map_filter = parsed_map; + } + + std::optional world_filter; + if (world_value.has_value()) { + ASSIGN_OR_RETURN(int parsed_world, + overworld::ParseWorldSpecifier(*world_value)); + world_filter = parsed_world; + } + + if (map_filter.has_value()) { + ASSIGN_OR_RETURN(int inferred_world, + overworld::InferWorldFromMapId(*map_filter)); + if (world_filter.has_value() && inferred_world != *world_filter) { + return absl::InvalidArgumentError( + absl::StrCat("Map 0x", + absl::StrFormat("%02X", *map_filter), + " belongs to the ", + overworld::WorldName(inferred_world), + " World but --world requested ", + overworld::WorldName(*world_filter))); + } + if (!world_filter.has_value()) { + world_filter = inferred_world; + } + } + + if (format != "json" && format != "text") { + return absl::InvalidArgumentError( + absl::StrCat("Unsupported format: ", format)); + } + + Rom rom_storage; + Rom* rom = nullptr; + if (rom_context != nullptr && rom_context->is_loaded() && !rom_override.has_value()) { + rom = rom_context; + } else { + ASSIGN_OR_RETURN(auto rom_or, + LoadRomFromPathOrFlag(rom_override)); + rom_storage = std::move(rom_or); + rom = &rom_storage; + } + + zelda3::Overworld overworld_data(rom); + auto load_status = overworld_data.Load(rom); + if (!load_status.ok()) { + return load_status; + } + + overworld::TileSearchOptions search_options; + search_options.map_id = map_filter; + search_options.world = world_filter; + + ASSIGN_OR_RETURN(auto matches, + overworld::FindTileMatches(overworld_data, + static_cast(tile_numeric), + search_options)); + + if (format == "json") { + std::cout << "{\n"; + std::cout << absl::StrFormat( + " \"tile\": \"0x%04X\",\n", tile_numeric); + std::cout << absl::StrFormat( + " \"match_count\": %zu,\n", matches.size()); + std::cout << " \"matches\": [\n"; + for (size_t i = 0; i < matches.size(); ++i) { + const auto& match = matches[i]; + std::cout << absl::StrFormat( + " {\"map\": \"0x%02X\", \"world\": \"%s\", " + "\"local\": {\"x\": %d, \"y\": %d}, " + "\"global\": {\"x\": %d, \"y\": %d}}%s\n", + match.map_id, overworld::WorldName(match.world), match.local_x, + match.local_y, + match.global_x, match.global_y, + (i + 1 == matches.size()) ? "" : ","); + } + std::cout << " ]\n"; + std::cout << "}\n"; + } else { + std::cout << absl::StrFormat( + "🔎 Tile 0x%04X → %zu match(es)\n", + tile_numeric, matches.size()); + if (matches.empty()) { + std::cout << " No matches found." << std::endl; + return absl::OkStatus(); + } + + for (const auto& match : matches) { + std::cout << absl::StrFormat( + " • Map 0x%02X (%s World) local(%2d,%2d) global(%3d,%3d)\n", + match.map_id, overworld::WorldName(match.world), match.local_x, + match.local_y, + match.global_x, match.global_y); + } + } + + return absl::OkStatus(); +} + absl::Status HandleOverworldDescribeMapCommand( const std::vector& arg_vec, Rom* rom_context) { std::optional map_value; diff --git a/src/cli/handlers/overworld.cc b/src/cli/handlers/overworld.cc index 925275a8..772f17fa 100644 --- a/src/cli/handlers/overworld.cc +++ b/src/cli/handlers/overworld.cc @@ -244,73 +244,13 @@ absl::Status OverworldFindTile::Run(const std::vector& arg_vec) { return ow_status; } - struct TileMatch { - int map_id; - int world; - int local_x; - int local_y; - int global_x; - int global_y; - }; + overworld::TileSearchOptions search_options; + search_options.map_id = map_filter; + search_options.world = world_filter; - std::vector worlds_to_search; - if (world_filter.has_value()) { - worlds_to_search.push_back(*world_filter); - } else { - worlds_to_search = {0, 1, 2}; - } - - std::vector matches; - - for (int world : worlds_to_search) { - int world_start = 0; - int world_maps = 0; - switch (world) { - case 0: - world_start = 0x00; - world_maps = 0x40; - break; - case 1: - world_start = 0x40; - world_maps = 0x40; - break; - case 2: - world_start = 0x80; - world_maps = 0x20; - break; - default: - return absl::InvalidArgumentError( - absl::StrCat("Unknown world index: ", world)); - } - - overworld.set_current_world(world); - - for (int local_map = 0; local_map < world_maps; ++local_map) { - int map_id = world_start + local_map; - if (map_filter.has_value() && map_id != *map_filter) { - continue; - } - - int map_x_index = local_map % 8; - int map_y_index = local_map / 8; - - int global_x_start = map_x_index * 32; - int global_y_start = map_y_index * 32; - - for (int local_y = 0; local_y < 32; ++local_y) { - for (int local_x = 0; local_x < 32; ++local_x) { - int global_x = global_x_start + local_x; - int global_y = global_y_start + local_y; - - uint16_t tile = overworld.GetTile(global_x, global_y); - if (tile == tile_id) { - matches.push_back({map_id, world, local_x, local_y, global_x, - global_y}); - } - } - } - } - } + ASSIGN_OR_RETURN(auto matches, + overworld::FindTileMatches(overworld, tile_id, + search_options)); if (format == "json") { std::cout << "{\n"; diff --git a/src/cli/handlers/overworld_inspect.cc b/src/cli/handlers/overworld_inspect.cc index c504a108..902477d7 100644 --- a/src/cli/handlers/overworld_inspect.cc +++ b/src/cli/handlers/overworld_inspect.cc @@ -295,6 +295,97 @@ absl::StatusOr> CollectWarpEntries( return entries; } +absl::StatusOr> FindTileMatches( + zelda3::Overworld& overworld, uint16_t tile_id, + const TileSearchOptions& options) { + if (options.map_id.has_value()) { + RETURN_IF_ERROR(ValidateMapId(*options.map_id)); + } + if (options.world.has_value()) { + if (*options.world < 0 || *options.world > 2) { + return absl::InvalidArgumentError( + absl::StrFormat("Unknown world index: %d", *options.world)); + } + } + + if (options.map_id.has_value() && options.world.has_value()) { + ASSIGN_OR_RETURN(int inferred_world, + InferWorldFromMapId(*options.map_id)); + if (inferred_world != *options.world) { + return absl::InvalidArgumentError( + absl::StrFormat( + "Map 0x%02X belongs to the %s World but --world requested %s", + *options.map_id, WorldName(inferred_world), + WorldName(*options.world))); + } + } + + std::vector worlds; + if (options.world.has_value()) { + worlds.push_back(*options.world); + } else if (options.map_id.has_value()) { + ASSIGN_OR_RETURN(int inferred_world, + InferWorldFromMapId(*options.map_id)); + worlds.push_back(inferred_world); + } else { + worlds = {0, 1, 2}; + } + + std::vector matches; + + for (int world : worlds) { + int world_start = 0; + int world_maps = 0; + switch (world) { + case 0: + world_start = 0x00; + world_maps = 0x40; + break; + case 1: + world_start = 0x40; + world_maps = 0x40; + break; + case 2: + world_start = 0x80; + world_maps = 0x20; + break; + default: + return absl::InvalidArgumentError( + absl::StrFormat("Unknown world index: %d", world)); + } + + overworld.set_current_world(world); + + for (int local_map = 0; local_map < world_maps; ++local_map) { + int map_id = world_start + local_map; + if (options.map_id.has_value() && map_id != *options.map_id) { + continue; + } + + int map_x_index = local_map % 8; + int map_y_index = local_map / 8; + + int global_x_start = map_x_index * 32; + int global_y_start = map_y_index * 32; + + for (int local_y = 0; local_y < 32; ++local_y) { + for (int local_x = 0; local_x < 32; ++local_x) { + int global_x = global_x_start + local_x; + int global_y = global_y_start + local_y; + + uint16_t current_tile = overworld.GetTile(global_x, global_y); + if (current_tile == tile_id) { + matches.push_back({map_id, world, local_x, local_y, global_x, + global_y}); + } + } + } + } + } + + return matches; +} + } // namespace overworld } // namespace cli } // namespace yaze diff --git a/src/cli/handlers/overworld_inspect.h b/src/cli/handlers/overworld_inspect.h index f38ad16f..128e2432 100644 --- a/src/cli/handlers/overworld_inspect.h +++ b/src/cli/handlers/overworld_inspect.h @@ -81,6 +81,20 @@ struct WarpQuery { std::optional type; }; +struct TileMatch { + int map_id; + int world; + int local_x; + int local_y; + int global_x; + int global_y; +}; + +struct TileSearchOptions { + std::optional map_id; + std::optional world; +}; + absl::StatusOr ParseNumeric(std::string_view value, int base = 0); absl::StatusOr ParseWorldSpecifier(std::string_view value); absl::StatusOr InferWorldFromMapId(int map_id); @@ -93,6 +107,10 @@ absl::StatusOr BuildMapSummary(zelda3::Overworld& overworld, absl::StatusOr> CollectWarpEntries( const zelda3::Overworld& overworld, const WarpQuery& query); +absl::StatusOr> FindTileMatches( + zelda3::Overworld& overworld, uint16_t tile_id, + const TileSearchOptions& options = {}); + } // namespace overworld } // namespace cli } // namespace yaze diff --git a/src/cli/service/agent/tool_dispatcher.cc b/src/cli/service/agent/tool_dispatcher.cc index 0cfe0ee4..7d35e4ca 100644 --- a/src/cli/service/agent/tool_dispatcher.cc +++ b/src/cli/service/agent/tool_dispatcher.cc @@ -38,6 +38,8 @@ absl::StatusOr ToolDispatcher::Dispatch( status = HandleResourceListCommand(args, rom_context_); } else if (tool_call.tool_name == "dungeon-list-sprites") { status = HandleDungeonListSpritesCommand(args, rom_context_); + } else if (tool_call.tool_name == "overworld-find-tile") { + status = HandleOverworldFindTileCommand(args, rom_context_); } else if (tool_call.tool_name == "overworld-describe-map") { status = HandleOverworldDescribeMapCommand(args, rom_context_); } else if (tool_call.tool_name == "overworld-list-warps") {