Enhance chat command functionality with ROM context integration and improve TUI initialization

docs/z3ed/AGENT-ROADMAP.md

@@ -24,7 +24,7 @@ This vision will be realized through a shared interface available in both the `z
 - **Components**:
     - `ConversationalAgentService`: The main class for managing the chat session.
     - Integration with existing `AIService` implementations (Ollama, Gemini).
-- **Status**: Not started.
+- **Status**: In progress — baseline service exists with chat history, tool loop handling, and structured response parsing. Next up: wiring in live ROM context and richer session state.
 
 ### 2. Read-Only "Tools" for the Agent
 - **Description**: To enable the agent to answer questions, we need to expand `z3ed` with a suite of read-only commands that the LLM can call. This is aligned with the "tool use" or "function calling" capabilities of modern LLMs.
@@ -37,14 +37,14 @@ This vision will be realized through a shared interface available in both the `z
     - `overworld set-area --map <id> --x <x> --y <y> --width <w> --height <h> --tile <id>`
     - `overworld replace-tile --map <id> --from <old_id> --to <new_id>`
     - `overworld blend-tiles --map <id> --pattern <name> --density <percent>`
-- **Status**: Some commands exist (`overworld get-tile`), but the suite needs to be expanded.
+- **Status**: Foundational commands (`resource-list`, `dungeon-list-sprites`) are live with JSON output. Focus is shifting to high-value Overworld and dialogue inspection tools.
 
 ### 3. TUI and GUI Chat Interfaces
 - **Description**: User-facing components for interacting with the `ConversationalAgentService`.
 - **Components**:
     - **TUI**: A new full-screen component in `z3ed` using FTXUI, providing a rich chat experience in the terminal.
     - **GUI**: A new ImGui widget that can be docked into the main `yaze` application window.
-- **Status**: Not started.
+- **Status**: In progress — CLI/TUI and GUI chat widgets exist, now rendering tables/JSON with readable formatting. Need to improve input ergonomics and synchronized history navigation.
 
 ### 4. Integration with the Proposal Workflow
 - **Description**: The final step is to connect the conversation to the action. When a user's prompt implies a desire to modify the ROM (e.g., "Okay, now add two more soldiers"), the `ConversationalAgentService` will trigger the existing `Tile16ProposalGenerator` (and future proposal generators for other resource types) to create a proposal.
@@ -60,26 +60,26 @@ This vision will be realized through a shared interface available in both the `z
 ## Consolidated Next Steps
 
 ### Immediate Priorities (Next Session)
-1.  **Implement Read-Only Agent Tools**:
+1.  **Share ROM Context with the Agent**:
-    - Add `resource list` command.
+    - Inject the active GUI/TUI ROM instance into `ConversationalAgentService`.
-    - Add `dungeon list-sprites` command.
+    - Ensure tool calls succeed without requiring `--rom` flags when running inside the editor.
-    - Ensure all new commands have JSON output options for machine readability.
+2.  **Expand Overworld Tool Coverage**:
-2.  **Stub out `ConversationalAgentService`**:
+    - Add read-only commands for tile searches, area summaries, and teleport destinations.
-    - Create the basic class structure.
+    - Guarantee each tool returns both JSON and human-readable summaries for the chat renderers.
-    - Implement simple chat history management.
+3.  **Document & Test the New Tooling**:
-3.  **Update `README.md` and Consolidate Docs**:
+    - Update the main `README.md` and relevant docs to cover the new chat formatting.
-    - Update the main `README.md` to reflect this new roadmap.
+    - Add regression tests (unit or golden JSON fixtures) for the new Overworld tools.
-    - Remove `IMPLEMENTATION-SESSION-OCT3-CONTINUED.md`.
-    - Merge any other scattered planning documents into this roadmap.
 
 ### Short-Term Goals (This Week)
-1.  **Build TUI Chat Interface**:
+1.  **Polish the TUI Chat Experience**:
-    - Create the FTXUI component.
+    - Tighten keyboard shortcuts, scrolling, and copy-to-clipboard behaviour.
-    - Connect it to the `ConversationalAgentService`.
+    - Align log file output with on-screen formatting for easier debugging.
-    - Implement basic input/output.
 2.  **Integrate Tool Use with LLM**:
     - Modify the `AIService` to support function calling/tool use.
     - Teach the agent to call the new read-only commands to answer questions.
+3.  **Land Overworld Tooling**:
+    - Ship at least two Overworld inspection commands with comprehensive tests.
+    - Record example transcripts demonstrating tool usage in both TUI and GUI.
 
 ### Long-Term Vision (Next Week and Beyond)
 1.  **Build GUI Chat Widget**:
@@ -109,11 +109,12 @@ We have made significant progress in laying the foundation for the conversationa
 - **Initial Agent "Tools"**: `resource-list` and `dungeon-list-sprites` commands are implemented.
 - **Tool Use Foundation**: The `ToolDispatcher` is implemented, and the AI services are aware of the new tool call format.
  - **Tool Loop Improvements**: Conversational flow now handles multi-step tool calls with default JSON output, allowing results to feed back into the chat without recursion.
+- **Structured Tool Output Rendering**: Both the TUI and GUI chat widgets now display tables and JSON payloads with friendly formatting, drastically improving readability.
 
 ### ✅ Build Configuration Issue Resolved
 The linker error is fixed. Both the CLI and GUI targets now link against `yaze_agent`, so the shared agent handlers (`HandleResourceListCommand`, `HandleDungeonListSpritesCommand`, etc.) compile once and are available to `ToolDispatcher` everywhere.
 
 ### 🚀 Next Steps
 1.  **Share ROM Context with the Agent**: Inject the active GUI ROM into `ConversationalAgentService` so tool calls work even when `--rom` flags are unavailable. Analyze the `src/app/rom.cc` and `src/app/rom.h` and `src/app/editor/editor_manager.cc` files for guidance on accessing the current project/ROM.
-2.  **Surface Tool Output in the UI**: Present JSON/table responses in the chat widgets with formatting instead of raw text dumps.
+2.  **Expand Tool Coverage**: Target Overworld navigation helpers (`overworld find-tile`, `overworld list-warps`, region summaries) and dialogue inspectors. Prioritize commands that unblock common level-design questions and emit concise table/JSON payloads.
-3.  **Expand Tool Coverage**: Analyze the yaze source code to identify Overworld features which can be implemented as tool calls and would be useful. Dungeon editing does not render in the GUI yet, so we will hold off on dungeon tools for now. Check `src/app/editor/overworld/` for the GUI interface, `src/app/zelda3/overworld/` for the core logic. We could also explore dialogue editing help as tool calls by looking at `src/app/editor/message/` since the MessageEditor is functional in the GUI.
+3.  **Publish Usage Examples**: Capture transcripts and screenshots that highlight structured chat output, then weave them into the docs so contributors can see the intended UX.

src/cli/handlers/agent.cc

@@ -66,7 +66,7 @@ absl::Status Agent::Run(const std::vector<std::string>& arg_vec) {
     return agent::HandleDungeonListSpritesCommand(subcommand_args);
   }
   if (subcommand == "chat") {
-    return agent::HandleChatCommand();
+    return agent::HandleChatCommand(rom_);
   }
 
   return absl::InvalidArgumentError(std::string(agent::kUsage));

src/cli/handlers/agent/commands.h

@@ -31,7 +31,7 @@ absl::Status HandleResourceListCommand(
 absl::Status HandleDungeonListSpritesCommand(
 	const std::vector<std::string>& arg_vec,
 	Rom* rom_context = nullptr);
-absl::Status HandleChatCommand();
+absl::Status HandleChatCommand(Rom& rom);
 
 }  // namespace agent
 }  // namespace cli

src/cli/handlers/agent/general_commands.cc

@@ -16,6 +16,7 @@
 #include "absl/strings/str_cat.h"
 #include "absl/strings/str_format.h"
 #include "absl/strings/str_replace.h"
+#include "absl/strings/string_view.h"
 #include "app/zelda3/dungeon/room.h"
 #include "cli/handlers/agent/common.h"
 #include "cli/modern_cli.h"
@@ -48,6 +49,29 @@ struct DescribeOptions {
   std::optional<std::string> last_updated;
 };
 
+absl::Status EnsureRomLoaded(Rom& rom, absl::string_view command_hint) {
+  if (rom.is_loaded()) {
+    return absl::OkStatus();
+  }
+
+  std::string rom_path = absl::GetFlag(FLAGS_rom);
+  if (rom_path.empty()) {
+    return absl::FailedPreconditionError(
+        absl::StrFormat(
+            "No ROM loaded. Pass --rom=<path> when running %s.\n"
+            "Example: z3ed %s --rom=zelda3.sfc",
+            command_hint, command_hint));
+  }
+
+  auto status = rom.LoadFromFile(rom_path);
+  if (!status.ok()) {
+    return absl::FailedPreconditionError(absl::StrFormat(
+        "Failed to load ROM from '%s': %s", rom_path, status.message()));
+  }
+
+  return absl::OkStatus();
+}
+
 absl::StatusOr<DescribeOptions> ParseDescribeArgs(
     const std::vector<std::string>& args) {
   DescribeOptions options;
@@ -114,21 +138,7 @@ absl::Status HandleRunCommand(const std::vector<std::string>& arg_vec,
   }
   std::string prompt = arg_vec[1];
 
-  if (!rom.is_loaded()) {
+  RETURN_IF_ERROR(EnsureRomLoaded(rom, "agent run --prompt \"<prompt>\""));
-    std::string rom_path = absl::GetFlag(FLAGS_rom);
-    if (rom_path.empty()) {
-      return absl::FailedPreconditionError(
-          "No ROM loaded. Use --rom=<path> to specify ROM file.\n"
-          "Example: z3ed agent run --rom=zelda3.sfc --prompt \"Your prompt "
-          "here\"");
-    }
-
-    auto status = rom.LoadFromFile(rom_path);
-    if (!status.ok()) {
-      return absl::FailedPreconditionError(absl::StrFormat(
-          "Failed to load ROM from '%s': %s", rom_path, status.message()));
-    }
-  }
 
   // 1. Create a sandbox ROM to apply changes to
   auto sandbox_or =
@@ -470,8 +480,10 @@ absl::Status HandleDescribeCommand(const std::vector<std::string>& arg_vec) {
   return absl::OkStatus();
 }
 
-absl::Status HandleChatCommand() {
+absl::Status HandleChatCommand(Rom& rom) {
-  tui::ChatTUI chat_tui;
+  RETURN_IF_ERROR(EnsureRomLoaded(rom, "agent chat"));
+
+  tui::ChatTUI chat_tui(&rom);
   chat_tui.Run();
   return absl::OkStatus();
 }
@@ -497,11 +509,7 @@ absl::Status HandleAcceptCommand(const std::vector<std::string>& arg_vec,
   auto proposal = proposal_or.value();
 
   // 2. Ensure the main ROM is loaded.
-  if (!rom.is_loaded()) {
+  RETURN_IF_ERROR(EnsureRomLoaded(rom, "agent accept --proposal-id <id>"));
-    return absl::FailedPreconditionError(
-        "No ROM loaded. Use --rom=<path> to specify the ROM to apply changes "
-        "to.");
-  }
 
   // 3. Apply the proposal to the main ROM.
   auto apply_status = generator.ApplyProposal(proposal, &rom);

src/cli/tui/chat_tui.cc

@@ -14,7 +14,16 @@ namespace tui {
 
 using namespace ftxui;
 
-ChatTUI::ChatTUI() = default;
+ChatTUI::ChatTUI(Rom* rom_context) : rom_context_(rom_context) {
+  if (rom_context_ != nullptr) {
+    agent_service_.SetRomContext(rom_context_);
+  }
+}
+
+void ChatTUI::SetRomContext(Rom* rom_context) {
+  rom_context_ = rom_context;
+  agent_service_.SetRomContext(rom_context_);
+}
 
 void ChatTUI::Run() {
   auto input = Input(&input_message_, "Enter your message...");

src/cli/tui/chat_tui.h

@@ -6,13 +6,17 @@
 #include "cli/service/agent/conversational_agent_service.h"
 
 namespace yaze {
+
+class Rom;
+
 namespace cli {
 namespace tui {
 
 class ChatTUI {
  public:
-  ChatTUI();
+  explicit ChatTUI(Rom* rom_context = nullptr);
   void Run();
+  void SetRomContext(Rom* rom_context);
 
  private:
   void Render();
@@ -21,6 +25,7 @@ class ChatTUI {
   ftxui::ScreenInteractive screen_ = ftxui::ScreenInteractive::Fullscreen();
   std::string input_message_;
   agent::ConversationalAgentService agent_service_;
+  Rom* rom_context_ = nullptr;
 };
 
 }  // namespace tui