feat: Introduce mock ROM mode for testing without actual ROM files

- Added a new feature to the `z3ed` AI agent allowing testing in mock ROM mode, which creates a minimal valid ROM structure with embedded labels but no actual game data. - Updated the `agent_test_suite.sh` script to default to mock ROM mode for easier testing. - Introduced `--mock-rom` command line flag to enable mock ROM mode in various agent commands. - Enhanced documentation to cover the usage and benefits of mock ROM mode for CI/CD and development testing. - Implemented necessary changes in the codebase to support mock ROM initialization and label management.
2025-10-10 17:10:09 -04:00
parent 83eccaa789
commit 6a49f7defa
9 changed files with 469 additions and 23 deletions
--- a/docs/C2-testing-without-roms.md
+++ b/docs/C2-testing-without-roms.md
@@ -0,0 +1,285 @@
+# Testing z3ed Without ROM Files
+
+**Last Updated:** October 10, 2025  
+**Status:** Active
+
+## Overview
+
+The `z3ed` AI agent now supports **mock ROM mode** for testing without requiring actual ROM files. This is essential for:
+
+- **CI/CD pipelines** - No ROM files can be committed to GitHub
+- **Development testing** - Quick iterations without ROM dependencies
+- **Contributors** - Test the agent without needing to provide ROMs
+- **Automated testing** - Consistent, reproducible test environments
+
+## How Mock ROM Mode Works
+
+Mock ROM mode creates a minimal but valid ROM structure with:
+- ✅ Proper SNES header (LoROM mapping, 1MB size)
+- ✅ All Zelda3 embedded labels (rooms, sprites, entrances, items, music, etc.)
+- ✅ Resource label manager fully initialized
+- ❌ No actual ROM data (tiles, graphics, maps are empty)
+
+This allows the AI agent to:
+- Answer questions about room names, sprite IDs, entrance numbers
+- Lookup labels and constants
+- Test function calling and tool dispatch
+- Validate agent logic without game data
+
+## Usage
+
+### Command Line Flag
+
+Add `--mock-rom` to any `z3ed agent` command:
+
+```bash
+# Simple chat with mock ROM
+z3ed agent simple-chat "What is room 5?" --mock-rom
+
+# Test conversation with mock ROM
+z3ed agent test-conversation --mock-rom
+
+# AI provider testing
+z3ed agent simple-chat "List all dungeons" --mock-rom --ai_provider=ollama
+```
+
+### Test Suite
+
+The `agent_test_suite.sh` script now defaults to mock ROM mode:
+
+```bash
+# Run tests with mock ROM (default)
+./scripts/agent_test_suite.sh ollama
+
+# Or with Gemini
+./scripts/agent_test_suite.sh gemini
+```
+
+To use a real ROM instead, edit the script:
+
+```bash
+USE_MOCK_ROM=false  # At the top of agent_test_suite.sh
+```
+
+## What Works with Mock ROM
+
+### ✅ Fully Supported
+
+**Label Queries:**
+- "What is room 5?" → "Tower of Hera - Moldorm Boss"
+- "What sprites are in the game?" → Lists all 256 sprite names
+- "What is entrance 0?" → "Link's House Main"
+- "List all items" → Bow, Boomerang, Hookshot, etc.
+
+**Resource Lookups:**
+- Room names (296 rooms)
+- Entrance names (133 entrances)
+- Sprite names (256 sprites)
+- Overlord names (14 overlords)
+- Overworld map names (160 maps)
+- Item names
+- Music track names
+- Graphics sheet names
+
+**AI Testing:**
+- Function calling / tool dispatch
+- Natural language understanding
+- Error handling
+- Tool output parsing
+- Multi-turn conversations
+
+### ⚠️ Limited Support
+
+**Queries Requiring Data:**
+- "What tiles are used in room 5?" → No tile data in mock ROM
+- "Show me the palette for map 0" → No palette data
+- "What's at coordinate X,Y?" → No map data
+- "Export graphics from dungeon 1" → No graphics data
+
+These queries will either return empty results or errors indicating no ROM data is available.
+
+### ❌ Not Supported
+
+**Operations That Modify ROM:**
+- Editing tiles
+- Changing palettes
+- Modifying sprites
+- Patching ROM data
+
+## Testing Strategy
+
+### For Agent Logic
+Use **mock ROM** for testing:
+- Function calling mechanisms
+- Tool dispatch and routing
+- Natural language understanding
+- Error handling
+- Label resolution
+- Resource lookups
+
+### For ROM Operations
+Use **real ROM** for testing:
+- Tile editing
+- Graphics manipulation
+- Palette modifications
+- Data extraction
+- ROM patching
+
+## CI/CD Integration
+
+### GitHub Actions Example
+
+```yaml
+name: Test z3ed Agent
+
+on: [push, pull_request]
+
+jobs:
+  test-agent:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      
+      - name: Install Dependencies
+        run: |
+          # Install ollama if testing local models
+          curl -fsSL https://ollama.ai/install.sh | sh
+          ollama pull qwen2.5-coder
+      
+      - name: Build z3ed
+        run: |
+          cmake -B build_test
+          cmake --build build_test --parallel
+      
+      - name: Run Agent Tests (Mock ROM)
+        run: |
+          ./scripts/agent_test_suite.sh ollama
+        env:
+          # Or use Gemini with API key
+          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
+```
+
+## Embedded Labels Reference
+
+Mock ROM includes all these labels from `zelda3::Zelda3Labels`:
+
+| Resource Type | Count | Example |
+|--------------|-------|---------|
+| Rooms | 296 | "Sewer - Throne Room" |
+| Entrances | 133 | "Link's House Main" |
+| Sprites | 256 | "Moldorm (Boss)" |
+| Overlords | 14 | "Overlord - Agahnim's Barrier" |
+| Overworld Maps | 160 | "Light World - Hyrule Castle" |
+| Items | 64+ | "Bow", "Boomerang", "Hookshot" |
+| Music Tracks | 64+ | "Title Theme", "Overworld", "Dark World" |
+| Graphics Sheets | 128+ | "Link Sprites", "Enemy Pack 1" |
+
+See `src/app/zelda3/zelda3_labels.h` for the complete list.
+
+## Troubleshooting
+
+### "No ROM loaded" error
+
+Make sure you're using the `--mock-rom` flag:
+
+```bash
+# Wrong
+z3ed agent simple-chat "test"
+
+# Correct
+z3ed agent simple-chat "test" --mock-rom
+```
+
+### Mock ROM fails to initialize
+
+Check the error message. Common issues:
+- Build system didn't include `mock_rom.cc`
+- Missing `zelda3_labels.cc` in build
+- Linker errors with resource labels
+
+### Agent returns empty/wrong results
+
+Remember: Mock ROM has **labels only**, no actual game data.
+
+Queries like "What tiles are in room 5?" won't work because there's no tile data.
+
+Use queries about labels and IDs instead: "What is the name of room 5?"
+
+## Development
+
+### Adding New Labels
+
+To add new label types to mock ROM:
+
+1. **Add to `zelda3_labels.h`:**
+```cpp
+static const std::vector<std::string>& GetNewResourceNames();
+```
+
+2. **Implement in `zelda3_labels.cc`:**
+```cpp
+const std::vector<std::string>& Zelda3Labels::GetNewResourceNames() {
+  static std::vector<std::string> names = {"Item1", "Item2", ...};
+  return names;
+}
+```
+
+3. **Add to `ToResourceLabels()`:**
+```cpp
+const auto& new_resources = GetNewResourceNames();
+for (size_t i = 0; i < new_resources.size(); ++i) {
+  labels["new_resource"][std::to_string(i)] = new_resources[i];
+}
+```
+
+4. **Rebuild:**
+```bash
+cmake --build build --parallel
+```
+
+### Testing Mock ROM Directly
+
+```cpp
+#include "cli/handlers/mock_rom.h"
+
+Rom rom;
+auto status = InitializeMockRom(rom);
+if (status.ok()) {
+  // ROM is ready with all labels
+  auto* label_mgr = rom.resource_label();
+  std::string room_name = label_mgr->GetLabel("room", "5");
+  // room_name == "Tower of Hera - Moldorm Boss"
+}
+```
+
+## Best Practices
+
+### DO ✅
+- Use mock ROM for CI/CD and automated tests
+- Use mock ROM for agent logic development
+- Use mock ROM when contributing (no ROM files needed)
+- Test with real ROM before releasing features
+- Document which features require real ROM data
+
+### DON'T ❌
+- Commit ROM files to Git (legal issues)
+- Assume mock ROM has actual game data
+- Use mock ROM for testing data extraction
+- Skip real ROM testing entirely
+
+## Related Documentation
+
+- [C1: z3ed Agent Guide](C1-z3ed-agent-guide.md) - Main agent documentation
+- [A1: Testing Guide](A1-testing-guide.md) - General testing strategy
+- [E3: API Reference](E3-api-reference.md) - ROM API documentation
+
+---
+
+**Implementation Status:** ✅ Complete  
+**Since Version:** v0.3.3  
+**Files:**
+- `src/cli/handlers/mock_rom.h`
+- `src/cli/handlers/mock_rom.cc`
+- `src/cli/flags.cc` (--mock-rom flag)
+- `scripts/agent_test_suite.sh` (updated)
+
--- a/docs/index.md
+++ b/docs/index.md
@@ -15,6 +15,7 @@ Welcome to the official documentation for yaze, a comprehensive ROM editor for T

 ## C: `z3ed` CLI
 - [C1: `z3ed` Agent Guide](C1-z3ed-agent-guide.md) - The AI-powered command-line interface.
+- [C2: Testing Without ROMs](C2-testing-without-roms.md) - Using mock ROM mode for testing and CI/CD.

 ## E: Development & API
 - [E1: Assembly Style Guide](E1-asm-style-guide.md) - 65816 assembly coding standards.
--- a/scripts/agent_test_suite.sh
+++ b/scripts/agent_test_suite.sh
@@ -10,8 +10,8 @@ BLUE='\033[0;34m'
 NC='\033[0m' # No Color

 Z3ED="./build_test/bin/z3ed"
-ROM="assets/zelda3.sfc"
 RESULTS_FILE="/tmp/z3ed_ai_test_results.txt"
+USE_MOCK_ROM=true  # Set to false if you want to test with a real ROM

 echo "=========================================="
 echo "  Z3ED AI Provider Test Suite"
@@ -22,7 +22,6 @@ echo ""
 > "$RESULTS_FILE"

 # --- Pre-flight Checks ---
-print_header "Performing Pre-flight Checks"

 if [ -z "$1" ]; then
  echo "❌ Error: No AI provider specified."
@@ -32,28 +31,34 @@ fi
 PROVIDER=$1
 echo "✅ Provider: $PROVIDER"

-# Check binaries and files
-for f in "$Z3ED_BIN" "$ROM_PATH" "$TEST_DIR/../prompt_catalogue.yaml" "$TEST_DIR/function_schemas.json"; do
-    if [ ! -f "$f" ]; then
-        echo -e "${RED}✗ Prerequisite file not found: $f${NC}"
-        exit 1
-    fi
-done
-echo "✅ Core binaries and files found."
-
-# Verify schemas
-if python3 -m json.tool "$TEST_DIR/function_schemas.json" > /dev/null 2>&1; then
-    echo "✅ Function schemas JSON is valid."
-else
-    echo "${RED}✗ Invalid JSON in function_schemas.json${NC}"
+# Check binary exists
+if [ ! -f "$Z3ED" ]; then
+    echo -e "${RED}✗ z3ed binary not found at: $Z3ED${NC}"
+    echo "Run: cmake --build build_test"
    exit 1
 fi
+echo "✅ z3ed binary found"

-# Verify manual tool execution
-if "$Z3ED_BIN" agent overworld-find-tile --tile 0x02E --format json --rom "$ROM_PATH" > /dev/null 2>&1; then
-    echo "✅ Manual tool execution successful."
+# Set ROM flags based on mode
+if [ "$USE_MOCK_ROM" = true ]; then
+    ROM_FLAGS="--mock-rom"
+    echo "✅ Using mock ROM mode (no ROM file required)"
 else
-    echo "${RED}✗ Manual tool execution failed.${NC}"
+    ROM="assets/zelda3.sfc"
+    if [ ! -f "$ROM" ]; then
+        echo -e "${RED}✗ ROM file not found: $ROM${NC}"
+        echo "Tip: Use mock ROM mode by setting USE_MOCK_ROM=true"
+        exit 1
+    fi
+    ROM_FLAGS="--rom=\"$ROM\""
+    echo "✅ Real ROM found: $ROM"
+fi
+
+# Verify z3ed can execute
+if "$Z3ED" --help > /dev/null 2>&1; then
+    echo "✅ z3ed executable works"
+else
+    echo "${RED}✗ z3ed failed to execute${NC}"
    exit 1
 fi

@@ -107,7 +112,7 @@ run_test() {
    echo "Query: $query"
    echo ""
    
-    local cmd="$Z3ED agent simple-chat \"$query\" --rom=\"$ROM\" --ai_provider=$provider $extra_args"
+    local cmd="$Z3ED agent simple-chat \"$query\" $ROM_FLAGS --ai_provider=$provider $extra_args"
    echo "Running: $cmd"
    echo ""
    
--- a/src/cli/agent.cmake
+++ b/src/cli/agent.cmake
@@ -97,6 +97,7 @@ set(YAZE_AGENT_SOURCES
  cli/service/resources/resource_context_builder.cc
  cli/handlers/overworld_inspect.cc
  cli/handlers/message.cc
+  cli/handlers/mock_rom.cc
  cli/flags.cc
  cli/service/rom/rom_sandbox_manager.cc
 )
--- a/src/cli/flags.cc
+++ b/src/cli/flags.cc
@@ -3,6 +3,9 @@
 #include "absl/flags/flag.h"

 ABSL_FLAG(std::string, rom, "", "Path to the ROM file");
+ABSL_FLAG(bool, mock_rom, false,
+          "Use mock ROM mode for testing without requiring an actual ROM file. "
+          "Loads all Zelda3 embedded labels but no actual ROM data.");

 // AI Service Configuration Flags
 ABSL_FLAG(std::string, ai_provider, "auto",
--- a/src/cli/handlers/agent/conversation_test.cc
+++ b/src/cli/handlers/agent/conversation_test.cc
@@ -1,6 +1,7 @@
 #include "cli/handlers/agent/commands.h"
 #include "app/rom.h"
 #include "app/core/project.h"
+#include "cli/handlers/mock_rom.h"

 #include "absl/flags/declare.h"
 #include "absl/flags/flag.h"
@@ -16,6 +17,7 @@
 #include "nlohmann/json.hpp"

 ABSL_DECLARE_FLAG(std::string, rom);
+ABSL_DECLARE_FLAG(bool, mock_rom);

 namespace yaze {
 namespace cli {
@@ -28,10 +30,23 @@ absl::Status LoadRomForAgent(Rom& rom) {
    return ::absl::OkStatus();
  }

+  // Check if mock ROM mode is enabled
+  bool use_mock = ::absl::GetFlag(FLAGS_mock_rom);
+  if (use_mock) {
+    // Initialize mock ROM with embedded labels
+    auto status = InitializeMockRom(rom);
+    if (!status.ok()) {
+      return status;
+    }
+    std::cout << "✅ Mock ROM initialized with embedded Zelda3 labels\n";
+    return ::absl::OkStatus();
+  }
+
+  // Otherwise load from file
  std::string rom_path = ::absl::GetFlag(FLAGS_rom);
  if (rom_path.empty()) {
    return ::absl::InvalidArgumentError(
-        "No ROM loaded. Pass --rom=<path> to z3ed agent test-conversation.");
+        "No ROM loaded. Pass --rom=<path> or use --mock-rom for testing.");
  }

  auto status = rom.LoadFromFile(rom_path);
--- a/src/cli/handlers/agent/tool_commands.cc
+++ b/src/cli/handlers/agent/tool_commands.cc
@@ -23,12 +23,14 @@
 #include "app/zelda3/dungeon/room.h"
 #include "app/zelda3/overworld/overworld.h"
 #include "cli/handlers/message.h"
+#include "cli/handlers/mock_rom.h"
 #include "cli/handlers/overworld_inspect.h"
 #include "cli/service/resources/resource_context_builder.h"
 #include "nlohmann/json.hpp"
 #include "util/macro.h"

 ABSL_DECLARE_FLAG(std::string, rom);
+ABSL_DECLARE_FLAG(bool, mock_rom);

 namespace yaze {
 namespace cli {
@@ -37,10 +39,22 @@ namespace agent {
 namespace {

 absl::StatusOr<Rom> LoadRomFromFlag() {
+  // Check if mock ROM mode is enabled
+  bool use_mock = absl::GetFlag(FLAGS_mock_rom);
+  if (use_mock) {
+    Rom rom;
+    auto status = InitializeMockRom(rom);
+    if (!status.ok()) {
+      return status;
+    }
+    return rom;
+  }
+
+  // Otherwise load from file
  std::string rom_path = absl::GetFlag(FLAGS_rom);
  if (rom_path.empty()) {
    return absl::FailedPreconditionError(
-        "No ROM loaded. Use --rom=<path> to specify ROM file.");
+        "No ROM loaded. Use --rom=<path> or --mock-rom for testing.");
  }

  Rom rom;
--- a/src/cli/handlers/mock_rom.cc
+++ b/src/cli/handlers/mock_rom.cc
@@ -0,0 +1,87 @@
+#include "cli/handlers/mock_rom.h"
+
+#include <vector>
+
+#include "absl/flags/declare.h"
+#include "absl/flags/flag.h"
+#include "absl/strings/str_format.h"
+#include "app/core/project.h"
+#include "app/zelda3/zelda3_labels.h"
+
+ABSL_DECLARE_FLAG(bool, mock_rom);
+
+namespace yaze {
+namespace cli {
+
+absl::Status InitializeMockRom(Rom& rom) {
+  // Create a minimal but valid SNES ROM header
+  // Zelda3 is a 1MB ROM (0x100000 bytes) in LoROM mapping
+  constexpr size_t kMockRomSize = 0x100000; // 1MB
+  std::vector<uint8_t> mock_data(kMockRomSize, 0x00);
+  
+  // SNES header is at 0x7FC0 for LoROM
+  constexpr size_t kHeaderOffset = 0x7FC0;
+  
+  // Set ROM title (21 bytes at 0x7FC0)
+  const char* title = "YAZE MOCK ROM TEST  "; // 21 chars including spaces
+  for (size_t i = 0; i < 21; ++i) {
+    mock_data[kHeaderOffset + i] = title[i];
+  }
+  
+  // ROM makeup byte (0x7FD5): $20 = LoROM, no special chips
+  mock_data[kHeaderOffset + 0x15] = 0x20;
+  
+  // ROM type (0x7FD6): $00 = ROM only
+  mock_data[kHeaderOffset + 0x16] = 0x00;
+  
+  // ROM size (0x7FD7): $09 = 1MB (2^9 KB = 512 KB = 1MB with header)
+  mock_data[kHeaderOffset + 0x17] = 0x09;
+  
+  // SRAM size (0x7FD8): $03 = 8KB (Zelda3 standard)
+  mock_data[kHeaderOffset + 0x18] = 0x03;
+  
+  // Country code (0x7FD9): $01 = USA
+  mock_data[kHeaderOffset + 0x19] = 0x01;
+  
+  // Developer ID (0x7FDA): $33 = Extended header (Zelda3)
+  mock_data[kHeaderOffset + 0x1A] = 0x33;
+  
+  // Version number (0x7FDB): $00 = 1.0
+  mock_data[kHeaderOffset + 0x1B] = 0x00;
+  
+  // Checksum complement (0x7FDC-0x7FDD): We'll leave as 0x0000 for mock
+  // Checksum (0x7FDE-0x7FDF): We'll leave as 0x0000 for mock
+  
+  // Load the mock data into the ROM
+  auto load_status = rom.LoadFromData(mock_data);
+  if (!load_status.ok()) {
+    return absl::InternalError(
+        absl::StrFormat("Failed to initialize mock ROM: %s", 
+                        load_status.message()));
+  }
+  
+  // Initialize embedded labels so queries work without actual ROM data
+  core::YazeProject project;
+  auto labels_status = project.InitializeEmbeddedLabels();
+  if (!labels_status.ok()) {
+    return absl::InternalError(
+        absl::StrFormat("Failed to initialize embedded labels: %s",
+                        labels_status.message()));
+  }
+  
+  // Attach labels to ROM's resource label manager
+  if (rom.resource_label()) {
+    rom.resource_label()->labels_ = project.resource_labels;
+    rom.resource_label()->labels_loaded_ = true;
+  }
+  
+  return absl::OkStatus();
+}
+
+bool ShouldUseMockRom() {
+  return absl::GetFlag(FLAGS_mock_rom);
+}
+
+} // namespace cli
+} // namespace yaze
+
--- a/src/cli/handlers/mock_rom.h
+++ b/src/cli/handlers/mock_rom.h
@@ -0,0 +1,35 @@
+#ifndef YAZE_CLI_HANDLERS_MOCK_ROM_H
+#define YAZE_CLI_HANDLERS_MOCK_ROM_H
+
+#include "absl/status/status.h"
+#include "app/rom.h"
+
+namespace yaze {
+namespace cli {
+
+/**
+ * @brief Initialize a mock ROM for testing without requiring an actual ROM file
+ * 
+ * This creates a minimal but valid ROM structure populated with:
+ * - All Zelda3 embedded labels (rooms, sprites, entrances, items, etc.)
+ * - Minimal header data to satisfy ROM validation
+ * - Empty but properly sized data sections
+ * 
+ * Purpose: Allow AI agent testing and CI/CD without committing ROM files
+ * 
+ * @param rom ROM object to initialize as mock
+ * @return absl::OkStatus() on success, error status on failure
+ */
+absl::Status InitializeMockRom(Rom& rom);
+
+/**
+ * @brief Check if mock ROM mode should be used based on flags
+ * @return true if --mock-rom flag is set
+ */
+bool ShouldUseMockRom();
+
+} // namespace cli
+} // namespace yaze
+
+#endif // YAZE_CLI_HANDLERS_MOCK_ROM_H
+