# IT-10: Collaborative Editing & Multiplayer Sessions **Priority**: P2 (High value, non-blocking) **Status**: 📋 Planned **Estimated Effort**: 12-15 hours **Dependencies**: IT-05 (Test Introspection), IT-08 (Screenshot Capture) **Target**: Enable real-time collaborative ROM editing with AI assistance --- ## Vision Enable multiple users to connect to the same YAZE session, see each other's edits in real-time, and collaborate with AI agents together. Think "Google Docs for ROM hacking" where users can: - **Connect to each other's sessions** over the network - **See real-time edits** (tiles, sprites, map changes) - **Share AI assistance** (one user asks AI, all users see results) - **Coordinate workflows** (e.g., one user edits dungeons, another edits overworld) - **Review changes together** with live cursors and annotations --- ## User Stories ### US-1: Session Host & Join **As a ROM hacker**, I want to host a collaborative editing session so my teammates can join and work together. ```bash # Host creates a session $ z3ed collab host --port 5000 --password "dev123" ✅ Collaborative session started Session ID: yaze-collab-f3a9b2c1 URL: yaze://connect/localhost:5000?session=yaze-collab-f3a9b2c1 Password: dev123 👥 Waiting for collaborators... # Remote user joins $ z3ed collab join yaze://connect/192.168.1.100:5000?session=yaze-collab-f3a9b2c1 🔐 Enter session password: *** ✅ Connected to session (Host: Alice) 👥 Active users: Alice (host), Bob (you) ``` **Acceptance Criteria**: - Host can create session with optional password - Clients can discover and join sessions - Connection state visible in GUI status bar - Maximum 8 concurrent users per session --- ### US-2: Real-Time Edit Synchronization **As a collaborator**, I want to see other users' edits in real-time so we stay synchronized. **Scenario**: Alice edits a tile in Overworld Editor ``` Alice's GUI: - Draws tile at (10, 15) → Sends edit event to all clients Bob's GUI (auto-update): - Receives edit event → Redraws tile at (10, 15) - Shows Alice's cursor/selection indicator ``` **Acceptance Criteria**: - Edits appear on all clients within 100ms - Conflict resolution for simultaneous edits - Undo/redo synchronized across sessions - Cursor positions visible for all users --- ### US-3: Shared AI Agent **As a team lead**, I want to use AI agents with my team so we can all benefit from automation. ```bash # Alice (host) runs an AI agent test $ z3ed agent test --prompt "Add treasure chest to room 12" --share 🤖 AI Agent: Analyzing request... Action: Click "Dungeon Editor" tab Action: Select Room 12 Action: Add object type 0x12 (treasure chest) at (5, 8) ✅ Proposal generated (ID: prop_3f8a) # All connected users see the proposal in their GUI Bob's Screen: ┌─────────────────────────────────────────┐ │ 🤖 AI Proposal from Alice │ │ │ │ Add treasure chest to room 12 │ │ • Click "Dungeon Editor" tab │ │ • Select Room 12 │ │ • Add treasure chest at (5, 8) │ │ │ │ [Accept] [Reject] [Discuss] │ └─────────────────────────────────────────┘ # Team vote: 2/3 accept → Proposal executes for all users ``` **Acceptance Criteria**: - AI agent results broadcast to all session members - Proposals require majority approval (configurable threshold) - All users see agent execution in real-time - Failed operations rollback for all users --- ### US-4: Live Cursors & Annotations **As a collaborator**, I want to see where other users are working so we don't conflict. **Visual Indicators**: ``` ┌─────────────────────────────────────────────┐ │ Overworld Editor │ │ │ │ 🟦 (Alice's cursor at map 0x40) │ │ 🟩 (Bob's cursor at map 0x41) │ │ 🟥 (Charlie editing palette) │ │ │ │ Active Editors: │ │ • Alice: Overworld (read-write) │ │ • Bob: Overworld (read-write) │ │ • Charlie: Palette Editor (read-only) │ └─────────────────────────────────────────────┘ ``` **Acceptance Criteria**: - Each user has unique color-coded cursor - Active editor window highlighted for each user - Text chat overlay for quick communication - Annotation tools (pins, comments, highlights) --- ### US-5: Session Recording & Replay **As a project manager**, I want to record collaborative sessions so we can review work later. ```bash # Host enables session recording $ z3ed collab host --record session_2025_10_02.yaml # Recording captures: # - All edit operations (tiles, sprites, maps) # - AI agent proposals and votes # - Chat messages and annotations # - User join/leave events # - Timestamps for audit trail # Later: Replay the session $ z3ed collab replay session_2025_10_02.yaml --speed 2x # Replay shows: # - Timeline of all edits # - User activity heatmap # - Decision points (proposals accepted/rejected) # - Final ROM state comparison ``` **Acceptance Criteria**: - All session activity recorded in structured format (YAML/JSON) - Replay supports speed control (0.5x - 10x) - Export to video format (optional, uses screenshots) - Audit log for compliance/review --- ## Architecture ### Components #### 1. Collaboration Server (New) **Location**: `src/app/core/collab/collab_server.{h,cc}` **Responsibilities**: - Manage WebSocket connections from clients - Broadcast edit events to all connected clients - Handle session authentication (password, tokens) - Enforce access control (read-only vs read-write) - Maintain session state (active users, current ROM) **Technology**: - **WebSocket** for low-latency bidirectional communication - **Protocol Buffers** for efficient serialization - **JWT tokens** for session authentication - **Redis** (optional) for distributed sessions **Key APIs**: ```cpp class CollabServer { public: // Start server on specified port absl::Status Start(int port, const std::string& password); // Handle new client connection void OnClientConnected(ClientConnection* client); // Broadcast edit event to all clients void BroadcastEdit(const EditEvent& event, ClientConnection* sender); // Handle AI proposal from client void BroadcastProposal(const AgentProposal& proposal); // Get active users in session std::vector GetActiveUsers() const; private: std::unique_ptr ws_server_; absl::Mutex clients_mutex_; std::vector> clients_; SessionState session_state_; }; ``` --- #### 2. Collaboration Client (New) **Location**: `src/app/core/collab/collab_client.{h,cc}` **Responsibilities**: - Connect to remote collaboration server - Send local edits to server - Receive and apply remote edits - Sync ROM state on join - Handle disconnection/reconnection **Key APIs**: ```cpp class CollabClient { public: // Connect to session absl::Status Connect(const std::string& url, const std::string& password); // Send local edit to server void SendEdit(const EditEvent& event); // Callback when remote edit received void OnRemoteEdit(const EditEvent& event); // Get list of active users std::vector GetUsers() const; // Disconnect from session void Disconnect(); private: std::unique_ptr ws_client_; CollabEventHandler* event_handler_; SessionInfo session_info_; }; ``` --- #### 3. Edit Event Protocol (New) **Location**: `src/app/core/proto/collab_events.proto` **Message Definitions**: ```protobuf syntax = "proto3"; package yaze.collab; // Generic edit event message EditEvent { string event_id = 1; // Unique event ID string user_id = 2; // User who made the edit int64 timestamp_ms = 3; // Unix timestamp oneof event_type { TileEdit tile_edit = 10; SpriteEdit sprite_edit = 11; PaletteEdit palette_edit = 12; MapEdit map_edit = 13; ObjectEdit object_edit = 14; } } // Tile edit (Tile16 Editor, Tilemap) message TileEdit { string editor = 1; // "tile16", "tilemap" int32 x = 2; int32 y = 3; int32 layer = 4; bytes tile_data = 5; // Tile pixel data or ID } // Sprite edit message SpriteEdit { int32 sprite_id = 1; int32 x = 2; int32 y = 3; bytes sprite_data = 4; } // Map edit (Overworld/Dungeon) message MapEdit { string map_type = 1; // "overworld", "dungeon" int32 map_id = 2; bytes map_data = 3; } // User cursor position message CursorEvent { string user_id = 1; string editor = 2; // Active editor window int32 x = 3; int32 y = 4; string color = 5; // Cursor color (hex) } // AI proposal event message ProposalEvent { string proposal_id = 1; string user_id = 2; // User who initiated agent string prompt = 3; repeated ProposalAction actions = 4; enum ProposalStatus { PENDING = 0; ACCEPTED = 1; REJECTED = 2; EXECUTING = 3; COMPLETED = 4; } ProposalStatus status = 5; // Voting map votes = 6; // user_id -> accept/reject int32 votes_needed = 7; } message ProposalAction { string action_type = 1; // "click", "type", "edit" map params = 2; } // Session state message SessionState { string session_id = 1; string host_user_id = 2; repeated UserInfo users = 3; bytes rom_checksum = 4; // SHA256 of ROM int64 session_start_ms = 5; } message UserInfo { string user_id = 1; string username = 2; string color = 3; // User's cursor color bool is_host = 4; bool read_only = 5; string active_editor = 6; } ``` --- #### 4. Conflict Resolution System **Challenge**: Multiple users edit the same tile/sprite simultaneously **Solution**: Operational Transformation (OT) with timestamps ```cpp class ConflictResolver { public: // Resolve conflicting edits EditEvent ResolveConflict(const EditEvent& local, const EditEvent& remote); private: // Last-write-wins with timestamp EditEvent LastWriteWins(const EditEvent& e1, const EditEvent& e2); // Merge edits if possible (e.g., different layers) std::optional TryMerge(const EditEvent& e1, const EditEvent& e2); }; ``` **Conflict Resolution Rules**: 1. **Same tile, different times**: Last write wins (based on timestamp) 2. **Same tile, same time (<100ms)**: Host user wins (host authority) 3. **Different tiles**: No conflict, apply both 4. **Different layers**: No conflict, apply both 5. **Undo/Redo**: Undo takes precedence (explicit user intent) --- #### 5. GUI Integration **Status Bar Indicator**: ``` ┌─────────────────────────────────────────────────────────────┐ │ File Edit View Tools Help 👥 3 users connected │ │ 🟢 Alice (Host) │ │ 🔵 Bob │ │ 🟣 Charlie │ └─────────────────────────────────────────────────────────────┘ ``` **Collaboration Panel**: ``` ┌─────────────────────────────────┐ │ Collaboration │ ├─────────────────────────────────┤ │ Session: yaze-collab-f3a9b2c1 │ │ Status: 🟢 Connected │ │ │ │ Users (3): │ │ 🟢 Alice (Host) - Dungeon │ │ 🔵 Bob (You) - Overworld │ │ 🟣 Charlie - Palette │ │ │ │ Activity: │ │ • Alice edited room 12 │ │ • Bob added sprite #23 │ │ • Charlie changed palette 2 │ │ │ │ [Chat] [Proposals] [Disconnect] │ └─────────────────────────────────┘ ``` **Cursor Overlay**: ```cpp // In canvas rendering void OverworldCanvas::DrawCollaborativeCursors() { for (const auto& user : collab_client_->GetUsers()) { if (user.active_editor == "overworld" && user.user_id != my_user_id) { ImVec2 cursor_pos = TileToScreen(user.cursor_x, user.cursor_y); ImU32 color = ImGui::GetColorU32(user.color); // Draw cursor indicator draw_list->AddCircleFilled(cursor_pos, 5.0f, color); // Draw username label draw_list->AddText(cursor_pos + ImVec2(10, -5), color, user.username.c_str()); } } } ``` --- ## CLI Commands ### Session Management ```bash # Host a session z3ed collab host [options] --port Port to listen on (default: 5000) --password Session password (optional) --max-users Maximum concurrent users (default: 8) --read-only Comma-separated list of read-only users --record Record session to file # Join a session z3ed collab join [options] --password Session password --username Display name (default: system username) --read-only Join in read-only mode # List active sessions (LAN discovery) z3ed collab list # Disconnect from session z3ed collab disconnect # Kick user (host only) z3ed collab kick ``` ### Session Replay ```bash # Replay recorded session z3ed collab replay [options] --speed Playback speed multiplier (default: 1.0) --start-time