scawful/yaze
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Enhance z3ed networking documentation and CLI capabilities

Browse Source 
- Updated NETWORKING.md to reflect the new versioning and provide a comprehensive overview of the z3ed networking, collaboration, and remote access features.
- Introduced detailed architecture descriptions, including communication layers and core components for collaboration.
- Added new CLI commands for hex and palette manipulation, enhancing the functionality of the z3ed CLI.
- Implemented a TODO management system within the CLI for better task tracking and execution planning.
- Improved README.md to include new CLI capabilities and enhancements related to the TODO management system.
This commit is contained in:
scawful
2025-10-05 01:15:15 -04:00
parent 40f44aa42a
commit 019c20e749
11 changed files with 1411 additions and 461 deletions
Download Patch File Download Diff File Expand all files Collapse all files

604
docs/z3ed/NETWORKING.md
View File

@@ -1,22 +1,20 @@
# Z3ED Networking & Collaboration
# z3ed Networking, Collaboration, and Remote Access
## Overview
**Version**: 0.2.0-alpha
**Last Updated**: October 5, 2025
Z3ED provides comprehensive networking capabilities across all three components:
- **yaze app**: GUI application with real-time collaboration
- **z3ed CLI**: Command-line interface for remote operations
- **yaze-server**: WebSocket server for coordination
## 1. Overview
## Architecture
This document provides a comprehensive overview of the networking, collaboration, and remote access features within the z3ed ecosystem. These systems are designed to enable everything from real-time collaborative ROM hacking to powerful AI-driven remote automation.
### Cross-Platform Design
The architecture is composed of three main communication layers:
1. **WebSocket Protocol**: A real-time messaging layer for multi-user collaboration, managed by the `yaze-server`.
2. **gRPC Service**: A high-performance RPC layer for programmatic remote ROM manipulation, primarily used by the `z3ed` CLI and automated testing harnesses.
3. **Collaboration Service**: A high-level C++ API within the YAZE application that integrates version management with the networking protocols.
All networking code is designed to work on:
- ✅ **Windows** - Using native Win32 sockets (ws2_32)
- ✅ **macOS** - Using native BSD sockets
- ✅ **Linux** - Using native BSD sockets
## 2. Architecture
### Components
### 2.1. System Diagram
```
┌─────────────┐ WebSocket ┌──────────────┐
@@ -24,8 +22,7 @@ All networking code is designed to work on:
│ (GUI) │ │ (Node.js) │
└─────────────┘ └──────────────┘
▲ ▲
│ │
│ WebSocket │
│ gRPC (for GUI testing) │ WebSocket
│ │
└─────────────┐ │
│ │
@@ -33,194 +30,113 @@ All networking code is designed to work on:
│ z3ed CLI │◄──────────────────────┘
│ │
└─────────────┘
gRPC (for GUI testing)
│
│
┌──────▼──────┐
│ yaze app │
│ (ImGui) │
└─────────────┘
```
## WebSocket Protocol
### 2.2. Core Components
### Connection
The collaboration system is built on two key C++ components:
```cpp
#include "app/net/websocket_client.h"
1. **ROM Version Manager** (`app/net/rom_version_manager.h`)
- **Purpose**: Protects the ROM from corruption and unwanted changes.
- **Features**:
- Automatic periodic snapshots and manual checkpoints.
- "Safe points" to mark host-verified versions.
- Corruption detection and automatic recovery to the last safe point.
- Ability to roll back to any previous version.
net::WebSocketClient client;
2. **Proposal Approval Manager** (`app/net/rom_version_manager.h`)
- **Purpose**: Manages a collaborative voting system for applying changes.
- **Features**:
- Multiple approval modes: `Host-Only` (default), `Majority`, and `Unanimous`.
- Automatically creates snapshots before and after a proposal is applied.
- Handles the submission, voting, and application/rejection of proposals.
// Connect to server
auto status = client.Connect("localhost", 8765);
3. **Collaboration Panel** (`app/gui/widgets/collaboration_panel.h`)
- **Purpose**: Provides a dedicated UI within the YAZE editor for managing collaboration.
- **Features**:
- Version history timeline with one-click restore.
- ROM synchronization tracking.
- Visual snapshot gallery.
- A voting and approval interface for pending proposals.
// Set up callbacks
client.OnMessage("rom_sync", [](const nlohmann::json& payload) {
// Handle ROM sync
});
## 3. Protocols
client.OnStateChange([](net::ConnectionState state) {
// Handle state changes
});
### 3.1. WebSocket Protocol (yaze-server v2.0)
Used for real-time, multi-user collaboration.
**Connection**:
```javascript
const ws = new WebSocket('ws://localhost:8765');
```
### Message Types
**Message Types**:
#### 1. Session Management
| Type | Sender | Payload Description |
| :--- | :--- | :--- |
| `host_session` | Client | Initiates a new session with a name, username, and optional ROM hash. |
| `join_session` | Client | Joins an existing session using a 6-character code. |
| `rom_sync` | Client | Broadcasts a base64-encoded ROM diff to all participants. |
| `proposal_share` | Client | Shares a new proposal (e.g., from an AI agent) with the group. |
| `proposal_vote` | Client | Submits a vote (approve/reject) for a specific proposal. |
| `snapshot_share` | Client | Shares a snapshot (e.g., a screenshot) with the group. |
| `proposal_update` | Server | Broadcasts the new status of a proposal (`approved`, `rejected`). |
| `proposal_vote_received`| Server | Confirms a vote was received and shows the current vote tally. |
| `session_hosted` | Server | Confirms a session was created and returns its code. |
| `session_joined` | Server | Confirms a user joined and provides session state (participants, history). |
**Host Session**:
```json
{
"type": "host_session",
"payload": {
"session_name": "My ROM Hack",
"username": "host",
"rom_hash": "abc123",
"ai_enabled": true
}
### 3.2. gRPC Service (Remote ROM Manipulation)
Provides a high-performance API for programmatic access to the ROM, used by the `z3ed` CLI and test harnesses.
**Status**: ✅ Designed and Implemented. Pending final build system integration.
**Protocol Buffer (`protos/rom_service.proto`)**:
```proto
service RomService {
// Core
rpc ReadBytes(ReadBytesRequest) returns (ReadBytesResponse);
rpc WriteBytes(WriteBytesRequest) returns (WriteBytesResponse);
rpc GetRomInfo(GetRomInfoRequest) returns (GetRomInfoResponse);
// Overworld
rpc ReadOverworldMap(ReadOverworldMapRequest) returns (ReadOverworldMapResponse);
rpc WriteOverworldTile(WriteOverworldTileRequest) returns (WriteOverworldTileResponse);
// Dungeon
rpc ReadDungeonRoom(ReadDungeonRoomRequest) returns (ReadDungeonRoomResponse);
rpc WriteDungeonTile(WriteDungeonTileRequest) returns (WriteDungeonTileResponse);
// Proposals
rpc SubmitRomProposal(SubmitRomProposalRequest) returns (SubmitRomProposalResponse);
rpc GetProposalStatus(GetProposalStatusRequest) returns (GetProposalStatusResponse);
// Version Management
rpc CreateSnapshot(CreateSnapshotRequest) returns (CreateSnapshotResponse);
rpc RestoreSnapshot(RestoreSnapshotRequest) returns (RestoreSnapshotResponse);
rpc ListSnapshots(ListSnapshotsRequest) returns (ListSnapshotsResponse);
}
```
**Join Session**:
```json
{
"type": "join_session",
"payload": {
"session_code": "ABC123",
"username": "participant"
}
}
**Use Case: Write with Approval via `z3ed`**
The `z3ed` CLI can submit a change as a proposal via gRPC and wait for a host to approve it in the YAZE GUI.
```bash
# 1. CLI connects and submits a proposal via gRPC
z3ed net connect --host server.example.com --port 50051
z3ed agent run --prompt "Make dungeon 5 harder" --submit-grpc-proposal
# 2. The YAZE GUI receives the proposal and the host approves it.
# 3. The z3ed CLI polls for the status and receives the approval.
```
#### 2. Proposal System (NEW)
## 4. Client Integration
**Share Proposal**:
```json
{
"type": "proposal_share",
"payload": {
"sender": "username",
"proposal_data": {
"description": "Place tile 0x42 at (5,7)",
"type": "tile_edit",
"data": {...}
}
}
}
```
### 4.1. YAZE App Integration
**Vote on Proposal** (NEW):
```json
{
"type": "proposal_vote",
"payload": {
"proposal_id": "prop_123",
"approved": true,
"username": "voter"
}
}
```
Response:
```json
{
"type": "proposal_vote_received",
"payload": {
"proposal_id": "prop_123",
"username": "voter",
"approved": true,
"votes": {
"host": true,
"user1": true,
"user2": false
},
"timestamp": 1234567890
}
}
```
**Update Proposal Status**:
```json
{
"type": "proposal_update",
"payload": {
"proposal_id": "prop_123",
"status": "approved" // or "rejected", "applied"
}
}
```
#### 3. ROM Synchronization
**Send ROM Sync**:
```json
{
"type": "rom_sync",
"payload": {
"sender": "username",
"diff_data": "base64_encoded_diff",
"rom_hash": "new_hash"
}
}
```
#### 4. Snapshots
**Share Snapshot**:
```json
{
"type": "snapshot_share",
"payload": {
"sender": "username",
"snapshot_data": "base64_encoded_image",
"snapshot_type": "screenshot"
}
}
```
## YAZE App Integration
### Using WebSocketClient
```cpp
#include "app/net/websocket_client.h"
// Create client
auto client = std::make_unique<net::WebSocketClient>();
// Connect
if (auto status = client->Connect("localhost", 8765); !status.ok()) {
// Handle error
}
// Host a session
auto session_info = client->HostSession(
"My Hack",
"username",
rom->GetHash(),
true // AI enabled
);
// Set up proposal callback
client->OnMessage("proposal_shared", [this](const nlohmann::json& payload) {
std::string proposal_id = payload["proposal_id"];
nlohmann::json proposal_data = payload["proposal_data"];
// Add to approval manager
approval_mgr->SubmitProposal(
proposal_id,
payload["sender"],
proposal_data["description"],
proposal_data
);
});
// Vote on proposal
client->VoteOnProposal(proposal_id, true, "my_username");
```
### Using CollaborationService
The `CollaborationService` provides a high-level API that integrates the version manager, approval manager, and WebSocket client.
```cpp
#include "app/net/collaboration_service.h"
@@ -228,297 +144,67 @@ client->VoteOnProposal(proposal_id, true, "my_username");
// High-level service that integrates everything
auto collab_service = std::make_unique<net::CollaborationService>(rom);
// Initialize with version manager and approval manager
// Initialize with managers
collab_service->Initialize(config, version_mgr, approval_mgr);
// Connect and host
collab_service->Connect("localhost", 8765);
collab_service->HostSession("My Hack", "username");
// Submit local changes as proposal
collab_service->SubmitChangesAsProposal(
"Modified dungeon room 5",
"username"
);
// Auto-sync is handled automatically
// Submit local changes as a proposal to the group
collab_service->SubmitChangesAsProposal("Modified dungeon room 5", "username");
```
## Z3ED CLI Integration
### 4.2. z3ed CLI Integration
### Connection Commands
The CLI provides `net` commands for interacting with the collaboration server.
```bash
# Connect to collaboration server
# Connect to a collaboration server
z3ed net connect --host localhost --port 8765
# Join session
# Join an existing session
z3ed net join --code ABC123 --username myname
# Leave session
z3ed net leave
```
# Submit an AI-generated change as a proposal and wait for it to be approved
z3ed agent run --prompt "Make boss room more challenging" --submit-proposal --wait-approval
### Proposal Commands
```bash
# Submit proposal from z3ed
z3ed agent run --prompt "Place tile 42 at (5,7)" --submit-proposal
# Check proposal status
# Manually check a proposal's status
z3ed net proposal status --id prop_123
# Wait for approval (blocking)
z3ed net proposal wait --id prop_123 --timeout 60
```
### Example Workflow
```bash
# 1. Connect to server
z3ed net connect --host localhost
# 2. Join session
z3ed net join --code XYZ789 --username alice
# 3. Submit AI-generated proposal
z3ed agent run --prompt "Make boss room more challenging" \
--submit-proposal --wait-approval
# 4. If approved, changes are applied
# If rejected, original ROM is preserved
```
## Windows-Specific Notes
### Building on Windows
The networking library automatically links Windows socket support:
```cmake
if(WIN32)
target_link_libraries(yaze_net PUBLIC ws2_32)
endif()
```
### vcpkg Dependencies
For Windows with vcpkg:
```powershell
# Install dependencies
vcpkg install openssl:x64-windows
# CMake will automatically detect and use them
```
### Windows Firewall
You may need to allow connections:
```powershell
# Allow yaze-server
netsh advfirewall firewall add rule name="YAZE Server" dir=in action=allow protocol=TCP localport=8765
# Or through UI: Windows Defender Firewall → Allow an app
```
## Security Considerations
### Transport Security
1. **Use WSS (WebSocket Secure)** in production:
```cpp
client->Connect("wss://server.example.com", 443);
```
2. **Server configuration** with SSL:
```javascript
const https = require('https');
const fs = require('fs');
const server = https.createServer({
cert: fs.readFileSync('cert.pem'),
key: fs.readFileSync('key.pem')
});
```
### Approval Security
1. **Host-only mode** (safest):
```cpp
approval_mgr->SetApprovalMode(ApprovalMode::kHostOnly);
```
2. **Verify identities**: Use authentication tokens
3. **Rate limiting**: Server limits messages to 100/minute
### ROM Protection
1. **Always create snapshots** before applying proposals:
```cpp
config.create_snapshot_before_sync = true;
```
2. **Mark safe points** after verification:
```cpp
version_mgr->MarkAsSafePoint(snapshot_id);
```
3. **Auto-rollback** on errors:
```cpp
if (error) {
version_mgr->RestoreSnapshot(snapshot_before);
}
```
## Platform-Specific Implementation
### httplib WebSocket Support
The implementation uses `cpp-httplib` for cross-platform support:
- **Windows**: Uses Winsock2 (ws2_32.dll)
- **macOS/Linux**: Uses BSD sockets
- **SSL/TLS**: Optional OpenSSL support
### Threading
All platforms use C++11 threads:
```cpp
#include <thread>
#include <mutex>
std::thread receive_thread([this]() {
// Platform-independent receive loop
});
```
## Error Handling
### Connection Errors
```cpp
auto status = client->Connect(host, port);
if (!status.ok()) {
if (absl::IsUnavailable(status)) {
// Server not reachable
} else if (absl::IsDeadlineExceeded(status)) {
// Connection timeout
}
}
```
### Network State
```cpp
client->OnStateChange([](ConnectionState state) {
switch (state) {
case ConnectionState::kConnected:
// Ready to use
break;
case ConnectionState::kDisconnected:
// Clean shutdown
break;
case ConnectionState::kReconnecting:
// Attempting reconnect
break;
case ConnectionState::kError:
// Fatal error
break;
}
});
```
## Performance
### Compression
Large messages are compressed:
```cpp
// ROM diffs are compressed before sending
std::string compressed = CompressDiff(diff_data);
client->SendRomSync(compressed, hash, sender);
```
### Batching
Small changes are batched:
```cpp
config.sync_interval_ms = 5000; // Batch changes over 5 seconds
```
### Rate Limiting
Server enforces:
- 100 messages per minute per client
- 5MB max ROM diff size
- 10MB max snapshot size
## Testing
### Unit Tests
```cpp
TEST(WebSocketClientTest, ConnectAndDisconnect) {
net::WebSocketClient client;
ASSERT_TRUE(client.Connect("localhost", 8765).ok());
EXPECT_TRUE(client.IsConnected());
client.Disconnect();
EXPECT_FALSE(client.IsConnected());
}
```
### Integration Tests
```bash
# Start server
cd yaze-server
npm start
# Run tests
cd yaze
cmake --build build --target yaze_net_tests
./build/bin/yaze_net_tests
```
## Troubleshooting
### "Failed to connect"
- Check server is running: `ps aux | grep node`
- Check port is available: `netstat -an | grep 8765`
- Check firewall settings
### "Connection timeout"
- Increase timeout: `client->SetTimeout(10);`
- Check network connectivity
- Verify server address
### "SSL handshake failed"
- Verify OpenSSL is installed
- Check certificate validity
- Use WSS URL: `wss://` not `ws://`
### Windows-specific: "ws2_32.dll not found"
- Reinstall Windows SDK
- Check PATH environment variable
- Use vcpkg for dependencies
## Future Enhancements
- [ ] WebRTC for peer-to-peer connections
- [ ] Binary protocol for faster ROM syncs
- [ ] Automatic reconnection with exponential backoff
- [ ] Connection pooling for multiple sessions
- [ ] NAT traversal for home networks
- [ ] End-to-end encryption for proposals
## See Also
- [Collaboration Guide](COLLABORATION.md) - Version management and approval
- [Z3ED README](README.md) - Main documentation
- [yaze-server README](../../../yaze-server/README.md) - Server setup
## 5. Best Practices & Troubleshooting
### Best Practices
- **For Hosts**: Enable auto-backups, mark safe points after playtesting, and use `Host-Only` approval mode for maximum control.
- **For Participants**: Submit all changes as proposals, wait for approval, and use descriptive names.
- **For Everyone**: Test changes in an emulator before submitting, make small atomic changes, and communicate with the team.
### Troubleshooting
- **"Failed to connect"**: Check that the `yaze-server` is running and the port is not blocked by a firewall.
- **"Corruption detected"**: Use `version_mgr->AutoRecover()` or manually restore the last known safe point from the Collaboration Panel.
- **"Snapshot not found"**: Verify the snapshot ID is correct and wasn't deleted due to storage limits.
- **"SSL handshake failed"**: Ensure you are using a `wss://` URL and that a valid SSL certificate is configured on the server.
## 6. Security Considerations
- **Transport Security**: Production environments should use WebSocket Secure (`wss://`) with a valid SSL/TLS certificate.
- **Approval Security**: `Host-Only` mode is the safest. For more open collaboration, consider a token-based authentication system.
- **ROM Protection**: The `RomVersionManager` is the primary defense. Always create snapshots before applying proposals and mark safe points often.
- **Rate Limiting**: The `yaze-server` enforces a rate limit (default: 100 messages/minute) to prevent abuse.
## 7. Future Enhancements
- **Encrypted WebSocket (WSS) support**
- **Diff visualization in UI**
- **Merge conflict resolution**
- **Branch/fork support for experimental changes**
- **AI-assisted proposal review**
- **Cloud snapshot backup**
- **Multi-host sessions**
- **Access control lists (ACLs)**
- **WebRTC for peer-to-peer connections**
- **Binary protocol for faster ROM syncs**
- **Automatic reconnection with exponential backoff**
- **Connection pooling for multiple sessions**
- **NAT traversal for home networks**
- **End-to-end encryption for proposals**

32
docs/z3ed/README.md
View File

@@ -135,6 +135,14 @@ The `z3ed` CLI is the foundation for an AI-driven Model-Code-Program (MCP) loop,
- `agent simple-chat`: A lightweight, non-TUI chat mode for scripting and automation.
- `agent test ...`: Commands for running and managing automated GUI tests.
- `agent learn ...`: **NEW**: Manage learned knowledge (preferences, ROM patterns, project context, conversation memory).
- `agent todo create "Description" [--category=<category>] [--priority=<n>]`: Create a new TODO item.
- `agent todo list [--status=<status>] [--category=<category>]`: List TODOs, with optional filters.
- `agent todo update <id> --status=<status>`: Update the status of a TODO item.
- `agent todo show <id>`: Display full details of a TODO item.
- `agent todo delete <id>`: Delete a TODO item.
- `agent todo clear-completed`: Remove all completed TODOs.
- `agent todo next`: Get the next actionable TODO based on dependencies and priority.
- `agent todo plan`: Generate a topologically-sorted execution plan for all TODOs.
### Resource Commands
@@ -239,7 +247,23 @@ All learned data is stored in `~/.yaze/agent/`:
- `projects.json`: Project contexts
- `memories.json`: Conversation summaries
## 9. CLI Output & Help System
## 9. TODO Management System
The TODO Management System enables the z3ed AI agent to create, track, and execute complex multi-step tasks with dependency management and prioritization.
### Core Capabilities
- ✅ Create and manage TODO items with priorities
- ✅ Track task status (pending, in_progress, completed, blocked, cancelled)
- ✅ Dependency tracking between tasks
- ✅ Automatic execution plan generation
- ✅ Persistent storage in JSON format
- ✅ Category-based organization
- ✅ Tools/functions tracking per task
### Storage Location
TODOs are persisted to: `~/.yaze/agent/todos.json` (macOS/Linux) or `%APPDATA%/yaze/agent/todos.json` (Windows)
## 10. CLI Output & Help System
The `z3ed` CLI features a modernized output system designed to be clean for users and informative for developers.
@@ -895,6 +919,7 @@ The AI response appears in your chat history and can reference specific details
- **Native Gemini Function Calling**: Upgraded from manual curl to native function calling API with automatic tool schema generation
- **Multimodal Vision Testing**: Comprehensive test suite for Gemini vision capabilities with screenshot integration
- **AI-Controlled GUI Automation**: Natural language parsing (`AIActionParser`) and test script generation (`GuiActionGenerator`) for automated tile placement
- **TODO Management System**: Full `TodoManager` class with CRUD operations, CLI commands, dependency tracking, execution planning, and JSON persistence.
#### Version Management & Protection
- **ROM Version Management System**: `RomVersionManager` with automatic snapshots, safe points, corruption detection, and rollback capabilities
@@ -908,11 +933,16 @@ The AI response appears in your chat history and can reference specific details
- **Collaboration UI Panel**: `CollaborationPanel` widget with version history, ROM sync tracking, snapshot gallery, and approval workflow
- **gRPC ROM Service**: Complete protocol buffer and implementation for remote ROM manipulation (pending build integration)
#### UI/UX Enhancements
- **Welcome Screen Enhancement**: Dynamic theme integration, Zelda-themed animations, and project cards.
- **Component Refactoring**: `PaletteWidget` renamed and moved, UI organization improved (`app/editor/ui/` for welcome_screen, editor_selection_dialog, background_renderer).
#### Build System & Infrastructure
- **gRPC Windows Build Optimization**: vcpkg integration for 10-20x faster Windows builds, removed abseil-cpp submodule
- **Cross-Platform Networking**: Native socket support (ws2_32 on Windows, BSD sockets on Unix)
- **Namespace Refactoring**: Created `app/net` namespace for networking components
- **Improved Documentation**: Consolidated architecture, enhancement plans, networking guide, and build instructions with JSON-first approach
- **Build System Improvements**: `mac-ai` preset, proto fixes, and updated GEMINI.md with AI build policies.
## 12. Troubleshooting