scawful/yaze
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor graphics optimizations documentation and add ImGui widget testing guide

Browse Source 
- Updated the gfx_optimizations_complete.md to streamline the overview and implementation details of graphics optimizations, removing completed status indicators and enhancing clarity on future recommendations.
- Introduced imgui_widget_testing_guide.md, detailing the usage of YAZE's ImGui testing infrastructure for automated GUI testing, including architecture, integration steps, and best practices.
- Created ollama_integration_status.md to document the current status of Ollama integration, highlighting completed tasks, ongoing issues, and next steps for improvement.
- Revised developer_guide.md to reflect the latest updates in AI provider configuration and input methods for the z3ed agent, ensuring clarity on command-line flags and supported providers.
This commit is contained in:
scawful
2025-10-04 03:24:42 -04:00
parent 9acd6ba17f
commit 5c863b1445
8 changed files with 1655 additions and 431 deletions
Download Patch File Download Diff File Expand all files Collapse all files

407
docs/imgui_widget_testing_guide.md Normal file
View File

@@ -0,0 +1,407 @@
# ImGui Widget Testing Guide
## Overview
This guide explains how to use YAZE's ImGui testing infrastructure for automated GUI testing and AI agent interaction.
## Architecture
### Components
1. **WidgetIdRegistry**: Centralized registry of all GUI widgets with hierarchical paths
2. **AutoWidgetScope**: RAII helper for automatic widget registration
3. **Auto* Wrappers**: Drop-in replacements for ImGui functions that auto-register widgets
4. **ImGui Test Engine**: Automated testing framework
5. **ImGuiTestHarness**: gRPC service for remote test control
### Widget Hierarchy
Widgets are identified by hierarchical paths:
```
Dungeon/Canvas/canvas:DungeonCanvas
Dungeon/RoomSelector/selectable:Room_5
Dungeon/ObjectEditor/input_int:ObjectID
Overworld/Toolset/button:DrawTile
```
Format: `Editor/Section/type:name`
## Integration Guide
### 1. Add Auto-Registration to Your Editor
**Before**:
```cpp
// dungeon_editor.cc
void DungeonEditor::DrawCanvasPanel() {
if (ImGui::Button("Save")) {
SaveRoom();
}
ImGui::InputInt("Room ID", &room_id_);
}
```
**After**:
```cpp
#include "app/gui/widget_auto_register.h"
void DungeonEditor::DrawCanvasPanel() {
gui::AutoWidgetScope scope("Dungeon/Canvas");
if (gui::AutoButton("Save##RoomSave")) {
SaveRoom();
}
gui::AutoInputInt("Room ID", &room_id_);
}
```
### 2. Register Canvas and Tables
```cpp
void DungeonEditor::DrawDungeonCanvas() {
gui::AutoWidgetScope scope("Dungeon/Canvas");
ImGui::BeginChild("DungeonCanvas", ImVec2(512, 512));
gui::RegisterCanvas("DungeonCanvas", "Main dungeon editing canvas");
// ... canvas drawing code ...
ImGui::EndChild();
}
void DungeonEditor::DrawRoomSelector() {
gui::AutoWidgetScope scope("Dungeon/RoomSelector");
if (ImGui::BeginTable("RoomList", 3, table_flags)) {
gui::RegisterTable("RoomList", "List of dungeon rooms");
for (int i = 0; i < rooms_.size(); i++) {
ImGui::TableNextRow();
ImGui::TableNextColumn();
std::string label = absl::StrFormat("Room_%d##room%d", i, i);
if (gui::AutoSelectable(label.c_str(), selected_room_ == i)) {
OnRoomSelected(i);
}
}
ImGui::EndTable();
}
}
```
### 3. Hierarchical Scoping
Use nested scopes for complex UIs:
```cpp
void DungeonEditor::Update() {
gui::AutoWidgetScope editor_scope("Dungeon");
if (ImGui::BeginTable("DungeonEditTable", 3)) {
gui::RegisterTable("DungeonEditTable");
// Column 1: Room Selector
ImGui::TableNextColumn();
{
gui::AutoWidgetScope selector_scope("RoomSelector");
DrawRoomSelector();
}
// Column 2: Canvas
ImGui::TableNextColumn();
{
gui::AutoWidgetScope canvas_scope("Canvas");
DrawCanvas();
}
// Column 3: Object Editor
ImGui::TableNextColumn();
{
gui::AutoWidgetScope editor_scope("ObjectEditor");
DrawObjectEditor();
}
ImGui::EndTable();
}
}
```
### 4. Register Custom Widgets
For widgets not covered by Auto* wrappers:
```cpp
void DrawCustomWidget() {
ImGui::PushID("MyCustomWidget");
// ... custom drawing ...
// Get the item ID after drawing
ImGuiID custom_id = ImGui::GetItemID();
// Register manually
gui::AutoRegisterLastItem("custom", "MyCustomWidget",
"Custom widget description");
ImGui::PopID();
}
```
## Writing Tests
### Basic Test Structure
```cpp
#include "imgui_test_engine/imgui_te_engine.h"
#include "imgui_test_engine/imgui_te_context.h"
ImGuiTest* t = IM_REGISTER_TEST(engine, "dungeon_editor", "canvas_visible");
t->TestFunc = [](ImGuiTestContext* ctx) {
ctx->SetRef("Dungeon Editor");
// Verify canvas exists
IM_CHECK(ctx->ItemExists("Dungeon/Canvas/canvas:DungeonCanvas"));
// Check visibility
auto canvas_info = ctx->ItemInfo("Dungeon/Canvas/canvas:DungeonCanvas");
IM_CHECK(canvas_info != nullptr);
IM_CHECK(canvas_info->RectFull.GetWidth() > 0);
};
```
### Test Actions
```cpp
// Click a button
ctx->ItemClick("Dungeon/Toolset/button:Save");
// Type into an input
ctx->ItemInputValue("Dungeon/ObjectEditor/input_int:ObjectID", 42);
// Check a checkbox
ctx->ItemCheck("Dungeon/ObjectEditor/checkbox:ShowBG1");
// Select from combo
ctx->ComboClick("Dungeon/Settings/combo:PaletteGroup", "Palette 2");
// Wait for condition
ctx->ItemWaitForVisible("Dungeon/Canvas/canvas:DungeonCanvas", 2.0f);
```
### Test with Variables
```cpp
struct MyTestVars {
int room_id = 0;
bool canvas_loaded = false;
};
ImGuiTest* t = IM_REGISTER_TEST(engine, "my_test", "test_name");
t->SetVarsDataType<MyTestVars>();
t->TestFunc = [](ImGuiTestContext* ctx) {
MyTestVars& vars = ctx->GetVars<MyTestVars>();
// Use vars for test state
vars.room_id = 5;
ctx->ItemClick(absl::StrFormat("Room_%d", vars.room_id).c_str());
};
```
## Agent Integration
### Widget Discovery
The z3ed agent can discover available widgets:
```bash
z3ed describe --widget-catalog
```
Output (YAML):
```yaml
widgets:
- path: "Dungeon/Canvas/canvas:DungeonCanvas"
type: canvas
label: "DungeonCanvas"
window: "Dungeon Editor"
visible: true
enabled: true
bounds:
min: [100.0, 50.0]
max: [612.0, 562.0]
actions: [click, drag, scroll]
description: "Main dungeon editing canvas"
```
### Remote Testing via gRPC
```bash
# Click a button
z3ed test click "Dungeon/Toolset/button:Save"
# Type text
z3ed test type "Dungeon/Search/input:RoomName" "Hyrule Castle"
# Wait for element
z3ed test wait "Dungeon/Canvas/canvas:DungeonCanvas" --timeout 5s
# Take screenshot
z3ed test screenshot --window "Dungeon Editor" --output dungeon.png
```
## Best Practices
### 1. Use Stable IDs
```cpp
// GOOD: Stable ID that won't change with label
gui::AutoButton("Save##DungeonSave");
// BAD: Label might change in translations
gui::AutoButton("Save");
```
### 2. Hierarchical Naming
```cpp
// GOOD: Clear hierarchy
{
gui::AutoWidgetScope("Dungeon");
{
gui::AutoWidgetScope("Canvas");
// Widgets here: Dungeon/Canvas/...
}
}
// BAD: Flat structure, name collisions
gui::AutoButton("Save"); // Which editor's save?
```
### 3. Descriptive Names
```cpp
// GOOD: Self-documenting
gui::RegisterCanvas("DungeonCanvas", "Main editing canvas for dungeon rooms");
// BAD: Generic
gui::RegisterCanvas("Canvas");
```
### 4. Frame Lifecycle
```cpp
void Editor::Update() {
// Begin frame for widget registry
gui::WidgetIdRegistry::Instance().BeginFrame();
// ... draw all your widgets ...
// End frame to prune stale widgets
gui::WidgetIdRegistry::Instance().EndFrame();
}
```
### 5. Test Organization
```cpp
// Group related tests
RegisterCanvasTests(engine); // Canvas rendering tests
RegisterRoomSelectorTests(engine); // Room selection tests
RegisterObjectEditorTests(engine); // Object editing tests
```
## Debugging
### View Widget Registry
```cpp
// Export current registry to file
gui::WidgetIdRegistry::Instance().ExportCatalogToFile("widgets.yaml", "yaml");
```
### Check if Widget Registered
```cpp
auto& registry = gui::WidgetIdRegistry::Instance();
ImGuiID widget_id = registry.GetWidgetId("Dungeon/Canvas/canvas:DungeonCanvas");
if (widget_id == 0) {
// Widget not registered!
}
```
### Test Engine Debug UI
```cpp
#ifdef IMGUI_ENABLE_TEST_ENGINE
ImGuiTestEngine_ShowTestEngineWindows(engine, &show_test_engine);
#endif
```
## Performance Considerations
1. **Auto-registration overhead**: Minimal (~1-2μs per widget per frame)
2. **Registry size**: Automatically prunes stale widgets after 600 frames
3. **gRPC latency**: 1-5ms for local connections
## Common Issues
### Widget Not Found
**Problem**: Test can't find widget path
**Solution**: Check widget is registered and path is correct
```cpp
// List all widgets
auto& registry = gui::WidgetIdRegistry::Instance();
for (const auto& [path, info] : registry.GetAllWidgets()) {
std::cout << path << std::endl;
}
```
### ID Collisions
**Problem**: Multiple widgets with same ID
**Solution**: Use unique IDs with `##`
```cpp
// GOOD
gui::AutoButton("Save##DungeonSave");
gui::AutoButton("Save##OverworldSave");
// BAD
gui::AutoButton("Save"); // Multiple Save buttons collide!
gui::AutoButton("Save");
```
### Scope Issues
**Problem**: Widgets disappearing after scope exit
**Solution**: Ensure scopes match widget lifetime
```cpp
// GOOD
{
gui::AutoWidgetScope scope("Dungeon");
gui::AutoButton("Save"); // Registered as Dungeon/button:Save
}
// BAD
gui::AutoWidgetScope("Dungeon"); // Scope ends immediately!
gui::AutoButton("Save"); // No scope active
```
## Examples
See:
- `test/imgui/dungeon_editor_tests.cc` - Comprehensive dungeon editor tests
- `src/app/editor/dungeon/dungeon_editor.cc` - Integration example
- `docs/z3ed/E6-z3ed-cli-design.md` - Agent interaction design
## References
- [ImGui Test Engine Documentation](https://github.com/ocornut/imgui_test_engine)
- [Dear ImGui Documentation](https://github.com/ocornut/imgui)
- [z3ed CLI Design](../docs/z3ed/E6-z3ed-cli-design.md)