scawful/yaze
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Add AI GUI controller and vision action refiner for enhanced automation

Browse Source 
- Introduced `AIGUIController` class to manage AI-driven GUI automation with vision feedback, enabling natural language command execution and iterative action refinement.
- Implemented `VisionActionRefiner` class to analyze screenshots and refine actions based on visual feedback, improving action success rates.
- Added header and implementation files for both classes, along with necessary methods for screenshot analysis, action verification, and UI element location.
- Updated CMake configuration to include new source files for the AI GUI controller and vision action refiner functionalities.
This commit is contained in:
scawful
2025-10-04 23:09:59 -04:00
parent 39edadb7b6
commit ec88f087a2
5 changed files with 1034 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/cli/agent.cmake
View File

@@ -73,6 +73,8 @@ set(YAZE_AGENT_SOURCES
cli/service/agent/learned_knowledge_service.cc
cli/service/ai/ai_service.cc
cli/service/ai/ai_action_parser.cc
cli/service/ai/vision_action_refiner.cc
cli/service/ai/ai_gui_controller.cc
cli/service/ai/ollama_ai_service.cc
cli/service/ai/prompt_builder.cc
cli/service/ai/service_factory.cc