ForgeAI: A Visual Workbench for Model Engineering

ForgeAI is a desktop tool designed to simplify local model engineering by providing a visual interface for inspecting, merging, and training AI models. After six months of development, this tool is built with Rust, Tauri v2, SvelteKit, and llama.cpp, and is available for Linux, macOS (both Intel and Apple Silicon), and Windows.

Key Features

• 3D Model Inspection: Visualize model architecture, memory usage, and layers in three dimensions, allowing for a more intuitive understanding of model structure.
• Model Merging: Offers a drag-and-drop interface for merging models using 12 different methods. The M-DNA Forge feature enables users to visually select and drag layers from different models to create a new 'offspring' model.
• Layer-Specific Training: Supports training of specific layers using LoRA/QLoRA techniques.
• Quantization: Allows the quantization of models into GGUF formats ranging from Q2 to Q8.

Technical Challenges and Learnings

The development revealed the complexity of cross-architecture model merging. Successful merges require models to be from the same family and within a 1.2x dimension difference. Arbitrary merging of models, such as attempting to merge a 268M (640d) model with a 999M (1152d) model, often results in poor outcomes due to dimension interpolation not equating to knowledge transfer across diverse architectures.

The tool is particularly useful for developers tired of juggling multiple command-line tools and YAML configurations, providing an all-in-one solution for local model engineering.