Routerly: Self-Hosted LLM Gateway with Runtime Routing Policies and Budget Control
Routerly is a self-hosted LLM gateway built to address gaps in existing solutions. The developer created it because OpenRouter is cloud-based, and they wanted something runnable on their own infrastructure, while LiteLLM's routing felt too manual despite handling budgeting well.
Core Features
Instead of hardcoding a specific model in your application, Routerly lets you define routing policies that determine model selection at runtime. Available policies include:
- Cheapest
- Fastest
- Most capable
- Combinations of these policies
Budget control operates at the project level with actual per-token tracking, providing granular cost management.
Compatibility and Use
Routerly is OpenAI-compatible, meaning it can drop into existing workflows without code changes. Specifically mentioned compatible tools include:
- Cursor
- LangChain
- Open WebUI
It works with "anything else" that uses the OpenAI API format.
Current Status
The developer acknowledges there are rough edges and is seeking community feedback on:
- What's broken
- What's missing
- Whether the routing logic makes sense in practice
- Whether it solves a real problem people have
The tool is completely free and open source, with no commercial sales pitch. The developer is focused on practical feedback from the technical community.
Resources
- GitHub Repository: https://github.com/Inebrio/Routerly
- Website: https://www.routerly.ai
📖 Read the full source: r/LocalLLaMA
👀 See Also
Clawforce: Open-Source Control Plane for Managing Clawbot Agent Teams
Clawforce is an open-source control plane for managing Clawbot agent teams that allows deployment in a few clicks. It provides configuration of characters, skills, MCP integrations, and tools through a UI, with agents capable of planning, coordinating, and executing tasks collaboratively.
hipEngine: Fast Native Qwen 3.6 Inference for RDNA3 (Strix Halo, 7900 XTX)
hipEngine is a new open-source (AGPLv3) ROCm-native inference engine for Qwen 3.6 MoE on RDNA3 GPUs. Benchmarks show prefill up to 2718 tok/s on 7900 XTX, competitive with llama.cpp, and INT8 KV cache enabling full 256K context in under 24GB.
Six open-source tools that address OpenClaw's security, cost, and complexity issues
A developer tested six community tools to solve OpenClaw's security gaps flagged by Cisco, spiraling costs, and complex setup. ClawSec provides security scanning and integrity verification, Antfarm enables deterministic multi-agent workflows, and LanceDB Pro improves memory retrieval with hybrid vector search.
Monitor Your Claude AI Usage with a New Linux Taskbar Widget
A new Linux taskbar widget helps users track their Claude AI subscription usage in real-time, with color-coded feedback and easy installation.