Routerly: Self-Hosted LLM Gateway with Runtime Routing Policies and Budget Control
Routerly is a self-hosted LLM gateway built to address gaps in existing solutions. The developer created it because OpenRouter is cloud-based, and they wanted something runnable on their own infrastructure, while LiteLLM's routing felt too manual despite handling budgeting well.
Core Features
Instead of hardcoding a specific model in your application, Routerly lets you define routing policies that determine model selection at runtime. Available policies include:
- Cheapest
- Fastest
- Most capable
- Combinations of these policies
Budget control operates at the project level with actual per-token tracking, providing granular cost management.
Compatibility and Use
Routerly is OpenAI-compatible, meaning it can drop into existing workflows without code changes. Specifically mentioned compatible tools include:
- Cursor
- LangChain
- Open WebUI
It works with "anything else" that uses the OpenAI API format.
Current Status
The developer acknowledges there are rough edges and is seeking community feedback on:
- What's broken
- What's missing
- Whether the routing logic makes sense in practice
- Whether it solves a real problem people have
The tool is completely free and open source, with no commercial sales pitch. The developer is focused on practical feedback from the technical community.
Resources
- GitHub Repository: https://github.com/Inebrio/Routerly
- Website: https://www.routerly.ai
📖 Read the full source: r/LocalLLaMA
👀 See Also
Agent Factory: Autonomous System Builds AI Agents from Online Problem Discussions
Agent Factory is an autonomous system that scrapes Reddit, HN, GitHub, and Twitter for real problems, scores them on demand, market gap, and feasibility, then builds standalone AI agents for promising ideas. The system uses a minimal Next.js template with 7 tools and runs Claude Code headless via a shell script.
VoidLLM: Zero-Knowledge Proxy for Ollama and vLLM with Team Access Control
VoidLLM is a proxy that sits between applications and local LLM servers like Ollama and vLLM, adding organization/team access control, API key management, usage tracking, and rate limiting without viewing prompts. It has <2ms proxy overhead and works with OpenAI-compatible SDKs.
Curated list of 260+ AI agents and tools with open-source and self-hosted focus
A comprehensive GitHub repository lists over 260 AI agents and frameworks, emphasizing open-source, self-hosted, and local-first options including Ollama, OpenClaw, and DeerFlow.
Building syntaqlite: A SQLite DevTools Project Created with AI Assistance
Lalit Maganti built syntaqlite, a set of developer tools for SQLite, over three months using AI coding agents after wanting it for eight years. The project required parsing SQL exactly like SQLite, which involves adapting SQLite's dense C codebase with over 400 grammar rules.