RepoLens: Interactive Local Codebase Packer and Token Optimizer (TUI/CLI) in Go

RepoLens is a new open-source, zero-dependency Go tool that packs a local codebase into a single file optimized for LLM context windows. It features an Elm-architecture TUI built on Charm's Bubble Tea, along with several smart context-optimization mechanisms.

Key Features

• Interactive File Explorer (TUI): Select files and folders recursively with the spacebar. Expand/collapse nested folders visually.
• Live Tiktoken Counter: A dynamic progress bar estimates context limit consumption (GPT-4o encoding) in real time as you select files.
• Smart Comment Stripping: Automatically removes comments and docstrings for Go, Python, TypeScript, JavaScript, Shell, HTML, and CSS, while keeping executable shebangs (#!/bin/bash). Benchmark on the spf13/cobra repo: prompt size shrunk from 621 KB to 518 KB (17.2% token savings).
• Token-based File Splitting: If your codebase exceeds a token threshold (e.g., --max-tokens 50000), RepoLens splits output into sequential parts, automatically prepending the tree layout diagram to every part.
• Local Secret Scanner: Flags potential AWS keys, OpenAI keys, Slack webhooks, and generic passwords. In TUI mode, blinks a yellow ⚠️ [SECRET!] warning next to the file.
• Robust XML Formatting: Safely wraps code payloads inside <![CDATA[ ... ]]> blocks and escapes nested brackets to prevent prompt formatting breakdown.

Quick Start

Install via Go:

go install github.com/catball912/repolens@latest

Or run in direct CLI mode:

repolens -n -d . -i "*_test.go,*.log" -o output.md

Who It's For

Developers who need to feed a local codebase into an LLM (e.g., for code review, refactoring, or documentation generation) without uploading files to a third-party service, and who want a lightweight, compiled tool with no runtime dependencies.