JIT Compiling Code in 5μs: Building a Fast JIT for Postgres pgrust

Historically, fast JIT compilation was a black art. Production databases either use LLVM or generate C/C++ code, both with high compile times. Now, AI assistance makes it practical to write a JIT compiler that directly targets assembly, achieving compile times around 5μs. This is what the author did for pgrust, a Postgres extension in Rust.

The key insight: JIT compilation is valuable when runtime information drastically alters behavior, like parsing data with unknown schema. For databases, JIT-compiling every query can yield 2-5x performance wins, but only if compile time is negligible. LLVM and C/C++ generation take milliseconds, limiting their use to complex queries only. A 5μs JIT can handle every query.

To demonstrate, the article builds a toy regular expression engine supporting literals and repetition (*). The regex b(an)* is used as a benchmark. The interpreter version is 10-20x slower than handwritten code. The JIT approach aims to close that gap.

Key points from the post:

• JIT compilation is the practice of generating compiled code at runtime, often giving 2-5x performance gains.
• Traditional JIT routes (LLVM, C generation) suffer from high compile times.
• AI assistance lowers the barrier to writing assembly-generating JITs.
• pgrust's JIT compiler compiles code in around 5μs, enabling JIT for every SQL query.
• The article provides a walkthrough of building a simple regex engine with JIT, skipping the parser for simplicity.

For developers interested in database internals or JIT techniques, this post offers practical insights into a new approach that could improve on legacy systems.