Codeflash Analysis: 118 Performance Bugs Found in Two PRs Written with Claude Code

Performance Analysis of AI-Generated Code

Codeflash used their own optimization tool to analyze two pull requests written with Claude Code. The features analyzed were Java language support (52,000 lines across parsers, context extractors, instrumentation, test runners, assertion transformers) and React framework support (24,000 lines covering component discovery, profiling, benchmarking, and code replacement).

Key Findings

Across just these two PRs, Codeflash identified 118 functions performing significantly worse than necessary. These weren't edge cases — they were functions in the hot path of their optimizer, running on every optimization job for every user.

Patterns of Inefficiency

• Catastrophically inefficient algorithms: A type extraction function in the Java context module was 446x slower than needed, implemented with naive string scanning instead of tree-sitter-based extraction. A helper function finder was 74x slower for similar reasons.
• Redundant computation: Functions were re-parsing data already parsed, re-traversing trees already walked, rebuilding strings character by character. An assertion target call builder was 19x slower due to recomputing source byte conversions on every invocation instead of caching. An import-insertion utility in the React PR was 36x slower due to redundant tree traversals.
• Missing caching: Functions called repeatedly with same inputs computed results from scratch each time. A type definition extractor in the React PR was 16x slower without memoizing intermediate results, and an export checker was 9x slower for same reason.
• Suboptimal data structures: Lists where sets should have been, linear searches where hash lookups would work, string concatenation in loops instead of joins. A brace-balancing parser was 3x slower due to inefficient data structure choices.

Concrete Example: 19x Performance Improvement

Claude Code wrote this function to convert byte offsets to character positions:

# Called for every AST node found in the file
start_char = len(content_bytes[:start_byte].decode("utf8"))
end_char = len(content_bytes[:end_byte].decode("utf8"))

Codeflash replaced it with:

# Build a lookup table once, then binary search for every node
from bisect import bisect_right
cum_bytes = [0]
for ch in source.decode("utf8"):
    cum_bytes.append(cum_bytes[-1] + len(ch.encode("utf8")))
start_char = bisect_right(cum_bytes, start_byte) - 1
end_char = bisect_right(cum_bytes, end_byte) - 1

The original code decodes the entire byte prefix from the beginning of the file on every call — O(n) per lookup. For a file with hundreds of AST nodes, this means re-decoding the same bytes hundreds of times. The optimized version builds a lookup table once and uses binary search — O(n) once, then O(log n) per lookup.

The article emphasizes this isn't about whether to use AI coding agents (they recommend using them), but about what happens to the code after you do. These performance issues represent a new category of technical debt that AI agents systematically introduce by focusing on correctness and readability over performance optimization.