Gemma 4 Chat Template Bug: Tool Parameters with anyOf/null Rendered as Empty type

A Reddit user discovered that Gemma 4 (gemma-4-31B-it) fails to parse tool parameters that use the JSON Schema pattern anyOf: [$ref, null] — a common pattern for nullable object references. The default chat template assumes a direct type field at the top level, so schemas like this:

{"anyOf": [{"$ref": "#/$defs/SomeObject"}, {"type": "null"}]}

get stripped of anyOf, $ref, and $defs, resulting in type: "" in the prompt. This breaks tool calling on multiple inference engines (llama-server, others) while Qwen3.5 and gpt-oss-20b handle it correctly.

Diagnosis and Fix

The user debugged with verbose logging from llama-server and had GPT-5.5-high (via codex CLI) compare logs between Qwen3.5-27B-Q4_K_M and gemma-4-31B-it-Q4_K_S on a MacBook Pro. The root cause was traced to the Gemma chat template's assumption that every parameter has a direct type key. A small change to the Jinja template now preserves anyOf, $ref, and $defs structures.

The corrected Jinja template is available on Pastebin: https://pastebin.com/p9z3BAC0

A PR has been submitted to the Hugging Face repository for gemma-4-31B-it.

Takeaway

If you use Gemma 4 for tool/function calling with nullable JSON Schema refs, apply the fixed chat template. Users of Qwen3.5 or gpt-oss-20b are unaffected.