TabFM: Google's Zero-Shot Foundation Model for Tabular Data Classification and Regression

Google Research released TabFM, a foundation model for tabular data that performs zero-shot classification and regression using in-context learning (ICL). Instead of training per dataset, you feed the entire table (training rows + target rows) as a prompt, and the model predicts in a single forward pass — no hyperparameter tuning or feature engineering.

How It Works

TabFM uses a hybrid architecture combining TabPFN and TabICL:

• Alternating row and column attention: A multilayer module attends across both rows (examples) and columns (features), capturing complex interactions without manual feature crafting.
• Row compression: Each row's cross-attended representation is compressed into a dense vector.
• ICL Transformer: Processes the compressed row vectors, reducing computational cost compared to raw grid attention.

Key Advantages

• No manual model training, hyperparameter tuning, or feature engineering.
• Works on previously unseen tables — zero-shot.
• Efficient scaling via row compression.

TabFM is now available on Hugging Face and GitHub.

For a deeper dive into the architecture and synthetic training data approach, check the source link below.