Tabular foundation models are a class of pretrained transformers that predict missing values in any spreadsheet zero-shot, analogous to how language models complete text. As of mid-2026, every single-model entry above the best tuned gradient-boosted decision tree on the TabArena community benchmark is one of these models. This article explains what they are, validates the strongest one with unrestricted open weights on personal hardware, and identifies scenarios where gradient-boosted trees still prevail.
Figure 1: Accuracy versus serving cost on TabArena (official board, snapshot 2026–07–15). TabICLv2 (blue) occupies the cheap-and-accurate frontier among publicly downloadable models; TabFM (hollow) tops the board but lacks an accompanying paper; and the tree family (squares) sits over 150 Elo lower at comparable serving cost. Source: image by author.
A fairly new class of model for tables—pretrained transformers that predict on any spreadsheet zero-shot, loosely termed “tabular LLMs”—has emerged. On the TabArena leaderboard, every single-model entry above the best tuned, ensembled GBDT configuration is one of these; the only entries above it that are not are AutoGluon’s 4-hour ensemble pipelines. This inverts the default answer of the past decade, which was to fit a tuned gradient-boosted tree. Leaderboards have previously been misleading in ways that only surface when an outsider re-runs them (as was my experience with time-series models). So, before accepting this one, I audited it: I recomputed the board’s scoring from its published artifacts, took TabICLv2—the strongest model with unrestricted open weights—and re-ran it from scratch on hardware I controlled.
My independent run covered all 51 datasets on the benchmark’s official Lite protocol (one train/test split per dataset, using the same split indices as the leaderboard) on a single AWS A10G. Scored against the official artifact rows on those identical splits, my TabICLv2 achieved an Elo of 1559, compared to the official 1575—adjacent ranks and well within the ±60–86 bootstrap intervals. Per task, 16 of the 51 metric values matched exactly to four decimal places, the median relative difference was 0.08%, and the worst dataset differed by 3.5%, consistent with GPU nondeterminism rather than a methodological discrepancy. The entire sweep, including environment setup, fit within a 2.1-hour GPU session costing just over $2.
What a tabular foundation model is
A tabular foundation model is a single pretrained model that predicts on any table zero-shot. You provide the rows whose labels you know as context—similar to pasting worked examples into an LLM prompt—and it predicts the labels of held-out rows in a single forward pass. No gradient training on your data occurs, and no hyperparameter search is needed; the step that other tabular methods call “training” becomes inference. The literature terms this in-context learning, and the TabPFN papers established its viability for tables.
The model functions like a learned k-nearest-neighbors approach: to label a held-out row, it attends over the rows with known labels and derives an answer weighted by similarity. The distinction from ordinary k-NN is that both the similarity metric and combination of neighbors are learned during pretraining rather than predetermined.
The family has expanded rapidly: TabPFN and its successors (Prior Labs), TabICL and TabICLv2 (Inria’s SODA team), TabDPT (Layer 6), and—just two weeks before this post—Google Research’s TabFM. All of these appear in the leaderboard standings.
The term “tabular LLM” is loosely applied. These models are transformers, but they do not model language and differ from text LLMs in every component. A text LLM tokenizes word pieces from a fixed vocabulary; a tabular model processes cells, columns, and rows of a table—numbers and categories with no fixed vocabulary, since your table’s columns may be entirely novel, and the model must adapt. A text LLM is pretrained on internet text using next-token prediction; a tabular foundation model is pretrained on synthetic or real tabular data using objectives like masked column prediction or in-context learning on diverse table structures.