Nums AI Releases Causilo: A Tabular Foundation Model That Tops TabArena Among Single Models

Nums AI has released Causilo, a pretrained tabular foundation model for classification and regression. Causilo ships with a scikit-learn-compatible interface, Apache-2.0 licensed code, and pretrained weights on Hugging Face. On TabArena, it holds the highest Elo among single models for both classification and regression.

Is it deployable? Yes—for research and evaluation today, on CUDA or CPU. Commercial, production, and hosted API use require a separate license from Nums AI.

What Causilo Does

Causilo is an in-context learning model. Calling fit does not update the pretrained weights. Instead, it stores the training rows as context and predicts query rows in a single forward pass. According to its TabArena submission, Nums AI pretrained Causilo exclusively on synthetic data, with no TabArena datasets included.

Inputs can be NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification supports up to 10 classes. Regression returns mean predictions by default. Version 1.0.1 adds median and quantile outputs, based on 999 native quantiles.

Architecture: Refinement, Compression, and In-Context Learning

Nums AI splits the network into three phases: refinement, compression, and in-context learning. The released code and configs show how each phase works.

Features are grouped in sets of three. Each value is embedded with 16 learned sine and cosine frequencies. Missing values receive their own learned vector.

Two column stages summarize each feature group. In each stage, 128 latent slots read only the training rows and pass that summary to every row. Between the two column stages, a row stage lets feature groups interact through four latent tokens. It uses cross-attention instead of full self-attention, which Nums AI says keeps cost linear in the feature count.

A pooling block then compresses each row into a fixed 512-dimensional vector. Labels are added to the training rows. A 12-layer prediction block lets query rows attend to those labeled rows. Query rows cannot modify the training context or one another.

By default, eight ensemble members share the same weights. Each member cycles through one of four normalization schemes—none, rank2gaussian, robust, or power—with seeded feature and class permutations.

TabArena Results

Nums AI used the official TabArena pipeline: 51 datasets and 816 Full splits, with 8 estimators and seed 42. A TabArena maintainer re-ran the full evaluation and reproduced the same overall Elo of 1794.

| Task | Causilo Elo | Next best single model | Causilo improvability |

|------|-------------|------------------------|-----------------------|

(Full results table available in the original release notes.)

2026 Context: Why Tabular Foundation Models Matter

Causilo arrives as tabular foundation models move from research curiosity to practical infrastructure. In 2026, TabArena has become the de facto leaderboard for tabular ML, much as MMLU and SWE-bench anchor LLM evaluation. Nums AI's decision to train only on synthetic data—and still reach top single-model Elo—signals that in-context learning is maturing as a replacement for per-dataset gradient boosting pipelines. For teams evaluating TabPFN alternatives, Causilo is now the strongest single-model baseline to beat.

via MarkTechPost