A Coding Guide to Cactus Whistle: Does a 16.9 MB Speech Model Hold Up Against Whisper Base? Word Error Rate, Speed, Streaming, Timestamps, Keywords, and Speech-to-Tool Calls on a CPU

By Sana Hassan — October 21, 2026

In this tutorial, we put Whistle — Cactus Compute's speech-to-text model — through a hands-on evaluation. At just 16.9 MB, Whistle runs on a CPU, transcribes seven languages, and returns word timestamps and probabilities. The model card claims it beats Whisper base on LibriSpeech, so we test that claim on seeded test-clean and test-other samples.

We compare Whistle with Whisper tiny and base on identical clips, judged by paired speaker-cluster intervals with Holm correction. The results: a lead over greedy Whisper base on test-other, no significant gap against beam search, and a speed advantage that narrows as clips grow.

Beyond the headline numbers, we probe a 30-second window where the pinned engine silently drops the middle of longer audio. We also examine streaming latency, timestamps, keyword biasing, other languages, noise, embeddings, and speech-to-tool calls.

A note on methodology: all figures are one-run samples from an Apple M4. Published full-split numbers are quoted, not measured.

Setup and Imports

import os
import io
import re
import sys
import json
import math
import time
import random
import hashlib
import shutil
import zipfile
import platform
import traceback
import subprocess
import collections
import urllib.error
import urllib.request

RESULTS = {}
VERDICTS = []


def banner(title):
    print("\n" + "=" * 78)
    ...

via MarkTechPost