Thinking Machines Lab has released Inkling-Small, an open‑weights Mixture‑of‑Experts model with 276 billion total parameters and 12 billion active parameters. The model was trained on NVIDIA GB300 NVL72 systems and can reason over text, images and audio natively. It offers a 1 million token context window and adjustable thinking effort, and its weights are available under the Apache 2.0 license on Hugging Face.
For teams worried about deployment cost, the BF16 checkpoint needs at least 600 GB of aggregated VRAM—met by four B300 GPUs or eight H200 GPUs. The quantized NVFP4 checkpoint drops that requirement to 180 GB. It runs as W4A4 on a single B300 (SM100+ required) or as W4A16 on two H200s, making the model accessible to startups on a single rented B300 instance and to mid‑size enterprises that already have H200 capacity. Supported runtimes include SGLang, vLLM, TokenSpeed, Unsloth and Hugging Face.
Architecturally, Inkling‑Small is a 42‑layer decoder‑only transformer with a sparse MoE feed‑forward backbone. Each token routes to six of 256 experts plus two shared experts that are always active. Attention combines local and global layers, and the model is encoder‑free. Images are split into 40×40‑pixel patches and processed by a four‑layer hMLP; audio is converted to dMel spectrograms. Both modalities, together with text, pass through a lightweight embedding layer and are handled jointly by the same decoder, producing text‑only output. The model supports BF16, MXFP8 and NVFP4 numerics, and accepts WAV audio at 16 kHz ideally under two minutes.
Benchmark results show Inkling‑Small outperforms its larger teacher on many reasoning and coding tasks: 31.6 % on Humanity’s Last Exam (text), 80.2 % on SWE‑bench Verified, 64.7 % on Terminal‑Bench 2.1, 54.4 % on Toolathlon Verified, and strong scores on GPQA Diamond (89.5 %), AIME 2026 (95.5 %) and IFBench (82.2%). Some knowledge‑recall benchmarks decline, reflecting the trade‑off of a smaller active parameter set.
Multimodal performance remains close to the original Inkling: MMMU Pro 74.0 %, CharXiv RQ 77.4 % (81.3 % with programmatic inspection), Audio MC 54.9 %, MMAU 77.0 % and VoiceBench 90.1 %. Epistemics improve slightly, with a ForecastBench Brier index of 61.3 ± 0.46. Safety scores are strong (StrongREJECT 98.4 %, FORTRESS benign 96.9 %), and the lab advises adding downstream moderation such as Llama Guard for consumer‑facing uses.
In short, Inkling‑Small gives organizations a powerful, multimodal model that can be self‑served on modest GPU hardware, lowering the barrier to advanced AI while maintaining high performance on coding, reasoning and agentic tasks. #AI #Product #MachineLearning #GPU #OpenSource #Innovation