
Inkling 2 Mini: 26B MoE Local Model with 1M-Token Context
By Ali Al-Montaser — Senior AI Technical Editor
Thinking Machines Lab launches Inkling 2 Mini, a 26B-parameter multimodal Mixture-of-Experts transformer with only 3B active parameters per token and a 1M-token context window. It uses a looped transformer architecture to match 48 logical layers with just 24 physical layers, ships in NVFP4 and MXFP8 formats, and supports transformers, llama.cpp, MLX, and SGLang. The model outperforms Gemma 4 31B and Qwen3.8-27B on 32 of 48 benchmarks, positioning it as a key option for local AI setups.
Executive Overview
Thinking Machines Lab has announced the release of Inkling 2 Mini, a multimodal Mixture-of-Experts (MoE) transformer designed for local AI setups. The model is available immediately through the transformers, llama.cpp, MLX, and SGLang frameworks. It features 26 billion total parameters with only 3 billion active parameters per token, a context window of up to 1 million tokens, and support for both image and text inputs. According to the official Hugging Face blog, Inkling 2 Mini outperforms Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL on 32 of 48 benchmarks, making it a pivotal option for local AI environments.
📊 Official Technical Specifications & Data Sheet
| Technical Aspect | Confirmed Official Data |
|---|---|
| 💰 Pricing & Usage Cost | Open-weights model, free to download and run locally via Hugging Face, with no official per-million-token pricing. Operating cost depends on the user's local or cloud hardware. |
| 🌐 Platforms & Immediate Availability | Hugging Face (original weights), transformers, llama.cpp / Llama App (macOS and Windows), MLX (mlx-lm via mlx-community), SGLang (Day-0 support for production deployment on data center hardware with SM103 requirement). |
| ⚡ Performance & Speed Benchmarks | Leads 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL. Leads FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0. Generation speed measured on a single NVIDIA B300 GPU with a 1024-token prompt and 1024-token generation at concurrency levels of 1, 4, 16, and 64. DSpark drafter support for additional acceleration at a small memory cost. |
| 🛡️ Security & Robustness | Leads the Safety and Alignment category among the 48 benchmarks. No separate numerical security ratings are disclosed in the official statement. |
| 🧠 Context Window | 1,000,000 tokens (1M-token context size), with a local sliding attention window of 1,024 tokens in each block of six layers. |
| 🌍 Arabic Language & Regional Support | The official statement does not mention a separate Arabic evaluation. The model is multilingual by nature as a multimodal transformer, but no confirmed Arabic performance figures are available in the source. |
Deep-Dive Features & Architecture
Inkling 2 Mini is built on a Looped Transformer architecture, its most notable architectural innovation. The model contains only 24 physical decoder layers, but the middle 12 layers (from 6 to 17) are executed three times in a loop, making it behave as if it has 48 logical layers (6 + 3×12 + 6) while reusing the same weights on each pass. This design reduces the number of distinct layers stored in the checkpoint while maintaining 48 layer visits per token, with each visit retaining its own KV state and short-convolution state.
The model uses a Mixture-of-Experts architecture where each token is routed to 8 out of 128 SwiGLU experts per layer (compared to 384 experts in Inkling 2 Small), with selected-score normalization and weight scaling by a factor of 8. The text decoder width is 2,304 (compared to 4,608 in Small), and the intermediate width per expert is 1,152. The attention mechanism uses 24 query heads and 8 KV heads, with learned relative positional logarithms, a learned sink, and an output gate, while full attention layers use dynamic convolutions for keys and values.
The model ships in mixed-precision format combining expert weights in NVFP4 and MXFP8, and includes a DSpark drafter for additional acceleration at a small memory cost. A research variant (TML-d24-varloop-research) shares the same architecture but is trained to accept a variable number of loop passes (1 or 2 instead of 3), significantly speeding up inference with a possible slight quality drop.
Benchmark & Competitive Performance
According to the official statement, Inkling 2 Mini leads its class on the Artificial Analysis Intelligence Index v5 (AAv5) and outperforms Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL on 32 of 48 benchmarks. It leads on FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and trails it on AutomationBench-AA v1.0.6. The model also leads the Safety and Alignment category. These results position Inkling 2 Mini as a strong contender for local AI workloads that require both efficiency and high performance.
Industry Impact & Enterprise Adoption
The release of Inkling 2 Mini signals a growing trend toward efficient, locally deployable multimodal models. With its 26B total parameters and only 3B active parameters per token, it offers a balance between capability and resource efficiency, making it attractive for enterprises and developers running AI on local hardware. The immediate support for popular frameworks like transformers, llama.cpp, MLX, and SGLang lowers the barrier to adoption, while the 1M-token context window enables long-document processing and complex reasoning tasks. The model's strong benchmark performance against established competitors like Gemma 4 31B and Qwen3.8-27B suggests it could become a preferred choice for organizations seeking open-weight, high-performance models for on-premise or edge deployments.
Conclusion
Inkling 2 Mini represents a significant advancement in local AI models, combining a novel looped transformer architecture with Mixture-of-Experts efficiency and a massive context window. Its availability across multiple frameworks and its competitive benchmark results make it a compelling option for developers and enterprises alike. As the demand for local, privacy-preserving AI continues to grow, models like Inkling 2 Mini are poised to play a central role in shaping the next generation of AI applications.
Media Source: Hugging Face | Official Company Statement: Original Source | Fact Verification & Analysis: AI Tools Oasis
Frequently Asked Questions
Inkling 2 Mini has a total physical size of approximately 26 billion parameters (26B), while only 3 billion parameters (3B) are active per token, thanks to its Mixture-of-Experts architecture that selects 8 experts out of 128 in each layer.
Inkling 2 Mini supports a context window of up to 1 million tokens (1M-token context size), with a local sliding attention window of 1,024 tokens in each block of six layers.
Inkling 2 Mini ships with immediate support for transformers, llama.cpp (via Llama App on macOS and Windows), and MLX (via mlx-lm), plus Day-0 SGLang support for production deployment on data center hardware. GGUF versions are available in IQ4_NL (4-bit) and Q8_0 (8-bit) formats.
Inkling 2 Mini outperforms Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL on 32 of 48 benchmarks, leads on FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and trails it on AutomationBench-AA v1.0.6.
Inkling 2 Mini has 24 physical decoder layers, but the middle 12 layers (6–17) are executed three times in a loop, making it behave like 48 logical layers (6 + 3×12 + 6) while reusing the same weights. This reduces checkpoint size while maintaining 48 layer visits per token.

AI Tools Oasis Team
Bringing you the latest news and analysis in the world of Artificial Intelligence with accuracy and credibility. Follow us for all updates.


