Thinking Machines Lab released Inkling 2 Mini on 20 November 2026, and it is the most interesting small open-weight model of the month. It has 26.4 billion parameters in total but only 3.4 billion active per token, it runs its middle layers three times for every token (a "looped" transformer), it accepts text and images with a context window of up to 1 million tokens, and the official checkpoint weighs about 20GB. That means a 32GB Mac or one decent desktop GPU can run it, today, with Ollama, llama.cpp or MLX. The lab's own numbers put it at 79.2% on SWE-bench Verified, ahead of every open model its size it compared against. For a startup, the practical answer is: install it this week as a private local coding and document model, test it on your own tasks, and do not replace a working API model in production until it passes your evals.
I am a fractional CTO for AI-first startups, and "can we run a model on our own machines?" comes up constantly. Inkling 2 Mini is the first small release that makes me revisit my usual answer, with caveats.
The short version:
- What it is: a 26.4B mixture-of-experts model (128 experts, 8 routed per token), 3.4B active parameters, 5.4B "weight activations" per token once you count the loops, 1M-token context, text and image input.
- Hardware: about 20GB for the official weights. Ollama says a Mac with 32GB or more; community 4-bit GGUF files start around 16.6GB, and 2-bit files fit a 12GB GPU at a quality cost.
- License: Apache 2.0 with an acceptable use policy, according to Ollama, VentureBeat and TNW.
- The catch: looping multiplies the KV cache, so long contexts eat memory fast, and day-zero runtimes are still being fixed.
Who should care about Inkling 2 Mini
Anyone who wants a capable model that never sends code or data off the laptop, and any startup paying API bills for high-volume, low-stakes tasks. If neither describes you, read the decision section and move on.
- Engineers who want a local coding assistant. The lab and Hugging Face both position it as a local coding companion, and Ollama ships launchers for agent tools such as OpenCode.
- Startups handling sensitive data. Health, legal, finance or enterprise customer data that you would rather not send to a third-party API is the strongest reason to run a model yourself. If that is you, my HIPAA guide explains why the vendor list matters so much.
- Teams with a large volume of simple calls. Classification, extraction, summarising documents and reading screenshots or PDFs are tasks where a small model can be good enough and cheap.
What actually shipped, with dates
Thinking Machines announced Inkling 2 Mini on its blog on 20 November 2026, with weights on Hugging Face and day-zero support in the main open-source runtimes. Hugging Face published a launch post the same day. Here are the facts that matter, with where they come from.
- Size: 26.4B total parameters and 3.4B active, per the lab. The lab adds that the model uses "5.4B weight activations per token to account for looping", which is why Ollama's page says 5.4B active. Both figures are correct; they count different things.
- Architecture: 24 stored layers in four blocks of six. The first and last blocks run once; the two middle blocks run three times with shared weights, so each token passes through 48 layers of compute while only 24 layers are stored. Most layers use a 1,024-token sliding attention window, with one full-attention layer per block.
- Training: the lab pruned the model from 384 experts per layer to 128 during training, then distilled about 1 trillion tokens from a larger internal model. Most weights are stored natively in NVFP4 (with four layers in MXFP8), so the lab says there is no post-training quantization loss in the official checkpoint.
- Context and input: up to 1 million tokens, per Hugging Face and Ollama, with text and image input and text output.
- Where to get it:
thinkingmachines/Inkling-2-Minion Hugging Face, GGUF files underggml-org/Inkling-2-Mini-GGUF(IQ4_NL and Q8_0) plus Unsloth's quantizations, MLX support upstreamed into mlx-lm, andinkling-2-miniin the Ollama library. The lab also serves it on its Tinker platform.
API prices, if you would rather not host it: VentureBeat reports $0.15 per million input tokens and $0.50 per million output tokens first-party, and lists OpenRouter at $0.08 / $0.35. Neurohive lists OpenRouter at $0.10 / $0.40. The sources disagree, so check the live pricing page before you budget.
What the benchmarks show, and what they don't
The lab's numbers are strong for agentic coding at this size, but they are the lab's numbers. In its launch table, Inkling 2 Mini scores 79.2% on SWE-bench Verified (bash only) against 77.8% for Qwen3.8-27B, 71.1% for Nemotron 3.5 Super VL and 46.1% for Gemma 4 31B. On DeepSWE v1.1 it reports 48.2% against Qwen's 42.2%, and on MCP Atlas, a tool-use benchmark, 82.7% against 77.3%.
The lab is also unusually frank about weaknesses. Its post says the model is "not the strongest small model for multi-turn chat or for instructions with many constraints", and Qwen3.8-27B beats it on Terminal Bench v2.1 (73.0% vs 71.6%) and AutomationBench (48.2% vs 41.8%). Looping does not add stored knowledge either: its non-hallucination rate on AA-Omniscience is 63.4% against Qwen's 71.4%, so pair it with search or retrieval for anything factual.
Independent signals so far:
- Artificial Analysis gives it 20.1 on its Intelligence Index v5.0, level with Qwen3.8-27B at its highest reasoning setting, and reports about $0.59 and 9 minutes per index task against $3.53 and 38 minutes for Qwen, as reported by VentureBeat and TNW.
- Community tool-calling tests are less flattering. One tester quoted by Aliteq ran the BFCL multi-turn benchmark and found Inkling behind an optimised Qwen setup at every quantization he tried (46.7% vs 58.7% with the official checkpoint), and concluded it was not yet a reason to replace Qwen.
- The model card's own warnings, as quoted by Aliteq: it "can get stuck in long reasoning loops", is "less reliable with tools than larger models", and "may take destructive actions without confirmation". The lab's internal destructive-actions score (10.2%, lower is better) is far better than the other models it tested, but it is not zero.
A benchmark tells you a model can do the job. Only your own eval set tells you whether it does your job.
How to run Inkling 2 Mini locally
The quickest path is Ollama on a Mac with 32GB or more; llama.cpp is the path for an NVIDIA GPU; MLX is the native Apple path. Treat everything below as a starting point: the runtimes added support between 20 and 21 November, and fixes are still landing. Check each project's release notes before you rely on a version.
Option 1: Ollama (simplest, Mac or Linux)
Ollama announced support in version 0.40.17, which was a pre-release at launch, so update first. The library page lists three tags, all about 20GB: inkling-2-mini (latest), inkling-2-mini:26b-a5b, and inkling-2-mini:26b-a5b-nvfp4 for MLX on Apple Silicon.
# Check your version, then pull and chat
ollama --version
ollama run inkling-2-mini
Ollama also serves an HTTP API on port 11434, which is how you wire the model into your own code:
curl http://localhost:11434/api/chat \
-d '{
"model": "inkling-2-mini",
"messages": [{"role": "user", "content": "Explain what this regex does: ^[a-z0-9-]+$"}]
}'
To use it as a coding agent, Ollama's page lists launchers such as:
ollama launch opencode --model inkling-2-mini
Option 2: llama.cpp (NVIDIA GPUs, smaller quantizations)
llama.cpp support landed on 20 November, and Aliteq reports a follow-up fix for Unsloth's quantizations merged on 21 November, so build from a recent commit. Hugging Face's post shows the command as llama serve; on a standard llama.cpp build the server binary is llama-server, which accepts the same -hf flag:
# 4-bit GGUF from ggml-org, downloaded on first run
llama-server -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL --jinja --ctx-size 32768
# 8-bit instead, if you have the memory
llama-server -hf ggml-org/Inkling-2-Mini-GGUF:Q8_0 --jinja --ctx-size 32768
Unsloth's recommended sampling, per Aliteq, is temperature 1.0, top-p 1.0, top-k 0 and min-p 0.0. Its file sizes give you a feel for what fits: about 8.5GB for UD-IQ2_XXS, 13.6GB for UD-Q3_K_XL, 16.6GB for UD-Q4_K_M and 22.9GB for UD-Q8_K_XL, plus a 0.64GB vision projector if you want image input.
Option 3: MLX (Apple Silicon, native)
Hugging Face says support is upstreamed into mlx-lm, with community quantizations under the mlx-community organisation:
pip install -U mlx-lm
mlx_lm generate --model thinkingmachines/Inkling-2-Mini \
--prompt "Write a Python function that validates an Indian GSTIN." -m 1000
Reasoning effort
The chat template takes a reasoning effort setting: none, minimal, low, medium, high (the default), xhigh and max. For quick edits and extraction, low or medium saves a lot of tokens and time; keep high for hard debugging.
The long-context catch
The 1M-token context is real on paper, but looping makes long contexts expensive in memory, so plan for 32K to 128K on a laptop. Each loop pass keeps its own key-value cache, so the cache grows faster than in a normal model of this shape. Aliteq's estimate is about 1GB of cache at 32K tokens in fp16, rising to about 32GB at 1M tokens (around 9GB with a q4_0 cache). That is its own derivation, not a vendor figure, but early users report the same pattern: one tester measured 31.2GiB of memory at 131K context for Inkling against 20.9GiB for Qwen3.8-27B, and 262K did not fit his machine.
In practice:
- Set the context size explicitly (
--ctx-sizein llama.cpp,num_ctxin Ollama) instead of accepting a huge default. - Quantize the KV cache to q8_0 if your runtime supports it for this architecture.
- Avoid offloading experts to system RAM; testers report it drops speed sharply.
Speed reports are all self-reported and vary by engine: EXO Labs, which says it worked with the lab, reports about 125 tokens per second on an M5 Max MacBook Pro and 210 on an M5 Ultra Mac Studio; community users report roughly 45 to 65 tokens per second on single RTX 3090 and 4090 cards with llama.cpp. As one of them put it, "speed is the engine, not the model".
Should your startup use it? A decision guide
Use it where privacy or volume matters and the task is checkable; keep frontier APIs where mistakes are expensive. Here is how I'd sort the work.
- Good fit now: a private coding assistant for engineers working on sensitive repos; summarising and extracting fields from documents, PDFs and screenshots (its document and screen benchmarks are strong); first-pass triage of support tickets or logs; offline demos and air-gapped customer environments.
- Test before you trust: multi-step agents that call tools. Its tool-calling results are mixed in community tests and the model card itself warns about loops and destructive actions.
- Keep on a frontier API for now: anything customer-facing where a wrong answer costs money or trust, long multi-turn conversations, instructions with many constraints, and closed-book factual questions.
On cost, remember that "free" weights are not free. A laptop model costs nothing per token for one engineer, but serving it to customers means GPUs, monitoring and someone on call. I worked through that math in self-hosting an LLM vs using an API; the short version is that hosted open models on a pay-per-token API often beat both. Compare this with Kimi K3.1 Pro earlier this month: a strong open model that almost no startup can self-host. Inkling 2 Mini is the opposite, which is exactly why it matters.
Guardrails if you let it act as an agent
A local model is still an agent with your permissions. Before you point OpenCode or any agent tool at a real repo:
- Run it in a container or a throwaway checkout, not your main working copy with production credentials in the environment.
- Keep secrets out of reach: no
.envfiles, cloud CLI sessions or SSH keys inside the sandbox. - Cap output tokens and agent turns, as the model card advises, so a reasoning loop stops itself.
- Require confirmation for destructive commands (deletes, force pushes, migrations).
- Log every tool call so you can see what it did.
My AI agent security guide covers this in more depth.
What I'd do as a CTO this week
Spend two hours, not two sprints. I would not change a production model on day three of a release, but I would run a small, honest test now.
- Install it on one engineer's machine with Ollama or llama.cpp and use it for real work for a week, on a sandboxed checkout.
- Run your existing eval set against it alongside your current model. If you don't have one, my guide to LLM evals for startups shows how to build 50 test cases in an afternoon.
- Pick one high-volume, low-risk task (extraction, tagging, summarising) and measure cost per passing task through a hosted endpoint.
- Keep your code provider-agnostic, so a model swap is a config change. That is the whole argument of avoiding LLM vendor lock-in, and releases like this are when it pays off.
If you want a second opinion on where open models fit in your product and what they would really cost, that is a normal part of my fractional CTO work, and a lighter-touch technical advisor retainer covers it too.
Frequently asked questions
What is Inkling 2 Mini?
Inkling 2 Mini is an open-weight AI model from Thinking Machines Lab, released on 20 November 2026. It is a 26.4-billion-parameter mixture-of-experts model with 3.4 billion active parameters per token, a looped architecture that runs its middle layers three times, a context window of up to 1 million tokens, and text and image input.
Can I run Inkling 2 Mini on a Mac?
Yes, if it has enough memory. The official weights are about 20GB and Ollama recommends a Mac with 32GB or more. Run ollama run inkling-2-mini after updating Ollama, or use mlx-lm for a native Apple Silicon setup. Keep the context size modest, because long contexts use a lot of extra memory on this model.
How much VRAM does Inkling 2 Mini need?
For the full-quality checkpoint, plan for about 20GB plus room for the KV cache, so a 24GB or 32GB GPU. Community 4-bit GGUF files are about 16.6GB, and a 2-bit file of about 8.5GB fits fully on a 12GB card at lower quality.
Is Inkling 2 Mini better than Qwen3.8-27B?
It depends on the task. On the lab's own benchmarks it leads on SWE-bench Verified, DeepSWE, BrowseComp and MCP Atlas, and Artificial Analysis rates the two level overall while Inkling uses far less compute per task. Qwen3.8-27B leads on Terminal Bench v2.1, AutomationBench, factual accuracy and, in community tests, multi-turn tool calling. Test both on your own work.
Is Inkling 2 Mini free for commercial use?
Ollama, VentureBeat and TNW report that it is released under Apache 2.0 with an acceptable use policy, which generally allows commercial use. Read the license and policy on the Hugging Face model card yourself before you ship it in a product.
If you are deciding whether a local or open model belongs in your product, book a 30-minute call and we can work through it with your actual workloads.
Sources
- Introducing Inkling 2 Mini (Thinking Machines Lab)
- Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup (Hugging Face)
- inkling-2-mini model library page (Ollama)
- Inkling 2 Mini is now on Ollama with 0.40.17 (Ollama on X)
- Thinking Machines wows with Inkling 2 Mini looped transformer model that runs locally on laptops (VentureBeat)
- Thinking Machines releases Inkling 2 Mini, a looped AI model for laptops (The Next Web)
- Inkling 2 Mini: Thinking Machines' 20GB open model (Neurohive)
- Can you run Inkling 2 Mini locally? (Aliteq)
- Thinking Machines' Inkling 2 Mini Redraws the Open-Weight Efficiency Frontier (Futurum Group)
Written By
Kunal Vohra
Technical Co-Founder & Fractional CTO
I've co-founded 6+ startups across India, the UAE, and the US, spanning AI, Web3, fintech, and cybersecurity. I write about the technical and strategic decisions that determine whether a startup thrives or stalls.
Comments
Loading comments…