Kimi K3: what it actually means to run it locally, use it remotely, or plug it into Claude Code

Published on the TEN INVENT blog · July 2026

Last week Moonshot AI announced Kimi K3. Yesterday they published the weights. Since then the internet has been full of articles saying the same three things: "2.8 trillion parameters", "the largest open-weight model in history", "run it locally". That last part is, for 99.9% of us, false.

This article tries to answer three practical questions for someone who builds software for a living, not for someone who collects benchmarks: what it costs to run it locally, what changes if you use it remotely instead of Claude Code, and how to wire it into Claude Code in five minutes.

1. The launch timeline, verified

The order of events matters, because there was a ten-day window in which the model was live but the weights were not:

The official repo is huggingface.co/moonshotai/Kimi-K3 — public, ungated, with real safetensors, a LICENSE and a complete model card. There is also a GitHub repo MoonshotAI/Kimi-K3 with the technical report as a PDF.

One warning worth stating explicitly: this week dozens of "Kimi-K3" mirrors and repos have appeared that are not Moonshot's. If you download anything, download from the moonshotai organization and pin the commit hash. The model requires trust_remote_code=True, which means you are executing Python code that comes from the repo. Read it first.

2. What K3 is, technically

From the official model card:

• 2.8T total parameters, 104B active per token — MoE with 896 experts, of which 16 are activated (plus 2 shared). That's ~1.8% routing density.
• 93 layers, of which 69 are KDA (Kimi Delta Attention — linear attention with a delta rule) and 24 are Gated MLA.
• Context of 1,048,576 tokens, flat, with no price tiering.
• Natively multimodal (text + image, 401M MoonViT-V2 encoder).
• Native MXFP4 quantization with MXFP8 activations — quantization-aware training applied from the SFT phase onwards.
• Thinking is always on. There is no non-thinking mode. reasoning_effort accepts low / high / max, default max.

On benchmarks: it is the first open-weight model that seriously enters the conversation with the frontier. It beats Claude Opus 4.8 and GPT-5.5 on most coding and agentic evals, sits a few points below Claude Fable 5 and GPT-5.6 Sol on most of the rest, and is above both on a few (SWE-Marathon 42.0, BrowseComp 91.2, MCPMark 94.5). Artificial Analysis puts it at 57 on the Intelligence Index — first place among open-weight models.

The architectural detail that matters operationally: KDA breaks conventional prefix caching. Moonshot contributed its own caching implementation to vLLM alongside the launch. If you serve K3 on an old vLLM build, it will hurt.

3. Local: the hard math

This is the part where we have to be honest.

The published weights occupy 96 shards, ~1.56 TB (1.42 TiB). Not 594 GB — that figure circulated widely in the pre-launch press and is simply wrong; you cannot store 2.8T parameters at 4 bits in less than ~1.4 TB, no matter how creative you get. And that is before the KV cache, which at 1M context becomes a serious consumer in its own right.

What that means in hardware:

• One node of 8× H100 80GB = 640 GB. Doesn't fit. Not even close.
• One node of 8× 192 GB accelerators = 1.536 TB. The model theoretically fits; nothing else does.
• Moonshot's recommendation: at least 64 accelerators (8 nodes × 8), with expert parallelism and serious inter-node interconnect.
• Someone reported booting K3 on 80× RTX 5090, getting 20 tok/s on a single stream, day one, untuned. That is a technical achievement, not a solution.

The officially recommended engines: vLLM, SGLang, TokenSpeed. The basic command looks mundane:

vllm serve moonshotai/Kimi-K3 \
  --tensor-parallel-size 8 \
  --max-model-len 131072 \
  --trust-remote-code

Practical tip if you actually get there: start at 128k–256k context, not 1M. The full window requires either a linear-attention kernel supported in your build, or enough GPUs to shard the KV cache across.

GGUF status: at the time of writing, there are no confirmed full-fidelity GGUF ports for llama.cpp / Ollama / LM Studio. They will come. But even a dynamic Q2 in the Unsloth style would remain in the hundreds-of-GB zone — it doesn't drop below the "server" threshold, only below the "datacenter" one.

Translation for a 1–5 person consulting firm

No. There is no scenario in which running K3 locally makes economic sense for us. A Mac Studio M3 Ultra, however well equipped, doesn't see this model even with binoculars. An RTX 4090 is a joke in this context.

What does make sense locally, if you want data sovereignty: Kimi K2.7 Code (1T parameters, 256k context, ~325 GB at dynamic Q2, ~605 GB at full precision) runs on a well-specced Mac Studio or a workstation with 256+ GB of unified memory. It is a good model. It is not K3, but it is a good model, and the difference between "good" and "frontier" matters less than you think for 80% of tasks.

The honest conclusion: "open weights" for K3 means sovereignty for companies with racks, not for developers with laptops. That distinction gets lost in the headlines.

4. Remote, instead of Claude Code: the real economics

This is where the discussion gets interesting, because it is the only practical option.

Pricing

K3 is at Sonnet level in list price. It is not the cheap Chinese model everyone expected after K2 — it is 3–4x more expensive than its own predecessor K2.6 ($0.95/$4).

The part that changes the math is the $0.30 cache hit — a 90% discount. In agentic loops, where you resend the same system prompt, the same files and the same repo context at every step, most of the input hits the cache. OpenRouter reports a ~92% cache hit rate on K3 traffic, which brings effective input to around ~$0.52/1M.

The counterpoint, and it is a serious one: K3 always thinks at maximum effort. Every reasoning token is billed at $15/1M. There is no "think less so I pay less" button. Blended (80/20 input/output) lands around $5.40/1M — versus ~$9 for Opus 4.8. Real, but not spectacular, and entirely dependent on how chatty the model is on your task.

Independent measurements by Artificial Analysis on the Moonshot endpoint: ~62 output tokens/second, ~2 seconds to first token. It is slower than you are used to. In a long agentic loop, per-task latency can cancel out the per-token price advantage.

Where your request actually runs

This is the question that matters for anyone working with US or EU clients under NDA.

• Moonshot's direct API (api.moonshot.ai) — infrastructure in China. For many contracts, an automatic non-starter, regardless of model quality.
• Fireworks / Baseten — US hosting, with Zero Data Retention options, certified by Moonshot. Identical pricing: $3/$15. Nobody undercuts, which is itself a clue about what the licensing agreements behind the scenes look like.
• Vercel AI Gateway — routes across US providers with failover, same model ID moonshotai/kimi-k3, plus a pricier and faster "Kimi K3 Fast" tier.
• OpenRouter — multiple providers, routing by price/speed/tool-calling accuracy.

If you have data residency clauses in your contract, the US-hosted option is not optional. And it is worth reading before, not after.

5. How to plug it into Claude Code

The good part: Claude Code doesn't check who it's talking to. It reads three environment variables — a URL, a token, a model name — and sends the request wherever they point. Moonshot exposes an Anthropic-compatible endpoint, so the harness stays identical: same hooks, same MCP servers, same skills, same habits.

The configuration (from Moonshot's official documentation)

Persistent, in ~/.claude/settings.json:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://api.moonshot.ai/anthropic",
    "ANTHROPIC_AUTH_TOKEN": "YOUR_MOONSHOT_API_KEY",
    "ANTHROPIC_MODEL": "kimi-k3[1m]",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "kimi-k3[1m]",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "kimi-k3[1m]",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "kimi-k3[1m]",
    "ANTHROPIC_DEFAULT_FABLE_MODEL": "kimi-k3[1m]",
    "CLAUDE_CODE_SUBAGENT_MODEL": "kimi-k3[1m]",
    "ENABLE_TOOL_SEARCH": "false",
    "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "1048576",
    "CLAUDE_CODE_EFFORT_LEVEL": "max"
  }
}

Or temporarily, for the current session only, with export on the same variables before running claude. Don't mix them — values in settings.json take priority over those exported in the terminal, and that is the number one reason "the configuration doesn't work".

Verification is done with /status, not /model. The /model menu is a fixed list of internal aliases and will never display Kimi models — it's not a bug, there is nothing to switch there. /status must show the Moonshot base URL and kimi-k3[1m].

The pitfalls, in the order you will hit them

1. A leftover ANTHROPIC_API_KEY in your shell. It conflicts with ANTHROPIC_AUTH_TOKEN and produces confusing 401 errors. Delete it.
2. Unset tier variables. If you only set ANTHROPIC_MODEL, background tasks (conversation titles, summarization) and sub-agents request Claude model names the Kimi endpoint doesn't recognize — and fail silently. Set them all.
3. ENABLE_TOOL_SEARCH must be false. The endpoint doesn't support the feature yet. Claude Code disables it by default anyway when the base URL isn't Anthropic, but explicit is better.
4. WebFetch doesn't work on the Kimi endpoint at the moment — it returns "temporarily unavailable" or empty content. Workaround: paste the content into the chat or use a scraping MCP.
5. CLAUDE_CODE_AUTO_COMPACT_WINDOW must match the model: 1048576 for K3, 262144 for K2.7 Code. Too small → it compacts prematurely and you lose context; too large → context length errors.
6. Preserved thinking history. K3 was trained in a mode that requires the assistant's complete message — including reasoning_content and tool_calls — to be sent back in messages, not just content. Claude Code handles it, but if you build your own harness or switch models mid-session, expect unstable output.
7. Web search is billed separately, ~$0.004 per call, on top of token pricing.

Alternatively, Moonshot has its own CLI — Kimi Code — which they say the model works best with, and which you can install in parallel. The difference shows in their own benchmarks: K3 scores 72.9 on Kimi Code Bench with their own harness and 73.7 with Claude Code, so the harness is not the deciding factor.

6. The license — read it

This is the part most articles skip.

K3 is not MIT and not Apache 2.0. It is a bespoke document, the "Kimi K3 License", labeled on Hugging Face as license:other. Artificial Analysis classified it as "Commercial Use Restricted".

The text is MIT-like for most of its length — you can download, run, modify, fine-tune, redistribute and use commercially with no license fee. Two clauses change things at scale:

• Model-as-a-Service: if you or your affiliates operate a business offering third parties access to inference or fine-tuning, and the group's aggregate revenue (not the revenue attributable to K3) exceeds $20M over any 12 consecutive months, you must sign a separate agreement with Moonshot before any commercial use.
• Attribution: any commercial product with more than 100 million monthly active users or more than $20M monthly revenue must display "Kimi K3" prominently in the interface.

Strictly internal use and access through Moonshot's products or their certified partners are exempt.

For a firm our size, both thresholds are theoretical. But if you are building a product you hope to sell, or if you resell inference, clause 2 is a contractual obligation, not a footnote. And the $20M group threshold is small enough that practically every serious inference provider falls under it — which explains why nobody undercuts Moonshot's pricing.

7. Verdict

Local: no, unless you have a cluster. For data sovereignty at small-firm scale, K2.7 Code on your own hardware is the real answer.

Remote, as a replacement for Claude Code: worth testing, with eyes open. The economics are real but smaller than the headlines suggest — ~40% below Opus on blended, conditional on a good cache hit rate, and partly paid for in latency, because the model always thinks at maximum. For client work under NDA, use a US-hosted provider with ZDR, not the direct endpoint.

As a second brain: this is where it gets most interesting to me. You already have the Claude Code subscription. The marginal cost of configuring a second profile that routes to K3 is five minutes and an API key. A second model, with an independent opinion on the same code, that excels exactly in the areas where it ranks first — frontend, design, very long agentic tasks — is not a replacement. It is a second opinion. And cc-switch or a simple alias in .zshrc makes switching instantaneous.

That, I think, is the real win of this week: not that we have a cheaper model, but that the harness has become portable. The tools we have built into habits — Claude Code, MCP, skills — are no longer tied to a single vendor. That is good news regardless of who you pick in the end.

A note, as usual: I believe AI is the future and that the frictions above — WebFetch not working, environment variables that must be set by hand in eight places, 1.5 TB of weights nobody can run at home — are growing pains that will get solved. The ecosystem has 18 months of real maturity. But as long as they are in front of us, they deserve honest discussion, because the infrastructure decisions we make now cost us real money in the coming months.

All data in this article was verified on July 28, 2026 against the official model card, the Moonshot platform documentation and independent reporting. In a launch week, things move daily — check the repo before any production decision.