Model ≠ harness
A K3 API call, a Fireworks agent loop, and a local MCP agent can produce different results even with the same weights.
Kimi K3 benchmark · coding agent evaluation
Kimi K3 benchmark results, costs, and an evaluation plan
A maintained reading guide for Kimi K3 coding-agent benchmarks: separate the model from the harness, pass rate from cost, and vendor claims from independent measurements. This is an independent guide, not a Moonshot/Kimi official page.
Release update: Kimi's official overview lists K3's July 16, 2026 release and July 27 full-weight availability. The open-weight repository now documents a 2.8T MoE model, 104B activated parameters, MXFP4 weights, MXFP8 activations, and 1,048,576-token context. Treat local serving as a cluster/inference-engine project, not a laptop install promise.
Research status: Semrush US Volume/KD/CPC/Intent and variants were not obtained in this round. Google Trends US past-7-day comparison supplied for this brief shows Kimi K3 around 31 relative mean versus opencode around 8; interest continued after July 16, but this page does not call it a confirmed Breakout.
Last reviewed: 2026-07-28. The search snapshot is an estimate for trend direction, not raw search volume. Similarweb project insight also suggested nxcode.io added Kimi K3 benchmark content and organic terms; that observation may be delayed or estimated.
Evidence table
Compare measurements without flattening them
K3, GPT, Claude, and Fable appear here as named measurement points. A score is only meaningful with its harness, benchmark, and evidence type attached.
| Model | Harness / benchmark | Metric / value | Pass rate | Cost | Latency / effort | Evidence and caveat |
|---|---|---|---|---|---|---|
| Kimi K3Coding agent | Fireworks agent harnessSWE-style repository tasks | Task solve rate92.4% | 92.4% task solve | Provider study says lower across this mix; up to 50× claim for routing | Not normalized; about 55 turns / 1.3M tokens on SWE | Fireworks study Provider study · 2026-07-28Same-harness comparison, not a universal model ranking. |
| Kimi K3Agentic knowledge | Artificial Analysis evaluation harnessAA-Briefcase | Elo1543 | Not a pass rate | Article says more expensive than Opus 4.8 to run | Nearly one hour per task on average | Artificial Analysis Independent · 2026-07-28Private realistic tasks and Elo measure knowledge work, not repo repair. |
| Kimi K3Model profile | Moonshot API / official model docsModel capability profile | Context / tool capability1M context; tools documented | Not reported | See official per-token pricing | Measure on your provider and region | Kimi K3 API docs Vendor-reported · 2026-07-28Official capability claims describe the model, not an independent evaluation. |
| Claude Fable 5Coding agent | Fireworks agent harnessSWE-style repository tasks | Task solve rate92.6% | 92.6% task solve | Reference point for the provider's routing analysis | About 21 turns / 130K tokens on SWE | Fireworks study Provider study · 2026-07-28Task mix and harness shape the result; do not transfer it to chat quality. |
| Claude Fable 5Agentic knowledge | Artificial Analysis evaluation harnessAA-Briefcase | Elo1574 | Not a pass rate | Comparator in the same article | Not reported in the page snapshot used here | Artificial Analysis Independent · 2026-07-28Second place here does not imply second place for coding agents. |
| GPT-5.6 SolModel profile | OpenLM comparison boardBoard snapshot | ListingListed; score not copied | Not comparable here | Provider and route dependent | Provider and route dependent | OpenLM board Board snapshot · 2026-07-28A board listing is a discovery signal, not a same-harness result. |
| Claude Opus 4.8Model profile | OpenLM comparison boardBoard snapshot | ListingListed; score not copied | Not comparable here | Provider and route dependent | Provider and route dependent | OpenLM board Board snapshot · 2026-07-28Keep this row separate from K3's Fireworks and AA measurements. |
| Kimi K2.7 / frontier comparatorsEvaluation method | Pydantic AI or vendor agent SDKsGrounded security benchmark methodology | Precision / recall / F1Method reference; not a K3 score | Not a K3 score | Cost per scan / true positive | Repeat runs to measure stability | Semgrep methodology Method guide · 2026-07-28Evaluation design reference, not evidence about K3 itself. |
How to read the table
Three distinctions prevent benchmark bait
Benchmark ≠ product
SWE-style repair, AA-Briefcase knowledge work, frontend Arena, and a security benchmark answer different questions.
Pass rate ≠ cost
Record tokens, cache hits, turns, retries, and wall-clock time. A lower rate can still win a narrow workload; a headline score does not price a task.
Kimi K3 pricing and limits
Use API rates as a cost input, not a quality claim
Moonshot's current Kimi API documentation describes K3 as a 1M-token-context model, always thinking, with configurable reasoning_effort and automatic context caching. The pricing page is the source of truth for cache-hit, cache-miss, and output rates; provider routes and subscriptions should not be mixed into one number.
$0.30 cache-hit input · $3 cache-miss input · $15 output per 1M tokens Check taxes, account tier, route, and the official rate card before budgeting.
Kimi K3 API, Kimi Code, or local weights?
These are three different operating surfaces
Hosted `kimi-k3` access with OpenAI/Anthropic-compatible requests, per-token billing, 1M context, and provider-side quota. Best for a fast baseline and controlled cost traces.
API quickstartA coding-agent harness that can select K3 and own the terminal workflow. Its task success, tool permissions, retries, and context policy must be benchmarked separately from the model.
Official Kimi Agent overviewFull weights are available in the official repository. The README recommends vLLM, SGLang, or TokenSpeed; MXFP4 and a 1M context do not imply a single-GPU or consumer-hardware minimum. Size GPU memory, interconnect, KV cache, throughput, and concurrency together.
Official Kimi K3 repositoryKimi K3 cost estimator
Model cache hits, retries, and long context before you budget
Enter workload assumptions to estimate Kimi K3 API spend. The calculator uses the reviewed K3 rate card: $0.30/M cache-hit input, $3/M cache-miss input, and $15/M output. Retries are counted as additional calls.
Illustrative baseline for difference onlyThese editable rates are not a live GPT/Claude quote.
Estimated monthly$60.541,000 billed calls
Per original request$0.06including retry assumption
Difference vs baseline−$61.96baseline monthly: $122.50
Not a quote: actual invoices can vary with provider route, account tier, rounding, cached prefix shape, tool-call output, taxes, retries, and long-context behavior. Verify the official rate card before making a purchasing decision. The long-context multiplier is a planning assumption, not a Kimi promise.
Prompts and workload scenarios
Evaluate K3 where your agent actually works
Measure tool-call validity, retries, test-backed task solve, tokens, and wall-clock latency with the same harness.
Prompt starter: Inspect this repository, propose a patch, run the tests, and explain every changed file.Use fixed screenshots, accessibility checks, responsive breakpoints, and a human rubric; visual polish is not a repository pass rate.
Prompt starter: Build the supplied UI brief without changing the acceptance criteria.Track typecheck, test coverage, diff size, regression rate, and rollback quality across a pinned fixture.
Prompt starter: Refactor only the named module and preserve its public API.Evaluate permission boundaries, network egress, secrets handling, approval pauses, tool schemas, and audit logs alongside task success.
Prompt starter: Use allowlisted tools only; stop and request approval before writes or network calls.Kimi K3 + MCP
Evaluate the boundary around the model
Kimi's API supports tool calls and dynamic tool loading. MCP is an agent integration and permission layer, not a benchmark score. If the harness can read files, execute commands, browse, or call a proxy, record those capabilities and approval rules beside the result.
model: kimi-k3 reasoning_effort: high tools: - read_repo - run_tests - mcp_proxy (allowlist only)
- Pin tool schemas and tool-result fixtures.
- Deny secrets, unrestricted egress, and destructive writes by default.
- Log tool calls, retries, tokens, latency, and human overrides.
Coding-agent evaluation · MCP safety checklist
Copy a repeatable K3 test template
Evaluation template
[ ] Pin the exact model ID, reasoning effort, provider, and date. [ ] Keep the agent harness, tool definitions, system prompt, and retry policy identical. [ ] Use a held-out task set with reproducible repositories, dependencies, and fixtures. [ ] Record pass rate, partial credit, test evidence, and human adjudication separately. [ ] Log input/output tokens, cache hits, turns, wall-clock latency, and failures. [ ] Run repeated trials to expose nondeterminism, timeout, and flaky tool-call behavior. [ ] Check security boundaries: secrets, network egress, MCP tools, and write permissions. [ ] Publish raw prompts, harness version, exclusions, and uncertainty with the conclusion.
Use the output as a test plan. It is not a certification that K3, GPT, Claude, or any agent is safe or best.
Sources and update method
What is current, what is estimated, and what to re-check
Page review: 2026-07-28. Search trend values are the supplied US 7-day relative snapshot. Model, API, benchmark, and provider pages can change.
Artificial Analysis and Semgrep use their own evaluation designs. Fireworks' K3/Fable result is a provider study. Moonshot's capability and pricing claims are vendor documentation.
Do not run an agent benchmark against production secrets or unrestricted network access. Pin prompts, harness versions, dependencies, and task fixtures.
Research linksOfficial Kimi Agent overview · release and weights timeline Official MoonshotAI Kimi K3 repository · weights and deployment Moonshot AI model overview Kimi K3 API and capability docs Kimi K3 API pricing NxCode evaluation guide Artificial Analysis AA-Briefcase Fireworks same-harness study OpenLM model board Semgrep benchmark methodology Reddit r/LocalLLaMA discussion Reddit Kimi discussion YouTube K3 benchmark / MCP search
Continue the engineering workflow
Connect benchmark results to implementation guardrails
AI code review toolsTurn agent output into a human merge gate.Claude Max vs ChatGPT ProCompare subscription workflows and usage limits.Dependency vulnerability scannerCheck generated manifests without pretending to scan CVEs.MCP proxyInspect tool permissions and network boundaries.Agent observability toolsMeasure turns, traces, cost, and failures.Agent skills directoryReview the skills and instructions surrounding a coding agent.Free AI tool directoryBrowse practical generators and coding helpers.
FAQ
Questions to answer before calling a benchmark result
What is the Kimi K3 benchmark?
Kimi K3 benchmark is not one test. This page compares coding-agent, agentic knowledge, model-board, and evaluation-method evidence while keeping each harness and score attached to its source.
Is Kimi K3 the best coding model?
No absolute conclusion is justified from the available snapshots. K3 is competitive on some coding and agentic measurements, but task mix, harness, cost, latency, and provider behavior can change the result.
How does Kimi K3 compare with Fable, GPT, or Claude?
Fireworks reports a near-tie between K3 and Claude Fable 5 on its same-harness task mix, while Artificial Analysis reports an AA-Briefcase Elo comparison. GPT and Claude board rows are kept separate when the same benchmark and harness are not available.
How much does Kimi K3 cost?
The official Kimi API pricing page should be checked before budgeting. The reviewed snapshot shows separate cache-hit input, cache-miss input, and output rates; Kimi subscriptions, Moonshot API, Fireworks, OpenRouter, and self-hosting are different cost surfaces.
Can I use Kimi K3 with MCP?
Kimi's API documents tool calls and dynamic tool loading. MCP support depends on the agent or proxy around the model, so evaluate permissions, tool schemas, network egress, logs, and approvals as part of the harness.
Are the Kimi K3 search figures official?
No. Semrush figures were not obtained. The page records a supplied US 7-day Google Trends relative snapshot of about 31 for Kimi K3 versus about 8 for opencode, plus sustained interest after July 16, without calling it a confirmed Breakout.