Kimi K3 vLLM Go-Live Acceptance Checklist

Kimi K3 vLLM should go live only after the compatibility chain, stress-test memory headroom, real prefix-cache hits, and Agent API behavior all pass together. A running process or one successful request is not enough. If the version chain fails, rebuild the environment. If pressure remains during sustained tests, change the topology or add capacity.

This checklist is for engineers moving Kimi K3 vLLM from testing to production, SRE and platform owners signing a go-live decision, and technical leads comparing self-hosted capacity with temporary or remote delivery options.

Last updated August 13, 2026. Version and feature claims were checked against the official Kimi K3 vLLM recipe updated August 6, 2026, the vLLM Kimi K3 release post published July 27, 2026, and NVIDIA CUDA compatibility documentation.

The pass decision: four outcomes, not one green light

A useful Kimi K3 vLLM go-live decision has four possible outcomes:

• Go live: The container, vLLM build, CUDA runtime, host driver, memory behavior, prefix caching, and Agent interface all pass with saved evidence.
• Observe with limits: The core path works, but a non-critical feature has a known and bounded defect. Traffic must be rate-limited, the affected feature must be documented, and an owner must monitor it.
• Rebuild the environment: CUDA 13, the NVIDIA R580 driver family, the image, the vLLM build, or the container runtime does not match the supported chain.
• Expand or redesign capacity: The environment is compatible, but sustained load causes OOM, queue growth, communication stalls, or unstable processes.

The evidence package should include:

1. Container image and digest.
2. vLLM version and launch command.
3. Host driver output.
4. Container-visible CUDA and GPU information.
5. Startup logs.
6. Memory curves before and after any OOM.
7. Cold and warm prefix-cache results.
8. Agent tool-call test results.
9. Test sample, concurrency, context settings, and reviewer name.

The test sample must resemble production Agent traffic. A short prompt can make an unstable deployment look healthy. Shared system prompts, tool definitions, long conversation state, structured output, retries, and multiple concurrent requests must appear in the acceptance sample.

Compatibility first: CUDA 13 and R580 must agree across three layers

The official Kimi K3 recipe currently labels the integration as pre-release. It specifies the vllm/vllm-openai:kimi-k3 Docker image, a CUDA 13 build, vLLM 0.27.1 or newer, and an R580-or-newer NVIDIA host driver. The same recipe states that the image does not provide a CUDA 12.9 tag for this path. (recipes.vllm.ai)

NVIDIA’s compatibility matrix independently lists CUDA 13.x with a minimum driver branch of R580. A newer driver can provide backward compatibility, but an older R575 host should not be treated as equivalent merely because nvidia-smi displays a CUDA capability line. (docs.nvidia.com)

Verify the chain separately:

Container layer

• Record the exact image tag and digest.
• Confirm that the image is the Kimi K3 image, not a general CUDA image.
• Inspect the installed PyTorch and CUDA runtime versions inside the container.
• Save the vLLM package version and the model-serving command.

Host layer

• Record the actual NVIDIA driver version from the host.
• Verify that every node in a multi-node deployment uses the expected driver family.
• Check Fabric Manager and interconnect-related services where the topology requires them.
• Confirm that the visible GPUs match the intended hardware class and count.

Startup layer

• Save the first complete engine initialization log.
• Check for CUDA initialization errors, NCCL failures, missing kernels, unsupported architecture messages, and fallback paths.
• Confirm that the selected tensor, expert, or data parallel settings are the ones used during testing.
• Record whether the engine enabled the intended tool-call and reasoning parsers.

The acceptance rule is simple: the image, runtime, host driver, and startup path must describe one coherent environment. If the host has R580 but the container was built from an untracked custom branch, the deployment is not equivalent to the official recipe.

For teams building against another CUDA environment, keep a separate evidence track. Record the source branch, commit, dependency lockfile, build command, generated image digest, and rollback image. Do not mix custom-build results with official-image results. That makes later incident review much harder.

The official recipe lists a minimum of eight GB300 GPUs for the NVIDIA path and recommends multi-node deployment for real production traffic. The vLLM release post also describes an eight-GPU quick-start path and reports benchmark results on 16 NVIDIA GB300 GPUs. These are deployment references, not universal memory guarantees for every request pattern. (recipes.vllm.ai)

What must be checked after the server starts?

A successful process start proves only that the basic initialization path completed. It does not prove that the server can survive production traffic.

The next checks should cover:

• One standard text request.
• One long-context request.
• One multimodal request if the application uses vision.
• One request with tools and structured output.
• Several repeated requests with the same system prompt.
• Multiple concurrent Agent sessions.
• A sustained run long enough to expose memory growth and queue behavior.

Save the request body, response body, HTTP status, timing data, and server log reference for every test class. If the gateway rewrites messages, tool schemas, or streaming fields, save the payload before and after the gateway. Otherwise, a protocol defect may be incorrectly blamed on the model runtime.

The vLLM release documentation describes Kimi K3 as a 2.8-trillion-parameter MoE model with 16 of 896 experts active per token and a context window of up to 1,048,576 tokens. Those figures explain why a short smoke test is a weak acceptance signal: model size, parallel execution, recurrent state, full-attention cache, and request length all affect the real capacity boundary. (recipes.vllm.ai)

Capacity stability: separate startup OOM from workload OOM

Kimi K3 deployment OOM events should not be treated as one generic failure. The cause determines whether the next action is a configuration change, an environment rebuild, or capacity expansion.

Run the pressure test in distinct phases.

Phase one: model-load stability

Start the server with the production image and intended parallel configuration. Record:

• Memory usage during weight loading.
• Peak allocation during engine initialization.
• GPU visibility and rank assignment.
• Initialization duration.
• Any allocator, NCCL, or kernel fallback messages.

If the process fails before serving a request, the likely decision is environment rebuild or topology correction, not traffic tuning. A model-load failure means the deployment has not reached an application-level capacity test.

Phase two: long-input behavior

Use fixed input samples with increasing context lengths. Keep the generation limit and sampling settings documented. Record the memory curve and whether the scheduler rejects requests, queues them, or terminates the process.

This phase identifies excessive max-model-len, unsuitable cache settings, and topology limits. Do not copy a memory threshold from another GPU cluster. The same model can behave differently across GPU memory sizes, parallel layouts, interconnects, quantization paths, and cache policies.

Phase three: concurrent Agent traffic

Use the production request shape:

• Repeated system instructions.
• Tool schemas.
• Tool results.
• Multi-turn state.
• Structured output requirements.
• Retries and timeout behavior.
• A mix of short and long requests.

Observe weight memory, KV or hybrid cache memory, request queue time, running-request count, completed-request rate, and process restarts. Capture the request parameters at the moment of any OOM.

Phase four: sustained operation

A deployment that survives a short burst can still fail after cache pressure accumulates. Run a sustained test with a fixed concurrency schedule and a known request mix. Track whether latency rises while throughput remains flat, whether cached blocks are evicted excessively, and whether the process recovers after rejected requests.

Do not approve a deployment from the final error line alone. The OOM record must include the preceding memory curve, request shape, concurrency, context configuration, and topology.

The result should map to one of three actions:

• Adjust configuration: The environment remains stable, but context or concurrency is clearly above the intended service envelope.
• Limit traffic: The service is stable only under a lower, documented concurrency ceiling.
• Expand capacity: OOM or queue collapse continues after reasonable context and concurrency controls.

For a broader planning method, the large-model capacity planning guide can be used to document memory assumptions, network constraints, and fallback capacity before requesting additional resources.

Prefix caching: prove reuse instead of checking a flag

The Kimi K3 vLLM release post shows --enable-prefix-caching in the serving command. The official recipe also explains that Kimi K3 uses hybrid caching across recurrent KDA state and full-attention cache. This means a visible launch flag is only the first check. It does not prove that production prompts are being reused. (vllm.ai)

To verify prefix caching:

1. Prepare a fixed shared system prompt.
2. Reuse the same tool definitions.
3. Send a cold request and save its timing and token counts.
4. Send a second request with the same prefix but a controlled user-message change.
5. Repeat the warm request several times.
6. Send a control request with a changed system prompt or tool schema.
7. Compare cache queries, cache hits, cached prompt tokens, and request latency.
8. Save the server metrics snapshot for each test window.

The vLLM metrics documentation exposes prefix-cache counters such as queries, hits, and cached prompt tokens. Those counters should be read together with latency and input-token data rather than interpreted in isolation. A hit counter without a meaningful change in cached prompt tokens may indicate that the test is too small, the prefix is not long enough, or the request structure is not the one expected by the cache manager. The official vLLM metrics documentation should be kept with the acceptance record.

A valid result needs both:

• Cold evidence: The first request pays the full prefix processing cost.
• Warm evidence: Repeated requests reuse a stable shared prefix and show corresponding cache metrics.

Control the differences carefully. Changing whitespace, tool ordering, system-message content, serialization, or multimodal metadata can invalidate the shared prefix. A test that changes several fields at once cannot explain why caching appears ineffective.

Kimi K3 adds another boundary. Its recurrent state cannot be retained at every token position without consuming substantial cache capacity. The vLLM release post describes prompt-end retention, interval-based retention, and selective retention policies for recurring prefixes. The deployment team should therefore document the retention policy instead of assuming that every partial prefix will be cached. (vllm.ai)

If the flag is present but warm requests show no reliable reuse, classify the result as cache not accepted. Check prompt identity, retention settings, metrics scraping, and the selected vLLM build before changing GPU capacity.

Agent correctness: compare model output with protocol output

Kimi K3 Agent acceptance must test more than readable text. The API consumer may depend on reasoning fields, structured output, tool_calls, argument JSON, streaming chunks, and retry behavior.

Use a fixed schema set and run each case repeatedly:

• A tool with one required argument.
• A tool with nested objects or arrays.
• A tool call followed by a tool result.
• A response that should not call a tool.
• Invalid or incomplete tool arguments.
• Structured output with schema validation.
• A timeout followed by a retry.
• A multi-turn request carrying previous tool results.

The official Kimi K3 recipe warns that the model can occasionally emit a tool-call format that its parser does not expect and recommends schema validation and retry handling. That warning makes one successful demonstration insufficient for production approval. (recipes.vllm.ai)

Separate the defect by layer:

• If the raw model response is malformed, investigate the model parser, prompt format, or vLLM integration.
• If the raw response is correct but the gateway changes it, investigate the gateway.
• If the gateway is correct but the client rejects it, investigate the client schema or SDK.
• If retries create duplicate tool calls, inspect idempotency and request identifiers.

A production pass requires repeatable schema-valid output under the intended sampling settings. A known, bounded parser defect may justify limited observation for a non-critical feature, but it should not be hidden inside a general “API passed” statement.

The sign-off checklist for a go-live decision

Use this checklist as the final review record:

• [ ] The Kimi K3 image tag and digest are recorded.
• [ ] The vLLM version and source channel are recorded.
• [ ] CUDA 13 is confirmed inside the container.
• [ ] Every host meets the R580-or-newer driver requirement.
• [ ] Host, container, and startup logs agree on the runtime chain.
• [ ] GPU count, topology, parallel settings, and interconnect settings are recorded.
• [ ] Model-load memory behavior is stable.
• [ ] Long-input tests have saved memory curves and request parameters.
• [ ] Concurrent Agent tests have documented queue and latency behavior.
• [ ] Sustained testing has no unexplained process termination.
• [ ] Cold prefix-cache results are saved.
• [ ] Warm prefix-cache results show cache metrics and controlled reuse.
• [ ] Tool schemas pass repeated validation.
• [ ] Reasoning, content, and tool_calls fields match the client contract.
• [ ] Retry and failure behavior is documented.
• [ ] Every failed item has an owner, deadline, and rollback action.
• [ ] A reviewer has signed the final go-live outcome.

The decision logic should remain explicit:

• Version or image mismatch: rebuild.
• Startup OOM: correct the environment or topology.
• Sustained OOM after controlled limits: expand capacity or redesign the deployment.
• Prefix cache flag present but no measured reuse: fix the cache test or runtime before approval.
• Non-critical, bounded Agent defect: observe with limits.
• Repeated protocol or parser failure on a critical path: do not go live.

The ProxyMac console can be used as part of an operational handoff when the team needs to keep environment details, access ownership, and delivery notes aligned with the acceptance record.

What to monitor after approval

The first production watch window should include:

• Engine process health.
• GPU memory and allocation failures.
• Running and waiting requests.
• Input and output token rates.
• Prefix-cache queries, hits, and cached prompt tokens.
• Request error codes.
• Tool-call parse failures.
• Retry counts.
• Inter-node communication errors.
• Container restarts and node-level driver errors.

Rollback should trigger when the service repeatedly exceeds the accepted queue envelope, produces schema-invalid tool calls on a critical workflow, shows unexplained cache regression after a deployment change, or experiences OOM under the approved traffic profile.

The rollback artifact should be the previously accepted image and launch configuration, not an untracked emergency rebuild. Keep the last accepted container digest, runtime variables, parallel settings, and parser options together.

A self-managed Kimi K3 cluster offers direct control, but it also leaves the team responsible for R580 upgrades, CUDA 13 image consistency, interconnect topology, cache metrics, and sustained-capacity failures. A general-purpose cloud instance can add another layer of variability through GPU availability, networking, storage, and image lifecycle controls. For a short validation window or a temporary production expansion, renting a prepared Mac environment through ProxyMac may provide a simpler delivery path than rebuilding a fragile environment under deadline pressure. It is still not the right fit for every long-term, heavy-load deployment or for workloads requiring dedicated physical interfaces. The sensible next step is to bring the current driver version, container details, Agent samples, and pressure-test records into the ProxyMac support workflow, then decide whether the evidence points to rebuilding, adding capacity, or changing topology.