How to Use the Kimi K3 API: Setup, Cache Optimization, and Code Examples

Kimi K3 is available through OpenRouter and Moonshot’s direct API. This guide covers both access methods, shows you how to optimize for cache hits (turning $3/M input into $0.30/M), and provides working code examples for common coding workflows.

Quick Start: OpenRouter (Recommended)

OpenRouter is the fastest path to K3 for most developers. It uses the OpenAI-compatible format, so existing code likely needs only a model name change.

Step 1: Get an API Key

1. Sign up at openrouter.ai
2. Add credits to your account
3. Generate an API key from the dashboard

Step 2: Make Your First Request

import openai

client = openai.OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="your-openrouter-key"
)

response = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=[
        {"role": "system", "content": "You are a senior software engineer."},
        {"role": "user", "content": "Write a Python function to merge two sorted lists."}
    ]
)

print(response.choices[0].message.content)

Step 3: Verify It Works

The response should contain a clean merge function. K3 is ranked #1 on Frontend Code Arena and scores 88.3% on Terminal-Bench, so code quality should be immediately noticeable.

Direct Moonshot API Access

For teams wanting to go directly through Moonshot:

import requests

url = "https://api.moonshot.cn/v1/chat/completions"
headers = {
    "Authorization": "Bearer your-moonshot-api-key",
    "Content-Type": "application/json"
}

payload = {
    "model": "kimi-k3",
    "messages": [
        {"role": "system", "content": "You are a senior software engineer."},
        {"role": "user", "content": "Explain how to implement a red-black tree in Rust."}
    ],
    "temperature": 0.7
}

response = requests.post(url, headers=headers, json=payload)
print(response.json()["choices"][0]["message"]["content"])

Cache Optimization: Getting 90%+ Hit Rates

K3’s prefix caching charges $0.30/M for cached tokens vs $3/M for fresh tokens. The difference is 10x. Here is how to maximize cache hits.

Understanding Prefix Caching

The cache works on prefixes. If your current request starts with the same tokens as a previous request, those matching tokens are served from cache. The cache matches from the beginning of the input, character by character, until it finds a difference.

Request 1: [System Prompt][File A][File B][User: "fix the bug"]
Request 2: [System Prompt][File A][File B][User: "add a test"]
                          ^ all of this is cached ^

In Request 2, everything before the new user message matches Request 1’s prefix, so it is served from cache.

Strategy 1: Static System Prompts

Never put dynamic content (timestamps, random IDs, changing context) in your system prompt. Keep it identical across all requests.

# Good: Static system prompt
SYSTEM_PROMPT = """You are a senior software engineer specializing in Python.
Follow these rules:
- Write clean, well-documented code
- Include type hints
- Handle edge cases
- Write unit tests when asked"""

# Bad: Dynamic content breaks cache
SYSTEM_PROMPT = f"""You are a senior software engineer.
Current time: {datetime.now()}  # This breaks the cache every second!
Session ID: {uuid4()}  # This breaks the cache every request!
"""

Strategy 2: Front-Load File Context

Put your codebase files immediately after the system prompt, before conversation history. This way the file context (which rarely changes) is part of the cacheable prefix.

def build_messages(system_prompt, files, conversation_history, new_message):
    messages = [
        {"role": "system", "content": system_prompt},
        # Files as a user message early in the conversation
        {"role": "user", "content": f"Here are the relevant files:\n\n{files}"},
        {"role": "assistant", "content": "I've reviewed the files. What would you like me to help with?"},
    ]
    # Add conversation history
    messages.extend(conversation_history)
    # Add new message
    messages.append({"role": "user", "content": new_message})
    return messages

Strategy 3: Consistent File Ordering

Always include files in the same order. If you load main.py, utils.py, config.py, keep that order every time. Reordering breaks the prefix match.

# Sort files deterministically
def load_context_files(file_paths):
    sorted_paths = sorted(file_paths)  # Always same order
    content = ""
    for path in sorted_paths:
        with open(path) as f:
            content += f"### {path}\n```\n{f.read()}\n```\n\n"
    return content

Strategy 4: Batch Follow-Up Questions

Instead of starting a new conversation for each question, continue the existing one. Each subsequent message benefits from a longer cached prefix.

conversation = []

def ask(question):
    conversation.append({"role": "user", "content": question})
    response = client.chat.completions.create(
        model="moonshotai/kimi-k3",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": CONTEXT_FILES},
            {"role": "assistant", "content": "Ready to help."},
            *conversation
        ]
    )
    answer = response.choices[0].message.content
    conversation.append({"role": "assistant", "content": answer})
    return answer

# Each subsequent call has a longer cached prefix
ask("What does the authenticate() function do?")
ask("Can you add rate limiting to it?")
ask("Write tests for the rate limiting logic.")

Working with Long Context (1M Tokens)

K3 supports 1 million tokens of context. Here is how to load a substantial codebase:

import os

def load_codebase(root_dir, extensions=[".py", ".ts", ".js"]):
    """Load all relevant files from a directory into a single context string."""
    files_content = []
    total_chars = 0
    max_chars = 3_000_000  # ~750K tokens, leaving room for conversation

    for dirpath, dirnames, filenames in os.walk(root_dir):
        # Skip common non-essential directories
        dirnames[:] = [d for d in dirnames if d not in [
            "node_modules", ".git", "__pycache__", "dist", "build"
        ]]
        for filename in sorted(filenames):
            if any(filename.endswith(ext) for ext in extensions):
                filepath = os.path.join(dirpath, filename)
                relative_path = os.path.relpath(filepath, root_dir)
                with open(filepath) as f:
                    content = f.read()
                    if total_chars + len(content) > max_chars:
                        break
                    files_content.append(f"### {relative_path}\n```\n{content}\n```")
                    total_chars += len(content)

    return "\n\n".join(files_content)

codebase = load_codebase("./src")

Vision: Sending Images

K3 has native vision support. Use it to debug UI issues from screenshots:

import base64

def encode_image(image_path):
    with open(image_path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=[
        {"role": "system", "content": "You are a frontend developer. Fix UI bugs based on screenshots."},
        {"role": "user", "content": [
            {"type": "text", "text": "This button is misaligned. Here's the screenshot and the CSS:"},
            {"type": "image_url", "image_url": {
                "url": f"data:image/png;base64,{encode_image('bug-screenshot.png')}"
            }},
            {"type": "text", "text": "```css\n.submit-btn { margin: 10px; float: left; }\n```"}
        ]}
    ]
)

Streaming Responses

For interactive applications, stream responses:

stream = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=[
        {"role": "system", "content": "You are a coding assistant."},
        {"role": "user", "content": "Write a WebSocket server in Node.js"}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Error Handling and Retries

import time
from openai import RateLimitError, APIError

def robust_request(messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(
                model="moonshotai/kimi-k3",
                messages=messages,
                temperature=0.7
            )
            return response.choices[0].message.content
        except RateLimitError:
            wait_time = 2 ** attempt
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)
        except APIError as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(1)
    return None

Cost Monitoring

Track your spending by monitoring token usage:

def tracked_request(messages):
    response = client.chat.completions.create(
        model="moonshotai/kimi-k3",
        messages=messages
    )

    usage = response.usage
    input_cost = (usage.prompt_tokens / 1_000_000) * 3.0
    output_cost = (usage.completion_tokens / 1_000_000) * 15.0
    # Note: cached tokens would be cheaper, but API may not distinguish in usage stats
    total_cost = input_cost + output_cost

    print(f"Tokens - Input: {usage.prompt_tokens}, Output: {usage.completion_tokens}")
    print(f"Cost (max estimate): ${total_cost:.4f}")

    return response.choices[0].message.content

Integration with Development Tools

K3 works with Aider and other coding tools through OpenRouter. For complete setup instructions with Aider and Claude Code, see our dedicated K3 Aider and Claude Code setup guide.

For the broader context of available AI coding tools, K3 fits anywhere that accepts OpenAI-compatible APIs.

FAQ

Do I need a separate account for Moonshot’s API vs OpenRouter?

Yes. OpenRouter is a third-party aggregator that provides access to K3 alongside many other models. Moonshot’s direct API is a separate service. Both offer the same K3 model at the same pricing. OpenRouter is easier for most developers since it uses the standard OpenAI format.

How do I know if my tokens are hitting cache?

Some API responses include cache statistics in the response metadata. Check the response headers or usage object for cache hit information. If this is not exposed, you can estimate based on prompt structure: if your prompt shares a long prefix with recent requests, most tokens are likely cached.

What temperature should I use for coding tasks?

For code generation, use 0.0 to 0.3 for maximum determinism. For creative tasks or brainstorming approaches, 0.7 to 1.0. For most coding workflows, 0.2 is a good default that provides consistent output while allowing minor variations.

Is there a rate limit on K3?

Rate limits depend on your account tier (both on OpenRouter and Moonshot directly). Check your account dashboard for current limits. For high-volume usage, contact the provider to increase limits.

Can I use K3 with function calling / tool use?

Yes, K3 supports function calling through the standard OpenAI-compatible format. This makes it compatible with agentic frameworks that rely on tool use. For models specifically optimized for agentic work, also consider Muse Spark 1.1 which has native subagent orchestration.