An AI job times out after the model produced its result but before the client received confirmation. The client retries. Without idempotency, the application may generate twice, charge twice, send two messages, or execute the same agent tool twice.

Idempotency means repeated attempts for the same intended operation converge on one recorded operation and outcome.

Where AI workflows need it

• starting an image, video, or document-generation job;
• creating an agent run;
• sending an agent-authored message;
• writing extracted data to a system of record;
• charging for a generation or purchasing credits;
• processing a webhook event;
• executing a consequential tool call;
• retrying work after a worker crash.

Read and search operations are usually safe to repeat. Side effects require an explicit policy.

Idempotency key contract

The caller creates a unique key for one logical operation and reuses it only when retrying that operation.

POST /v1/agent-runs
Idempotency-Key: 01J60Y8M5X7M4TQW9Y3N6M2K8P
Content-Type: application/json

The server stores:

• tenant or caller identity;
• idempotency key;
• request fingerprint;
• operation status;
• resource or job ID;
• final response reference;
• creation and expiry timestamps.

Scope the key to the authenticated tenant. Two customers may coincidentally send the same value and must not share a result.

State machine

received → running → succeeded
                  ↘ failed_retryable
                  ↘ failed_final

When the same key arrives:

• same fingerprint + running: return the existing job/status;
• same fingerprint + succeeded: return the recorded result reference;
• different fingerprint: reject the key reuse;
• retryable failure: resume according to explicit policy;
• final failure: return the recorded failure unless a new operation is requested.

This contract is stronger than merely caching an HTTP response.

Request fingerprints prevent accidental reuse

Hash a canonical representation of fields that define the operation. Exclude volatile transport metadata, but include inputs that change the intended work: tenant, task, source document, model policy, tool target, or requested action.

Do not store raw sensitive prompts solely for idempotency if a keyed hash or durable input reference is sufficient.

Atomic claim before work

Only one worker should claim a new key. Use a unique database constraint and transaction rather than “check, then insert,” which races under concurrency.

CREATE TABLE idempotency_records (
  tenant_id text NOT NULL,
  key text NOT NULL,
  request_hash text NOT NULL,
  status text NOT NULL,
  job_id text,
  response_ref text,
  created_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (tenant_id, key)
);

The winner creates the job. Other attempts read the existing record.

Worker leases and crash recovery

An operation can remain running after a worker dies. Add a lease or heartbeat so another worker can recover abandoned work.

Recovery does not mean every step can safely repeat. Split a long agent workflow into durable steps and record completion before advancing. External tools may require their own idempotency key or reconciliation logic.

Model generation versus side effects

Calling a model twice may waste money or produce different text, but it is not the same risk as sending two emails or issuing two refunds.

Separate:

1. generation attempt;
2. validated proposed action;
3. authorized side effect;
4. confirmed result.

An agent retry may regenerate a plan, but it must not repeat an already confirmed side effect. Record tool-call identifiers and business operation IDs outside the model conversation.

Webhook deduplication

Webhook providers may deliver an event more than once. Store the provider’s stable event ID before executing its effect. If the same event returns, acknowledge it without repeating the action.

Your own outgoing webhooks also need stable event IDs across delivery attempts. See Webhook Architecture for AI Workflows.

Retries and idempotency are separate

Retries decide when to attempt again. Idempotency decides whether a repeated attempt represents new work. You normally need both.

Only retry transient failures, use backoff and jitter, and keep a retry budget. Connect that policy through Handling AI API Failures.

Expiry and retention

Keep an idempotency record long enough to cover realistic client, queue, and webhook retries. Expiring too early can turn a delayed retry into duplicate work.

Retention depends on impact:

• low-value generation preview: shorter window may be acceptable;
• billing or external message: longer audit and reconciliation window;
• regulated or consequential action: follow domain record requirements.

Do not retain sensitive model content merely because the idempotency record remains. Store references and minimal outcome metadata where possible.

Agent tool policy

For each tool, declare whether it is:

• read-only and repeatable;
• idempotent by resource state;
• idempotent with a supplied operation key;
• non-idempotent and confirmation-gated;
• unsafe to retry automatically.

Examples:

Tool action	Safe strategy
Set ticket label to urgent	State-based idempotency
Create issue	Idempotency key mapped to created issue
Send email	Stable message operation ID and provider reconciliation
Increment credit	Transactional ledger entry, never blind replay
Delete resource	Stable resource identity plus authorization and audit

Observability

Track:

• duplicate attempts suppressed;
• conflicting key reuse;
• operations stuck in running state;
• recovered leases;
• replayed tool calls;
• idempotency record latency and failures;
• cost avoided through generation reuse.

Production checklist

• Require idempotency keys for consequential create/action endpoints.
• Scope keys by authenticated tenant.
• Reject the same key with different input.
• Claim keys atomically.
• Persist job and side-effect state outside model context.
• Use worker leases for crash recovery.
• Separate generation retries from tool side effects.
• Deduplicate incoming events and stabilize outgoing event IDs.
• Retain records for the complete retry horizon.
• Test concurrent duplicates and crashes between every important step.

Idempotency is a core part of AI Application Architecture. Combine it with Authentication for AI Applications, rate limiting, and an AI gateway to keep retries, identity, cost, and side effects under one reliable policy.