Customer-managed storage is an architecture requirement, not a checkbox next to a cloud region. A platform may let you export traces to your bucket while keeping the primary trace database, prompts and indexes in its own account.
My take: shortlist Langfuse and Phoenix when you want a documented self-hosted starting point. Evaluate Braintrust’s self-hosted data plane or LangSmith’s licensed deployment options when their workflow benefits justify additional commercial and architectural diligence. Do not buy a BYOC promise until every data store and outbound connection is mapped. This is a documentation review, not a deployment test.
Start with four different deployment promises
| Promise | What it can mean | What it does not establish |
|---|---|---|
| Regional SaaS | Vendor operates in a selected region | Customer owns the account or storage keys |
| Export to customer storage | Copies or archives leave the platform | Primary traces are deleted from vendor storage |
| Customer-managed data plane | Selected processing and storage run in your environment | Control plane, telemetry and support never leave it |
| Full self-hosting | You operate the application stack | Automatic security, compliant configuration or zero egress |
Treat each of these as a different procurement option. The question is where inputs, outputs, metadata, evaluations and backups reside throughout their lifecycle, not merely where the largest JSON payload sits.
Langfuse: an explicit multi-store architecture
Langfuse’s self-hosting overview documents application containers together with PostgreSQL, ClickHouse, Redis or Valkey, and object storage. It also describes optional model connectivity and an optional gateway. Some additional features require a license.
That is a useful starting point for a storage inventory. It is not a single-bucket deployment. Assign an owner to database backups, object retention and queue recovery before accepting production traffic.
The blob storage documentation describes supported storage configuration and deletion permissions for retention. Verify those permissions in a disposable environment. An application-level deletion request and a backup expiration policy are separate operational controls.
Choose this route when your platform team can maintain the underlying services. A small Compose demonstration does not establish high availability, disaster recovery or the staffing required for a large ingestion workload.
Phoenix: a smaller documented storage starting point
Phoenix’s self-hosting documentation describes a containerized service with a SQL backend. SQLite is a simple starting configuration; PostgreSQL is supported for production deployment.
That makes Phoenix worth considering for a team that already operates PostgreSQL and wants tracing and evaluation in its environment. It does not remove the need to inventory attachments, exports, credentials, SDK behavior and evaluator model calls.
Keep Phoenix and Arize’s commercial offerings distinct when requesting a quote. Confirm which product, support terms and deployment rights appear in the contract. Similar branding does not establish identical licensing or operational responsibilities.
Braintrust: inspect the data-plane boundary
Braintrust publishes an official Helm chart repository for its self-hosted data plane. That is concrete evidence of an operator-managed deployment path, rather than only a marketing phrase.
A data plane still needs a boundary review. Ask which user identities, project definitions, evaluation configurations and operational metadata remain in the control plane. Request an architecture diagram for the quoted version, including support access and outbound dependencies.
Do not assume that the chart proves an air-gapped configuration or that every managed-service feature works without external connectivity. Test the documented contract with your own network controls before approving sensitive production traces.
LangSmith: distinguish agents from observability
LangSmith’s data-plane documentation separates cloud, hybrid and self-hosted deployment behaviors. It describes custom persistence options, tracing choices and licensing or telemetry differences.
An agent runtime deployed in your account does not automatically mean all observability is self-hosted. Trace destination is a separate choice. Ask the vendor to distinguish Agent Server state, observability storage, orchestration metadata and license checks in the proposed configuration.
That distinction is especially important if your application team is purchasing runtime hosting while the security team believes it is approving local trace storage. Use a shared diagram and contract terminology to resolve the difference.
Compare costs without inventing a universal monthly price
| Route | Software or commercial basis | Additional costs to include |
|---|---|---|
| Langfuse self-hosted | Open-source core; licensed additions may apply | Multiple stores, backups, upgrades, on-call support |
| Phoenix self-hosted | Self-hosted software; confirm current license terms | Database, compute, authentication and operations |
| Braintrust data plane | Obtain the applicable commercial terms | Kubernetes, storage, connectivity and support |
| LangSmith deployment | Obtain the applicable licensed deployment quote | Runtime, trace destination and operational ownership |
A public SaaS entry price is not a quote for the required private architecture. Compare the same retention, ingestion volume, availability and support coverage across options. Evaluator model usage belongs in the budget even if the platform software is free.
Map the data lifecycle before running a pilot
Create a table for prompts, completions, tool results, embeddings, media, scores, identity data, indexes, caches, backups and exported datasets. For each record the account, region, encryption-key owner, retention period and deletion mechanism.
Then map outbound connections. A self-hosted interface can still call an external model to score sensitive traces. Error reporting or support bundles can also contain payload fragments. Review the actual configuration and contractual controls rather than assuming those paths are disabled.
Use synthetic traces first. Put a unique marker in a prompt, a tool result and an attachment. Confirm where each marker appears, how authorization restricts it and whether a deletion removes it from the relevant live stores.
Test operational failure as well as privacy
Stop a worker, interrupt the database and deny access to object storage in an isolated environment. Determine whether ingestion blocks the application, queues safely, drops data or duplicates it after recovery.
Restore a backup into a separate environment and verify that traces, evaluations and permissions still align. A recovery procedure that restores payloads but loses their organization boundary is not an acceptable result.
Review SDK defaults with the same care. Sampling, redaction and endpoint configuration affect both cost and confidentiality. Client-side masking should be tested against nested tool arguments and exception messages, not just the main prompt field.
Where other observability products fit
Helicone, LangWatch and Datadog can remain procurement candidates when their documented deployment model matches your requirement. This guide does not infer customer-managed primary storage from a generic security page or an export integration.
Ask every candidate the same questions and request evidence for the exact offering. If a vendor cannot identify the authoritative store, deletion behavior and control-plane boundary, keep its status as unverified rather than filling the gaps with assumptions.
Related Articles
- LLM observability for developers
- AI agent observability platforms
- Production RAG design
- Evaluate AI vendors
- AI evaluation checklist
FAQ
Does BYOC mean nothing leaves my cloud?
No. It describes a deployment arrangement whose exact boundary must be verified. Control-plane metadata, licensing and evaluator calls may still leave the environment.
Is an S3 export customer-managed primary storage?
No. An export is a copy unless the architecture explicitly makes your bucket authoritative and defines the remaining databases and indexes.
Is self-hosting cheaper than SaaS?
It depends on workload and operating capability. Include upgrades, backups, incident response, security work and model evaluation calls in the comparison.
Can a private trace store call public model APIs?
Yes, if configured that way. Data storage and model processing are separate decisions. Review both before sending sensitive payloads.
What should block procurement?
An unexplained data store, untestable deletion path or unclear support-access boundary should block approval until the vendor and your operators resolve it.