The independent guide to AI evals tools

Eval platforms, open-source frameworks, and LLM observability tools. Filter by hosting model, open-source license, online evals, and published certifications, with every fact linked to its source.

Explore the landscape Read the buyer’s guide

18vendors profiled: Managed services, expert networks, platforms, and dataset sellers.
72primary sources: Vendor sites, trust portals, and company releases. No listicles.
0paid placements: Nobody pays to appear or to change their position.

Who covers what

Filter vendors

Published workload coverage for 18 AI eval tools. Click a column to see those vendors.

Which workloads each vendor publicly describes
Vendor	Tracing	Offline evals & CI	Online evals	LLM-as-a-judge	Guardrails & red team	Open source	Self-hosted	Prompt management
Arize AI (AX and Phoenix)	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
Arthur	No	Yes	No	No	Yes	Yes	Yes	Yes
Braintrust	Yes	Yes	Yes	Yes	No	No	Yes	Yes
Confident AI (DeepEval)	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
Datadog LLM Observability	Yes	Yes	Yes	Yes	No	No	No	Yes
Deepchecks	Yes	No	Yes	No	No	No	Yes	No
Evidently AI	Yes	No	Yes	Yes	No	Yes	Yes	No
Fiddler AI	Yes	Yes	No	Yes	Yes	No	Yes	No
Galileo (Splunk Agent Observability)	Yes	Yes	Yes	Yes	Yes	No	Yes	No
HoneyHive	Yes	Yes	Yes	No	No	No	Yes	Yes
Laminar	Yes	No	No	Yes	No	Yes	Yes	No
Langfuse	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
LangSmith (LangChain)	Yes	Yes	Yes	Yes	No	No	Yes	Yes
MLflow (Databricks Managed MLflow)	Yes	Yes	Yes	Yes	No	Yes	No	Yes
Openlayer	Yes	Yes	Yes	No	Yes	No	Yes	No
Opik (Comet)	Yes	Yes	Yes	Yes	No	Yes	Yes	No
Promptfoo	No	Yes	No	Yes	Yes	Yes	Yes	No
W&B Weave (CoreWeave)	Yes	Yes	Yes	Yes	No	Yes	No	Yes
Vendors	16	15	14	14	6	10	15	9

Based on each vendor’s published categories and capabilities. A filled dot means the vendor describes this workload publicly; it is not a quality judgment.

Vendor landscape

18 tools, checked November 8, 2026. Use filters for workload, buying model, published certifications, and public pricing. Select up to four to compare side by side.

AI eval tools in alphabetical order with buying model, ownership, published certifications, and pricing
Vendor	Buying model	Ownership	SOC 2	ISO 27001	HIPAA	GDPR	Pricing
Arize AI (AX and Phoenix)Hosted SaaS platform (Arize AX)	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Subsidiary	Published	Published	Published	Published	Published
ArthurHosted SaaS platform	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Privately held	Not found	Not found	Not found	Not found	Published
BraintrustHosted SaaS platform	Hosted SaaS, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Published	Published
Confident AI (DeepEval)Hosted SaaS platform	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Published	Published
Datadog LLM ObservabilityPart of a larger observability platform (Datadog)	Hosted SaaS, Part of a larger platform	Publicly listed	Published	Published	Published	Not found	Published
DeepchecksHosted SaaS platform	Hosted SaaS, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Published	On request
Evidently AIOpen-source framework (Evidently, Apache 2.0)	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Privately held	Not found	Not found	Not found	Not found	On request
Fiddler AIHosted SaaS platform	Hosted SaaS, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Not found	Published
Galileo (Splunk Agent Observability)Part of a larger observability platform (Splunk)	Hosted SaaS, Self-hosted, VPC, or on-prem, Part of a larger platform	Subsidiary	Published	Not found	Published	Not found	Published
HoneyHiveHosted SaaS platform	Hosted SaaS, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Published	Published
LaminarOpen-source core (Apache 2.0)	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Not found	Published
LangfuseOpen-source core (MIT)	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Subsidiary	Published	Published	Published	Published	Published
LangSmith (LangChain)Hosted SaaS platform	Hosted SaaS, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Published	Published
MLflow (Databricks Managed MLflow)Open-source framework (MLflow, Apache 2.0)	Hosted SaaS, Open-source or source-available core, Part of a larger platform	Privately held	Published	Not found	Not found	Not found	On request
OpenlayerHosted SaaS platform	Hosted SaaS, Self-hosted, VPC, or on-prem	Privately held	Published	Not found	Published	Published	Published
Opik (Comet)Open-source core (Apache 2.0)	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Privately held	Published	Published	Published	Published	Published
PromptfooOpen-source framework (MIT)	Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem	Privately held	Published	Published	Not found	Not found	Published
W&B Weave (CoreWeave)Hosted SaaS platform	Hosted SaaS, Open-source or source-available core, Part of a larger platform	Subsidiary	Published	Published	Published	Not found	Published

Alphabetical. A check means the vendor publishes it on its own site or trust portal; a dash means we found nothing public, so ask the vendor.

Research and comparisons

GuideNov 8, 2026 How to choose AI evaluation software Seven decisions, in order: what to evaluate, offline vs online, judges, hosting model, pricing unit, security evidence, and ownership. ComparisonNov 8, 2026 Langfuse vs LangSmith ComparisonNov 8, 2026 Braintrust vs Arize AI ComparisonNov 8, 2026 Promptfoo vs Confident AI (DeepEval)

Browse by workload

How this guide works

Every claim links to a public source, with the date we checked it.
No paid rankings. Tools are listed alphabetically or by the filters you choose.
Pricing shown exactly as published. Otherwise: contact vendor.

Read the methodology