AI and LLM evaluation platforms and tools
18 vendors in alphabetical order. Filter by workload, buying model, ownership, certifications, and public pricing, then select up to four to compare.
| Vendor | Buying model | Ownership | SOC 2 | ISO 27001 | HIPAA | GDPR | Pricing | Compare |
|---|---|---|---|---|---|---|---|---|
| Arize AI (AX and Phoenix)Hosted SaaS platform (Arize AX) | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Subsidiary | Published | Published | Published | Published | Published | |
| ArthurHosted SaaS platform | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Privately held | Not found | Not found | Not found | Not found | Published | |
| BraintrustHosted SaaS platform | Hosted SaaS, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Published | Published | |
| Confident AI (DeepEval)Hosted SaaS platform | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Published | Published | |
| Datadog LLM ObservabilityPart of a larger observability platform (Datadog) | Hosted SaaS, Part of a larger platform | Publicly listed | Published | Published | Published | Not found | Published | |
| DeepchecksHosted SaaS platform | Hosted SaaS, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Published | On request | |
| Evidently AIOpen-source framework (Evidently, Apache 2.0) | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Privately held | Not found | Not found | Not found | Not found | On request | |
| Fiddler AIHosted SaaS platform | Hosted SaaS, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Not found | Published | |
| Galileo (Splunk Agent Observability)Part of a larger observability platform (Splunk) | Hosted SaaS, Self-hosted, VPC, or on-prem, Part of a larger platform | Subsidiary | Published | Not found | Published | Not found | Published | |
| HoneyHiveHosted SaaS platform | Hosted SaaS, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Published | Published | |
| LaminarOpen-source core (Apache 2.0) | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Not found | Published | |
| LangfuseOpen-source core (MIT) | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Subsidiary | Published | Published | Published | Published | Published | |
| LangSmith (LangChain)Hosted SaaS platform | Hosted SaaS, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Published | Published | |
| MLflow (Databricks Managed MLflow)Open-source framework (MLflow, Apache 2.0) | Hosted SaaS, Open-source or source-available core, Part of a larger platform | Privately held | Published | Not found | Not found | Not found | On request | |
| OpenlayerHosted SaaS platform | Hosted SaaS, Self-hosted, VPC, or on-prem | Privately held | Published | Not found | Published | Published | Published | |
| Opik (Comet)Open-source core (Apache 2.0) | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Privately held | Published | Published | Published | Published | Published | |
| PromptfooOpen-source framework (MIT) | Hosted SaaS, Open-source or source-available core, Self-hosted, VPC, or on-prem | Privately held | Published | Published | Not found | Not found | Published | |
| W&B Weave (CoreWeave)Hosted SaaS platform | Hosted SaaS, Open-source or source-available core, Part of a larger platform | Subsidiary | Published | Published | Published | Not found | Published |
No vendors match these filters.
Alphabetical. A check means the vendor publishes it on its own site or trust portal; a dash means we found nothing public, so ask the vendor.
Who covers what
Published workload coverage by vendor.
| Vendor | Tracing | Offline evals & CI | Online evals | LLM-as-a-judge | Guardrails & red team | Open source | Self-hosted | Prompt management |
|---|---|---|---|---|---|---|---|---|
| Arize AI (AX and Phoenix) | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes |
| Arthur | No | Yes | No | No | Yes | Yes | Yes | Yes |
| Braintrust | Yes | Yes | Yes | Yes | No | No | Yes | Yes |
| Confident AI (DeepEval) | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
| Datadog LLM Observability | Yes | Yes | Yes | Yes | No | No | No | Yes |
| Deepchecks | Yes | No | Yes | No | No | No | Yes | No |
| Evidently AI | Yes | No | Yes | Yes | No | Yes | Yes | No |
| Fiddler AI | Yes | Yes | No | Yes | Yes | No | Yes | No |
| Galileo (Splunk Agent Observability) | Yes | Yes | Yes | Yes | Yes | No | Yes | No |
| HoneyHive | Yes | Yes | Yes | No | No | No | Yes | Yes |
| Laminar | Yes | No | No | Yes | No | Yes | Yes | No |
| Langfuse | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes |
| LangSmith (LangChain) | Yes | Yes | Yes | Yes | No | No | Yes | Yes |
| MLflow (Databricks Managed MLflow) | Yes | Yes | Yes | Yes | No | Yes | No | Yes |
| Openlayer | Yes | Yes | Yes | No | Yes | No | Yes | No |
| Opik (Comet) | Yes | Yes | Yes | Yes | No | Yes | Yes | No |
| Promptfoo | No | Yes | No | Yes | Yes | Yes | Yes | No |
| W&B Weave (CoreWeave) | Yes | Yes | Yes | Yes | No | Yes | No | Yes |
| Vendors | 16 | 15 | 14 | 14 | 6 | 10 | 15 | 9 |
Based on each vendor’s published categories and capabilities. A filled dot means the vendor describes this workload publicly; it is not a quality judgment.