Evaluation

A zero-dependency Node.js CLI that scans websites for AI-agent discovery and interoperability signals, reports a grade, and suggests fixes.

An open-source privacy evaluation toolkit that analyzes agent traces across tool calls, shared memory, inter-agent messages, and logs, producing redacted reports and CI gates.

AgentOps aims to improve AI agent development with tools for observability, evaluations, and replay analytics, offering a streamlined process for testing and debugging compliant AI agents through a user-friendly interface and comprehensive documentation

Arize Phoenix is a source-available AI observability and evaluation platform, built on OpenTelemetry and OpenInference, for tracing, evaluating and debugging AI applications and agents in your own environment.

BrowserBash is a CLI and MCP server for running plain-English browser tests in Chrome and returning deterministic assertions and exit codes for agent-driven UI validation.

ClawBench is an open-source benchmark for browser agents, with everyday tasks across live websites, execution traces, and request-level outcome evaluation.

House Party Protocol is a local first Python harness for Claude Code and Codex CLI teams that records criterion command results outside the model, refuses verdicts from the builder's own lane, and binds approvals to a spec hash.

An open-source replay-based evaluation toolkit for AI agents that records or imports production traces as sessions and reruns them against model, prompt, or code changes.

Trace agent runs, manage prompts and evaluate outputs in an AI engineering platform.

Opik is an open-source platform for LLM and agent observability, tracing, evaluation, prompt management, and production monitoring.

Project Telos is a local-first MCP and CLI workbench for inspectable agent workflows, with provenance, routing ledgers, action receipts, context packs, diagnostics, and replayable verification packets.

Self Auditing Agent pairs audit claims with rerunnable commands and includes gatecheck, a mutation tester for quality gates available as a GitHub Action and pre-commit hook.

The Agentic Leaderboard publishes weekly rankings of open-source AI agents across reliability, tool selection, autonomous iteration, and community signals, with public data and methodology.