MCP Benchmark Report

November 5, 2025 - Performance Analysis

Total Queries
12
Total Runs
61
Success Rate
100%

📊 Scoring Methodology

Formula:

Score = 1,000,000 × (successes × mcp_success_rate) / ((failures + 1) × tokens × duration × turns)

✓ Higher is Better

  • Successes (completing task correctly)
  • MCP Success Rate (tool calls that work)

✗ Lower is Better

  • Failures (task failures)
  • Tokens (API usage cost)
  • Duration (time to complete)
  • Turns (conversation rounds)
⚠️

Note: Scores vary with data volume and complexity. For valid A/B testing, run tests at the same time under identical data conditions (e.g., same Jira board state).

Score Distribution

MCP Success Rate

Average Tokens Used

Average Duration (seconds)

Query Details

🔍 MCP Errors Analysis