Start free

Connect Scorecard MCP so your agent can check evaluation runs and scores during a call.

Scorecard MCP keeps your evaluation projects, testsets, runs and scores in one place. With Scorecard connected, your AgentCalls agent can look up a run's status, pull the latest scores for a system, and answer questions about testcases while the caller is on the line. Connect your Scorecard account once from your AgentCalls dashboard, using credentials from Scorecard's settings. Your agent then works with the tool as your team does, and only with what you allow.

Start freeSee pricing

What your agent can do with Scorecard MCP

  • Look up the status and latest results of an evaluation run during a call
  • Pull current scores and metrics for a named system
  • Answer questions about testsets and testcases in your project
  • Search Scorecard documentation to answer a caller's technical question

Scorecard MCP exposes 2 actions. You choose which ones the agent may use and whether each runs during or after the call.

Example calls

Status check from a project lead

  1. Caller

    Hi, can you tell me whether last night's run for the support bot finished?

  2. Agent

    Yes, it finished at 2:14 this morning. It ran 480 testcases and passed 463 of them.

  3. Caller

    And what was the accuracy score?

  4. Agent

    The accuracy metric came in at 96.5 percent, up from 94.1 percent on the previous run.

Question about a testset

  1. Caller

    We added new refund questions last week. Are they in the main testset?

  2. Agent

    They are in the refunds testset. It holds 37 testcases, and 35 of them were added last Tuesday.

  3. Caller

    Great, can you confirm the two that were left out?

  4. Agent

    Yes, the two left out are the partial refund cases. They sit in a draft testset until they are reviewed.

Follow-up call about a failed run

  1. Agent

    Hello, this is calling from the evaluation team. The nightly run for the billing agent failed at 3 AM. Do you have a few minutes?

  2. Caller

    Yes, what failed?

  3. Agent

    Twelve testcases timed out. Would you like me to send the list to your email so your engineers can review it?

  4. Caller

    Yes, please send it to the engineering inbox.

How to connect Scorecard MCP

  1. 1

    Create your agent

    Sign up to AgentCalls, pick an English or Arabic voice and write what the agent should know and do in plain language.

  2. 2

    Connect Scorecard MCP

    From Integrations in the dashboard, choose Scorecard MCP and enter the connection details from your Scorecard MCP account settings.

  3. 3

    Choose the actions

    In the agent editor, enable the Scorecard MCP actions the agent may use and set each to run during the call or afterwards. Then test it from your browser.

Related integrations

Google SheetsVerifiedSpreadsheets & databasesGoogle Sheets is a cloud-based spreadsheet tool enabling real-time collaboration, data analysis, and integration with…SupabaseSpreadsheets & databasesSupabase is an open-source backend-as-a-service providing a Postgres database, authentication, storage, and real-time…AirtableSpreadsheets & databasesAirtable merges spreadsheet functionality with database power, enabling teams to organize projects, track tasks, and…Code InterpreterSpreadsheets & databasesCodeInterpreter extends Python-based coding environments with integrated data analysis, enabling developers to run…SerpApiSpreadsheets & databasesSerpApi provides a real-time API for structured search engine results, allowing developers to scrape, parse, and…ExaSpreadsheets & databasesExa focuses on data extraction and search, helping teams gather, analyze, and visualize information from websites…

All Spreadsheets & databases integrations

Questions, answered.

Does the agent use Scorecard during the call or after it?

Both. During a call your agent can look up runs, scores and testsets in real time. It can also leave a summary in your records after the call ends.

What do I need to connect Scorecard MCP?

You need a Scorecard account and its credentials from Scorecard's settings. Sign in to your AgentCalls dashboard and add the connection under tools.

Which plan includes this integration?

Tool integrations, including Scorecard MCP, are included from the Growth plan.

Page drafted from the tool's public description and reviewed automatically for accuracy of claims

Put your agent to work with Scorecard MCP.

Connect in minutes, test from your browser, go live on a number in the US, UK or Gulf. Free to start, no card needed.

Start freeSee pricing