AI Interfaces

Hosted MCP Server and Claude Plugin

The backend hosts a Model Context Protocol server at /api/v1/mcp through which Claude operates the platform as the signed-in user: it reads and changes accounts, projects, agents and their assets, reads analytics and conversation data, runs tests. The Studio frontend is the OAuth authorization server that signs Claude in, and the Promethist Claude plugin registers the endpoints of the three platform instances. The customer-facing description is in the public part of this documentation (Promethist for Claude).

Claude ──► https://{host}/api/v1/mcp            backend: resource server + MCP server
   │  401 + resource_metadata
   ├──► /.well-known/oauth-protected-resource    Studio: points to the authorization server
   ├──► /.well-known/oauth-authorization-server  Studio: endpoints, PKCE S256
   ├──► /oauth/register                          Studio: dynamic client registration
   ├──► /oauth/authorize                         Studio: needs a Studio session (Keycloak sign-in)
   └──► /oauth/token                             Studio: access token (1 h), refresh token (30 days)

Endpoint and Security Chain

McpServerConfiguration builds a stateless MCP server (McpServer.async over WebFluxStatelessServerTransport of org.springframework.ai:mcp-spring-webflux, MCP Java SDK 2.0) with the message endpoint /api/v1/mcp. Every request is independent — there is no MCP session to keep — so the server scales with the backend’s replicas. It advertises only the tools capability, and its server instructions tell the model to call list_tenants first and to load the relevant guide with get_guide before any domain work.

The endpoint has its own Spring Security filter chain, ordered ahead of the application-wide one:

  • every request must be authenticated;

  • bearer tokens are validated against the Studio’s signing keys (promethist.mcp.jwks-uri), the issuer (promethist.mcp.issuer), the audience — which must contain promethist.mcp.audience, the endpoint’s own URL — and the token kind, which must be access;

  • a request without a valid token gets 401 with WWW-Authenticate: Bearer resource_metadata="{promethist.mcp.resource-metadata-url}", which is how Claude discovers where to sign in.

The JWT decoder of this chain is deliberately not a Spring bean: as a bean it would replace the Keycloak decoder of the REST API. Keycloak tokens are therefore not accepted at /api/v1/mcp, and MCP tokens are not accepted anywhere else.

A WebFilter with the highest precedence implements the specification’s Origin check: a request that carries an Origin header is refused with 403 unless the origin is the issuer, https://claude.ai or https://claude.com. Requests without Origin (command-line clients) pass. The SDK’s own validator is switched off because it is incompatible with Spring Framework 7’s header type.

OAuth Authorization Server

The Studio frontend implements the authorization server (frontend/lib/mcp-oauth.ts and the routes in frontend/app/oauth/ and frontend/app/api/mcp-oauth/). It does not involve Keycloak in issuing tokens: Keycloak only signs the user in to the Studio, and the Studio then issues its own tokens for the MCP resource.

Endpoint Behavior

GET /.well-known/oauth-protected-resource

RFC 9728 metadata: the resource (MCP_RESOURCE), the authorization server (MCP_OAUTH_ISSUER) and the scope mcp. Served by a regular route through a rewrite in next.config.ts, because Next.js does not route dot-prefixed directories.

GET /.well-known/oauth-authorization-server

RFC 8414 metadata: the endpoints below, response type code, grants authorization_code and refresh_token, PKCE method S256 only, token endpoint authentication none (public clients).

POST /oauth/register

RFC 7591 dynamic client registration, stateless: it returns a fresh client_id and stores nothing.

GET /oauth/authorize

Validates the request — the redirect URI must be on the allowlist, response_type=code, a PKCE challenge with method S256 is required — and then needs a Studio session: without one it sends the user through /login and back. It answers with a signed authorization code valid for 60 seconds.

POST /oauth/token

authorization_code: verifies the code, that the redirect URI matches, the PKCE verifier, and that the code has not been redeemed before. refresh_token: verifies the refresh token. Both issue a new access token (one hour, audience MCP_RESOURCE) and a new refresh token (30 days), so a client that keeps refreshing stays signed in. Responses are not cacheable.

GET /oauth/jwks

The public signing key.

Properties of the design:

Stateless tokens

Authorization codes, access tokens and refresh tokens are all RS256 JWTs signed with one key per environment (MCP_OAUTH_PRIVATE_JWK), distinguished by a kind claim that every consumer checks. Nothing is persisted, so there is no per-token revocation: rotating the environment’s key invalidates all of its tokens at once. There is deliberately no default key.

Redirect allowlist

Without stored registrations, the allowlist is the control that keeps codes from being sent elsewhere: https://claude.ai, https://claude.com and loopback addresses (http://localhost, http://127.0.0.1, any port) for command-line clients.

Single-use codes

A redeemed code’s jti is remembered until the code expires — in the memory of the Next.js process. That suffices because the frontend runs one replica; with more replicas or during a rolling update the check spans only one process, and PKCE remains the protection.

Identity

The tokens carry the user’s email (lowercased) as sub and email. The backend resolves the platform user from it exactly as it does for a Keycloak token, so an MCP caller has precisely the roles the same person has in the Studio, evaluated at every call.

Tool Calls

Each call goes through the same steps (McpServerConfiguration.invokeTool):

  1. Argument contract — an argument the tool’s schema does not declare is refused before anything else happens, with the list of the arguments the tool accepts (McpArgumentContract). Without this, a misspelled field would be silently ignored and the call reported as a success.

  2. Confirmation — a destructive tool runs only with confirm: true; otherwise it answers that the user has to confirm first.

  3. Identity — the bearer token, stashed by the transport, is decoded again and the tool runs with an authenticated JwtAuthenticationToken in the reactive security context.

  4. Handler — the tool’s code runs.

  5. Error mapping — failures become one sentence for the model (McpErrorTranslator, below).

As a rule, a tool calls the REST controller method that serves the same operation, through its Spring proxy, so it is subject to the same @PreAuthorize check as the HTTP API. Because bean validation of @Valid request bodies is done by the WebFlux argument resolver and not by the proxy, tools pass request bodies through McpToolSupport.validated first. Where there is no endpoint — evaluation elements and groups, test_agent — the tool states its permission check explicitly.

Results are the REST response objects serialized with the backend’s Jackson mapper, so a tool returns the same shape as the HTTP API. Objects that have a page in the Studio carry a webUrl ({promethist.mcp.web-url}/dashboard/studio/accounts/{tenantId}/projects/{projectId}/...). An empty list is answered with a sentence, because models retry a bare []. Credentials stored in integrations are redacted from what the model sees.

Errors

McpErrorTranslator decides what the model may read:

  • a denied permission becomes "Permission denied: your account does not have the required role for this operation.";

  • a McpToolException, a validation failure or a "not found" is relayed verbatim — unless the message looks internal (a SpEL expression, a fully qualified class name, a stack frame, a connection string), in which case a generic message is returned and the detail is logged;

  • anything unexpected becomes <tool> could not be completed right now. Please try again. and is logged.

Tools

list_tenants and get_guide are built into McpServerConfiguration; all other tools come from McpToolProvider beans in ai.promethist.platform.component (Mcp*Tools.kt) — 125 tools in total.

Provider Tools Area

McpWorkspaceTools

22

Accounts, projects and their terms and conditions, members, invitations, end users of a project.

McpAgentTools

12

Agents, revisions and lifecycle, voices, visuals, published templates.

McpTemplateTools

7

Template authoring (staff only).

McpEvaluationTools

15

Evaluations, their revisions and lifecycle, their output data elements and groups.

McpKnowledgeTools

8

Knowledge sources, web ingestion, file upload links.

McpMultimodalTools

6

Interactive content, media upload links.

McpIntegrationTools

11

Webhooks, MCP servers, connectors, live tool probe.

McpSkillTools

9

Skills: project assignments readable by viewers, administration staff only.

McpAnalyticsTools

13

Analytics cards, summary, export; subscription, invoices and usage (read only).

McpConversationDataTools

6

Sessions and transcripts, raw turns and LLM logs, bulk export.

McpTestSuiteTools

8

Test suites, runs and results.

McpMemorySnapshotTools

2

Memory snapshots of a simulated user: save and restore.

McpTestAgentTools

1

test_agent, live conversations (Live Agent Tests).

McpTestHarnessTools

3

Raw bridge documents (staff only).

The 19 confirmation-gated tools are those that irreversibly remove something: revert_agent, archive_agent, archive_template, delete_template, delete_evaluation, revert_evaluation, remove_insight, remove_group, delete_knowledge, delete_multimodal_interaction, delete_integration, delete_connector, delete_skill, remove_skill_from_project, delete_test_suite, remove_tenant_member, remove_project_member, revoke_tenant_invitation and revoke_project_invitation. Promotion is deliberately not gated.

Files cross the protocol through capability links (Capability URLs): export_analytics and export_conversation_data return a 24-hour download link, and upload_knowledge_file and upload_interactive_content_file return a single-use curl command with a placeholder for the local file path, valid for one hour, with limits of 10 MB for knowledge, 25 MB for images and 500 MB for videos. A video uploaded this way goes through the Studio’s upload path and is re-encoded like any other (Re-encoding of Uploaded Videos).

Operating Guides

The model learns how Promethist works from guides, not from tool descriptions. They are Markdown files in backend/src/main/resources/mcp-guides/, loaded at startup by McpGuideService and served by get_guide(topic):

  • The file name is the topic (kebab-case). YAML front matter sets title, summary and optionally audience: staff, which restricts the guide to staff accounts; it is still listed, labelled "(Promethist staff only)".

  • The tool’s description and its topic enum are generated from the files, so adding a guide is adding a file. An unknown topic is answered with the list of topics.

  • Guides ship with the backend: changing what the model is told is a backend release, not a plugin release.

Topic Covers

analytics

What every analytics number means, its unit, common misreadings.

authoring

Agent fields, limits, completeness rules, voices and visuals, templates, guardrails, business process.

billing

Conversational minutes, account usage, plans, invoices.

memory-testing

Several sessions as one returning user, spread over time, and re-running the session under test from a memory snapshot.

relational-metrics

The v2 checks a tester is judged by (RQ v2, SIM v2, CON v1), their ids per cluster, how to report a pass rate, and the legacy sets.

data-export

Where transcripts, raw turns and LLM logs live and what each store will not tell you.

domain

Accounts, projects, agents, revisions, packages, assets.

evaluation

Evaluation design, element types, groups, featured metric, execution modes, destinations, revisions.

integration

Pre-, during- and post-session integrations and connectors: fields, references, tool selection.

knowledge

Knowledge sources, web ingestion and refresh, retrieval behavior, references.

multimodal

Interactive-content types, handover rules, references.

packages

Project packages and how to build Empower coaching and Engage kiosks.

quality

The readiness rubric for agents and projects.

skills

What a skill is, how agents activate it, references.

test-automation

Building simulated users and running them as test suites.

testing

Planning and running live roleplay tests and turning failures into fixes.

workspace

Account and project hierarchy, roles, members, billing access.

Keep the guides consistent with the tools: when a tool’s behavior changes, the guide that describes it changes in the same commit.

Contracts Enforced by Tests

Test Fails when

McpToolRegistryTest

A tool appears, disappears or is renamed without updating its expected list; the set of confirmation-gated tools changes; a destructive tool lacks the confirm argument or a non-destructive one has it; an argument name is not snake_case; a schema is not an object schema; a tool description exceeds 420 characters or the whole tool listing (descriptions and schemas) exceeds 120,000 characters; a tool name suggests a forbidden capability (login, logout, payment, checkout, portal, Stripe, webhook callback).

McpGuideServiceTest

A guide’s body exceeds 16,000 characters, its title 60 or its summary 90 characters.

EvaluationScaleConsistencyTest

A prompt or a guide states the retired evaluation scale or pass threshold.

McpErrorTranslatorTest, McpStrictArgumentsTest

Internal detail could reach a client, or an unknown argument is not refused.

McpServerIntegrationTest (integrationTests)

The running endpoint stops challenging unauthenticated calls or loading the guides.

The budgets exist because every connected client pays for the whole tool listing on every connection, and a guide is read in full every time it is loaded.

Adding a Tool

  1. Add an McpTool to the provider of its area — a @Component in ai.promethist.platform.component, the package McpToolRegistryTest discovers providers in. Build its schema with McpSchemaBuilder, snake_case argument names, and a description of at most 420 characters that, by convention, names the REST operation it wraps.

  2. Call the guarding controller method rather than a service, or state the permission check explicitly when no endpoint exists. Pass @Valid request bodies through McpToolSupport.validated.

  3. Mark it destructive and add McpSchemaBuilder.confirm if it irreversibly removes something.

  4. Throw McpToolException for anything the model should read verbatim; never put internal detail in it.

  5. Add the tool to EXPECTED_TOOLS (and DESTRUCTIVE_TOOLS) in McpToolRegistryTest, write a test in backend/src/test/kotlin/ai/promethist/platform/mcp/, and update the guide that covers the area — within its size budget.

The Claude Plugin

The plugin is a Claude plugin in the organization’s plugin marketplace repository, promethistai/agent-plugins on GitLab (marketplace promethist-agent-plugins, plugin directory plugins/promethist). It contains:

  • .mcp.json, registering three remote MCP servers of type http:

  • skills: promethist (choose the instance, how sign-in works on each Claude surface, call list_tenants first and get_guide before domain work), promethist-evaluation-report (an HTML heatmap from evaluation results), promethist-knowledge-format (preparing documents for knowledge) and promethist-test-and-improve-agent (a test, diagnose and fix loop built on simulated users).

The plugin runs no server and holds no credentials — besides the connector registrations it has only the skills, two of which ship helper scripts that run in Claude’s environment. Claude’s MCP client performs the OAuth flow. Because the behavior lives in the server’s tools and guides, most changes do not need a new plugin version. Installing the plugin does not connect the servers: in Claude Desktop and Cowork each connector is enabled and connected once in the plugin’s Connectors tab; in Claude Code it is authenticated with /mcp.

Configuration

Setting Value in a deployment of host {host}

promethist.mcp.issuer / MCP_OAUTH_ISSUER

https://{host}

promethist.mcp.jwks-uri

https://{host}/oauth/jwks

promethist.mcp.audience / MCP_RESOURCE

https://{host}/api/v1/mcp

promethist.mcp.resource-metadata-url

https://{host}/.well-known/oauth-protected-resource

promethist.mcp.web-url, promethist.mcp.engine-url

https://{host}

MCP_OAUTH_PRIVATE_JWK

The environment’s signing key, from the MCP_OAUTH_PRIVATE_JWK_EU, _US or _PREVIEW CI variable.

The backend values come from the Helm ConfigMap, the frontend values are written at build time. The issuer and the audience must agree on both sides; a mismatch makes every call fail with 401. The backend refuses to start without the six promethist.mcp.* URLs.