Bridge Application

Results and Evaluation

Every run is stored as a BridgeRun, in MongoDB and in the Elasticsearch index bridge-run. The Elasticsearch copy is what makes results chartable in Kibana — success rates and scores over time, per test case or per configuration.

Run Verdict

status is one of:

Status Meaning

Status	Meaning
`Running`	The run has started. Written when the run begins, so a crashed run is still visible.
`Passed`	The conversation completed and the tester ended it without reporting a failure.
`Failed`	The tester ended the session with a failure reason, or the run hit an unrecoverable error — a device that could not be provisioned, a fatal socket error, or the overall timeout.

Running

The run has started. Written when the run begins, so a crashed run is still visible.

Passed

The conversation completed and the tester ended it without reporting a failure.

Failed

The tester ended the session with a failure reason, or the run hit an unrecoverable error — a device that could not be provisioned, a fatal socket error, or the overall timeout.

reason carries the explanation, which for a tester-driven end is the text the tester supplied when it ended the session. This is the field to read first when a run fails.

BridgeRun Document

Field Content

Field	Content
`id`	Run ID — what the run endpoints return, one per line.
`testCaseId`	Test case the run belongs to, when it was started from one.
`properties`	The `BridgeProperties` the run actually used, after variable substitution. Runs are therefore self-describing and reproducible.
`deviceType`	Device type resolved from the device URI.
`deviceImage`	Final screenshot of the device as JPEG, when the device provides screenshots.
`sessionId`	Engine session ID of the tester’s conversation — the handle for looking the conversation up through the Session API or the session debugger.
`status`, `reason`	The verdict, as above.
`startTime`, `endTime`	Run boundaries.
`evaluationsPassed`	Whether the engine’s evaluations of the conversation passed.
`evaluations`	Each evaluation metric as its own field, numbers kept as numbers so they can be aggregated and charted per metric.
`evaluationsJson`	The raw evaluation payload as returned by the engine.
`weightedAverageScore`	A single score computed from `evaluations` using the run’s `aggregate-weights`, when weights were configured.
`evaluationsPending`	Set while evaluation results are still being awaited.

id

Run ID — what the run endpoints return, one per line.

testCaseId

Test case the run belongs to, when it was started from one.

properties

The BridgeProperties the run actually used, after variable substitution. Runs are therefore self-describing and reproducible.

deviceType

Device type resolved from the device URI.

deviceImage

Final screenshot of the device as JPEG, when the device provides screenshots.

sessionId

Engine session ID of the tester’s conversation — the handle for looking the conversation up through the Session API or the session debugger.

status, reason

The verdict, as above.

startTime, endTime

Run boundaries.

evaluationsPassed

Whether the engine’s evaluations of the conversation passed.

evaluations

Each evaluation metric as its own field, numbers kept as numbers so they can be aggregated and charted per metric.

evaluationsJson

The raw evaluation payload as returned by the engine.

weightedAverageScore

A single score computed from evaluations using the run’s aggregate-weights, when weights were configured.

evaluationsPending

Set while evaluation results are still being awaited.

Evaluation Fetching

Evaluations run in the engine after the conversation ends, so they are not available when the run finishes. When a run completes with a session ID, the bridge starts a background watcher that polls the engine’s evaluation status endpoint (/api/evaluation/status?sessionId=…) until it reports a terminal state — up to 24 attempts, 5 seconds apart, so roughly two minutes.

The results are then flattened onto the run: each insight becomes a field in evaluations, the raw payload is kept in evaluationsJson, and evaluationsPassed records the outcome.

If the bridge is restarted while runs are still waiting, it resumes watching the runs marked evaluationsPending on startup, so a redeploy does not lose results.

Weighted Score

When a configuration defines aggregate-weights — a map of evaluation metric ref to weight — the bridge computes weightedAverageScore from the numeric evaluation values. That single number is what a trend chart or a pass threshold can be built on, instead of comparing every metric separately.

"aggregate-weights": {
  "taskCompletion": 0.6,
  "userEngagement": 0.4
}

Metrics that are missing or non-numeric in a given run are skipped rather than counted as zero.

Recordings

With promethist.bridge.video-recording-enabled (the default), each run is recorded to recording/bridge/{runId}.ts in the configured storage — the device screen with the audio of both sides. Fetch it through GET /run/recording/{runId}.ts. Watching the recording is usually the fastest way to understand why a conversation went wrong.

In CLI mode --video-file writes the recording to a local path instead.

Where to Look When a Run Fails

reason on the run — the tester’s own explanation, or the error that ended it.
The recording, for what the device actually showed and said.
The engine session (sessionId) — turns and the LLM log through the Session API, or the session debugger for the full timeline.
properties on the run, to confirm the test ran against the environment and agent you expected after variable substitution.
For a bs:// run, the BrowserStack session, which carries the verdict and its own video and device logs.