Engine Application

Feedback Scheduling

How the engine decides which feedback definition to show a user, and what it remembers about it. The request and response shapes are documented for integrators in Feedback API; this page covers the logic behind them.

Everything lives in FeedbackService, behind FeedbackController. The controller resolves the caller with UserService.getUser(deviceRef): the authenticated user, or, failing that, the user of the deviceRef device, or a 404.

Request Flow

Both specification endpoints narrow the project’s FeedbackDefinitions the same way, then pick one:

Take the newest Published revision of each ref, and keep it only if it is not in test mode and has the requested trigger point — a newer revision that moved to the other trigger point or to test mode thus takes the definition away from the older ones. After a session, also require shortSessionEnabled or a session of at least sessionLengthThreshold turns.
Drop a definition the user already answered — only when it has askOncePerUser or is evaluation based after a session, always before a session. Answered means at least one answer with a value: a submission without answers is a postponement, not an answer.
Resolve each remaining definition to postponed, due or dropped (below) and build its specification from the entries that fit — the entries of the session’s length branch after a session, the entries without a branch before one.
Of the specifications built, keep one: postponed before due, and within each group the lowest priority number. A definition that yields no specification does not take part, so the next one is served instead.
Create a feedback for the winner only, and return its specification.

Resolving a definition goes through the user’s Feedback records:

A skipped feedback of the definition, of any of its revisions, drops it for good.
Otherwise, if the latest feedback of the current revision is postponed, the definition is served whatever its schedule says.
Otherwise the definition is served only if it is due by its scheduling mode.

Revisions

The revisions of a definition are separate documents. Each has its own id and its own revision number, and they share a ref. Only the newest Published revision of a ref is served, so publishing a new one replaces the previous revision for every user. The definitions of the project are read once per request and grouped by ref; the ids of all the revisions of a ref, whatever their state, are kept with the one that is served.

What the engine looks up decides whether it carries over to the new revision:

Lookup Scope

Lookup	Scope
Answered and skipped	Every revision of the `ref`: the query takes all of its ids. A user who answered or skipped any revision is not asked again.
The last request, a postponed feedback and the rotation of a carousel	The revision served: the lookup is by the user, the definition `id` and the `revision`. A new revision therefore looks never requested, which restarts its schedule — it is due at once unless its mode says otherwise — and its carousel rotation, and drops a postponement.

Answered and skipped

Every revision of the ref: the query takes all of its ids. A user who answered or skipped any revision is not asked again.

The last request, a postponed feedback and the rotation of a carousel

The revision served: the lookup is by the user, the definition id and the revision. A new revision therefore looks never requested, which restarts its schedule — it is due at once unless its mode says otherwise — and its carousel rotation, and drops a postponement.

A feedback holds the id of the revision it was requested for, so it stays correct when the definition later gets a newer revision; a submission for an older revision is still recorded and counts, through the ref, for the newer one. No migration is needed, but the documents of earlier revisions must be kept: deleting them would make their feedbacks unreachable from the ref.

Feedback Records

A Feedback is the single record of one request and the user’s reaction to it. It is created when a specification is served, with the time it was requested, the revision of the definition, no submitted time and no answers. The creation is fire-and-forget — it is subscribed on its own, and a failure is logged without affecting the response. Every time a user is shown a form is thereby recorded: ignored and repeated showings add up as separate feedbacks, which is intended. Readers of the collection must therefore look at the state, because it also holds feedbacks that nobody reacted to. A feedback does not record which questions were shown, only the answers given.

Its state is derived, never stored:

state When

`state`	When
`Requested`	It has no `submitted` time: nothing was submitted for it. This is what a served form is until the user reacts, and what an ignored one stays.
`Postponed`	It was submitted without answers.
`Skipped`	It has answers and every `value` is null.
`Answered`	It has at least one answer with a value.

Requested

It has no submitted time: nothing was submitted for it. This is what a served form is until the user reacts, and what an ignored one stays.

Postponed

It was submitted without answers.

Skipped

It has answers and every value is null.

Answered

It has at least one answer with a value.

Submitting fills in the latest feedback of the current revision if it is still only requested, setting its answers and submitted time. Otherwise — nothing was requested, the background creation did not land, or the last feedback was already submitted — the answers get a feedback of their own, requested and submitted at that moment. An answer with a null value is a skipped question and is kept; an invalid one is dropped, which can leave a submission with no answers, and so postponed.

Skipping and postponing record their reaction the same way, through the same code as a submission, and only differ in the answers they set. The skip endpoint answers every question of the definition once — those of all its entries, whatever the session length branch, in the order of the entries — with a null value. The postpone endpoint sets no answers. Neither reads a body, and neither is affected by emptyAnswersSkip, which only turns a submission without answers into a skip.

Because the state is not stored, the conditions behind Answered and Skipped also exist as queries in FeedbackRepository (existsAnswered, existsSkipped), and must be kept in step with the property. The FeedbackRepositoryIntegrationTest checks them against it on a real MongoDB, and runs with the integrationTests task. The other lookups sort by requested and are backed by the compound index {userId, definitionId, revision, requested}. Feedbacks are never deleted, and they are not part of the user data purge.

Scheduling Modes

"The last request" below is the requested time of the user’s latest feedback of the current revision, whatever became of it; with none, the period starts at the beginning of time. A form the user ignored therefore follows its schedule like any other, and only a postponed one is served ahead of it. A definition missing the setting its mode needs is treated as due.

schedulingMode Due when

`schedulingMode`	Due when
`Periodic`	`periodicInterval` — a day, a week or a month — has passed since the last request. `AfterEveryConversation` has no waiting time.
`UsageBased`	`usageBasedThreshold` is reached since the last request, counted either in engaged sessions or in minutes. An engaged session has at least five turns and is looked up among the user’s 100 most recent sessions. Minutes are the user’s active duration from `EDRService`, for the tenant of the definition’s project.
`Intelligent`	Since the last request, the user has had at least five engaged sessions and spent at least 45 minutes and the first session in that period is at least seven days old.
`EvaluationBased`	The user has not answered the definition and `evaluationCondition` is satisfied — see below.

Periodic

periodicInterval — a day, a week or a month — has passed since the last request. AfterEveryConversation has no waiting time.

UsageBased

usageBasedThreshold is reached since the last request, counted either in engaged sessions or in minutes. An engaged session has at least five turns and is looked up among the user’s 100 most recent sessions. Minutes are the user’s active duration from EDRService, for the tenant of the definition’s project.

Intelligent

Since the last request, the user has had at least five engaged sessions and spent at least 45 minutes and the first session in that period is at least seven days old.

EvaluationBased

The user has not answered the definition and evaluationCondition is satisfied — see below.

Evaluation-Based Scheduling

The condition is the one navigation specifications use, Condition: an evaluation insight or a featured evaluation result that reaches a value, or stays at or under it. A featured condition names its evaluation by evaluationRef, so it holds for every revision of that evaluation; a condition stored before that field existed is first resolved to the ref of the evaluation revision it was saved against (withEvaluationRefs, with the refs from EvaluationQueryService.legacyEvaluationRefs). Its matches takes two lookups, which here are filled from all of the user’s results in the project by EvaluationQueryService:

insight values from EvaluationInsightResult, keyed by the insight ref. A result saved without a ref is matched through its insight definition, the way EvaluationService does it;
featured values from EvaluationResult, keyed by the evaluation ref. A result saved without a ref is matched through its evaluation definition.

This differs from navigation, where NavigationStatusService persists the state an evaluation run moves an entry to when the run finishes, and reads the same lookups on a read (in a session only at its start) only while an entry could still move up, to catch up on earlier evaluations. Feedback evaluates at request time, so it reads two result collections per request; both are queried by project and user, which the {projectId, userId} indexes created on those collections serve. They are not declared on the models.

The form is due for as long as the condition holds and the user has not answered; skipping or answering ends it.

Question Selection

The entries of a specification are ordered by Entry.order. A SingleQuestion entry carries all of its questions. A Carousel entry carries one question out of its pool: the one after the pool question the user answered last, wrapping around to the first. The answered question comes from the user’s latest feedback of the current revision that has an answer to a question of the pool; with none, or when that question left the pool, the first question is used. A postponed or ignored form keeps its question, because nothing has been answered in the meantime.

The pool order is the order of the entry’s questions list.