Core Deployment

Helm Charts

Everything needed to deploy the platform lives in deploy/. Each service has its own chart under deploy/{service}/helm; the supporting infrastructure is either a chart in the same tree or a third-party chart referenced from the helmfile.

Path Chart

Path	Chart
`deploy/engine/helm`	`promethist-engine` — the platform server. The chart that matters most; detailed below.
`deploy/bridge/helm`	`promethist-bridge` — the automated conversation testing service.
`deploy/relay/helm`	`promethist-relay` — the Socket.IO relay.
`deploy/elasticsearch/helm`	Elasticsearch and Kibana, deployed through the Elastic Cloud on Kubernetes operator.
`deploy/neo4j/helm`	Neo4j, as a StatefulSet with a persistent volume.
`deploy/activemq/helm`	Apache ActiveMQ Artemis broker. Has its own `README.md` and `TESTING.md`.
`deploy/otel-collector/helm`	OpenTelemetry collector configuration.
`deploy/model-serving/triton/helm`	`promethist-triton` — NVIDIA Triton Inference Server for embeddings, animation and self-hosted models. Includes its own image build.
`deploy/model-serving/kserve/helm`, `deploy/model-serving/openllm/helm`	KServe serving runtimes and a JVM-based model server.
`deploy/unreal/helm`	Unreal Engine rendering service.

deploy/engine/helm

promethist-engine — the platform server. The chart that matters most; detailed below.

deploy/bridge/helm

promethist-bridge — the automated conversation testing service.

deploy/relay/helm

promethist-relay — the Socket.IO relay.

deploy/elasticsearch/helm

Elasticsearch and Kibana, deployed through the Elastic Cloud on Kubernetes operator.

deploy/neo4j/helm

Neo4j, as a StatefulSet with a persistent volume.

deploy/activemq/helm

Apache ActiveMQ Artemis broker. Has its own README.md and TESTING.md.

deploy/otel-collector/helm

OpenTelemetry collector configuration.

deploy/model-serving/triton/helm

promethist-triton — NVIDIA Triton Inference Server for embeddings, animation and self-hosted models. Includes its own image build.

deploy/model-serving/kserve/helm, deploy/model-serving/openllm/helm

KServe serving runtimes and a JVM-based model server.

deploy/unreal/helm

Unreal Engine rendering service.

Cluster add-ons that are applied directly rather than through a chart of their own (GPU operator, device plugins, local path provisioner) are in deploy/drivers/ and the deploy/values-*.yaml files.

Engine Chart

deploy/engine/helm deploys the engine as a Deployment behind a Service and an nginx Ingress.

Values

Value Default Description

Value	Default	Description
`replicas`	`3`	Pod count. With more than one replica the chart adds required pod anti-affinity by hostname, so replicas land on different nodes.
`app.name` / `app.suffix`	`engine` / —	Compose the host name: `{name}[-{suffix}].{domain}`.
`app.domain` / `app.domainSecondary`	`promethist.ai` / —	Primary and optional secondary ingress domain. The secondary domain gets its own Ingress with identical routing.
`app.image` / `app.tag`	registry image / `latest`	Container image and tag. `app.tag` is also published as `app.version` in the rendered configuration.
`app.containerPort`	`9010`	HTTP port. The Socket.IO port `9011` is fixed.
`app.javaOpts`	heap and direct memory limits	JVM options. The default caps heap at 40% of the container memory (2 GiB of the 5 Gi limit) and direct memory at 1 GiB — see Memory Budget.
`app.javaToolOpts` / `app.otelExporter` / `app.otelMetricsExporter`	OpenTelemetry agent and collector endpoints	Applied when `app.serviceMonitor` is enabled.
`app.serviceMonitor`	`true`	Enables the OpenTelemetry environment block and renders a Prometheus Operator `ServiceMonitor` scraping `/actuator/prometheus`.
`app.localConfigName`	`promethist-engine`	Name of the secret (or CSI-projected volume key) holding `application-default.properties`.
`app.storage`	`promethist` / `promethist-engine`	Identity prefix and ref of the resource storage container.
`app.resources`	5 Gi memory	Requests and limits. `values-develop.yaml` lowers the request for development environments.
`aws`, `azure`, `gcp`, `firebase`	enabled	Toggle the credential wiring for each provider. `azure.enabled` switches the secret volume to the Secrets Store CSI driver; `gcp.enabled` mounts the Google key and sets `GOOGLE_APPLICATION_CREDENTIALS`; `aws.enabled` maps the `aws-credentials` secret into the environment.
`canary.enabled` / `canary.weight`	`false` / `10`	Renders an additional canary Ingress with an nginx traffic-split annotation and a `-canary` host.
`spotInstance`	`false`	Schedules the pods onto spot nodes — used by ephemeral environments.
`clusterIssuer` / `ingressClassName` / `imagePullSecrets`	`letsencrypt` / `nginx` / `promethistai-registry`	Certificate issuer, ingress class and image pull secret.

replicas

3

Pod count. With more than one replica the chart adds required pod anti-affinity by hostname, so replicas land on different nodes.

app.name / app.suffix

engine / —

Compose the host name: {name}[-{suffix}].{domain}.

app.domain / app.domainSecondary

promethist.ai / —

Primary and optional secondary ingress domain. The secondary domain gets its own Ingress with identical routing.

app.image / app.tag

registry image / latest

Container image and tag. app.tag is also published as app.version in the rendered configuration.

app.containerPort

9010

HTTP port. The Socket.IO port 9011 is fixed.

app.javaOpts

heap and direct memory limits

JVM options. The default caps heap at 40% of the container memory (2 GiB of the 5 Gi limit) and direct memory at 1 GiB — see Memory Budget.

app.javaToolOpts / app.otelExporter / app.otelMetricsExporter

OpenTelemetry agent and collector endpoints

Applied when app.serviceMonitor is enabled.

app.serviceMonitor

true

Enables the OpenTelemetry environment block and renders a Prometheus Operator ServiceMonitor scraping /actuator/prometheus.

app.localConfigName

promethist-engine

Name of the secret (or CSI-projected volume key) holding application-default.properties.

app.storage

promethist / promethist-engine

Identity prefix and ref of the resource storage container.

app.resources

5 Gi memory

Requests and limits. values-develop.yaml lowers the request for development environments.

aws, azure, gcp, firebase

enabled

Toggle the credential wiring for each provider. azure.enabled switches the secret volume to the Secrets Store CSI driver; gcp.enabled mounts the Google key and sets GOOGLE_APPLICATION_CREDENTIALS; aws.enabled maps the aws-credentials secret into the environment.

canary.enabled / canary.weight

false / 10

Renders an additional canary Ingress with an nginx traffic-split annotation and a -canary host.

spotInstance

false

Schedules the pods onto spot nodes — used by ephemeral environments.

clusterIssuer / ingressClassName / imagePullSecrets

letsencrypt / nginx / promethistai-registry

Certificate issuer, ingress class and image pull secret.

Memory Budget

The container memory limit has to hold more than the Java heap. Under production load an engine replica uses:

Part Size Notes

Part	Size	Notes
Heap	2 GiB	`MaxRAMPercentage=40` of the 5 Gi limit. Under load G1 commits the whole maximum, and live data after GC reaches 1.4–1.8 GiB during the day.
Direct memory	~0.7 GiB	Netty’s pooled buffers. They grow during the first hours and then level off; the cap is `MaxDirectMemorySize=1g`.
Non-heap	~0.4 GiB	Metaspace and code cache.
Other native memory	~0.75 GiB	Thread stacks (400–500 threads), GC structures and malloc arenas. The JVM metrics do not show it; it is the container RSS minus the three parts above.

Heap

2 GiB

MaxRAMPercentage=40 of the 5 Gi limit. Under load G1 commits the whole maximum, and live data after GC reaches 1.4–1.8 GiB during the day.

Direct memory

~0.7 GiB

Netty’s pooled buffers. They grow during the first hours and then level off; the cap is MaxDirectMemorySize=1g.

Non-heap

~0.4 GiB

Metaspace and code cache.

Other native memory

~0.75 GiB

Thread stacks (400–500 threads), GC structures and malloc arenas. The JVM metrics do not show it; it is the container RSS minus the three parts above.

That adds up to about 3.9 GiB, or 4.2 GiB with direct memory at its cap. With the former 4 Gi limit and a 50% heap, replicas ran at the limit and the kernel OOM-killed them. When you change the heap percentage, the direct memory cap or the limit, keep heap plus direct memory at least 1.5 GiB below the limit. Development environments are the one exception: values-develop.yaml keeps a 4 Gi limit with the default JVM options, so 1.6 GiB of heap plus 1 GiB of direct memory falls slightly short of this margin, which their low load tolerates. The parts are exported as jvm_memory_committed_bytes, jvm_buffer_memory_used_bytes{id="direct"} and, for the whole container, container_memory_working_set_bytes.

Templates

Template Renders

Template	Renders
`deployment.yaml`	Deployment with a rolling update strategy, both container ports (`9010`, `9011`), the configuration and secret volume mounts, and Actuator-based probes: `/actuator/health/liveness` and `/actuator/health/readiness`.
`configmap.yaml`	`application.properties` derived from the values — domain, namespace, version, storage settings, Mongo database name and graceful shutdown configuration.
`service.yaml`	ClusterIP Service exposing port `80` → `9010` and port `81` → `9011`.
`ingress.yaml`	Ingress with TLS, WebSocket support, cookie affinity, disabled buffering and unlimited body size. Routes `/socket.io` and `/socket/io` to the Socket.IO port and everything else to the HTTP port. Also creates a `-stable` host alias, and the canary Ingress when enabled.
`ingress-secondary.yaml`	The same routing for `app.domainSecondary`, when set.
`servicemonitor.yaml`	Prometheus Operator ServiceMonitor, when `app.serviceMonitor` is enabled.

deployment.yaml

Deployment with a rolling update strategy, both container ports (9010, 9011), the configuration and secret volume mounts, and Actuator-based probes: /actuator/health/liveness and /actuator/health/readiness.

configmap.yaml

application.properties derived from the values — domain, namespace, version, storage settings, Mongo database name and graceful shutdown configuration.

service.yaml

ClusterIP Service exposing port 80 → 9010 and port 81 → 9011.

ingress.yaml

Ingress with TLS, WebSocket support, cookie affinity, disabled buffering and unlimited body size. Routes /socket.io and /socket/io to the Socket.IO port and everything else to the HTTP port. Also creates a -stable host alias, and the canary Ingress when enabled.

ingress-secondary.yaml

The same routing for app.domainSecondary, when set.

servicemonitor.yaml

Prometheus Operator ServiceMonitor, when app.serviceMonitor is enabled.

Prerequisites

The chart expects the following to exist in the target namespace:

a secret named after app.localConfigName (default promethist-engine) containing application-default.properties, and — when the corresponding toggle is on — the Google service account key and Firebase admin key. In Azure clusters this is projected from Key Vault by the azure-secret-provider SecretProviderClass (deploy/secret-provider.yaml);
an aws-credentials secret when aws.enabled is set;
the image pull secret named by imagePullSecrets;
a reachable OpenTelemetry collector when app.serviceMonitor is enabled;
the external data stores the engine connects to (MongoDB, Elasticsearch, optionally Neo4j) and a Keycloak realm, configured through application-default.properties.

Installing

helm -n $NAMESPACE upgrade --install promethist-engine deploy/engine/helm \
  --set app.namespace=$NAMESPACE \
  --set app.domain=promethist.dev \
  --set app.image=registry.gitlab.com/promethistai/promethist/engine \
  --set app.tag=$IMAGE_TAG

For a development environment add -f deploy/engine/helm/values-develop.yaml --set replicas=1. The same invocation is what CI runs — see Core Deployment.

The chart is also packaged and published to the internal chart repository on every release.

Bridge Chart

deploy/bridge/helm deploys the bridge service (container port 9013) with a Deployment, Service and Ingress. In addition to the service itself it renders one CronJob per entry in e2eCron.jobs, which periodically call the bridge’s own run endpoints to execute scheduled end-to-end conversation tests. The production value files (values-production-eu.yaml, values-production-us.yaml) define those schedules per region; each job may carry a variable group and a bearer token from e2eCron.authSecret.

See Bridge for what those runs actually do.

Relay Chart

deploy/relay/helm deploys the relay service (container port 9012) with a Deployment, Service and Ingress. It is the simplest of the three service charts — no configuration files, no secrets and the smallest resource footprint.

Platform Infrastructure with Helmfile

deploy/helmfile.yaml.gotmpl assembles the shared platform infrastructure — Keycloak, PostgreSQL, MinIO, the Elastic stack and the telemetry stack (OpenTelemetry collector, Prometheus, Grafana, Tempo, Loki, Promtail) — alongside the engine release.

Environments are defined at the top of the helmfile (production, preview, issue, develop, local); each layers environments/defaults.yaml.gotmpl, its own environments/{name}/values.yaml.gotmpl and environments/secrets.yaml.gotmpl. Per-release values are templates under deploy/configs/.

cd deploy
helmfile apply --environment develop

Individual releases can be selected by label, which is the usual way to touch one component only:

helmfile apply --environment develop -l name=engine -l name=minio

Which components are installed is driven by the components.* toggles of the environment — all, runtime, telemetry, analytics, keycloak, postgresql, minio. Values can be overridden ad hoc with --state-values-file or --state-values-set.

The helmfile expects credentials for PostgreSQL, Keycloak, Azure storage, Grafana and MinIO in the environment; deploy/README.md lists them together with the Keycloak client setup and the steps for adding a new environment.

The engine can be deployed both through the helmfile and, as CI does, by a direct helm upgrade --install of deploy/engine/helm. Both paths use the same chart, but they are separate release histories — pick one per environment rather than mixing them.