Skip to main content
The Compose sandbox and the sample conf/config.toml favor a fast start. Work through this page before a deployment holds real data.

​
Replace the sandbox defaults

​
Checklist

1

Provision PostgreSQL and object storage

Use PostgreSQL 17 with automated backups. Create a dedicated database and user for MoleSignal.Keep [meta_database].max_connections multiplied by the number of nodes below the max_connections limit of PostgreSQL.Create a private bucket in the same region as the nodes. Do not add lifecycle rules that expire objects. MoleSignal applies retention and deletes objects itself.
2

Generate and store the cipher key

MS_CIPHER_KEY is the root key for the secrets that MoleSignal stores in PostgreSQL: API tokens, share links, remote cluster tokens, notification connector credentials, pipeline connector credentials, Mole Agent provider keys, synthetic monitoring secrets, and organization field keys. Generate it once:
Store the value in a secret manager and give the same value to every node. Back it up separately from the database. A database restore without the key leaves those secrets unreadable.
Never replace the key in place. Data that was encrypted with the old key becomes unreadable. To rotate the key, stop all writes, export the new value as MS_CIPHER_KEY, and run the offline command with the old key:
The command re-encrypts every affected column and prints the number of rows per table. Verify the counts before retiring the old key.
A deployment that started without MS_CIPHER_KEY used an all-zero development key. Pass AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= as --old-key to move such a deployment to a real key.
3

Secure the root account

The root account is created on the first start of an empty database, from [auth].root_email and [auth].root_password. Set both before that first start. Later changes to root_password do not change the password of the existing account.Password reset and Account settings → Security & sign-in → Change password work by email. They need [notify.smtp] and [http].external_url. Without them, no email is sent. Configure both before relying on them.The account with root_email becomes the platform administrator at every start. Keep root_email set to an existing user.
4

Keep secrets out of the repository

Pass the database DSN, the object storage keys, and MS_CIPHER_KEY through environment variables or a secret manager. Use MS_META_DATABASE_DSN, MS_OBJECT_STORE_ACCESS_KEY, and MS_OBJECT_STORE_SECRET_KEY. Leave the matching keys in config.toml empty.Optionally pin the JWT signing secret with MS_AUTH_JWT_SECRET_OVERRIDE. Otherwise MoleSignal generates one in PostgreSQL on the first start. Rotate it with POST /api/v1/auth/jwt/rotate, which needs the sys.settings.manage permission. Tokens signed with the retired secret stay valid for 24 hours.
5

Expose the right ports

Set [http].external_url to the public address. Onboarding snippets, password reset emails, and status page domains use it.
6

Terminate TLS and configure the proxy

Terminate TLS in a reverse proxy or enable [http.tls] for automatic ACME certificates.Set the proxy body limit to at least 2 MiB, because MoleSignal accepts request bodies up to that size. For /api/v1/query/stream, disable response buffering and allow long reads. The reference nginx configuration of the project uses a 600-second timeout for that path.Restrict /metrics to internal callers. Expose the intake paths /api/v1/_bulk, /api/v1/_kinesis_firehose, /api/v1/_cloudflare, and /api/v1/_heroku only when a sender needs them.
7

Protect the write-ahead log

Mount a persistent local volume at [wal].dir on every node that runs the intake role. The default flush_strategy = "batch" with sync_level = "data" syncs every 64 records or 50 ms. Use every_write with sync_level = "all" when no record may be lost at the cost of latency.Give the process time to drain on shutdown. Set the termination grace period of the orchestrator above [node].drain_timeout_secs, which defaults to 30 seconds.
8

Set retention

The default retention is 30 days, set in Settings → Data retention of the _sys organization. Data older than that is deleted from every stream without its own retention. Raise the value or set retention per stream before data accumulates. See Streams.
9

Size the alert and compactor roles

The alert and compactor roles can run several instances, which divide the work. The alert nodes divide it by rule, report, pipeline, and connector, and one primary node runs the jobs that must run once. The compactor nodes divide it by stream and by organization. Set a unique [node].id on every node and keep the clocks synchronized. A standalone node already includes both roles. Use standalone unless splitting roles is required, and read Role-based deployment before splitting.
10

Monitor the deployment

Scrape /metrics with Prometheus. Probe GET /api/v1/healthz, which is public and answers 503 while the WAL replays or the object store is unreachable. The container image already runs this check as its HEALTHCHECK.Keep [telemetry.self_collect] enabled so the service records its own metrics, traces, and profiles. Create alert rules on those metrics. See Self-observability.
11

Plan backups and upgrades

Back up PostgreSQL, the object store, and the cipher key as described in Backup and restore. Rehearse a restore before relying on it. Read Upgrade before the first version change.

​
Verify before go-live

  • GET /api/v1/healthz returns 200 from the public address.
  • The startup log reports object_store_credentials_source and does not print MS_CIPHER_KEY missing.
  • A test record sent with a service account token appears in a query.
  • The default sign-in [email protected] with password admin is rejected.
  • Restoring the latest PostgreSQL backup into a scratch database succeeds.
Last modified on October 26, 2026