Configuration - MoleSignal Docs

MoleSignal reads one TOML file and MS_ environment variables. Start from the sample conf/config.toml that ships with each release. The sample lists every setting, and this page documents each one.

molesignal --config /etc/molesignal/config.toml

How configuration is loaded

Settings apply in this order. Later sources override earlier ones.

Built-in defaults.
The TOML file passed with --config.
MS_ environment variables.

Restart the process after changing a setting. A running process logs changes to the file and does not apply them.

Most sections ignore keys that they do not define, so a typo has no effect and produces no warning. The sections [meta_database], [object_store] (including [object_store.retry]), [telemetry] (including [telemetry.trace] and [telemetry.self_collect]), [apm], [probe], [otlp_grpc], [notify], and [scheduled_reports.renderer] reject unknown keys, so a typo stops the process at startup.

The Default column in the reference tables shows the value from the sample conf/config.toml. Where the built-in value used for a missing key differs, the description says so.

Environment variables

Every setting has an environment variable. Write the MS_ prefix, then each level of the setting path in upper case, joined by single underscores.

Setting	Environment variable
`[http].port`	`MS_HTTP_PORT`
`[meta_database].dsn`	`MS_META_DATABASE_DSN`
`[cluster].advertise_addr`	`MS_CLUSTER_ADVERTISE_ADDR`
`[node].roles`	`MS_NODE_ROLES`

Write lists in TOML syntax, for example MS_NODE_ROLES='["intake","querier"]'. Names that contain a dot or a double underscore are not recognized. These variables are read separately from the settings:

Variable	Purpose
`MS_CIPHER_KEY`	Root key for the secrets that MoleSignal encrypts in PostgreSQL. Provide 32 random bytes encoded as base64. Without it, the process falls back to an all-zero development key and logs a warning. Never change the key without running `rewrap-kek`. See Production checklist.
`MS_AUTH_JWT_SECRET_OVERRIDE`	Pins the JWT signing secret instead of the one generated in PostgreSQL on first start.
`MS_SELF_TELEMETRY_CLUSTER_TOKEN`	Shared secret that authenticates the trace candidates and self-telemetry that role-split nodes forward to each other. Set the same value on every role-split node while trace capture (on by default) or `[telemetry.self_collect]` is enabled. A node without the `intake` role refuses to start without it.
`MS_AGENT_OPENAI_API_KEY`, `MS_AGENT_ANTHROPIC_API_KEY`, `MS_AGENT_COMPATIBLE_API_KEY`	Development fallback API keys for Mole Agent model providers. Each has a matching `_BASE_URL` variable. Configure production providers in the product instead.
`RELEASE_CHANNEL`	Labels the deployment maturity: `alpha`, `beta`, `rc`, or `stable`.
`RUST_LOG`	Overrides `[telemetry].log_level` when set.
`AZURE_STORAGE_ACCOUNT`, `AZURE_STORAGE_ACCESS_KEY`	Azure Blob credentials for the `azure` object backend.
`GOOGLE_APPLICATION_CREDENTIALS`	Path to the service account JSON for the `gcs` object backend.

Minimal production settings

A production node needs these settings at a minimum. The rest of the file can stay at its defaults.

[http]
external_url = "https://molesignal.example.com"

[meta_database]
dsn = "postgres://molesignal:<password>@db.internal:5432/molesignal"

[object_store]
backend = "s3"
bucket = "molesignal"
region = "us-east-1"
endpoint = ""          # empty for AWS S3, the service URL for S3-compatible storage
access_key = ""        # prefer MS_OBJECT_STORE_ACCESS_KEY
secret_key = ""        # prefer MS_OBJECT_STORE_SECRET_KEY

[wal]
dir = "/var/lib/molesignal/wal"

[auth]
root_email = "[email protected]"
root_password = "<initial password>"

Also set MS_CIPHER_KEY in the environment. See the Production checklist for the full list of steps before going live.

Node

[node] selects the roles that a process runs. See Role-based deployment.

Key	Default	Description
`roles`	`["standalone"]`	Roles to run: `standalone`, `router`, `intake`, `querier`, `compactor`, `alert`. `standalone` runs all of them in one process.
`id`	empty	Node id, at most 64 characters. An empty value uses the OS hostname. Set a distinct id for every process when several run on one host.
`drain_timeout_secs`	`30`	Time to wait for pending intake data to flush after `SIGTERM` or `SIGINT`. The process exits when the time elapses.

Listeners

HTTP

Key	Default	Description
`bind`	`0.0.0.0`	Listen address.
`port`	`5080`	HTTP port when `[http.tls]` is disabled.
`gzip`	`true`	Compresses the web UI files the process serves, with gzip, for clients that accept it. API responses are not compressed. Set `false` when a reverse proxy compresses.
`external_url`	empty	Public origin, for example `https://molesignal.example.com`. When empty, the web UI uses the browser origin. Generated onboarding snippets use this value, and status page custom domains use it as the CNAME target. Mole Agent’s account links and chat links in IM use it, and the Microsoft Teams callback address is under it, which then needs it to be https; see Mole Agent in IM. Restart after a change.

HTTP TLS and ACME

[http.tls] starts a TLS listener with automatic certificates from an ACME service such as Let’s Encrypt.

Key	Default	Description
`enabled`	`false`	Master switch. All other keys are ignored when `false`.
`plain_port`	`80`	Plain HTTP port for ACME HTTP-01 challenges and the redirect to HTTPS.
`port`	`443`	HTTPS port.
`acme_directory`	`production`	`production`, `staging` (Let’s Encrypt staging), or a custom directory URL.
`account_email`	empty	ACME account email. Let’s Encrypt requires it.
`key_storage_dir`	`/var/lib/molesignal/acme`	Directory for the ACME account key and the per-domain keys.
`issue_poll_secs`	`60`	Interval for scanning domains that wait for a certificate.
`renewal_retry_secs`	`21600`	Interval for renewal retries and expiry scans.

Internal gRPC

[grpc] carries node-to-node traffic. Keep the port on a trusted network.

Key	Default	Description
`bind`	`0.0.0.0`	Listen address.
`port`	`5082`	Port. Also used by peers through `[cluster].advertise_addr`.
`max_message_size`	`32`	Largest gRPC message in MiB.

OTLP gRPC

[otlp_grpc] configures the OTLP gRPC receiver. The receiver has no enable switch and listens on every node that runs the intake role, including standalone.

Key	Default	Description
`bind`	`0.0.0.0`	Listen address.
`port`	`4317`	Port.
`max_message_size`	`32`	Largest gRPC message in MiB.

Arrow Flight SQL

[flight_sql] serves SQL clients such as DBeaver, JDBC, and ADBC. Every call requires an API token. See Database clients.

Key	Default	Description
`enabled`	`false`	Starts the listener.
`bind`	`0.0.0.0`	Listen address.
`port`	`5083`	Port.
`default_lookback_hours`	`24`	Lookback window in hours for SQL that has no time filter.
`max_message_size`	`32`	Largest message in MiB.

Probe Agent

[probe] configures the registration and control endpoints for Probe Agents. The endpoints start on standalone and alert nodes. Registration uses server-authenticated TLS, and the control stream uses short-lived mutual TLS identities. See Synthetic monitoring.

Key	Default	Description
`bind`	`0.0.0.0`	Listen address.
`register_port`	`5084`	Registration port. Must differ from `control_port`.
`control_port`	`5085`	Control stream port.
`register_endpoint`	`https://localhost:5084`	Public `https://` URL that agents use to register. The one-time register command prints it. Set it behind a proxy or load balancer.
`control_endpoint`	`https://localhost:5085`	Public `https://` URL of the control stream.
`server_names`	`["localhost", "127.0.0.1"]`	DNS names and IP addresses in the server certificate. Include the public host names of the endpoints.
`certificate_days`	`30`	Validity of issued certificates, from 1 to 365.
`max_message_size`	`32`	Largest message in MiB.

pprof

[profiling] starts a node-local pprof listener for diagnostics.

Key	Default	Description
`enabled`	`false`	Starts the listener.
`bind`	`127.0.0.1`	Listen address. Must be a loopback address unless `allow_remote` is `true`.
`port`	`5084`	Port. Change it when the Probe Agent registration port uses the same value on the same host.
`allow_remote`	`false`	Allows a non-loopback `bind`. Remote exposure can leak runtime information.

Storage

Metadata database

[meta_database] configures the PostgreSQL database that holds metadata, identities, the file catalog, and product resources.

Key	Default	Description
`dsn`	`postgres://molesignal:molesignal@localhost:5432/molesignal`	PostgreSQL connection string. Replace the sample credentials.
`min_connections`	`2`	Connections opened at startup. Limited to `max_connections`.
`max_connections`	`16`	Connection pool limit.

MoleSignal applies its embedded schema migrations to the database on start. See Upgrade.

Object storage

[object_store] configures the store for Parquet files, search indexes, and profile blobs. Backends: local, s3, azure, gcs. Use s3 for AWS S3 and S3-compatible services such as RustFS, Cloudflare R2, and Alibaba Cloud OSS.

Key	Default	Description
`backend`	`local`	`local`, `s3`, `azure`, or `gcs`.
`root`	`./data/objects`	Directory of the `local` backend. Ignored by the other backends.
`bucket`	empty	Bucket name for `s3` and `gcs`, container name for `azure`. Required for those backends.
`region`	empty	S3 region. S3-compatible services state the value that they expect.
`endpoint`	empty	S3 endpoint URL. Leave empty for AWS S3. Use `http://host:9000` style URLs for RustFS and similar services.
`path_style`	`false`	`false` uses virtual-hosted addressing, which AWS S3 and Huawei Cloud OBS need. Set `true` for path-style addressing, which RustFS and most self-hosted services need.
`access_key`	empty	S3 access key, or the Azure storage access key.
`secret_key`	empty	S3 secret key.
`credentials_file`	empty	Path of a file with the lines `access_key=<value>` and `secret_key=<value>`. See the order below.
`multipart_threshold_size`	`32`	Objects at least this large, in MiB, upload in parts. Manifest objects and writes with a version condition always upload in one request. The minimum is 1.
`multipart_part_size`	`8`	Size of one part of a multipart upload, in MiB, from 5 to 1024. Very large objects use larger parts so that an upload stays within 10,000 parts.
`range_threshold_size`	`16`	A read of a byte range at least this large, in MiB, splits into chunk requests that run in parallel. The minimum is 1.
`range_chunk_size`	`8`	Size of one chunk request, in MiB, from 1 to 1024.
`max_concurrency`	`8`	Concurrent object store operations per process.
`op_timeout_secs`	`30`	Timeout for waiting for a concurrency slot and for one backend operation.
`health_probe_interval_secs`	`30`	Interval of the background write, read, and delete probe. The minimum is 5.

The S3 and Azure clients take their credentials from the first of these sources that provides both an access key and a secret key:

The variables MS_OBJECT_STORE_ACCESS_KEY and MS_OBJECT_STORE_SECRET_KEY.
The file that credentials_file names.
The keys access_key and secret_key.

The startup log reports the winner as object_store_credentials_source: env, file, inline, or none. Prefer an environment variable or a credentials file over config.toml for the secret. Every part of a multipart upload and every chunk of a ranged read is an object store operation that takes a concurrency slot, retries on its own, and shows in the object_store_* metrics. One transfer holds at most half of the slots, so large objects do not starve other operations. Transient errors of the cloud backends are retried with exponential backoff:

Key	Default	Description
`[object_store.retry].max_attempts`	`4`	Total attempts per operation.
`[object_store.retry].base_backoff_ms`	`100`	First wait.
`[object_store.retry].max_backoff_ms`	`5000`	Largest wait.
`[object_store.retry].jitter_ratio`	`0.2`	Random jitter applied to each wait, from `0` to `1`.

Write-ahead log

[wal] configures the write-ahead log of nodes that run the intake role. Put the directory on a persistent local volume. See Backup and restore.

Key	Default	Description
`dir`	`./data/wal`	WAL directory.
`segment_size`	`256`	Segment size in MiB. A full segment starts a new one.
`flush_strategy`	`batch`	When to `fsync`. `none` never syncs and relies on the page cache. `every_write` syncs each record, which gives the strongest durability and the highest latency. `batch` syncs after `batch_max_pending` records or `batch_max_delay_ms`.
`sync_level`	`data`	Depth of each sync. `none` skips it. `data` syncs file data. `all` also syncs metadata and the parent directory.
`batch_max_pending`	`64`	Records that trigger a sync in `batch` mode.
`batch_max_delay_ms`	`50`	Longest time between syncs in `batch` mode.
`encrypt`	`false`	Encrypts each record payload with the root key (`MS_CIPHER_KEY`) before writing it to disk. Absent from the sample file.
`max_replay_size`	`512`	Memory limit for startup replay in MiB. Replay flushes once the limit is exceeded.

Intake

[intake] controls the in-memory buffers of an intake node.

Key	Default	Description
`buffer_max_size`	`256`	Hard limit of the raw buffer of one stream in MiB. The adaptive threshold never exceeds it.
`max_buffer_memory_size`	`1024`	Total memory reserved by the active and detached buffers of the process, in MiB.
`flush_interval_secs`	`30`	Longest time between flushes.
`flush_parallelism`	`4`	Concurrent flushes.

[intake.rotation] sizes the Parquet files adaptively:

Key	Default	Description
`adaptive_enabled`	`true`	Adjusts the next raw threshold of each stream from the encoded-to-raw ratio of finished files.
`target_file_size`	`128`	Target size of encoded Parquet files in MiB.
`min_buffer_size`	`16`	Lower limit of the adaptive threshold in MiB.
`ewma_alpha`	`0.2`	Smoothing factor of the ratio, greater than 0 and at most 1.

[intake.prometheus] limits Prometheus remote_write input:

Key	Default	Description
`max_labels_per_series`	`64`	Labels per series, not counting `__name__`.
`max_label_name_length`	`128`	Longest label name in UTF-8 bytes.
`max_label_value_length`	`2048`	Longest label value in UTF-8 bytes.
`max_samples_per_batch`	`16384`	Samples in one internal batch.

[intake.prometheus.cardinality] admits series in the process that receives the remote_write request, before the batch is forwarded to an intake node. Each process counts the series it receives, so with several routers every router applies the limits on its own:

Key	Default	Description
`enabled`	`true`	Applies the limits.
`max_active_series_per_process`	`1000000`	Active series per process.
`max_new_series_per_minute`	`20000`	New series that the process accepts per minute.
`idle_ttl_secs`	`900`	A series idle for this long releases its capacity.

The limits per organization and per metric are a platform setting: Settings → Metric cardinality in the _sys organization. Every node reads them again every 30 seconds and applies them to the series that it counts.

Compaction and retention

[compactor] merges files and applies retention. It runs on standalone and compactor nodes.

Key	Default	Description
`interval_secs`	`300`	Time between compaction rounds.
`target_size`	`512`	Largest total size, in MiB, of the files one merge combines, from `1` to `65536`. A merged file is about this size or smaller. A merge reads its inputs a part at a time and uploads its output in parts, so its memory does not grow with this size. One merge reads at most 64 files; the rest are merged with its result in a later round.
`max_concurrent_groups`	`4`	Streams that one node processes at the same time in one round.

How long a stream without a retention of its own is kept is a platform setting: Settings → Data retention in the _sys organization, 30 days by default. Downsampling is set for each metrics stream on its Retention tab. See Streams.

Catalog maintenance

The [storage.*] sections control background maintenance of the file catalog and the object store.

Key	Default	Description
`[storage.catalog].seal_after_hours`	`48`	Seals a partition this many hours after it ends: writes its immutable manifest and the Bloom filters of its exact fields, which let lookups of an exact value skip the partition’s files. A sealed partition is no longer compacted, so the delay leaves time for the hours of a day to be merged. `0` disables sealing.
`[storage.catalog].manifest_cache_size`	`256`	Memory for the manifest cache in MiB.
`[storage.catalog].interval_secs`	`300`	Sealing interval.
`[storage.catalog].batch_size`	`64`	Partitions per round.
`[storage.index].enabled`	`true`	Builds missing search indexes and rebuilds the ones written in an older format, a few per round. Indexes of sealed partitions are rebuilt by the catalog maintenance and published with the next generation of their manifest.
`[storage.index].interval_secs`	`30`	Interval.
`[storage.index].batch_size`	`64`	Files per round.
`[storage.index].max_concurrency`	`2`	Concurrent index builds.
`[storage.index].retry_after_secs`	`300`	Wait before retrying a failed build.
`[storage.gc].enabled`	`true`	Deletes objects that the catalog no longer references.
`[storage.gc].interval_secs`	`30`	Interval.
`[storage.gc].grace_period_secs`	`3600`	Every object waits this long in the delete queue. Before the final delete, the catalog reference and the checksum are checked again.
`[storage.gc].lease_secs`	`900`	Lease on a claimed delete batch.
`[storage.gc].batch_size`	`128`	Objects per round.
`[storage.gc].max_concurrency`	`4`	Concurrent deletes.
`[storage.reconciler].enabled`	`true`	Compares the catalog with the object store.
`[storage.reconciler].interval_secs`	`300`	Interval.
`[storage.reconciler].batch_size`	`256`	Objects per round.
`[storage.reconciler].orphan_grace_secs`	`86400`	Age after which an unreferenced object counts as an orphan.
`[storage.reconciler].verify_checksums`	`true`	Verifies object checksums while reconciling.

Query

Limits and asynchronous jobs

Key	Default	Description
`[querier].concurrency`	`0`	Query engine parallelism. `0` uses the number of CPUs.
`[querier].max_scan_rows`	`100000000`	Largest number of rows that one query may read: the rows of the files that the query has to read plus the rows in memory buffers. A larger query is rejected with `400` before it reads data. A PromQL query applies the limit to each selector and counts the rows it reads of the selector’s metrics, stopping once it is over. `0` removes the limit.
`[querier].auto_async_threshold_rows`	`50000000`	Queries whose estimated rows exceed the value become asynchronous search jobs. `0` disables the conversion.
`[querier].estimate_throughput_per_sec`	`1000`	Conservative rows-per-second figure that the size estimate uses.
`[search].max_result_rows`	`0`	Row cap for queries without `LIMIT` that are not aggregations. `0` disables the cap. Absent from the sample file.
`[search_jobs].workers`	`2`	Search jobs that run at the same time on one node.
`[search_jobs].idle_poll_secs`	`2`	Initial fallback polling delay. It backs off while idle.
`[search_jobs].cleanup_interval_secs`	`3600`	Interval for removing expired jobs.

Admission control

[search.admission] limits concurrent queries per workgroup. All values default to 0 or empty, which means no limit. The section is absent from the sample file.

Key	Description
`default_max_concurrent`	Per-node limit for workgroups without an entry in `groups`.
`groups`	Table that maps a workgroup name to its per-node limit.
`role_map`	Table that maps a role (`owner`, `admin`, `editor`, `viewer`) to a workgroup. Unmapped roles use `default`.
`cluster_default_max_concurrent`	Soft cluster-wide limit for workgroups without an entry in `cluster_groups`.
`cluster_groups`	Table that maps a workgroup name to its cluster-wide soft limit.
`cluster_sync_interval_secs`	Interval for publishing node load. `0` uses 5.
`cluster_stale_secs`	Nodes silent for longer drop out of the cluster view. `0` uses 30.

Caches

Each layer is an LRU cache with a time-to-live. A capacity of 0 disables the layer. Hit ratios appear at /metrics as cache_<layer>_hit_ratio.

Section	Default capacity	Default TTL	Holds
`[cache.index_handle]`	`10000`	`600` s	Opened search index handles.
`[cache.query_result]`	`1000`	`60` s	Results of SQL over closed time windows.
`[cache.tantivy_result]`	`1000000`	`600` s	Term counts that prune files.
`[cache.tantivy_footer]`	`100000`	`3600` s	Index footers that save range reads.

Each section takes capacity and ttl_secs. [cache.parquet_metadata] keeps parsed Parquet footers, page indexes included, in memory. A query that opens a cached file skips the footer read. Its only key is max_size, in MiB, 256 by default. 0 disables it. [cache.object] caches byte ranges of remote objects on local disk. The local object backend bypasses it.

Key	Default	Description
`enabled`	`true`	Enables the cache. A `max_size` of `0` also disables it.
`root`	`./data/cache/objects`	Cache directory.
`max_size`	`10240`	Largest cache size in MiB, 10 GiB.
`block_size`	`4`	Block size in MiB, from 1 to 16.
`memory_size`	`256`	Verified blocks kept in memory, in MiB. Later reads of a block skip the disk. `0` keeps blocks on disk only.
`min_free_size`	`1024`	Free disk space to keep in MiB, 1 GiB.
`max_concurrent_fetches`	`16`	Concurrent block fetches. Must be greater than zero.
`max_concurrent_fetches_per_object`	`4`	Concurrent fetches for one object. Must be greater than zero.

When the cache grows past max_size or free disk space falls below min_free_size, the least recently used blocks go first. When the garbage collector deletes an object, the node that deletes it also drops the object’s blocks, provided it has read the object since it started. Other nodes evict those blocks like any other.

Incremental range cache

[search.stream_agg_cache] caches closed windows of range PromQL queries such as rate(metric[5m]), so a dashboard refresh recomputes only the newest window.

Key	Default	Description
`max_size`	`256`	Memory for the cache in MiB, estimated from the series and windows each query keeps. `0` disables the cache.
`ttl_secs`	`300`	Entry lifetime. Covers late data beyond `safe_lookback_secs`.
`safe_lookback_secs`	`300`	Only windows that end before now minus this value are cached.
`max_series_per_query`	`10000`	Series cached per fingerprint. `0` removes the limit.

Cluster and rate limits

Key	Default	Description
`[cluster].advertise_addr`	`127.0.0.1:5082`	Address of the internal gRPC port that peers use. Written to the `cluster_nodes` table. Set it to an address that every peer can reach.
`[cluster].heartbeat_interval_secs`	`5`	Heartbeat interval.
`[cluster].peer_timeout_secs`	`15`	A peer without a heartbeat for this long counts as gone. A new compactor node also waits this long before it takes work.
`[router.rate_limit].intake_qps`	`1000`	Intake requests per second for each organization on the `router` role. `0` removes the limit.
`[router.rate_limit].query_qps`	`100`	Query requests per second for each organization on the `router` role. `0` removes the limit.
`[router.rate_limit].burst_multiplier`	`2`	Token bucket capacity as a multiple of the rate.

The rate limit applies to the router role only, and each router replica counts separately. See Role-based deployment.

Alerting and email

Key	Default	Description
`[alert].eval_interval_secs`	`30`	Rule evaluation interval.
`[alert].dispatch_interval_secs`	`10`	Escalation dispatcher interval.
`[alert].eval_timeout_secs`	`10`	Timeout for one rule evaluation. A timeout increments `alert_rule_eval_timeout_total` and does not block the next round.
`[alert].default_ack_timeout_secs`	`300`	`ack_timeout_secs` of an escalation step that leaves the field out. The value is saved with the step, so a later change does not alter saved steps.

[notify.smtp] configures platform email such as sign-up approval. Notification channels of type email keep their own credentials as encrypted connectors. SMTP stays disabled while host is empty.

Key	Default	Description
`host`	empty	SMTP server. A non-empty value enables SMTP.
`port`	`587`	Port. The built-in value `0` falls back to `587`.
`username`	empty	Login name. Leave empty for no authentication.
`password`	empty	Login password. Used only when `username` is set.
`from`	`[email protected]`	Sender address. Required when SMTP is enabled. The built-in value is empty.
`tls`	`starttls`	`none`, `starttls`, or `tls`.
`timeout_secs`	`10`	Connection and send timeout. The built-in value `0` means 1 second.

Authentication

Key	Default	Description
`token_ttl_secs`	`86400`	Default lifetime of login tokens (JWT) and API tokens, 24 hours.
`root_email`	`[email protected]`	Email of the root account. The built-in value is empty.
`root_password`	`admin`	Password of the root account. The built-in value is empty. Replace the sample value.

MoleSignal creates the root account on the first start, when the database holds no users and both values are set. Changing root_password later does not change the password of an existing account. Change it in the web UI instead. At every start, the account with root_email becomes the platform administrator, so root_email must belong to an existing user. The JWT signing secret is generated on first start and stored in PostgreSQL. The [auth].jwt_secret key from older releases is ignored. Use MS_AUTH_JWT_SECRET_OVERRIDE to pin a secret, and POST /api/v1/auth/jwt/rotate to rotate it.

Self-observability

Process logs and self-collected telemetry use [telemetry]. See Self-observability.

Key	Default	Description
`[telemetry].log_level`	`info,tantivy=warn`	Log filter in `EnvFilter` syntax. The built-in value is `info`. `RUST_LOG` overrides it.
`[telemetry].log_format`	`text`	`text` or `json`.
`[telemetry].log_output`	`console`	`console` or `file`.
`[telemetry].log_directory`	`logs`	Directory of log files when `log_output` is `file`.
`[telemetry].log_rotation`	`daily`	`minutely`, `hourly`, `daily`, or `never`.
`[telemetry].log_max_files`	`7`	Rotated files to keep.

The last three keys are commented out in the sample file. The values shown apply when log_output is file.

Self-collected telemetry

[telemetry.self_collect] writes the metrics, traces, and profiles of the process into the reserved _molesignal stream family.

Key	Default	Description
`enabled`	`true`	Master switch. The built-in value is `false`. Traces also require `[telemetry.trace].enabled`.
`metrics_enabled`	`true`	Collects process metrics.
`metrics_interval_secs`	`15`	Metrics collection interval.
`queue_capacity`	`8192`	Bounded in-memory queue of each signal.
`batch_max_events`	`256`	Largest batch.
`batch_max_delay_ms`	`1000`	Longest wait to fill a batch.
`flush_timeout_secs`	`5`	Time to flush the queues on shutdown.
`profiles_enabled`	`false`	Collects process profiles.
`profile_kinds`	`["cpu"]`	`cpu`, `heap`, or both.
`profile_interval_secs`	`600`	Profile interval.
`profile_duration_secs`	`10`	Length of one CPU profile, from 1 to 120.

The _molesignal streams are created with a retention of 7 days. Change it on the stream’s Retention tab in the _sys organization; a restart keeps the value.

Trace capture and tail sampling

[telemetry.trace] controls the spans that the service records about itself and the tail sampler that decides which traces to keep.

Key	Default	Description
`enabled`	`true`	Default switch for capture and tail sampling. The runtime policy can still change while `force_disabled` is `false`.
`force_disabled`	`false`	Highest-priority deployment switch. The runtime policy at `/api/v1/system/telemetry` cannot override it.
`filter`	`info`	`EnvFilter` for span capture and export. Independent of `RUST_LOG` and `log_level`.
`deployment_environment`	`production`	Environment label. `development`, `dev`, and `test` use `development_sample_ratio`. Other values use `normal_sample_ratio`.
`normal_sample_ratio`	`0.10`	Base sampling ratio of normal traces outside development, from 0.0 to 1.0.
`development_sample_ratio`	`1.0`	Base sampling ratio in development environments, from 0.0 to 1.0.
`decision_window_secs`	`30`	Time that the sampler aggregates a trace before deciding, from 5 to 120.
`root_grace_millis`	`1000`	Time to wait for late child spans after the root span ends.
`decision_cache_secs`	`300`	Lifetime of a sampling decision. At least `decision_window_secs`.
`tail_max_traces`	`10000`	Undecided traces that the sampler may hold.
`tail_memory_size`	`256`	Memory limit for undecided traces in MiB.
`max_spans_per_trace`	`1000`	Spans kept per trace.
`max_attributes_per_span`	`128`	Attributes kept per span.
`max_events_per_span`	`128`	Events kept per span.
`max_links_per_span`	`128`	Links kept per span.
`max_string_length`	`4096`	Longest attribute, event, or link string in UTF-8 bytes.
`shutdown_timeout_secs`	`10`	Time to stop the pipeline on shutdown.

[telemetry.trace.slow_thresholds] marks spans as slow, in milliseconds: http_ms = 1000, query_ms = 5000, batch_intake_ms = 2000, database_ms = 200, object_store_ms = 500, external_ms = 1000, background_ms = 30000. [telemetry.trace.external] exports traces to an external OTLP receiver. It stays off while endpoint is empty.

Key	Default	Description
`endpoint`	empty	OTLP receiver address.
`protocol`	`grpc`	`grpc` or `http/protobuf`. The value is never inferred from the URL.
`timeout_ms`	`5000`	Connection and export timeout.
`queue_capacity`	`8192`	Export queue capacity.
`batch_size`	`256`	Traces per batch.
`gzip`	`false`	Compresses the payload.
`allow_self_export`	`false`	Allows export to this MoleSignal cluster. Disabled to prevent loops.
`headers`	none	OTLP headers. Reference secrets with `env:NAME` or `secret:NAME` and never write the value.
`custom_ca_file`	none	CA file for the HTTPS server certificate.
`client_certificate_file`	none	mTLS client certificate. Set together with `client_key_file`.
`client_key_file`	none	mTLS client key. Set together with `client_certificate_file`.

APM

[apm] controls the application performance monitoring projection. It starts on nodes that own trace candidates or run the alert role rollup. See APM.

Key	Default	Description
`queue_capacity`	`65536`	Input queue, from 1024 to 1048576.
`flush_interval_ms`	`5000`	Aggregate flush interval, from 100 to 60000.
`flush_max_snapshots`	`10000`	Snapshots per flush, from 1 to 100000.
`shutdown_drain_secs`	`10`	Time to drain the queue on shutdown, from 1 to 300.
`late_grace_secs`	`300`	Time that late trace data is still accepted. Greater than 0 and shorter than the hot retention.
`hot_retention_hours`	`24`	Retention of high-resolution data, from 1 to 168.
`rollup_retention_days`	`30`	Retention of rollups, from 1 to 3650.
`max_query_range_days`	`30`	Longest range of one APM query. At most `rollup_retention_days`.
`max_exemplars_per_bucket`	`3`	Exemplars per aggregate bucket, from 1 to 16.
`max_error_samples_per_group`	`8`	Error samples per group, from 1 to 64.
`[apm.histogram].schema_version`	`2`	Version of the latency bucket layout. Increase it whenever `buckets_per_octave` changes. At least `2`: version `1` is the fixed buckets of earlier aggregates, which stay readable.
`[apm.histogram].buckets_per_octave`	`8`	Buckets to each doubling of latency, from 1 to 16, covering 125 µs to about 131 s; `+Inf` is added automatically. With `8`, neighbouring bounds are about 9% apart and a percentile is within about 4%.
`[apm.version_comparison].min_requests_per_version`	`1000`	Requests that a version needs to join version comparison.

The limits on how many services, endpoints, dependencies, operations, error groups, versions, and instances APM keeps apart an hour are a platform setting: APM → Settings → Cardinality limits, for whoever holds sys.settings.manage. Every node reads them again every 30 seconds. See APM settings. A configuration file that still has an [apm.cardinality] section fails to load.

Features and integrations

Key	Default	Description
`[functions].llm_eval_enabled`	`false`	Allows `llm` pipeline steps. Every matching event triggers one model call, so cost and latency grow with intake volume. Use it on low-volume streams only.
`[mmdb].db_path`	`/usr/share/molesignal/mmdb/dbip-city-lite.mmdb`	DB-IP City Lite database. The container image and the release archive (in `mmdb/`, see Deploy MoleSignal) carry the monthly edition that was current when they were built. RUM intake looks up each event’s country from the client IP; without the file, events keep the country the client sent.

IP geolocation data by DB-IP, licensed under CC BY 4.0. The release binary includes the JavaScript function runtime and VRL. Neither needs a setting. See Functions. Scheduled reports in PDF or PNG form render pages with headless Chrome:

Key	Default	Description
`[scheduled_reports.renderer].base_url`	`http://127.0.0.1:5173`	Origin that headless Chrome loads report pages from. Must be an absolute `http` or `https` URL. The default points at the development server of the web UI, so set the address of the web UI that the node can reach.
`[scheduled_reports.renderer].concurrent_renders`	`2`	Concurrent renders. `0` counts as 1, and values above 4 become 4.
`[scheduled_reports.renderer].render_timeout_secs`	`30`	Timeout of one render. `0` counts as 30, and values above 60 become 60.
`[scheduled_reports.renderer].viewport_width`	`1280`	Width in pixels of the viewport that a report page loads in. The PDF page or image has the same width.
`[scheduled_reports.renderer].viewport_height`	`800`	Height in pixels of the viewport that a report page loads in. The PDF page or image is as tall as the content of the page.

When Chrome is unavailable, PDF and PNG exports return an error and no placeholder file. Scheduled log inspections run on nodes with the alert role. See Log inspections:

Key	Default	Description
`[log_inspection].min_interval_secs`	`3600`	Shortest time between two runs of one inspection. Values below 60 count as 60.
`[log_inspection].max_per_org`	`20`	Scheduled inspections that one organization can keep. `0` allows none.
`[log_inspection].concurrent_runs`	`2`	Inspections that one node runs at the same time. `0` counts as 1.
`[log_inspection].run_timeout_secs`	`300`	Time limit of one run, including every model call and query. Values below 30 count as 30, and values above 480 become 480.
`[log_inspection].report_retention_days`	`30`	Days that reports are kept. Expired reports are removed hourly. `0` counts as 1.

The daily Mole Agent token budget applies to each organization. An organization administrator can set a limit of its own in Mole Agent → Settings → Usage. See Mole Agent. The skill folders serve every organization; see Skills:

Key	Default	Description
`[agent].daily_token_budget`	`0`	Model tokens that one organization can use per day, input plus output, counted per UTC day. `0` means no limit.
`[agent].budget_warning_percent`	`80`	Share of the limit, in percent, at which the chat starts to warn. Values below 1 count as 1, and values above 100 become 100.
`[agent].mcp_stdio_commands`	`[]`	Programs that the organization’s stdio MCP servers may start on the MoleSignal server, compared as written with the first word of a server’s command, such as `npx` or `/opt/mcp/bin/server`. Empty means no stdio server runs. The command runs with the MoleSignal process’s user rights, so list only what you trust. As an environment variable: `MS_AGENT_MCP_STDIO_COMMANDS='["npx","uvx"]'`.
`[agent].skill_paths`	`[]`	Server folders whose subfolders holding `SKILL.md` are skills for every organization, read-only in the product. Each node reads them at startup and again when their files change. As an environment variable: `MS_AGENT_SKILL_PATHS='["/var/lib/molesignal/skills"]'`.

Settings stored in the database

These settings live in PostgreSQL, and the web UI or API changes them. They have no TOML keys:

SSO providers (OIDC, SAML, and LDAP).
Remote clusters for federated queries.
Status page domains and certificates, and the Slack app that visitors add a page to Slack with.
Notification channels, schedules, and escalation policies.
Mole Agent model providers, prompts, tools, and approval policy.
Organization cipher keys, field masking rules, and sharing policy.

​How configuration is loaded

​Environment variables

​Minimal production settings

​Node

​Listeners

​HTTP

​HTTP TLS and ACME

​Internal gRPC

​OTLP gRPC

​Arrow Flight SQL

​Probe Agent

​pprof

​Storage

​Metadata database

​Object storage

​Write-ahead log

​Intake

​Compaction and retention

​Catalog maintenance

​Query

​Limits and asynchronous jobs

​Admission control

​Caches

​Incremental range cache

​Cluster and rate limits

​Alerting and email

​Authentication

​Self-observability

​Self-collected telemetry

​Trace capture and tail sampling

​APM

​Features and integrations

​Settings stored in the database

How configuration is loaded

Environment variables

Minimal production settings

Node

Listeners

HTTP

HTTP TLS and ACME

Internal gRPC

OTLP gRPC

Arrow Flight SQL

Probe Agent

pprof

Storage

Metadata database

Object storage

Write-ahead log

Intake

Compaction and retention

Catalog maintenance

Query

Limits and asynchronous jobs

Admission control

Caches

Incremental range cache

Cluster and rate limits

Alerting and email

Authentication

Self-observability

Self-collected telemetry

Trace capture and tail sampling

APM

Features and integrations

Settings stored in the database