Skip to main content
A stream is the organization-scoped contract used by intake, query, pipelines, and retention. Open Streams to see what arrives, and to inspect or configure each stream.

​
Stream types

The same stream name may exist for different stream types. Always include stream_type in API requests and automation.

​
The stream list

Streams lists every stream of the workspace. Switch the tabs to show All streams or one type, search by name or description, and filter by runtime state. The head counts the streams that receive normally and those that need attention, and sums the compressed data written in the period and the current storage. A stream marked Not queryable keeps receiving data but is hidden from query selectors. Each row has Query or Explore, View details, and Copy stream ID. Search jobs in the head opens the search jobs.

​
Create a stream

MoleSignal creates a stream on the first write to it, so most streams appear by themselves. Create one in advance to set its retention, its encrypted fields, or its query access from the start. Select Create stream:

​
Stream details

Select a stream to open it. Query data opens it in its explorer, and the menu has Copy stream ID and Delete stream. The tabs: Changes on the tabs collect into a draft. Save and apply applies them at once, and Discard changes drops them. Before saving, the page warns about changes that cannot be taken back: a shorter retention deletes expired data, and downsampling deletes the original samples. Max query range limits how many hours one SQL or PromQL query of the stream may cover. A query over a longer range is rejected, so a mistaken “last 90 days” cannot scan a large stream end to end. Empty or 0 means no limit. Set Queryable off when a catch-all source should keep receiving but must not appear in query selectors. A routing pipeline can still read the stream. Manage extend data through Extend tables rather than the generic telemetry query path.

​
Schema and fields

A field has a name, a data type, a nullable flag, an index type, and an encrypted flag. The data types are bool, int64, float64, utf8, timestamp, and json. A new field is added to the schema when it first appears in incoming data, and a value of another type is rejected with type mismatch on field. The index type decides which helper structure MoleSignal builds for a field: A field that first appears in incoming data gets its index type from its name: Every other new field gets none. An exact level lets the logs histogram count errors from the index instead of reading the files. A changed index type applies to data written after the change; older files are indexed the new way when the compactor merges them. To change a field, open Fields & indexes, select the edit action of the field, and choose:
  • the Index type: Not indexed, Full text search, Exact match, Bloom filter, or Skip index. Full-text search applies to text fields;
  • the Stream-level policy of Result field masking: Inherit global rule, Custom stream rule, or Explicitly unmasked for stream. See Field masking. Metrics streams do not take part in masking.
Select Apply to draft, then Save and apply. The type of a field never changes.

​
Encrypted fields

Mark a field encrypted when creating the stream to encrypt it before it is written to Parquet. The stored column holds ciphertext, and a plain query returns the ciphertext. Read the value back with decrypt(field) in SQL:
The flag cannot be changed later, because existing data would become unreadable. The key comes from the organization cipher key and the cipher root key MS_CIPHER_KEY. Losing the root key makes encrypted fields unrecoverable. See Security and multi-tenancy.

​
Retention and deletion

Retention controls when the compactor removes data. Set it on the Retention tab as Data retention (days), or leave it empty for the platform default, 30 days, set in Settings → Data retention of the _sys organization. Data older than the retention is deleted in the next cleanup, so shortening the retention deletes the data that is then past it, and deleted files never come back. A metric of a metrics stream that has not been seen within the retention also leaves the metric list. On the Retention tab of a metrics stream, Downsample older samples rolls the samples of every metric in the stream older than a number of days up into time buckets from 1 minute to 1 day: the samples of a series in one bucket become one point, values averaged and delta counts summed, and the original samples are deleted. Queries over long ranges then read far less, and older data keeps only the bucket resolution. The original samples cannot be recovered. For logs and traces, the stream setting Merge small files by day (on by default) merges the files of a finished day into one an hour after the day ends, when together they are no larger than the compactor’s [compactor].target_size. A query over many days then opens far fewer files. Retention then applies by the day: a merged file is deleted once its whole day has expired. Turn the setting off in the stream’s Settings tab to keep hourly files. Delete stream deletes the stream and its configuration, and cannot be undone. It can break dashboards, saved views, alerts, reports, and pipelines: check the Usage tab first.

​
Search jobs

A search job is a query that runs in the background and keeps its result until the job expires. Jobs come from three places:
  • a query sent with Prefer: respond-async, or a query whose result is expected to be too large for an interactive request;
  • the backfill of a pipeline;
  • a query that Mole Agent submits.
Select Search jobs in the header of Streams to open the list. The views at the head of the list filter by state, each with its count: All, Queued, Running, Done, Failed, and Cancelled. While a job is queued or running, the list refreshes by itself. Each row shows the type of the job (Query or Backfill), the statement, the state, the queried range, the submit time, the duration, the result rows, the attempt, and the expiry. One action fits the state of the job: Delete removes a job with its result in any state. A member sees and manages the jobs that the member submitted. An administrator with org.settings.manage sees and manages every job of the organization. See Query for the job API.

​
Permissions

Data intake

Connect OTLP, Prometheus, Loki, Elasticsearch, and cloud senders.

Pipelines

Transform, enrich, and route stream data.
Last modified on October 26, 2026