Skip to main content
Planned for Enterprise and MoleSignal Cloud. The capability is in design and is not available in any release. The open source edition does not include it. Names, scope, and behavior on this page can change before release.
The Iceberg REST catalog publishes the telemetry of an organization as Apache Iceberg tables. A query engine that runs on infrastructure chosen by the organization connects to the catalog, finds streams as tables, and reads the Parquet files directly from object storage. MoleSignal manages the tables, metadata, permissions, tenant boundary, and data location. The engine, the queries, and the compute stay with the organization.

​
Who does what

MoleSignal manages:
  • Tables. Every exposed stream appears as a read-only Iceberg table with the schema of the stream.
  • Metadata. Snapshots, data files, and statistics stay consistent with compaction and retention.
  • Permissions. The catalog decides which principal can list and read which tables.
  • Tenant isolation. An organization sees only its own tables and files.
  • Data location. MoleSignal decides where the files live and issues short-lived, read-only credentials for them.
The organization runs:
  • The engine. Apache Spark, Trino, Apache Flink, DuckDB, or any other client of the Iceberg REST catalog specification.
  • The workloads. Queries, ETL pipelines, machine learning jobs, and BI dashboards.
  • The compute and its cost. Clusters, scheduling, and result storage.

​
How a connection works

1

Authenticate

The engine presents an API token or service account credential to the catalog endpoint.
2

Find tables

The catalog lists the streams that the principal may read, grouped by stream type.
3

Load a table

The catalog returns the schema, the partitioning, and the data files of one consistent snapshot, together with short-lived read-only credentials for the storage location of the table.
4

Read files

The engine reads the Parquet files directly from object storage. File content does not pass through MoleSignal.

​
Choose an access path

Use Flight SQL or the query API for governed, interactive questions. Use the catalog when a job scans large time ranges with engine-side compute.

​
What the catalog exposes

  • One table per stream. Namespaces mirror the stream types logs, metrics, traces, and profiles.
  • Columns follow the stream schema. _timestamp comes first, with microsecond precision in UTC. JSON fields are strings. Encrypted fields stay ciphertext.
  • Files are partitioned by event time. Time-range filters prune files.
  • Not exposed. Extend tables, which live in memory, and the derived read models that MoleSignal keeps for internal features.

​
Access control

  • Authentication. API tokens (ms_<prefix>_<secret>) and service accounts, the same credentials as the HTTP API and Flight SQL.
  • Authorization. A dedicated catalog permission, named at release, authorizes access. Reading a table requires this permission and streams.query.
  • Tenant isolation. Each organization has its own catalog scope. Storage credentials cover only the files of the requested table, expire after a short period, and never allow writes or deletes.
  • Audit. Credential issuance and exposure changes are recorded as audit events.

​
Data protection boundaries

An engine that reads Parquet files sees the stored values. Field masking rules apply to MoleSignal queries and to Flight SQL results. They do not apply to files that an engine reads directly. Encrypted fields stay encrypted in the files, so the engine receives ciphertext.
Catalog access therefore equals raw data access to the exposed streams:
  • Grant the catalog permission only to principals that may read unmasked data.
  • Administrators choose which streams the catalog exposes. A stream with an active field masking rule is not exposed until an administrator explicitly allows it.
  • Metrics streams do not take part in field masking and follow the normal exposure setting.

​
Data freshness and consistency

  • A snapshot lists committed files only. Records that are still in the intake buffer become visible after the next flush.
  • Compaction and retention appear in the next snapshot. Files that compaction replaces stay in object storage for a grace period, so a job that planned against an earlier snapshot can finish. Plan again for jobs that run longer than the grace period.

​
Supported engines

The catalog follows the open Iceberg REST catalog specification. These engines are the verification targets: Verified versions and connection settings are published at release.

​
Limits of the first release

  • Read-only. The catalog rejects writes, table creation, and schema changes. Change schemas and retention on the stream.
  • Current snapshot only. Time travel, branches, and tags are not available.
  • No views and no multi-table transactions.
  • No organization-owned tables. An engine can attach several catalogs at once, so results can be joined with and written to a catalog that the organization operates.
  • Object stores that can issue scoped access only. The local object store backend cannot serve files to external engines.

​
Availability

Connect with a database client

Query MoleSignal over Arrow Flight SQL today.

Security and multi-tenancy

Tenant isolation, field masking, and field encryption.
Last modified on October 9, 2026