> ## Documentation Index > Fetch the complete documentation index at: https://docs.molesignal.com/llms.txt > Use this file to discover all available pages before exploring further. # Iceberg REST catalog > Query MoleSignal telemetry from Spark, Trino, Flink, and DuckDB through a managed Apache Iceberg REST catalog. > [!WARNING] > > Planned for Enterprise and MoleSignal Cloud. The capability is in design and is not available in any release. The open source edition does not include it. Names, scope, and behavior on this page can change before release. The Iceberg REST catalog publishes the telemetry of an organization as [Apache Iceberg](https://iceberg.apache.org) tables. A query engine that runs on infrastructure chosen by the organization connects to the catalog, finds streams as tables, and reads the Parquet files directly from object storage. MoleSignal manages the tables, metadata, permissions, tenant boundary, and data location. The engine, the queries, and the compute stay with the organization. ## Who does what MoleSignal manages: * **Tables.** Every exposed stream appears as a read-only Iceberg table with the schema of the stream. * **Metadata.** Snapshots, data files, and statistics stay consistent with compaction and retention. * **Permissions.** The catalog decides which principal can list and read which tables. * **Tenant isolation.** An organization sees only its own tables and files. * **Data location.** MoleSignal decides where the files live and issues short-lived, read-only credentials for them. The organization runs: * **The engine.** Apache Spark, Trino, Apache Flink, DuckDB, or any other client of the Iceberg REST catalog specification. * **The workloads.** Queries, ETL pipelines, machine learning jobs, and BI dashboards. * **The compute and its cost.** Clusters, scheduling, and result storage. ## How a connection works **1. Authenticate** The engine presents an API token or service account credential to the catalog endpoint. **2. Find tables** The catalog lists the streams that the principal may read, grouped by stream type. **3. Load a table** The catalog returns the schema, the partitioning, and the data files of one consistent snapshot, together with short-lived read-only credentials for the storage location of the table. **4. Read files** The engine reads the Parquet files directly from object storage. File content does not pass through MoleSignal. ## Choose an access path | | Iceberg REST catalog | [Arrow Flight SQL](https://docs.molesignal.com/en-US/query/database-clients.md) | | - | - | - | | Where queries run | In the engine of the organization | In MoleSignal | | Typical use | Bulk analysis, ETL, machine learning, and BI over time ranges | Interactive SQL from database clients | | Field masking | Not applied. See [Data protection boundaries](#data-protection-boundaries). | Applied | | Editions | Enterprise and Cloud (planned) | All editions | Use Flight SQL or the query API for governed, interactive questions. Use the catalog when a job scans large time ranges with engine-side compute. ## What the catalog exposes * **One table per stream.** Namespaces mirror the stream types `logs`, `metrics`, `traces`, and `profiles`. * **Columns follow the stream schema.** `_timestamp` comes first, with microsecond precision in UTC. JSON fields are strings. Encrypted fields stay ciphertext. * **Files are partitioned by event time.** Time-range filters prune files. * **Not exposed.** Extend tables, which live in memory, and the derived read models that MoleSignal keeps for internal features. ## Access control * **Authentication.** API tokens (`ms__`) and service accounts, the same credentials as the HTTP API and Flight SQL. * **Authorization.** A dedicated catalog permission, named at release, authorizes access. Reading a table requires this permission and `streams.query`. * **Tenant isolation.** Each organization has its own catalog scope. Storage credentials cover only the files of the requested table, expire after a short period, and never allow writes or deletes. * **Audit.** Credential issuance and exposure changes are recorded as audit events. ## Data protection boundaries > [!WARNING] > > An engine that reads Parquet files sees the stored values. Field masking rules apply to MoleSignal queries and to Flight SQL results. They do not apply to files that an engine reads directly. Encrypted fields stay encrypted in the files, so the engine receives ciphertext. Catalog access therefore equals raw data access to the exposed streams: * Grant the catalog permission only to principals that may read unmasked data. * Administrators choose which streams the catalog exposes. A stream with an active field masking rule is not exposed until an administrator explicitly allows it. * Metrics streams do not take part in field masking and follow the normal exposure setting. ## Data freshness and consistency * A snapshot lists committed files only. Records that are still in the intake buffer become visible after the next flush. * Compaction and retention appear in the next snapshot. Files that compaction replaces stay in object storage for a grace period, so a job that planned against an earlier snapshot can finish. Plan again for jobs that run longer than the grace period. ## Supported engines The catalog follows the open Iceberg REST catalog specification. These engines are the verification targets: | Engine | Status | | - | - | | Apache Spark | Planned | | Trino | Planned | | Apache Flink | Planned | | DuckDB | Planned | | PyIceberg and other REST catalog clients | Expected to work, not verified | Verified versions and connection settings are published at release. ## Limits of the first release * **Read-only.** The catalog rejects writes, table creation, and schema changes. Change schemas and retention on the stream. * **Current snapshot only.** Time travel, branches, and tags are not available. * **No views and no multi-table transactions.** * **No organization-owned tables.** An engine can attach several catalogs at once, so results can be joined with and written to a catalog that the organization operates. * **Object stores that can issue scoped access only.** The `local` object store backend cannot serve files to external engines. ## Availability | Edition | Availability | | - | - | | Open source | Not included. Use [Arrow Flight SQL](https://docs.molesignal.com/en-US/query/database-clients.md) for external SQL access. | | Enterprise | Planned. Needs an object store that the engines can reach and a license that includes the capability. | | MoleSignal Cloud | Planned. Each region serves the catalog next to the data of its organizations. | > **[Connect with a database client](https://docs.molesignal.com/en-US/query/database-clients.md)** > > Query MoleSignal over Arrow Flight SQL today. > **[Security and multi-tenancy](https://docs.molesignal.com/en-US/security.md)** > > Tenant isolation, field masking, and field encryption. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.