> ## Documentation Index
> Fetch the complete documentation index at: https://docs.molesignal.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Log inspections

> Have Mole Agent inspect a log stream on demand or on a schedule, keep every report, and notify when a report says something is wrong.

A log inspection has Mole Agent read the logs of a period and write a report: whether something is
wrong, what it is, and which logs show it. An inspection only reads logs. It changes no data and no
configuration.

| Kind | Use |
| - | - |
| **On demand** | Inspect the logs that the current query matches, once, from the **AI inspection** view of **Logs**. The report streams onto the page. The inspection is a Mole Agent chat, so it can continue in **Mole Agent**. It is not kept with the scheduled reports. |
| **Scheduled** | Save a stream, a query, and what to attend to. MoleSignal inspects on the schedule, keeps every report, and notifies when a report says something is wrong. |

## Before starting

* Enable at least one AI model under **Mole Agent → Settings → Models & reasoning**. See
  [Mole Agent](https://docs.molesignal.com/en-US/agent#set-up-a-model-provider). Without an enabled model, saved inspections
  stay listed and cannot run.
* Check the permissions:

| Task | Permission |
| - | - |
| Inspect on demand, and read scheduled inspections and their reports | `agent.use` and `streams.query` |
| Create, edit, switch on or off, delete, and run now | `inspections.manage`, granted to the owner, admin, and editor roles by default |
| Choose a notification channel in the form | `alerts.read`, which lists the notification connectors |

Mole Agent reads the same inspections and reports in a chat with `list_log_inspections`,
`list_log_inspection_runs`, and `get_log_inspection_run`, under the same permissions.
`run_log_inspection` runs one now: it needs `inspections.manage` and, by default, a confirmation in
the chat. See [Tool catalog and execution](https://docs.molesignal.com/en-US/tools.md).

## Inspect on demand

1. Open **Logs**, select a stream and a time range, and run a query. Leave the query empty to
   inspect every log of the stream.
2. Open the **AI inspection** view.
3. Choose the **AI model**. Optionally describe **What to attend to**, such as the service to watch
   or an outcome to treat as expected.
4. Select **Start inspection**.

The inspection sends query results of these logs to the chosen model. Select **Continue in Mole
Agent** to ask follow-up questions about the report.

## Save a scheduled inspection

Open **Logs → AI inspection**, then select either entry:

* **Save as scheduled inspection** opens the form with the current stream, query, requirements, and
  model. The inspection can be saved even when the query matches nothing now.
* **Scheduled inspections** opens the list, where **New inspection** starts an empty form.

| Field | Meaning |
| - | - |
| **Name** | Unique in the organization. At most 120 characters. |
| **Stream** | One log stream. The inspection reads this stream only. |
| **Query** | Optional. Only logs that match are inspected. The syntax is the same as on **Logs**. |
| **What to attend to** | Optional, at most 2,000 characters. What matters, and what counts as normal. |
| **Schedule** | **At an interval**, **Daily**, or **Weekly** at a time of day. The shortest interval is an instance setting, 1 hour by default. |
| **Period covered** | How much time before each run is inspected: 15 minutes to 7 days in the form. The default follows the schedule. |
| **AI model** | One enabled model of the organization. |
| **When to notify** | **Notify when abnormal** (default), **Notify on attention or worse**, or **Notify every time**. |
| **Notification channels** | Optional, at most 10. Each is a fixed group or address on a notification connector. |
| **Run on the schedule once saved** | When off, the inspection is saved and runs only on **Run now**. |

A daily or weekly time is read in the time zone of the interface at the time of creation: the time
zone of the personal preferences, or the time zone of the browser when none is set. The form names
the time zone, and a later edit does not change it. Notifications and reports use the interface
language at the time of creation.

An organization keeps at most 20 scheduled inspections by default.

## How a run works

| Aspect | Behavior |
| - | - |
| Identity | A run reads the logs as the user who saved the inspection last. Before every run, MoleSignal checks that this user can still sign in, is still a member of the organization, and still has `agent.use` and `streams.query`. Otherwise the run fails with the reason that the identity can no longer be used, and another user with the permission saves the inspection again to run it as that user. An API Token cannot save a scheduled inspection. |
| Tools | The model uses three read-only tools: `query_logs`, `get_stream_schema`, and `search_field_values`, and only on the stream of the inspection. A tool that the Tool Policy of the organization disables or puts behind approval is not used, because nobody approves in the background. |
| Time window | A run covers the period just before it starts. Runs that a stopped node missed are not made up: once the node is back, a due inspection runs once and the schedule continues from there. |
| Limits | A run takes at most 5 minutes by default, after which it counts as timed out. A tool result above 64 KiB is not handed to the model, which is told to narrow the query instead. A failed run is not retried before the next scheduled run. |
| Concurrency | One inspection has one run at a time. One node runs at most 2 inspections at once by default. |
| **Run now** | The run is queued and starts within 15 seconds, under the identity of the inspection. The record names the user who started it. A switched-off inspection can also run this way. |

## Reports and verdicts

Select an inspection in the list to open its reports. Past runs are on the left. The selected run
shows the report, the period covered, the duration, the tools called, the tokens used, and the
notifications sent. **Open the logs of this period** opens **Logs** on the same stream, query, and
period.

A scheduled report carries a verdict:

| Verdict | Meaning |
| - | - |
| **Normal** | Nothing that needs action. |
| **Needs attention** | Something deserves a look, and nothing is failing. |
| **Abnormal** | Something is wrong and needs action. |
| **No verdict** | The answer of the model carried no readable verdict. The report is kept, and notification treats it as abnormal. |

A run that did not finish has no report. The record shows its state and the reason:

| State | Reason |
| - | - |
| **Timed out** | The run exceeded its time limit. |
| **Failed** | The identity can no longer be used; the chosen AI model is missing, disabled, or has no key; the model call failed; the organization's Mole Agent token budget for today is used up; the node stopped during the run; no node picked up the run; or an internal error occurred. |

Reports are kept for 30 days by default and removed hourly after that. Deleting an inspection
deletes all its reports.

## Notifications

* When the verdict of a report meets **When to notify**, each channel receives one notification:
  the title states the verdict and the name of the inspection, and the body holds the stream, the
  period, the verdict, the summary of the report, and a link to the report. The link needs
  `http.external_url` to be set.
* A failed run notifies once, when the run before it was not a failure. Repeated failures notify
  only the first time, whatever **When to notify** says.
* Without a channel, nothing is sent and the reports are still kept.
* Every notification appears under **Settings → Notify → Delivery log** with the stage
  **Direct**. A direct delivery does not pass through notification policies and cannot be retried
  from the delivery log.
* A notification connector that a scheduled inspection uses cannot be deleted.

## Instance settings

| Setting | Default | Purpose |
| - | - | - |
| `[log_inspection].min_interval_secs` | `3600` | The shortest time between two runs of one inspection. Values below 60 count as 60. |
| `[log_inspection].max_per_org` | `20` | Scheduled inspections that one organization can keep. `0` allows none. |
| `[log_inspection].concurrent_runs` | `2` | Inspections that one node runs at the same time. |
| `[log_inspection].run_timeout_secs` | `300` | The time limit of one run, including every model call and query. Clamped to 30–480. |
| `[log_inspection].report_retention_days` | `30` | How long reports are kept. |

Nodes with the `alert` role run scheduled inspections and check every 15 seconds for a due or
queued run. With several `alert` nodes, each run is picked up by one node only. See
[Role-based deployment](https://docs.molesignal.com/en-US/deployment/role-based.md).

Two metrics follow the runs:

| Metric | Meaning |
| - | - |
| `log_inspection_runs_total{outcome}` | Finished runs by outcome: `succeeded`, `failed`, or `timed_out`. |
| `log_inspection_run_duration_seconds` | The duration of a run. |

## API

The endpoints are under `/api/v1/logs/inspections`:

| Method and path | Purpose |
| - | - |
| `GET /logs/inspections` | List the scheduled inspections of the organization. |
| `POST /logs/inspections` | Create an inspection. |
| `GET /logs/inspections/limits` | The limits that the instance sets. |
| `GET`, `PUT`, `DELETE /logs/inspections/{id}` | Read, replace, or delete an inspection with its reports. |
| `POST /logs/inspections/{id}/enabled` | Switch an inspection on or off. |
| `POST /logs/inspections/{id}/run` | Run once now. |
| `GET /logs/inspections/{id}/runs` | Past runs, newest first, without the report text. `limit` defaults to 50 and is at most 200. `before` pages to older runs. |
| `GET /logs/inspections/{id}/runs/{run_id}` | One run with its report. |

> **[Logs](https://docs.molesignal.com/en-US/logs)**
>
> Write the query that an inspection reads.

> **[Notification channels](https://docs.molesignal.com/en-US/notification-channels)**
>
> Connect the channels that receive the notifications.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.