> ## Documentation Index
> Fetch the complete documentation index at: https://docs.molesignal.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Incidents

> Work an incident from the first page to recovery: acknowledge it, read its evidence, ask Mole Agent for the root cause, silence it, and look back at the history and trends.

An incident is opened when a [rule](https://docs.molesignal.com/en-US/alerting/rules.md) fires, or when a [synthetic check fails](https://docs.molesignal.com/en-US/synthetics.md#alert-incidents). It records the rule, the values, the
labels, and every step until it ends. Open **Alerts → Incidents**, or press `⌘⌥A`. Reading incidents
requires `alerts.read`; acknowledging and resolving require `alerts.acknowledge`, and silencing requires
`alerts.silence`. Alert notifications received through a Feishu app or Slack app can also be
acknowledged and resolved on the card; see [Alert cards](https://docs.molesignal.com/en-US/im.md#alert-cards).

## Incident states

| State | List label | Meaning |
| - | - | - |
| `open` | **Waiting** | Firing, and nobody has acknowledged it. Escalation goes on. |
| `acknowledged` | **In progress** | Someone owns it. Escalation stops. |
| `resolved` | **Recovered** | Over: the rule recovered, or someone resolved it. |
| `closed` | **Closed** | Over. |

An open incident resolves by itself when the rule meets its [recovery condition](https://docs.molesignal.com/en-US/alerting/rules.md#recovery-and-missing-data).
An acknowledged incident stays acknowledged until someone resolves it. Each rule, or each group of a
split rule, has at most one active incident at a time.

## The incident list

The head of **Incidents** counts the affected services, the incidents that wait for acknowledgement,
and the incidents recovered in the last 24 hours. A summary says how many incidents fire now, how long
the oldest has been firing, and how many rules are evaluated. **Recently recovered** lists what ended
lately.

Switch between **Firing** and **Waiting**, the incidents that have no owner yet. Each row shows the
severity, the incident and its service, the state, the duration, and the owner, with three actions:

| Action | Effect |
| - | - |
| **Acknowledge** | Takes ownership. The incident becomes **In progress**, and escalation stops. |
| **Silence** | Pauses further notifications and escalation of this incident only. See [Silence one incident](https://docs.molesignal.com/en-US/alerting/silences.md#silence-one-incident). |
| **Runbook** | Opens the response guide of the rule, when it has one. |

## Incident details

Select an incident to open its details. The head shows the state, the time it fired, the rule, and how
long it has fired, with **Acknowledge**, **Silence**, and **Resolve**.

| Section | Content |
| - | - |
| **Timeline** | When it fired, who acknowledged it, and who resolved it, with the times in between. |
| **Investigate** | **Open triggering query** replays the exact query in its explorer. **Related logs** and **Related metrics** open the affected context in the same time range. An incident of a RUM figure has **Open in RUM**. |
| **AI root cause** | The analysis of Mole Agent. See [AI root cause](#ai-root-cause). |
| **Cross-signal handles** | The services, trace IDs, and hosts found in the evidence, each a link. |
| **Triggering query** | The query and the values it returned. **Show sample values** shows the sample rows. |
| **Metadata** | The labels and annotations of the incident. |
| **Assignees** | Who the escalation reached. |
| **Blast radius** | For an incident that names a service, what the service reaches along the service map. See [The blast radius of an incident](https://docs.molesignal.com/en-US/apm/insights.md#the-blast-radius-of-an-incident). |
| **Sample sessions** | For an incident of a RUM figure, the sessions that show it, each with a replay. See [RUM alerts](https://docs.molesignal.com/en-US/rum/alerts.md#incidents). |

## AI root cause

When Mole Agent is set up with a model provider, it analyzes new incidents in the background and
stores an evidence-based result with the incident. **Generate** asks for one at once, and
**Regenerate** replaces it. Rate a result with **Helpful** or **Not helpful**; the ratings feed
[Quality](https://docs.molesignal.com/en-US/quality.md).

Generating requires Mole Agent permissions. Once the Mole Agent token budget of the organization for
the day is used up, automatic analysis pauses, and generation on demand follows the budget setting. See
[Usage and budget](https://docs.molesignal.com/en-US/agent#usage-and-budget).

## Groups of incidents

[Semantic groups](https://docs.molesignal.com/en-US/alerting/silences.md#semantic-groups) collapse related incidents, for example all
critical incidents of one service, into one group. Acknowledging or resolving a group with
`POST /api/v1/alerts/incident_groups/{id}/ack` or `/resolve` does the same to every member.

## History

**Alerts → History** lists past incidents with their current state, the time they fired and
recovered, and how long they lasted. Search by incident, service, or incident ID, and filter by
severity and state.

## Insights

**Alerts → Insights** shows how the workspace responds, over the last 24 hours, 7 days, or 30 days:

| Part | Shows |
| - | - |
| Summary | How many incidents still need attention, the **Average recovery time** of the recovered ones, and **Rapid recoveries**: the share recovered within 60 seconds, a sign of noisy rules. |
| **Incident timeline** | Incidents by hour for 24 hours, and by day for longer ranges, in the browser timezone, with the peak. |
| **Most affected services** | The services that incidents name most often. Rules need a `service` label for this. |
| **Top firing rules** | The rules that opened the most incidents. Start tuning there. |

## Incidents API

| Operation | Endpoint |
| - | - |
| List incidents | [`GET /api/v1/alerts/incidents`](https://docs.molesignal.com/en-US/api/alerting/list-incidents) |
| Acknowledge | [`POST /api/v1/alerts/incidents/{id}/ack`](https://docs.molesignal.com/en-US/api/alerting/ack-incident) |
| Resolve | [`POST /api/v1/alerts/incidents/{id}/resolve`](https://docs.molesignal.com/en-US/api/alerting/resolve-incident) |
| Root cause analysis | [`GET /api/v1/alerts/incidents/{id}/rca`](https://docs.molesignal.com/en-US/api/alerting/incident-rca) |
| Insights | [`GET /api/v1/alerts/insights`](https://docs.molesignal.com/en-US/api/alerting/insights) |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.