> ## Documentation Index
> Fetch the complete documentation index at: https://docs.molesignal.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Alerting and incident response

> Rules, incidents, anomaly detection, templates, schedules, escalations, silences, subscriptions, and insights.

Open **Alerts** to move from detection to acknowledgement, escalation, and resolution.

## Alert center

| Section               | Purpose                                                                                |
| --------------------- | -------------------------------------------------------------------------------------- |
| **Incidents**         | Triage active incidents, inspect context, acknowledge, resolve, silence, or start RCA. |
| **Rules**             | Create scheduled, real-time, or anomaly rules and severity thresholds.                 |
| **History**           | Review past incident and rule activity.                                                |
| **Insights**          | Inspect volume, resolution time, fast-resolution ratio, and activity by hour.          |
| **Escalations**       | Define ordered targets and acknowledgement timeouts.                                   |
| **On-call schedules** | Build timezone-aware rotations and temporary overrides.                                |
| **Silences**          | Suppress matching notifications for a controlled time window.                          |
| **Groups**            | Normalize labels into reusable incident-grouping semantics.                            |

## Rule types

<CardGroup cols={3}>
  <Card title="Scheduled" icon="clock">
    Runs a SQL or metric query on an interval and evaluates the configured trigger and severity thresholds.
  </Card>

  <Card title="Real-time" icon="bolt">
    Records `real_time` rule intent. The current backend evaluates real-time intent through the scheduled
    evaluator; an intake-path matcher is not active yet.
  </Card>

  <Card title="Anomaly" icon="wave-square">
    Compares current values with a historical MAD or EWMA baseline, optionally with weekly
    seasonality.
  </Card>
</CardGroup>

Use the rule workbench to select the stream, query, evaluation range, thresholds, labels,
annotations, template, and escalation policy. Test the query before enabling the rule.

Anomaly rules accept a 1–30 day lookback, a positive deviation multiplier, and either `mad` or
`ewma`. EWMA also uses an alpha value in the range `(0, 1]`.

## Incident lifecycle

An incident starts active. A responder can acknowledge the incident, silence matching delivery,
and resolve the incident after recovery. Incident detail preserves rule context, trace ids, labels, delivery state, and
related RCA when available.

Semantic grouping combines incidents with the same normalized fingerprint. Acknowledging or
resolving a group cascades to all member incidents.

Mole Agent RCA requires the `agent` feature and Agent permissions. Mole Agent stores the
evidence-backed result with the incident instead of replacing the incident lifecycle.

## Escalation and on-call

The `alert_manager` role evaluates rules and drives escalation:

1. resolve the active step's users, teams, schedules, and channels;
2. dispatch each target and record an independent delivery result;
3. wait for acknowledgement until `ack_timeout_secs`;
4. advance to the next step until acknowledged, resolved, or exhausted.

Schedules use an explicit timezone, rotations, and date-bounded overrides. Use the on-call preview
before assigning a schedule to production escalation.

## Notification delivery

Configure SMTP, Slack, Lark, and webhook credentials under **Settings → Notify management →
Connectors**. Connector configuration is encrypted at rest, and sensitive values are never returned
in plaintext.

Use **Users** to bind personal provider identities, **Templates** to build reusable messages,
**Policies** to match events and resolve recipients, and **Fallback routes** for team or organization
delivery. See [Notification channels and delivery](/en-US/notification-channels) for the complete
workflow.

## Silences

A silence matches labels during a fixed window and pauses delivery without changing the rule.
Silencing one active incident can create a matching silence rule.

## Permissions

| Permission           | Capability                                                                                          |
| -------------------- | --------------------------------------------------------------------------------------------------- |
| `alerts.read`        | Read rules, incidents, notification resources, insights, groups, and silences.                      |
| `alerts.manage`      | Create and change rules, connectors, templates, policies, fallback routes, escalations, and groups. |
| `alerts.acknowledge` | Acknowledge and resolve incidents or incident groups.                                               |
| `alerts.silence`     | Create, change, and remove silences.                                                                |
| `schedules.read`     | Read schedules and on-call results.                                                                 |
| `schedules.manage`   | Create schedules, rotations, and overrides.                                                         |

<CardGroup cols={2}>
  <Card title="Notification channels" icon="bell" href="/en-US/notification-channels">
    Configure connectors, users, templates, policies, fallbacks, and delivery operations.
  </Card>

  <Card title="Webhook connectors" icon="webhook" href="/en-US/alert-webhook-channels">
    Configure payloads, target overrides, signing, and webhook security.
  </Card>
</CardGroup>