DocumentaciónHelp

APM alerts and incidents

Which alerts the APM generates, how to adjust their thresholds, and how they open and close incidents in Core.

On this page

The APM watches each service continuously and, when something goes out of the ordinary, opens an incident in Core. From there, people are notified according to your notification rules, just like any other incident.

Available alerts

Alert

When it triggers

Incident severity

How it closes

High p95 latency

The p95 exceeds the threshold, with a minimum number of requests in the window

Warning

Automatically, when latency returns to normal

High error rate

The percentage of requests with errors exceeds the threshold, with a minimum number of requests

Critical

It doesn't close when it drops below the threshold. It closes after a new version without the error, after the days without recurring (14 by default), with Declare fix, or manually

Stopped sending data

The service sends no telemetry for the specified minutes

Critical

Automatically, when it starts sending again

New error

An exception that hadn't been seen before appears

Warning

After a new version without the error, after the days without recurring (14 by default), with Declare fix, or manually

Error reappeared

An error that was resolved appears again (because its incident was closed or you marked it Resolved)

Warning

Same as a new error

How error incidents close

Latency and a service that stops sending data close automatically when they return to normal. Error rate and errors stay open even if they stop occurring, and they close in one of these ways:

  • New version without the error: you deploy a new version of the service and the error doesn't appear again in the next 30 minutes. For the error rate, the service must also be below the threshold.

  • Days without recurring: the error doesn't appear again within the days set in Close errors that don't recur (days) (14 by default). For the error rate, the service must also be below the threshold at that moment.

  • Declare fix: you describe what was fixed and the incident closes.

  • Resolve manually from the incident.

When an error incident closes, in any of these ways, the error becomes Resolved in APM → Errors. If it appears again later, ARGUNIS sends an Error reappeared notification and opens a new incident.

When the error rate drops below the threshold, the incident stays open: it's logged as an event, with no recovery notification. That way you don't lose track of a problem that only calmed down because of low traffic.

View the alerts

  1. Go to APM → Alerts.

  2. Review:

    • Company thresholds: the values that apply to all services without their own thresholds.

    • Services in alert or with their own settings: each service with its active alerts and whether it's muted or has own thresholds.

    • APM incidents: open ones and those from the last 7 days, with their Ticket, Service, Title, Start, and Status. Click the ticket to open the incident.

Change the company thresholds

You need the Administer permission in the APM.

  1. In APM → Alerts, click Change (or go to APM → Configuration and settings and look for Company alert thresholds).

  2. Click Edit.

  3. Adjust:

    • Maximum p95 (ms).

    • Maximum errors (%).

    • Minimum requests in the window: prevents alerts from a single error in a low-traffic service.

    • Window (min): how many minutes it's measured over.

    • Notify if a service sends no data for (min).

    • Close errors that don't recur (days): from 1 to 90, 14 by default. An error incident closes automatically if the error doesn't appear again within those days.

  4. Click Save.

To give a service different thresholds or mute it, use Configure → Thresholds on its page. See Read a service's detail page.

What happens next

Each APM incident also appears in Core → Incidents and reaches the people defined by your notification rules and resolver groups. From the incident you can acknowledge it, snooze it, declare the fix, or resolve it.

Related

Search Argunis Monitoring

Type at least 2 characters to search Argunis Monitoring.

↑↓ to navigate · Enter to open · Esc to close