Notifications: Difference between revisions
(Created page with "<!--PVE_IMPORT_START_MARKER--> <!-- Do not edit - this is autogenerated content --> {{#pvedocs:notifications-plain.html}} Category:Reference Documentation <pvehide> O...") |
No edit summary |
||
Line 117: | Line 117: | ||
Notification Matchers | Notification Matchers | ||
Notification matchers route notifications to notification targets based | Notification matchers route notifications to notification targets based | ||
on their matching rules. These rules can match | on their matching rules. These rules can match certain properties of a | ||
notification, such as the timestamp (match-calendar), the severity of | |||
the | the notification (match-severity) or metadata fields (match-field). | ||
If a matcher | If a notification is matched by a matcher, all targets configured for the | ||
will receive the notification. | matcher will receive the notification. | ||
An arbitrary number of matchers can be created, each with with their own | An arbitrary number of matchers can be created, each with with their own | ||
matching rules and targets to notify. | matching rules and targets to notify. |
Latest revision as of 17:30, 6 March 2024
Overview
There are a number of different notification events, each with their own set of metadata fields that can be used in notification matchers.
A notification matcher determines which notifications shall be sent where. A matcher has match rules, that can be used to match on certain notification properties (e.g. timestamp, severity, metadata fields). If a matcher matches a notification, the notification will be routed to a configurable set of notification targets.
A notification target is an abstraction for a destination where a notification should be sent to - for instance, a Gotify server instance, or a set of email addresses. There are multiple types of notification targets, including sendmail, which uses the system’s sendmail command to send emails, or gotify, which sends a notification to a Gotify instance.
The notification system can be configured in the GUI under Datacenter → Notifications. The configuration is stored in /etc/pve/notifications.cfg and /etc/pve/priv/notifications.cfg - the latter contains sensitive configuration options such as passwords or authentication tokens for notification targets.
Notification Targets
Sendmail
The sendmail binary is a program commonly found on Unix-like operating systems that handles the sending of email messages. It is a command-line utility that allows users and applications to send emails directly from the command line or from within scripts.
The sendmail notification target uses the sendmail binary to send emails.
In standard Proxmox VE installations, the sendmail binary is provided by Postfix. For this type of target to work correctly, it might be necessary to change Postfix’s configuration so that it can correctly deliver emails. For cluster setups it is necessary to have a working Postfix configuration on every single cluster node. |
The configuration for Sendmail target plugins has the following options:
-
mailto: E-Mail address to which the notification shall be sent to. Can be set multiple times to accomodate multiple recipients.
-
mailto-user: Users to which emails shall be sent to. The user’s email address will be looked up in users.cfg. Can be set multiple times to accomodate multiple recipients.
-
author: Sets the author of the E-Mail. Defaults to Proxmox VE.
-
from-address: Sets the from address of the E-Mail. If the parameter is not set, the plugin will fall back to the email_from setting from datacenter.cfg. If that is also not set, the plugin will default to root@$hostname, where $hostname is the hostname of the node.
-
comment: Comment for this target The From header in the email will be set to $author <$from-address>.
Example configuration (/etc/pve/notifications.cfg):
sendmail: example mailto-user root@pam mailto-user admin@pve mailto max@example.com from-address pve1@example.com comment Send to multiple users/addresses
SMTP
The configuration for SMTP target plugins has the following options:
-
mailto: E-Mail address to which the notification shall be sent to. Can be set multiple times to accomodate multiple recipients.
-
mailto-user: Users to which emails shall be sent to. The user’s email address will be looked up in users.cfg. Can be set multiple times to accomodate multiple recipients.
-
author: Sets the author of the E-Mail. Defaults to Proxmox VE.
-
from-address: Sets the From-addresss of the email. SMTP relays might require that this address is owned by the user in order to avoid spoofing. The From header in the email will be set to $author <$from-address>.
-
username: Username to use during authentication. If no username is set, no authentication will be performed. The PLAIN and LOGIN authentication methods are supported.
-
password: Password to use when authenticating.
-
mode: Sets the encryption mode (insecure, starttls or tls). Defaults to tls.
-
server: Address/IP of the SMTP relay
-
port: The port to connect to. If not set, the used port defaults to 25 (insecure), 465 (tls) or 587 (starttls), depending on the value of mode.
-
comment: Comment for this target
Example configuration (/etc/pve/notifications.cfg):
smtp: example mailto-user root@pam mailto-user admin@pve mailto max@example.com from-address pve1@example.com username pve1 server mail.example.com mode starttls
The matching entry in /etc/pve/priv/notifications.cfg, containing the secret token:
smtp: example password somepassword
Gotify
Gotify is an open-source self-hosted notification server that allows you to send and receive push notifications to various devices and applications. It provides a simple API and web interface, making it easy to integrate with different platforms and services.
The configuration for Gotify target plugins has the following options:
-
server: The base URL of the Gotify server, e.g. http://<ip>:8888
-
token: The authentication token. Tokens can be generated within the Gotify web interface.
-
comment: Comment for this target
The Gotify target plugin will respect the HTTP proxy settings from the datacenter configuration |
Example configuration (/etc/pve/notifications.cfg):
gotify: example server http://gotify.example.com:8888 comment Send to multiple users/addresses
The matching entry in /etc/pve/priv/notifications.cfg, containing the secret token:
gotify: example token somesecrettoken
Notification Matchers
Notification matchers route notifications to notification targets based on their matching rules. These rules can match certain properties of a notification, such as the timestamp (match-calendar), the severity of the notification (match-severity) or metadata fields (match-field). If a notification is matched by a matcher, all targets configured for the matcher will receive the notification.
An arbitrary number of matchers can be created, each with with their own matching rules and targets to notify. Every target is notified at most once for every notification, even if the target is used in multiple matchers.
A matcher without any matching rules is always true; the configured targets will always be notified.
matcher: always-matches target admin comment This matcher always matches
Matcher Options
-
target: Determine which target should be notified if the matcher matches. can be used multiple times to notify multiple targets.
-
invert-match: Inverts the result of the whole matcher
-
mode: Determines how the individual match rules are evaluated to compute the result for the whole matcher. If set to all, all matching rules must match. If set to any, at least one rule must match. a matcher must be true. Defaults to all.
-
match-calendar: Match the notification’s timestamp against a schedule
-
match-field: Match the notification’s metadata fields
-
match-severity: Match the notification’s severity
-
comment: Comment for this matcher
Calendar Matching Rules
A calendar matcher matches the time when a notification is sent agaist a configurable schedule.
-
match-calendar 8-12
-
match-calendar 8:00-15:30
-
match-calendar mon-fri 9:00-17:00
-
match-calendar sun,tue-wed,fri 9-17
Field Matching Rules
Notifications have a selection of metadata fields that can be matched.
-
match-field exact:type=vzdump Only match notifications about backups.
-
match-field regex:hostname=^.+\.example\.com$ Match the hostname of the node.
If a matched metadata field does not exist, the notification will not be matched. For instance, a match-field regex:hostname=.* directive will only match notifications that have an arbitraty hostname metadata field, but will not match if the field does not exist.
Severity Matching Rules
A notification has a associated severity that can be matched.
-
match-severity error: Only match errors
-
match-severity warning,error: Match warnings and error
The following severities are in use: info, notice, warning, error, unknown.
Examples
matcher: workday match-calendar mon-fri 9-17 target admin comment Notify admins during working hours matcher: night-and-weekend match-calendar mon-fri 9-17 invert-match true target on-call-admins comment Separate target for non-working hours
matcher: backup-failures match-field exact:type=vzdump match-severity error target backup-admins comment Send notifications about backup failures to one group of admins matcher: cluster-failures match-field exact:type=replication match-field exact:type=fencing mode any target cluster-admins comment Send cluster-related notifications to other group of admins
The last matcher could also be rewritten using a field matcher with a regular expression:
matcher: cluster-failures match-field regex:type=^(replication|fencing)$ target cluster-admins comment Send cluster-related notifications to other group of admins
Notification Events
Event | type | Severity | Metadata fields (in addition to type) |
---|---|---|---|
System updates available |
package-updates |
info |
hostname |
Cluster node fenced |
fencing |
error |
hostname |
Storage replication failed |
replication |
error |
- |
Backup finished |
vzdump |
info (error on failure) |
hostname |
Mail for root |
system-mail |
unknown |
- |
Field name | Description |
---|---|
type |
Type of the notifcation |
hostname |
Hostname, including domain (e.g. pve1.example.com) |
System Mail Forwarding
Certain local system daemons, such as smartd, generate notification emails that are initially directed to the local root user. Proxmox VE will feed these mails into the notification system as a notification of type system-mail and with severity unknown.
When the forwarding process involves an email-based target (like sendmail or smtp), the email is forwarded exactly as received, with all original mail headers remaining intact. For all other targets, the system tries to extract both a subject line and the main text body from the email content. In instances where emails solely consist of HTML content, they will be transformed into plain text format during this process.
Permissions
In order to modify/view the configuration for notification targets, the Mapping.Modify/Mapping.Audit permissions are required for the /mapping/notifications ACL node.
Testing a target requires Mapping.Use, Mapping.Audit or Mapping.Modify permissions on /mapping/notifications