Events and SNMP Messages

The following conditions are monitored for which SNMP traps can be sent. The trap levels and message strings are shown for the condition:

  • Script install failures
    • ‘error’,’upgrade failed’,’upgrade failed as other activity is in progress
  • Backup Success/Failure
    • ‘error’, ‘Backup failed’,
    • ‘info’, ‘Backup completed’
    • ‘error’, ‘ERROR: The last backup was more than 2 days ago’, ‘Backup list:
    • ‘info’, ‘INFO: Backups now runs regularly’, ‘Backup list:
  • Restore Success/Failure
    • ‘info’, ‘Backup restored
    • ‘error’, ‘Backup restoration failed
  • Nginx reconfiguration - If a webproxy is unable to contact one of the upstream systems
    • “error”, “nginx upstream failure”, “upstream %s server %s failed: %s”
    • “info”, “nginx upstreams OK”, “nginx upstream servers returned to normal”
  • Disk full/cleared (if a monitored disk is above 80%, it will send a trap, and also when this is cleared)
    • level, ‘DISK ALMOST FULL: Disk <disk name> is more than 80 percent full’
    • level, ‘DISK STATUS: Disk <disk name> is now running below 80 percent’
    • ‘error’, ‘DISK ALMOST FULL: Disk /var/log is more than 80 percent full’,’Use log purge to purge all rotated logsnnCurrent disk status:
    • ‘audit’, ‘DISK ALMOST FULL: Disk /var/log is more than 80 percent full’,’Use log purge to purge all rotated logsnnCurrent disk status:
  • Email failure (if a monitoring email was set up, and system cannot reach it)
    • ‘error’, ‘ERROR: Trouble sending health email’, ‘Trouble sending health email’
    • ‘info’, ‘INFO: Health emails is now being sent’, ‘Health emails is now being sent’
    • ‘info’, ‘INFO: Messages for <username> auto archived as it reached more than 500’ % user, ‘Use the following command to view archived messages:nnlog view <username>’ %
    • ‘info’, ‘INFO: The total local messages for <username> is now under 200’
    • ‘warn’, ‘WARNING: Not all notify levels is configured with an external email address ‘
    • ‘info’, ‘INFO: All notify levels is now configured with an external email address’
  • Database usage
    • ‘warn’, reason=’WARN: Database <database name> exceeded threshold’
    • ‘info’, reason=’INFO: Database <database name> returned to normal’
  • High disk latency
    • ‘error’, ‘ERROR: Disk slow ‘, ‘Disk latency info:
    • ‘info’, ‘INFO: The disk latency returned to normal’, ‘The disk latency returned to normal.’
  • Database failover (if the DB fails over from one node to another twice in a 5 minute period)
    • ‘error’, ‘ERROR: The db is failing over constantly within 5 min’, ‘Cluster failover status:
    • ‘info’, ‘INFO: The db failover status returned to normal’, ‘Cluster failover status:
  • Large log file warning
    • ‘error’, ‘ERROR: Log files larger than 1Gig found in /var/log ‘, ‘Logrotation was executed to rotate the following logs: <log filename>’
    • ‘info’, ‘INFO: /var/log rotated’, ‘/var/log rotated’
  • Network
    • ‘error’, ‘ERROR: Network Failures’, ‘The following network failures occured: <network errors>’
    • ‘info’, ‘INFO: Network failures resolved’, ‘Network failures resolved’,
  • Service failures
    • ‘error’, ‘ERROR: Service Failures’
    • ‘info’, ‘INFO: Services started successfully’
  • Security updates available
    • ‘warn’, ‘WARNING: Security Updates available’, ‘<number> updates available’
    • ‘info’, ‘INFO: Security Updates applied’, ‘There are 0 security updates available
  • High memory and CPU usage
    • ‘error’, ‘ERROR: High memory usage’, ‘Memory activity:
    • ‘info’, ‘INFO: Memory usage returned to normal’, ‘Memory more than 1024MB’
    • ‘warn’, ‘WARNING: High CPU usage’, ‘CPU activity:
    • ‘error’, ‘ERROR: Extremely high CPU usage’, ‘CPU activity:
    • ‘info’, ‘INFO: CPU usage returned to normal’
  • High swap usage
  • NTP configuration issues
    • ‘warn’, ‘WARNING: The ntp daemon has stopped on <server name>’, ‘Run ‘app start services:time’ to restart ntpd’,
    • ‘warn’, ‘WARNING: The ntp offset exceeds 1 second on %s’ % system_info, ‘ntp offset exceeds 1 secondnCurrent ntp offset value: <ntp offset>’
    • ‘info’, ‘INFO: ntp is now configured for <server name>’, ‘NTP cleared’, value=0
    • ‘info’, ‘INFO: The ntp offset restored to normal on <server name>’, ‘ntp offset clearednCurrent ntp offset value: <ntp offset>’
  • DNS configuration issues
    • ‘warn’, ‘WARNING: No dns configured for <server name>’, ‘It is recommended that the dns is configured.nnTo configure dns use the following command:nnetwork dns <server1> <server2>’
    • ‘info’, ‘INFO: dns is now configured for <server name>’, ‘DNS cleared’, value=0
  • Domain configuration issues
    • ‘info’, ‘INFO: domain is now configured for <server name>, ‘Domain cleared’, value=0
    • ‘warn’, ‘WARNING: No domain configured for <server name>, ‘It is recommended that the domain is configured.nnTo configure the domain use the following command:nnetwork domain <server1> <server2>’,
  • Log processing failure
    • ‘error’, ‘System unable to send <event type> messages to <IP>’
  • Monthly report export failure: (example at Internal Report Schedules)
    • ERROR: License file generation failed. The license audit report scheduled for <month> <year> was not successful. Please contact your VOSS account manager.

For details, refer to the topic on the individual SNMP trap.