Events and SNMP Messages
------------------------

The following conditions are monitored for which SNMP traps can be sent.
The trap levels and message strings are shown for the condition:

* Script install failures

  * 'error','upgrade failed','upgrade failed as other activity is in progress

* Backup Success/Failure

  * 'error', 'Backup failed',
  * 'info', 'Backup completed'
  * 'error', 'ERROR: The last backup was more than 2 days ago', 'Backup list:
  * 'info', 'INFO: Backups now runs regularly', 'Backup list:

* Restore Success/Failure

  * 'info', 'Backup restored
  * 'error', 'Backup restoration failed  

* Nginx reconfiguration - If a webproxy is unable to contact one of the upstream systems

  * "error", "nginx upstream failure", "upstream %s server %s failed: %s"
  * "info", "nginx upstreams OK", "nginx upstream servers returned to normal"

* Disk full/cleared (if a monitored disk is above 80%, it will send a trap, and also when this is cleared)

  * level, 'DISK ALMOST FULL: Disk <disk name> is more than 80 percent full'
  * level, 'DISK STATUS:  Disk <disk name> is now running below 80 percent'

  * 'error', 'DISK ALMOST FULL: Disk /var/log is more than 80 percent full','Use log purge to purge all rotated logs\n\nCurrent disk status:

* Email failure (if a monitoring email was set up, and system cannot reach it)

  * 'error', 'ERROR: Trouble sending health email', 'Trouble sending health email'
  * 'info', 'INFO: Health emails is now being sent', 'Health emails is now being sent'
  * 'info', 'INFO: Messages for <username> auto archived as it reached more than 500' %  user,  'Use the following command to view archived messages:\n\nlog view <username>' %
  * 'info', 'INFO: The total local messages for <username> is now under 200'
  * 'warn', 'WARNING: Not all notify levels is configured with an external email address '
  * 'info', 'INFO: All notify levels is now configured with an external email address'

* Database usage

  * 'warn', reason='WARN: Database <database name> exceeded threshold'
  * 'info', reason='INFO: Database <database name> returned to normal'

* High disk latency

  * 'error', 'ERROR: Disk slow ', 'Disk latency info:
  * 'info', 'INFO: The disk latency returned to normal', 'The disk latency returned to normal.'  

* Database failover (if the DB fails over from one node to another twice in a 5 minute period)

  * 'error', 'ERROR: The db is failing over constantly within 5 min', 'Cluster failover status:
  * 'info', 'INFO: The db failover status returned to normal', 'Cluster failover status:  

* Large log file warning

  * 'error', 'ERROR: Log files larger than 1Gig found in /var/log ', 'Logrotation was executed to rotate the following logs: <log filename>'
  * 'info', 'INFO: /var/log rotated', '/var/log rotated'
 
* Network

  * 'error', 'ERROR: Network Failures', 'The following network failures occured: <network errors>'
  * 'info', 'INFO: Network failures resolved', 'Network failures resolved', 

* Service failures

  * 'error', 'ERROR: Service Failures'
  * 'info', 'INFO: Services started successfully'

* Security updates available

  * 'warn', 'WARNING: Security Updates available', '<number> updates available' 
  * 'info', 'INFO: Security Updates applied', 'There are 0 security updates available

* High memory and CPU usage

  * 'error', 'ERROR: High memory usage', 'Memory activity:
  * 'info', 'INFO: Memory usage returned to normal', 'Memory more than 1024MB'
  * 'warn', 'WARNING: High CPU usage', 'CPU activity:
  * 'error', 'ERROR: Extremely high CPU usage', 'CPU activity:
  * 'info', 'INFO: CPU usage returned to normal'

* High swap usage

* NTP configuration issues

  * 'warn', 'WARNING: The ntp daemon has stopped on <server name>', 'Run \'app start services:time\' to restart ntpd',
  * 'warn', 'WARNING: The ntp offset exceeds 1 second on  %s' % system_info, 'ntp offset exceeds 1 second\nCurrent ntp offset value: <ntp offset>'
  * 'info', 'INFO: ntp is now configured for <server name>', 'NTP cleared', value=0
  * 'info', 'INFO: The ntp offset restored to normal on <server name>', 'ntp offset cleared\nCurrent ntp offset value: <ntp offset>'

* DNS configuration issues

  * 'warn', 'WARNING: No dns configured for <server name>', 'It is recommended that the dns is configured.\n\nTo configure
    dns use the following command:\nnetwork dns <server1> <server2>'

  * 'info', 'INFO: dns is now configured for <server name>', 'DNS cleared', value=0

* Domain configuration issues

  * 'info', 'INFO: domain is now configured for <server name>, 'Domain cleared', value=0
  * 'warn', 'WARNING: No domain configured for <server name>, 'It is recommended that the domain is configured.\n\nTo
    configure the domain use the following command:\nnetwork domain <server1> <server2>',

* Log processing failure

  * 'error', 'System unable to send <event type> messages to <IP>'

* Monthly report export failure: (example at :ref:`internal_schedules`)

  * ERROR: License file generation failed. The license audit report scheduled for <month> <year> was not successful.
    Please contact your VOSS account manager.


For details, refer to the topic on the individual SNMP trap.