Managing Actions And Notifications

---

17.1 Overview

NetVigil has a very flexible action and notification engine which can send email, alphanumeric pages, SNMP traps and be extended to run any other program (such as restarting a process or deleting log files, etc.). The module has a built in escalation engine so that notifications can be sent to different people as devices remain in a non-OK state for an extended period of time, The notifications can also be customized based on the time of day and week by applying custom `schedules' to the action profiles.

All notifications are handled by the DGEs themselves (and not a centralized BVE), which allows for a highly fault tolerant and distributed architecture.

Details on setting up system wide configurations are described in the "NetVigil Administrator Reference". This chapter describes configuring actions & notifications using the web interface by the end users.

17.2 Managing Action Profiles

Typically, actions are some form of notification that a test result has crossed a defined threshold into OK, WARNING, CRITICAL or UNKNOWN status. An Action Profile is a list of up to five actions, allowing the user to define multiple notification recipients and specific notification rules for each recipient.

There are many types of built in notification and action mechanisms in NetVigil:

Email

This is the simplest notification- it sends an email to the specified email address using the mail gateways specified by the NetVigil administrator.

When creating an action profile, you may specify multiple email recipient for each of the action item by separating them by comma (,). For example:

admin@your.domain, noc@your.isp 

Note that the maximum length for the message recipient (for each action item) is 255 characters. So if you are going to be sending email to a large number of recipients, it may be easier to setup an email alias on your mail server and use the new alias as the target recipient for the action profile.

The notification content can be customized on a global basis by the NetVigil administrator and described in the "Administrator Reference".

Alphanumeric Paging

To send a pager using a directly attached modem, select Alphanumeric Pager from the Notify Using list. Then, specify a Message Recipient in the format PIN@PC, where PIN is the recipient's Personal Identification Number (usually the pager number) and PC is a `paging central' location defined by your NetVigil Administrator for the DGE that will generate the notification. See "To create an action profile:" on page 235 for detailed instructions.

For example, assume that an Administrator has set up a `paging central' location called nextel that is used when NetVigil sends pages to Nextel customers. A typical pager message recipient might be 7325551212@nextel.

The notification content can be customized on a global basis by the NetVigil administrator and described in the "Administrator Reference".

RT or Remedy Ticketing System

(This feature might require purchasing a license from Fidelia). You can directly open a trouble ticket in RT or Remedy, and have a URL to this ticket displayed in the Status page.

Send SNMP trap

You can send an SNMP trap to another SNMP trap handler if desired.

Run External Scripts

NetVigil's plug-in architecture allows you to create any number of additional "plugins" that will be displayed in the drop down list. For details on how to create Plug-in Actions, please see the "Administrator Reference".

Figure 17.1 below illustrates how a test result can trigger multiple actions.

Figure 17.1 Example of Test Results Triggering Actions

Action Profiles can be created by either end-users or an Administrator to notify up to five separate recipients when a test status changes, or when a test status has been in a particular state for a predetermined number of test cycles. In the above example, the action has been configured with three separate sub-actions in a typical escalation scenario. When the Ping RTT test on the NT Server reaches a WARNING status, the NOC Administrator receives an E-mail notification of the problem. If the test crosses the upper threshold to CRITICAL status, the NOC Manager is notified. Once the test has remained in CRITICAL status for three (3) test cycles, the senior management is notified.

Note Action Profiles must first be defined before they are available to be assigned to one or more tests.

Note When you receive an E-mail notification that a test result for one of your devices has crossed into WARNING or CRITICAL status, that E-mail has only been sent to you or to the distribution list of which you are a member. The thresholds your administrator uses to trigger notifications may be different from your personal thresholds. Please do not create actions to alert the administrator or IT staff.

To create an action profile:
Click on the MANAGE tab on main navigation bar.
Click on the ACTIONS tab on the secondary navigation bar. You will be taken to the Manage Action Profiles page.
Click on the Create An Action Profile link in the information window and you will be taken to the Create Action Profile page.
Enter a unique action name (required) and a description (recommended).
For each sub-action (maximum of 5), choose the type of notification in the Notify Using drop down list (i.e. E-mail or instant messenger provider) and the message recipient's address. This is usually yourself or someone else who is responsible for monitoring your system's performance. You can enter multiple message recipients, separated by commas (e.g., jdoe@acme.com, fcheng@acme.com).
Check the box for each instruction you want followed ("Trigger on Warning" and "Trigger on Critical" are checked by default).
If desired, check either the box to delay initial notification and/or the box to identify a repeat interval, and the number of applicable test cycles for each.
Repeat steps 4. - 7. as desired. If you are notifying the same person via different actions, remember to avoid overlapping logic between the sub-actions, otherwise the recipient may receive duplicate notifications for a single test event.
Click on the Create Action Profile button to create the new action.
To update an action profile:
Click on the MANAGE tab on main navigation bar.
Click on the ACTIONS tab on the secondary navigation bar and you will be taken to the Manage Actions page.
Click on the Update link in the row for the action you wish to update. You will be taken to the Update Action page for that action.
Make the desired changes to any one of the sub-actions.
If you have not already configured five (5) sub-actions, you may add more by filling in the fields as described above in "To create an action profile:" on page 235.
You can also delete sub-actions by checking the box marked Delete this Action next to the unwanted sub-action.
Click on the Update Action Profile button at the bottom of the page to save your changes.
To assign an action profile to multiple devices or tests:
Click on the MANAGE tab on main navigation bar.
Click on the ACTIONS tab on the secondary navigation bar and you will be taken to the Manage Actions page.
Click on the Select Devices for Action link in the row for the action you wish to assign. You will be taken to the Assign Actions page.
For each device in the list, checking the ALL TESTS checkbox will assign the action to all the tests on the selected device(s).
For each device in the list, checking the SELECT TESTS checkbox will display another window for you to individually select the test(s) to which you want the action assigned.
Click on the Assign Action button at the bottom of the page to save your changes.


To permanently delete an action profile:
Click on the MANAGE tab on main navigation bar.
Click on the ACTIONS tab on the secondary navigation bar and you will be taken to the Manage Actions page.
Click on the Delete link for the action you wish to delete and you will be taken to a confirmation screen.
Click on the Delete button to confirm the deletion or Cancel to return to the Manage Actions page.
To modify actions for all tests on a device

admin -> actions -> "select devices for action" link -> select devices -> select tests

17.2.1 Assigning Time Schedules to Actions

You can create schedules and assign them to actions to run them at a specified day of week or hour of day (these schedules are similar to the ones that can be applied to tests as described in Section 16.6, "Configuring Test Schedules" on page 227).

To create a schedule
Go to Manage -> Other
Click on Manage Schedules
Click on Create a Schedule

These schedules are displayed in the user's configured timezone (set in Manage -> Prefs)

17.2.2 Suspending Actions for Acknowledged Tests

It is possible to suppress notifications while acknowledging or suppressing a test from the Event Manager by clicking on the appropriate checkbox.

17.3 Smart Notifications

NetVigil correlates OK notifications with prior non-OK notifications. So if you first receive an email for critical state, when the test returns to OK state, you will also get a notification.However, if your action profile was setup in a way (e.g. wait 2 test cycles) that the rest returned from critical to OK state before notification for critical state was not sent, no notification for OK state would be sent out either. If you want to disable this behavior (i.e. be notified of OK state regardless), all you have to do is disable the "smart notification" option for the device in question via manage -> devices -> update.

NetVigil's Smart Notification was designed to eliminate sending multiple notifications when a device goes down or is unavailable. Often, many configured tests on a device have action profiles assigned to them to notify various recipients when test status reaches Warning, Critical, Unknown, or all three.

Smart notification relies on the inherent dependency between the ping packet loss test results and the availability of the device. If the ping packet loss test returns 100%, then communication with the device has somehow been lost. This could result in all tests sending notifications every test cycle, especially if they are configured to notify on Unknown, which is what the test results will likely be in this situation.

To configure Smart Notification during new device configuration:
Click on the MANAGE tab. You will be taken to the Manage Devices page.
Click on the Create A Device link. For a detailed description of device creation, see "Manage NetVigil" in "Web User Guide"
Fill out the required information in the Create Device page.
Check the box labeled Smart Notification.
Proceed with test discovery by selecting the Create Device button.

Note You MUST configure a Packet Loss test for the device. If you don't, this feature will not work correctly.

If you are receiving notifications when the device is down you should check the following items: · Confirm that you have in fact configured Smart Notification on the device by navigating to the Manage Devices page and selecting the Update link for the device; the Smart Notification box should be checked. · Confirm that you have a Packet Loss test configured on the device by navigating to the Manage Devices page and selecting the Tests link for the device; the list of test should include the Packet Loss test. · If both of the above are configured properly, the notifications that you have received are likely a result of having been queued up prior the Packet Loss returning 100%. That is, if tests are scheduled in the queue ahead of the Packet Loss test, they will be executed prior to the trigger that suppresses all further notifications when Packet Loss = 100%. To avoid notifications in this case it is advisable that you change your action profiles to either 1) not notify on UNKNOWN; or 2) only notify after 2 or 3 test cycles have passed.