Application Delivery Management

Create event rules

You can configure rules to monitor specific events. Rules make it easier to monitor a large number of events generated across your infrastructure.

You can filter a set of events by configuring rules with specific conditions and assigning actions to the rules. When the events generated meet the filter criteria in the rule, the action associated with the rule is run. The conditions for which you can create filters are: severity, Citrix Application Delivery Controller (NetScaler) instances, category, failure objects, configuration commands, and messages.

You can assign the following actions to the events:

  • Send e-mail Action: Send an email for the events that match the filter criteria.

  • Send Trap Action: Send or forward SNMP traps to an external trap destination

  • Run Command Action: Run a command when an incoming event meets the configured rule.

  • Execute Job Action: Run a job is for events that match the filter criteria that you’ve specified.

  • Suppress Action: Suppresses drop an event for a specific time period.

  • Send Slack Notifications: Send notifications on the configured Slack channel for the events that match the filter criteria.

  • Send PagerDuty Notifications: Send event notifications based on the PagerDuty configurations for the events that match the filter criteria.

  • Send ServiceNow Notifications: Auto-generate ServiceNow incidents for an event that match the filter criteria.

For more information, see Add event rule actions

You can also have notifications resent at a specified interval until an event is cleared. And you can customize the email with a specific subject line, user message, and attachment.

Monitor events with rules

For example, as an administrator you might want to monitor “high CPU usage” events for specific NetScaler instances if those events can lead to an outage of your NetScaler instances. You can:

  • Create a rule to monitor the instances and specify an action that sends you an email notification when an event in the “high CPU usage” category occurs.

  • Schedule the rule to run at a specific time, such as between 11 AM to 11 PM, so that you are not notified every time there is an event generated.

Configuring an event rule involves the following tasks:

  1. Define the rule

  2. Choose the severity of the event that the rule detects

  3. Specify the category of the event

  4. Specify NetScaler instances to which the rule applies

  5. Select failure objects

  6. Specify advanced filters

  7. Specify actions to be taken when the rule detects an event

Step 1 - Define an event rule

Navigate to Infrastructure > Events > Rules, and click Add. If you want to enable your rule, select the Enable Rule check box.

You can set the Event Age option to specify the time interval (in seconds) after which NetScaler ADM refreshes an event rule.

Note:

The minimum value for the event age is 60 seconds. If you keep the Event Age field blank, the event rule is applied immediately after the event is occurred.

Based on the example above, you may want to be notified by email every time your NetScaler instance has a “high CPU usage” event for 60 seconds or longer. You can set the event age as 60 seconds, so that every time your NetScaler instance has a “high CPU usage” event for 60 seconds or more, you receive an email notification with details of the event.

Create event rule

You can also filter event rules by Instance Family to track the NetScaler instance from which NetScaler ADM receives an event.

If you want to include a regular expression other than asterisk (*) pattern matching, select Enable Advanced Filter with Regex Matching.

Step 2 - Choose the severity of the event

You can create event rules that use the default severity settings. Severity specifies the current severity of the events you which you want to add the event rule.

You can define the following levels of severity: Critical, Major, Minor, Warning, Clear, and Information.

Define severity levels

Note

You can configure severity for both generic and Advanced-specific events. To modify event severity for NetScaler instances managed on NetScaler ADM, navigate to Infrastructure > Events > Event Settings. Choose the Category for which you want to configure event severity and click Configure Severity. Assign a new severity level and click OK.

Step 3 - Specify the event category

You can specify the category or categories of the events generated by your NetScaler instances. All categories are created on NetScaler instances. These categories are then mapped with NetScaler ADM that can be used to define event rules. Select the category you want to consider and move it from the Available table to the Configured table.

In the example above, you must choose “cpuUsageHigh” as the event category from the table displayed.

Define event category

Step 4 - Specify NetScaler instances

Select the IP addresses of the NetScaler instances for which you want to define the event rule. In the Instances section, click Select Instances. In the Select Instances page, choose your instances, and click Select.

Specify instances

Step 5 - Select failure objects

You can either select a failure object from the list provided or add a failure object for which an event has been generated. You can also specify a regular expression to add failure objects. Depending on the specified regular expression, the failure objects are automatically added to the list. Failure objects are entity instances or counters for which an event has been generated.

Important

To list failure objects using regular expression, select Enable Advanced Filter with Regex Matching in Step 1.

The failure object affects the way that an event is processed and ensures it reflects the exact problem as notified. With this filter, you can track issues on the failure objects quickly and identify the cause for an issue. For example, if a user has login issues, then the failure object here is the user name or password, such as nsroot.

This list can contain counter names for all threshold‐related events, entity names for all entity‐related events, certificate names for certificate‐related events, and so on.

Object list

Step 6 - Specify advanced filters

You can further filter an event rule by:

  • Configuration Commands - You can specify the complete configuration command, or specify a regular expression to filter events.

    You can further filter the event rule by the command’s authentication status and/ or its execution status. For example, for a NetscalerConfigChange event, type [.]*bind system global policy_name[.]*.

    Advance configuration commands filter

  • Messages - You can specify the complete message description, or specify a regular expression to filter the events.
    For example, for a NetscalerConfigChange event, type [.]*ns_client_ipaddress :10.122.132.142[.]* or ns_client_ipaddress :^([.]*10.122.132.142[.]*).

    Advance message filters

Step 7 - Add event rule actions

You can add event rule actions to assign notification actions for an event. These notifications are sent or performed when an event meets the defined filter criteria that you’ve set above. You can add the following event actions:

  • Send email Action

  • Send Trap Action

  • Run Command Action

  • Run Job Action

  • Suppress Action

  • Send Slack Notifications

  • Send PagerDuty Notifications

  • Send ServiceNow Notifications

To set email Event Rule Action

When you choose the Send email Action event action type, an email is triggered when the events meet the defined filter criteria. You must either create an email distribution list by providing mail server or mail profile details or you can select an email distribution list that you’ve previously created.

Due to a high number of virtual servers being configured in NetScaler ADM, you might receive a high number of emails every day. The emails have a default subject line that provides information about the severity of the event, the category of the event and the failure object. But the subject line does not carry any information about the name of the virtual server where these events originate from. You now have an option to include some additional information like the name of the affected entity, name of the failure object.

You can also add a customized subject line and a user message, and upload an attachment to your email when an incoming event matches the configured rule.

While sending emails for event notifications, you might want to send a test email to test the configured settings. The “Test” button now allows you to send a test email after configuring an email server, associated distributed lists, and other settings. This feature ensures that settings are working fine.

You can also ensure that all critical events are addressed and no important email notifications are missed, by selecting the Repeat Email Notification until the event is cleared check box to send repeated email notifications for event rules that meet the criteria you’ve selected. For example, if you’ve created an event rule for instances that involve disk failures, and you want to be notified until the issue is resolved, you can opt to receive repeated email notifications about those events.

Set email event rule action

To set Trap Event Rule Action

When you choose the Send Trap Action event action type, SNMP traps are sent or forwarded to an external trap destination. By defining a trap distribution list (or a trap destination and trap profile details), trap messages are sent to specific trap listeners when events meet the defined filter criteria.

To set the Run Command Action

When you choose the Run Command Action event action, you can create a command or a script that can be run on NetScaler ADM for events matching a particular filter criterion.

You can also set the following parameters for the Run Command Action script:

   
Parameter Description
$source This parameter corresponds to the source IP address of the received event.
$category This parameter corresponds to the type of traps defined under category of the filter
$entity This parameter corresponds to the entity instances or counters for which an event has been generated. It can include the counter names for all threshold-related events, entity names for all entity-related events, and certificate names for all certificate-related events.
$severity This parameter corresponds to the severity of the event.
$failureobj The failure object affects the way that an event is processed and ensures that the failure object reflects the exact problem as notified. This can be used to track down problems quickly and to identify the reason for failure, instead of simply reporting raw events.

Note

During command execution, these parameters are replaced with actual values.

For example, consider that you want to set a run command action when a load balancing virtual server status is Down. As an administrator, you might want to consider providing a quick workaround by adding another virtual server. In NetScaler ADM, you can:

  • Write a script (.sh) file.

    The following is a sample script (.sh) file:

     #!/bin/sh
     source=$1
     failureobj=$2
     payload='{"params":{"warning":"YES"},"lbvserver":{"name":"'$failureobj'","servicetype":"HTTP","ipv46":"x.x.x.x","port":"80","td":"","m":"IP","state":"ENABLED","rhistate":"PASSIVE","appflowlog":"ENABLED","
     bypassaaaa":"NO","retainconnectionsoncluster":"NO","comment":""}}'
     url="http://$source/nitro/v1/config/lbvserver"
     curl --insecure -basic -u nsroot:nsroot -H "Content-type: application/json" -X POST -d $payload $url
    
     <!--NeedCopy-->
    
  • Save the .sh file in any persistent location on NetScaler agent. For example, /var.

  • Provide the .sh file location in NetScaler ADM to run when the rule criteria are met.

To set the Run Command action for creating a new virtual server:

  1. Define the rule

  2. Select the severity of the event

  3. Select the event category entitydown

  4. Select the instance that has the virtual server configured

  5. Select or create a failure object for the virtual server

  6. Under Event Rule Actions, click Add Action and select Run Command Action from the Action Type list.

  7. Under Command Execution List, click Add.

    The Create Command Distribution List page is displayed.

    1. In Profile Name, specify a name of your choice

    2. In Run Command, specify the NetScaler agent location, where the script must be run. For example: /sh/var/demo.sh $source $failureobj.

    3. Select Append Output and Append Errors

      Note

      You can enable the Append Output and Append Errors options if you want to store the output and errors generated (if any) when you run a command script in the NetScaler ADM server log files. If you do not enable these options, NetScaler ADM discards all outputs and errors generated while running the command script.

    4. Click Create.

  8. In the Add Event Action page, click OK.

    Add event action

    Note

    You can enable the Append Output and Append Errors options if you want to store the output and errors generated (if any) when you run a command script in the NetScaler ADM server log files. If you do not enable these options, NetScaler ADM discards all outputs and errors generated while running the command script.

To set the Execute Job Action

By creating a profile with configuration jobs, a job is run as a built-in job or a custom job for NetScaler and NetScaler SDX instances, for events and alarms that match the filter criteria you’ve specified.

  1. Under Event Rule Actions, click Add Action and select Execute Job Action from the Action Type drop-down list.

  2. Create a profile with a job that you want to run when the events meet the defined filter criteria.

  3. While creating a job, specify a profile name, the instance type, the configuration template, and what action you’d like to perform if the commands on the job fail.

  4. Based on the instance type selected and the configuration template chosen, specify your variables values and click Finish to create the job.

Run job action

To set the Suppress Action

When you choose the Suppress Action event action, you can configure a time period, in minutes, for which an event is suppressed or dropped. You can suppress the event for a minimum of 1 minute.

Set suppress action

To set Slack notifications from NetScaler ADM

Configure the required Slack channel by providing the profile name and the webhook URL in NetScaler ADM GUI. The event notifications are then sent to this channel. You can configure multiple Slack channels to receive these notifications

  1. In NetScaler ADM, navigate to Infrasturcture > Events > Rules, and click Add to create a rule.

  2. On the Create Rule page, set the rule parameters such as severity and category. Select instances and also failure objects that must be monitored.

  3. Under Event Rule Actions, click Add Action. Then, select Send Slack Notifications from the Action Type list and select Slack Profile List.

  4. You can also add a Slack profile list by clicking Add next to the Slack Profile List field.

  5. Type the following parameters to create a profile list:

    1. Profile Name. Type a name for the profile list to be configured on NetScaler ADM

    2. Channel Name. Type the name of the Slack channel to which the event notifications are to be sent.

    3. Webhook URL. Type the Webhook URL of the channel that you have entered earlier. Incoming Webhooks are a simple way to post messages from external sources into Slack. The URL is internally linked to the channel name and all event notifications are sent to this URL to be posted on the designated Slack channel. An example of a webhook is as follows: https://hooks.slack.com/services/T0******E/B9X55DUMQ/c4tewWAiGVTT51Fl6oEOVirK

  6. Click Create and click OK in the Add Event Action window.

Note:

You can also add the Slack profiles by navigating to System > Notifications > Slack Profiles. Click Add and create the profile as described in the earlier section.

You can view the status of the Slack profiles that you have created.

Your event rule is now created with appropriate filters and well defined event rule actions.

To set PagerDuty notifications from NetScaler ADM

You can add a PagerDuty profile as an option in NetScaler ADM to monitor the incident notifications based on your PagerDuty configurations. PagerDuty enables you to configure notifications through email, SMS, push notification, and phone call on registered number.

Before you add a PagerDuty profile in NetScaler ADM, ensure you have completed the required configurations in PagerDuty. For more information, see PagerDuty documentation.

You can select your PagerDuty profile as one of the options to get notifications for the following features:

  • Events – List of events that are generated for NetScaler instances.

  • Licenses – List of licenses that are currently active, about to expire, and so on.

  • SSL Certificates – List of SSL certificates that are added to NetScaler instances.

To add a PagerDuty profile in ADM:

  1. Log on to NetScaler ADM using administrator credentials.

  2. Navigate to Settings > Notifications > PagerDuty Profiles.

  3. Click Add to create a new profile.

  4. In the Create PagerDuty Profile page:

    1. Provide a profile name of your choice.

    2. Enter the Integration Key.

      You can get the Integration Key from your PagerDuty portal.

    3. Click Create.

    Use case:

    Consider a scenario that you:

    • want to send notifications to your PagerDuty profile.

    • have configured phone call as an option in PagerDuty to receive notifications.

    • want to get phone call alerts for NetScaler events.

    To configure:

    1. Navigate to Events > Rules

    2. On the Create Rule page, configure all other parameters to create a rule.

    3. Under Create Rule Actions, click Add Action.

      The Add Event Action page is displayed.

      1. Under Action Type, select Send PagerDuty Notifications.

        PagerDuty notification

      2. Select your PagerDuty profile and click OK.

        Create PagerDuty profile

    After the configuration is complete, whenever a new event is generated for NetScaler instance, you will receive a phone call. From the phone call, you can decide to:

    • Acknowledge the event

    • Mark it as resolved

    • Escalate to another team member

To auto-generate ServiceNow incidents from NetScaler ADM

You can auto-generate ServiceNow incidents for NetScaler ADM events by selecting the ServiceNow profile on the NetScaler ADM GUI. You must choose the ServiceNow profile in NetScaler ADM to configure an event rule.

Before you configure an event rule to auto-generate ServiceNow incidents, integrate NetScaler ADM with a ServiceNow instance. For more information, see Configure ITSM adapter for ServiceNow.

To configure an event rule, navigate to Events > Rules.

  1. On the Create Rule page, configure all other parameters to create a rule.

  2. Under Create Rule Actions, click Add Action.

    The Add Event Action page is displayed.

    1. In Action Type, select Send ServiceNow Notifications.

    2. In ServiceNow Profile, select the Citrix_Workspace_SN profile from the list.

    3. Click OK.