Product Documentation
ADC
14.1 - Current Release 13.1 13.0 12.1
View PDF

Export management logs directly from NetScaler to Splunk

August 27, 2024
Contributed by: 
C

Note:

Export of management logs directly from NetScaler to Splunk is only supported in NetScaler 13.1-37.199 and later FIPS releases.

You can now export management logs (non-packet engine logs) on a category basis such as shell, access, and nsmgmt logs from NetScaler to industry standard log aggregator platforms such as Splunk. Using the visualization tools at Splunk, you can get meaningful insights about the exported data.

There are multiple ways to export management logs from NetScaler to Splunk. You can either configure Splunk as an HTTP server or a syslog server. In the HTTP server configuration, you can use the HTTP event collector to send management logs over HTTP (or HTTPS) directly to the Splunk platform from your NetScaler. In the syslog server configuration, management logs are sent as syslog payloads from NetScaler to Splunk.

Export management logs to Splunk configured as an HTTP server

To configure the export of management logs you must perform the following steps:

  1. Configure an HTTP event collector on Splunk.
  2. Create a collector service and a time series analytics profile on NetScaler.

Note:

  • Management logs are not supported for non-time series profiles.
  • Management logs are not exported out of secondary nodes by default. Use time series over NSIP (set appflow param -timeseriesovernsip enabled) when the destination is reachable using NSIP and secondary export is required.

Configure an HTTP event collector on Splunk

You can forward management logs to Splunk by configuring an HTTP event collector. Configuring the HTTP event collector involves creating an authentication token and associating an event index with the token where events are sent, and setting the HTTP port number.

For information on how to configure the HTTP event collector, see the Splunk documentation.

Once you have configured the HTTP event collector, copy the authentication token and save it for reference. You need to specify this token while configuring the analytics profile on NetScaler.

Configure a time series analytics profile on NetScaler using the CLI

Do the following to export NetScaler management logs to Splunk.

  1. Configure the authentication token in AppFlow parameters.

    set appflow param -analyticsAuthToken "Splunk <The token that you received from Splunk>"
<!--NeedCopy-->

  2. Create a collector service for Splunk.

    add service <collector> <splunk-server-ip-address> <protocol> <port>

    Example:

    add service splunk_service 10.102.34.155 HTTP 8088

    In this configuration:

    • ip-address: Splunk server IP address.
    • collector-name: Name of the collector.
    • protocol: Specify the protocol as HTTP or HTTPS
    • port: Port number.

  3. Create a time series analytics profile.

    add analytics profile `profile-name` -type time series -managementlog <management-log-type> -collectors `collector-name` -analyticsEndpointMetadata `meta-data-for-endpoint`

    Example:

     add analytics profile managementlogs_profile -type timeseries -managementlog ACCESS -collectors splunk -analyticsEndpointMetadata '{"source":"netscaler","sourcetype:"mgmtlog","index":"ns-mgmtlog-index"}'

    In this configuration:

    • managementlog: Types of management logs that you must export. The following options are available:
      • ALL: Includes all categories of management and host logs.
      • SHELL: Includes bash.log and sh.log.
      • Access: Includes logs such as auth.log, nsvpn.log, vpndebug.log, httpaccess.log,httperror.log, httpaccess-vpn.log, and httperror-vpn.log.
      • NSMGMT: Includes ns.log and notice.log.
      • NONE: None of the logs are exported.

    • analyticsEndpointMetadata: Meta data that is endpoint specific. Management logs use HEC format as documented at Format events for HTTP Event Collector. The payload format is as follows:

       {
     "host": "<hostname set on the box by default>",
     "event": {
         "Loglevel": <int>,
         "Managementlog string": <string>,
         "Message Name": <string>,
         "Module Name": <string>
         }
 }
 <!--NeedCopy-->

      HEC format defines several meta fields that can be included in the payload other than the actual event including host, index, source, sourcetype, and so on.

      By default, NetScaler uses the FreeBSD host name as “host” and does not set any fields such as index, source, sourcetype in the payload. This setting can be changed by explicitly providing the required fields in -analyticsEndpointMetadata.

      For example, using '{"host":"my-netscaler","index":"my-splunk-index","sourcetype":"managementlog","source":"netscaler"}' as -analyticsEndpointMetadata crafts the below payload for every log:

       {
     "host": "my-netscaler",
     "index": "my-splunk-index",
     "sourcetype": "managementlog",
     "source": "netscaler",
     "event": {
         "Loglevel": <int>,
         "Managementlog string": <string>,
         "Message Name": <string>,
         "Module Name": <string>
      }
 }
 <!--NeedCopy-->

      Notice how the host is overwritten. (This behavior might not be desirable when logs are sent from both primary and secondary nodes because both carry the same host name. In this scenario, it is preferable not to configure the host in analyticsEndpointMetadata so that the FreeBSD host names are used by the respective nodes.)

      Also, notice that the fields index, sourcetype, source are added to the payload.

      Note:

      Ensure that -analyticsEndpointMetadata is a proper JSON.

  4. Verify the analytics profile configuration using the show analytics profile command.

    # show analytics profile splunkexport
1)    Name: audit_profile
      Collector: splunk
      Profile-type: timeseries
            Output Mode: avro
            Metrics: DISABLED
              Schema File: schema.json
              Metrics Export Frequency: 30
            Events: DISABLED
            Auditlog: DISABLED
            Serve mode: Push
       Endpoint URL: /services/collector/event
       Endpoint Content-type: Application/json
       Endpoint Metadata: Event:
       Reference Count: 0
       Managementlog: ACCESS

Once the configuration is successful, the management logs are sent as HTTP payloads to Splunk and you can view it on the Splunk application user interface.

Note:

You can modify the time series analytics profile parameters using the set analytics profile command.

Configure a time series analytics profile on NetScaler using the GUI

Perform the following steps:

  1. Create a collector service.
    1. Navigate to Traffic Management > Load Balancing > Services and click Add.
    2. On the Load Balancing Service page, enter the details in the required fields, click OK and then click Done.

  2. Create a time series analytics profile.

    1. Navigate to System > Profiles > Analytics Profiles and click Add.

    2. On the Create Analytics Profile page, provide the following details:

      1. Enter a name for the profile.
      2. From the Collectors list, select the service that you created.
      3. Select a TIME SERIES from the Type list.
      4. Enter the Analytics Auth Token that you received from Splunk with the prefix “Splunk”.
      5. Enter the details for Analytics Endpoint URL, Analytics Endpoint Content Type, and Analytics Endpoint Metadata.
      6. Select the Management Logs that you want to export and also the Output Mode in which you want to export.
      7. Click Create.

Export management logs to Splunk configured as a syslog server

To configure the export of management logs, you must perform the following steps:

Step 1: Configure Splunk as an external syslog server

You can forward management logs to Splunk by configuring an external syslog server on Splunk.

For information on how to configure the syslog port, see the Splunk documentation. Once you have configured the syslog port, save it for reference. You need to specify this port while configuring the audit syslogaction on NetScaler.

Step 2: Create an audit syslog action

For information on the steps to create an audit syslog action on NetScaler with the management log option, see Configure syslog audit action.

Step 3: Create a syslog audit policy

For information on the steps to create a syslog audit policy with the syslog action, see Configure syslog audit policy.

Step 4: Bind audit log policy

For information on how to bind audit log policy, see Bind audit log policy.

Additional information

This section provides more information on the three types of logs specified in this topic:

  • shell logs: Includes bash.log and sh.log.
  • access logs: Includes httpaccess.log,httperror.log,httpaccess-vpn.log, httperror-vpn.log, vpndebug.log, nsvpn.log, and auth.log.
  • nsmgmt logs: Includes notice.log, and ns.log (only includes non-packet engine logs).

For troubleshooting tips related to management logs export, see Troubleshoot issues related to management logs.

Export management logs directly from NetScaler to Splunk
August 27, 2024
Contributed by: 
C
August 27, 2024
Contributed by: 
C

In this article

Site feedback | Your privacy choices footer linkYour Privacy Choices | Privacy and legal terms | Cookie preferences | Consent settings | docs.cloud.com
© 1999- Cloud Software Group, Inc. All rights reserved.