Product Documentation
ADC
14.1 - Current Release 13.1 13.0 12.1
View PDF

Export transaction logs directly from NetScaler to Splunk

August 7, 2024
Contributed by: 
C C

You can now export transaction logs from NetScaler to industry-standard log aggregator platforms such as Splunk. The transaction log is the record of application traffic flow events on the NetScaler such as HTTP requests and responses, connection start and end. For more information on transaction logs, see AppFlow.

You can export transaction logs in JSON format for different insights such as Web Insight, security, gateway, HDX insights. To export transaction logs to Splunk, you must configure Splunk as an HTTP server and use the HTTP event collector to send transaction logs over HTTP (or HTTPS) directly to the Splunk platform from your NetScaler. Using the visualization tools at Splunk, you can get meaningful insights about the exported data.

Note:

The IP addresses that are exported as part of the transaction logs appear in the decimal format instead of the standard format. For example, if your NetScaler IP address is 10.102.154.153, the same in the transaction logs on Splunk is displayed as 174496409. You can use the inbuilt expressions available on Splunk to convert the IP address from decimal format to standard format.

Export transaction logs from NetScaler to Splunk configured as an HTTP server

To configure the export of transaction logs you must perform the following steps:

  1. Configure an HTTP event collector on Splunk.
  2. Create a collector service and an analytics profile on NetScaler.

Configure an HTTP event collector on Splunk

You can forward transaction logs to Splunk by configuring an HTTP event collector. Configuring the HTTP event collector involves creating an authentication token and associating an event index with the token where events are sent, and setting the HTTP port number. For information on how to configure the HTTP event collector, see the Splunk documentation.

Once you have configured the HTTP event collector, copy the authentication token and save it for reference. You need to specify this token while configuring the analytics profile on NetScaler.

Configure analytics profile on NetScaler

Do the following to export NetScaler transaction logs to Splunk.

  1. Create a collector service for Splunk.

    add service <collector> <splunk-server-ip-address> <protocol> <port>

    Example:

    add service splunk_service 10.102.34.155 HTTP 8088

    In this configuration:

    • ip-address: Splunk server IP address.
    • collector-name: Name of the collector.
    • protocol: Specify the protocol as HTTP or SSL.
    • port: Port number.

  2. Create an analytics profile.

    add analytics profile `profile-name` -type <insight> -collectors `collector-name` -analyticsAuthToken `<auth-scheme> <authorization-parameters>` -analyticsEndpointContentType `application/json` -analyticsEndpointUrl `endpoint-url`

    Example:

    add analytics profile transaction-log-profile -type webinsight -collectors splunk_collector -analyticsAuthToken "Splunk 1234-5678-12345" -analyticsEndpointContentType "application/json" -analyticsEndpointUrl "/services/collector/event"

    Note:

    The -allHttpHeaders option is supported for Splunk transaction log export in NetScaler 14.1-25.x and later.

    add analytics profile <profile-name> -type webinsight -allHttpHeaders

    set analytics profile <profile-name> -type webinsight -allHttpHeaders

    In this configuration:

    • insight: Types of insights that you can export The following options are available:
      • botinsight
      • CIinsight
      • Gatewayinsight
      • hdxinsight
      • lsninsight
      • securityinsight
      • tcpinsight
      • udpinsight
      • videoinsight
      • webinsight

    • -analyticsAuthToke auth-scheme authorization-parameters: Specify the authentication token to be included in the authorization header with the auth-scheme “Splunk” while sending logs to Splunk. This token is the authentication token created on the Splunk server while configuring the HTTP event collector.

    • analyticsEndpointContentType: The Content-Type header. The hardcoded value is application/json.

    • analyticsEndpointUrl: The path to HEC on Splunk (/services/collector/event or /services/collector).

      NOTE:

      You can modify the analytics profile parameters using the set analytics profile command.

    • dataFormatFile: The file that contains the details of what data in transaction log must be exported and in which format. Each endpoint expects JSON payload to be encoded in a specific format. For Splunk, the format is defined in the splunk_format.txt file which is present under /var/analytics_conf directory. You can refer to the splunk_format.txt file and create your own customized data format file specific to your use case. If the format is not specified, then by default the splunk_format.txt is selected.

  3. Verify the analytics profile configuration using the show analytics profile command.

    > sh analytics profile 
    
ns_analytics_default_http_profile
1)      Name: ns_analytics_default_http_profile
        Collector: splunk
        Profile-type: webinsight
                Page Tracking: DISABLED
                Client Side Measurements: DISABLED
                URL Logging: ENABLED
                Host Header Logging: ENABLED
                Method Logging: ENABLED
                Cookie Header Logging: DISABLED
                Referer Header Logging: DISABLED
                User Agent Logging: DISABLED
                Content Type Logging: DISABLED
                Authorization Header Logging: DISABLED
                Via Header Logging: DISABLED
                Location Header Logging: DISABLED
                URL Category Logging: DISABLED
                Log All HTTP Headers: DISABLED
                X-Forwarded-For Header Logging: DISABLED
                Set-Cookie Header Logging: DISABLED
                Set-Cookie2 Header Logging: DISABLED
                Domain Name Logging: DISABLED
                URL-Query Logging: DISABLED
                Integrated Cache Logging: DISABLED
                gRPC Status Logging: DISABLED
                API Spec Info Export Logging: DISABLED
        Authentication Token:
        Endpoint URL: /services/collector/event
        Endpoint Content-type: text/json
        Reference Count: 1
        Managementlog: NONE

  4. Bind the analytics profile to the virtual server.

    bind lb vserver <vserver-name> -analyticsProfile transaction-log-profile

    Example:

    bind lb vserver sample-virtualserver -analyticsProfile transaction-log-profile

After the configuration is successful, based on traffic, transactions will be logged and exported to HEC on Splunk.

Field-based filtering of data records

By default, NetScaler exports hundreds of fields in transaction log even when the endpoints do not require all of the exported data. Also, each endpoint expects the JSON payload to be encoded in a specific format such as the start and end of a data record, the delimiter between data records, and buffer start and end.

Splunk expects the JSON payload coming from NetScaler to be encoded in the following format:

  • Buffer start and end: No value required for BUFFER-START and BUFFER-END.

  • Data record start and end: The data record must start with {"event":{ and end with }}. All the fields that get exported must be between DATA-START and DATA-END.

    The data records must start with the following:

     RECORD-START
 {"event":{
 DATA_START

    The data records must end with the following:

     DATA-END
 }}
 RECORD-END
  • Delimiter between data records: No delimiter required.

By default, splunk_format.txt is available at var/analytics_conf folder that contains the JSON payload format that Splunk expects and also contains a few default fields for which the data gets exported.

The following is a sample data format file for Splunk:

    BUFFER-START
    RECORD-START
    {"event":{
    DATA-START
    153 observationPointId
    547 nsPartitionId
    154 exportingProcessId
    159 transactionId
    801 httpReqUrl
    685 httpReqMethod
    683 httpReqHost
    472 svrDstIpv4Address
    579 srvSrcPort
    580 srvDstPort
    13 backendSvrIpv6Address
    473 cltIpv4Address
    474 cltDstIpv4Address
    577 cltSrcPort
    578 cltDstPort
    14 transCltIpv6Address
    15 transCltDstIpv6Address
    DATA-END
    }}
    RECORD-END
    RECORD-DELIMITER

    RECORD-DELIMITER-END
    BUFFER-END

The JSON_fields.txt file under var/analytics_conf is a reference master file that contains the complete list of fields along with their identification numbers. The fields in the master file are categorized based on the insights. For example, if you want to know the fields associated with HDX insight, you can look at the HDX insights category of JSON_fields.txt file.

You can create a customized data format file based on your requirement by referrring to splunk_format.txt. For example, you can create my_splunk_format.txt. If your requirement is to export HDX insights, you can look at the HDX insights category in JSON_fields.txt file and add the required fields in your my_splunk_format.txt file. Similarly, you can delete any fields associated with the data that you do not wish to export.

Note:

Do not update the default splunk_format.txt file; instead, use it as a reference. If you update the default splunk_format.txt, then the contents of the file is overwritten upon upgrade.

After customizing the data format file, run the following command to update the analytics profile:

update analytics profile <profile-name> -dataFormatFile <data-format-file-name>

Example:

update analytics profile ns _analytics_ default_http_profile -dataFormatFile my_splunk_format.txt

You can also specifiy the value of data format file using the GUI. Navigate to System > AppFlow > Analytics Profiles and click Add. On the Create Analytics Profile page, if you select one of the following options for Type, then the Data Format File field appears where you can specificy the file name:

  • GLOBAL
  • WEB INSIGHT
  • TCP INSIGHT
  • SECURITY INSIGHT
  • VIDEO INSIGHT
  • HDX INSIGHT
  • GATEWAY INSIGHT
  • LSN INSIGHT
  • BOT INSIGHT
  • TIME SERIES

Sample outputs

This section contains sample outputs for different transaction logs.

HTTP transaction log sample output

The following is a sample output for the HTTP transaction log.

{
    appName: VS1
    clientMss: 1460
    clntFastRetxCount: 0
    clntTcpJitter: 0
    cintTcpPacketsRetransmited: 0
    clntTcpRtoCount: 0
    clntTcpZeroWindowCount: 0
    cltDstIpv4Address: 174496411
    cltIpv4Address: 174496407
    connEndTimestamp: 0
    connStartTimestamp: 7329468222993076980
    exportingProcessId: 0
    httpReqHost: 10.102.154.155
    httpReqMethod: GET
    httpReqUrl: /big.html
    httpRspLen: 114380
    httpRspStatus: 200
    mainPageCoreId: 0
    mainPageId: 0
    nsPartitionId: 0
    observationPointId: 174496409
    originRspLen: 0
    srvrIcpPacketsRetransmited: 0
    srvrTcpZeroWindowCount: 0
    svrDstIpv4Address: 174496415
    svrIpv4Address: 174496408
    tepSrvrConnRstCode: 0
    transClntRTT: 0
    transCltDstPort: 20480
    transCltFlowEndUsecRx: 7329468222993084980
    transCltFlowEndUsecTx: 7329468222993084980
    transCltFlowStartUsecRx: 7329468222993076980
    transCltFlowStartUsecTx: 7329468222993077984
    transCltSrcPort: 60315
    transCltTotRx0ctCnt: 1766
    transCltTotTx0ctCnt: 117580
    transSrvDstPort: 36895
    transSrvSrcPort: 15213
    transSrvrRTT: 0
    transSvrFlowEndUsecRx: 7329468222993084980
    transSvrFlowEndUsecTx: 7329468222993084980
    transSvrFlowStartUsecRx: 7329468222993077984
    transSvrFlowStartUsecTx: 0
    transSvrTotRx0ctCnt: 117580
    transSvrTotTx0ctCnt: 1766
    transactionId: 4890
}

TCP transaction log sample output

The following is a sample output for TCP transaction log.

{
    appName: vs1
    clientConnEndTimestamp: 7333165210582386064
    clientConnStartTimestamp: 7333165210582386054
    clientMss: 1460
    clntFastRetxCount: 0
    clntTcpJitter: 0
    clntTcpPacketsRetransmited: 0
    clntTcpRtoCount: 0
    clntTcpZeroWindowCount: 0
    cltDstIpv4Address: 174496411
    cltDstPort: 20480
    cltIpv4Address: 174496407
    cltSrcPort: 42939
    connectionChainHopCount: 0
    exportingProcessId: 0
    nsPartitionId: 0
    observationPointId: 174496409
    serverConnEndTimestamp: 7333165201992708470
    serverConnStartTimestamp: 7333165201992708459
    srvDstPort: 36895
    srvSrcPort: 51973
    srvrTcpPacketsRetransmited: 0
    srvrTcpZeroWindowCount: 0
    svrDstIpv4Address: 174496415
    svrIpv4Address: 174496408
    tcpClntConnRstCode: 0
    tcpSrvrConnRstCode: 0
    transClntRTT: 0
    transCltTotRxOctCnt: 208
    transCltTotTxOctCnt: 331
    transSrvrRTT: 0
    transSvrTotRxOctCnt: 331
    transSvrTotTxOctCnt: 208
    transactionId: 330
    vlanNumber: 1
}

SSL transaction log sample output

The following is a sample output for SSL transaction log.

{
    appName: sslvs
    clientConnEndTimestamp: 0
    clientConnStartTimestamp: 7333182669624439854
    clientMss: 1460
    clntFastRetxCount: 0
    clntTcpJitter: 0
    clntTcpPacketsRetransmited: 0
    clntTcpRtoCount: 0
    clntTcpZeroWindowCount: 0
    cltDstIpv4Address: 174496411
    cltDstPort: 47873
    cltIpv4Address: 174496407
    cltSrcPort: 17499
    connectionChainHopCount: 0
    exportingProcessId: 0
    httpContentType: text/html
    httpReqHost: 10.102.154.155
    httpReqMethod: GET
    httpReqUrl: /index.html
    httpReqUserAgent: curl/7.69.1
    httpRspLen: 291
    httpRspStatus: 200
    nsPartitionId: 0
    observationPointId: 174496409
    originRspLen: 0
    serverConnEndTimestamp: 0
    serverConnStartTimestamp: 7333182665330184556
    srvDstPort: 36895
    srvSrcPort: 34802
    srvrTcpPacketsRetransmited: 0
    srvrTcpZeroWindowCount: 0
    sslCipherValueBE: 0
    sslCipherValueFE: 50331701
    sslClientCertSizeBE: 0
    sslClientCertSizeFE: 0
    sslClntCertSigHashBE: 0
    sslClntCertSigHashFE: 0
    sslFLagsBE: 0
    sslFLagsFE: 1096
    sslServerCertSizeBE: 0
    sslServerCertSizeFE: 4096
    sslSessionIDBE: 0
    sslSessionIDFE: 2433458443
    sslSigHashAlgBE: 0
    sslSigHashAlgFE: 0
    sslSrvrCertSigHashBE: 0
    sslSrvrCertSigHashFE: 668
    svrDstIpv4Address: 174496415
    svrIpv4Address: 174496408
    tcpClntConnRstCode: 0
    tcpSrvrConnRstCode: 0
    transClntRTT: 0
    transCltFlowEndUsecRx: 7333182669624447854
    transCltFlowEndUsecTx: 7333182669624446854
    transCltFlowStartUsecRx: 7333182669624439854
    transCltFlowStartUsecTx: 7333182669624439854
    transCltTotRxOctCnt: 1501
    transCltTotTxOctCnt: 2223
    transSrvrRTT: 0
    transSvrFlowEndUsecRx: 7333182669624446854
    transSvrFlowEndUsecTx: 7333182669624446854
    transSvrFlowStartUsecRx: 7333182669624446854
    transSvrFlowStartUsecTx: 0
    transSvrTotRxOctCnt: 331
    transSvrTotTxOctCnt: 168
    transactionId: 2640
    vlanNumber: 1
}

Web Insight transaction log sample output

The following is a sample output for Web Insight transaction log.

{
    appName: vs1
    clientConnEndTimestamp: 0
    clientConnStartTimestamp: 7333336201820249485
    clientMss: 1460
    clntFastRetxCount: 0
    clntTcpJitter: 0
    clntTcpPacketsRetransmited: 0
    clntTcpRtoCount: 0
    clntTcpZeroWindowCount: 0
    cltDstIpv4Address: 174496411
    cltDstPort: 20480
    cltIpv4Address: 174758625
    cltSrcPort: 46824
    connectionChainHopCount: 0
    exportingProcessId: 0
    httpContentType: text/html
    httpReqHost: 10.102.154.155
    httpReqMethod: GET
    httpReqUrl: /
    httpRspLen: 291
    httpRspStatus: 200
    nsPartitionId: 0
    observationPointId: 174496409
    originRspLen: 0
    serverConnEndTimestamp: 0
    serverConnStartTimestamp: 7333336201820250487
    srvDstPort: 36895
    srvSrcPort: 6465
    srvrTcpPacketsRetransmited: 0
    srvrTcpZeroWindowCount: 0
    svrDstIpv4Address: 174496415
    svrIpv4Address: 174496408
    tcpClntConnRstCode: 0
    tcpSrvrConnRstCode: 0
    transClntRTT: 0
    transCltFlowEndUsecRx: 7333336201820251488
    transCltFlowEndUsecTx: 7333336201820251488
    transCltFlowStartUsecRx: 7333336201820249485
    transCltFlowStartUsecTx: 7333336201820250487
    transCltTotRxOctCnt: 190
    transCltTotTxOctCnt: 371
    transSrvrRTT: 0
    transSvrFlowEndUsecRx: 7333336201820251488
    transSvrFlowEndUsecTx: 7333336201820250487
    transSvrFlowStartUsecRx: 7333336201820250487
    transSvrFlowStartUsecTx: 7333336201820250487
    transSvrTotRxOctCnt: 371
    transSvrTotTxOctCnt: 202
    transactionId: 11218
    vlanNumber: 1
}

Sample dashboards for transaction logs

The following dashboards available on Splunk provide the data related to transaction logs:

  • NetScaler HTTP Insight Dashboard
  • NetScaler SSL Insight Dashboard
  • NetScaler TCP Insight Dashboard

For more information, see Sample dashboards on Splunk.

Export transaction logs directly from NetScaler to Splunk
August 7, 2024
Contributed by: 
C C
August 7, 2024
Contributed by: 
C C

In this article

Site feedback | Your privacy choices footer linkYour Privacy Choices | Privacy and legal terms | Cookie preferences | Consent settings | docs.cloud.com
© 1999- Cloud Software Group, Inc. All rights reserved.