Product Documentation
ADC
14.1 - Current Release 13.1 13.0 12.1
View PDF

Processed insights

August 29, 2024
Contributed by: 
C

NetScaler records the application traffic flow events such as HTTP requests and responses in the form of counters and transactions. When this data is sent to NetScaler Console or other observability tools, the receiving tool processes the data and provides actionable insights. For example, NetScaler Console processes all transactions and generates Web Insight reports. However, this method poses challenges as significant data must be transferred from NetScaler to observability tools and logics must be created to obtain insights.

From NetScaler release 14.1-29.x onwards, NetScaler can process the raw data and generate processed insights that can be sent outside NetScaler. The following are the advantages of NetScaler generating processed insights:

  • The data transferred from NetScaler to NetScaler Console or any other observability tool is minimum.
  • Aggregation and filtering of data can be done according to the use case from the source itself. For example, aggregate data based on a client or URL.
  • The data transferred is easily consumable on observability tools without any processing.
  • Dashboards on NetScaler Console or other observability tools can be built efficiently.

Processed insights from NetScaler using stream analytics

The performance of a website or an application depends on how well you optimize the delivery of the content. Stream analytics aggregates the real-time statistics of the website or application traffic based on the expressions provided in an entity called, selector. For example, a selector can be a client IP or a URL. You can configure a stream identifier that specifies the selector to be used, the statistics collection interval, the sample count, and the field on which the records must be sorted. You can sort the data based on the requests and create a list of top records, such as, top URLs, top clients. These top records are known as top N, where N is the number or count of records listed.

NetScaler processes the top N records that are generated by the stream analytics and provides insights. You can also export the stream analytics data to NetScaler Console or any other observability tool.

Based on the data generated by the stream identifier, you can take the required actions, such as optimizing the traffic, limiting bandwidth consumption for user or client. For more information, see Configure policy for optimizing traffic and How to limit bandwidth consumption for user or client device.

Notes:

  • You can bind one stream identifier to multiple virtual servers and collect statistics from multiple virtual servers for the same stream selector expressions.
  • You can export a maximum of 1000 Top N records per stream identifier.
  • Stream identifier aggregates result data as stream sessions, based on the evaluation of the selector expression. The total length of the stream session is limited to 60 characters and the length is equally shared among the number of selector expressions. For example, if the selector has 5 expressions, each expression output gets 12 characters.

Configure stream analytics using the CLI

To configure stream analytics using the CLI, perform the following steps:

  1. Configure stream selector.

    add stream selector <name> <rule> ...
<!--NeedCopy-->

    Example: To collect statistics aggregated at client IP and URL, run the following command:

    > add stream selector myselector HTTP.REQ.URL CLIENT.IP.SRC
 Done
> show stream selector myselector
    Name: myselector
    Expressions:
        1) HTTP.REQ.URL
        2) CLIENT.IP.SRC
 Done
>
<!--NeedCopy-->

  2. Configure an identifier for the selector.

    add stream identifier <identifier name> <selector name> -interval <int in mins> -sort <req> -Log SYSLOG -LogInterval <int in mins> -LogLimit <int>
<!--NeedCopy-->

    Example: To get statistics for each Client IP and URL and export them to SYSLOG, run the following command:

    add stream identifier TopTalkers myselector -interval 5  -log SYSLOG -loginterval 10 -loglimit 10
<!--NeedCopy-->

  3. Create a responder policy to collect stats for the given identifier.

    add responder policy <resp policy name> "analytics.stream(\"<identifier name>\").collect_stats" NOOP
<!--NeedCopy-->

    Example:

    add responder policy TopTalkers "analytics.stream(\"TopTalkers\").collect_stats" NOOP
<!--NeedCopy-->

  4. Bind the responder policy to the target virtual server for which the traffic is to be analyzed by the identifier. To enable the same stream identifier to process traffic from multiple virtual servers, bind the responder policy to all the virtual servers.

    bind lb vserver dnslb -policyName <resp policy name> -priority 1 -gotoPriorityExpression NEXT -type REQUEST
<!--NeedCopy-->

    Example:

    bind lb vserver dnslb -policyName TopTalkers -priority 1 -gotoPriorityExpression NEXT -type REQUEST
<!--NeedCopy-->

Configure stream analytics using the GUI

To configure stream analytics through the GUI, perform the following steps:

  1. Navigate to AppExpert > Action Analytics > Selectors.

  2. In the details pane, click Add and set the Name and Expressions parameter.

  3. Click Insert and then click Create.

  4. Navigate to AppExpert > Action Analytics > Stream Identifiers.

  5. In the details pane, click Add. In the Configure Stream Identifier page, set the following parameters, and click Create.

    • Name
    • Selector
    • Interval
    • Sample Count
    • Sort

  6. Navigate to AppExpert > Responder > Policies. On the Responder Policies page, click Add.

  7. On the Create Responder Policy page, enter a name for the responder policy.

  8. In the Expression editor, configure the condition for the responder policy. For example, ‘sys.check_limit(“limit_one_login”)’ and click Create.

  9. Navigate to Traffic Management > Load Balancing > Virtual Servers.

  10. On the Load Balancing Virtual Servers page, select the virtual server to which you want to bind the responder policy, and then click Open.

  11. In the Configure Virtual Server (Load Balancing) dialog box, select the Policies tab. Select the checkbox next to the name of the policy that you want to bind to this virtual server and click OK.

Export of stream analytics data using syslog through CLI

To export the stream analytics data using syslog, perform the following steps by using the NetScaler CLI:

  1. Create a syslog server endpoint. You can create multiple syslog servers for multiple log levels.

    add audit syslogAction <syslog servername> <server ip address>  -logLevel ALL -managementlog ALL -mgmtlogLevel ALL -dateFormat YYYYMMDD -userDefinedAuditlog YES -transport TCP -streamAnalytics <ALL/NONE>
<!--NeedCopy-->

    Example:

    add audit syslogAction topk_syslog_server 10.106.44.6 -logLevel ALL -managementlog ALL -mgmtlogLevel ALL -dateFormat YYYYMMDD -userDefinedAuditlog YES -transport TCP -streamAnalytics  <enabled/disabled>
<!--NeedCopy-->

  2. Create a syslog policy for the syslog server created in step 5.

    add audit syslogPolicy <syslog policy> <syslog servername>
<!--NeedCopy-->

    Example:

    add audit syslogPolicy topk_syslog true topk_syslog_server
<!--NeedCopy-->

  3. Bind the syslog policy at the right level.

    bind syslogGlobal -policyName <syslog policy> -priority 1
<!--NeedCopy-->

    Example:

    bind syslogGlobal -policyName topk_syslog -priority 1
<!--NeedCopy-->

Export of stream analytics data using syslog through GUI

To export the stream analytics data using syslog, perform the following steps by using the NetScaler GUI:

  1. Navigate to Configuration > System > Auditing > Syslog.

  2. In the Policies tab, click Add.

  3. Enter a name for the policy, select Advanced Policy and click Add to create an auditing server.

  4. Enter the required details in the Create Auditing server, select Stream Analytics, and click Create.

  5. On the Syslog Auditing page, select the policy, and click Advanced Policy Global Bindings.

  6. Select the policy, priority, and SYSTEM_GLOBAL from the Global Bind Type drop-down list. Click Bind.

Sample stream analytics data sent in syslog 

New audit log message module type for Stream Analytics = SA.

Apr  2 15:30:14 10.106.44.24  04/02/2024:09:58:28 GMT HA1 0-PPE-0 : default SA Message 3285 0 : "stream_iden_name: TopKTalkers, stream_sess_name: 1.1.1.1, Requests: 5836754, Bandwidth: 1635886, RespTime: 51, Connections: 9"
Apr  2 15:30:14 10.106.44.24  04/02/2024:09:58:28 GMT HA1 0-PPE-0 : default SA Message 3286 0 : "stream_iden_name: TopKTalkers, stream_sess_name: 2.2.2.2, Requests: 4939608, Bandwidth: 1384438, Resptime: 120, Connections: 7"
Apr  2 15:30:14 10.106.44.24  04/02/2024:09:58:28 GMT HA1 0-PPE-0 : default SA Message 3287 0 : "stream_iden_name: TopKTalkers, stream_sess_name: 3.3.3.3, Requests: 4293067, Bandwidth: 1203231, RespTime: 73, Connections: 8"
Apr  2 15:30:14 10.106.44.24  04/02/2024:09:58:28 GMT HA1 0-PPE-0 : default SA Message 3289 0 : "stream_iden_name: TopKTalkers, stream_sess_name: 4.4.4.4, Requests: 3834040, Bandwidth: 1074577, ResptTime: 109, Connections: 5"
<!--NeedCopy-->

The following image explains the stream analytics data sent in syslog:

stream analytics data

Processed insights
August 29, 2024
Contributed by: 
C
August 29, 2024
Contributed by: 
C

In this article

Site feedback | Your privacy choices footer linkYour Privacy Choices | Privacy and legal terms | Cookie preferences | Consent settings | docs.cloud.com
© 1999- Cloud Software Group, Inc. All rights reserved.