NetScaler Observability Exporter troubleshooting

This document explains how to troubleshoot issues that you may encounter while using NetScaler Observability Exporter.

• How do I verify that NetScaler sends application data logs to NetScaler Observability Exporter?

  Run the following command to verify that NetScaler sends application data logs to NetScaler Observability Exporter:

   nsconmsg -g lstream_tot_trans_written -d current
  

  The counter value indicates that the number of application transactions (for example, HTTP transactions) which have been sent to NetScaler Observability Exporter.

  If the application traffic rate (for example, HTTP req/sec) that is sent to NetScaler Observability Exporter is not equal to lstream_tot_trans_written, you can verify the same using the following command:

   nsconmsg -g nslstream_err_ulf_data_not_sendable -d current
  

  The counter value indicates that NetScaler cannot send the data to NetScaler Observability Exporter due to network congestion, unavailability of network bandwidth, and so on and the data is stored in the available buffers.

  Information about various transaction data and individual fields, and their datatype are available in the following location on the NetScaler:

   shell
   /netscaler/appflow/ns_ipfix.yaml
  

  To verify the current record type exported from NetScaler to NetScaler Observability Exporter, use the following command:

   nsconmsg -g appflow_tmpl -d current
  

  Location of metrics data export logs to NetScaler for time series data:

   /var/nslog/metrics_prom.log
  

  To verify kafka related counters, run the following command:

   kubectl exec -it <cpx-pod-name> [-c <cpx-container-name>] [-n <namespace-name>] -- bash
  
   tail -f /var/ulflog/counters/lstrmd\_counters\_codes.log | grep -iE \"\(http\_reqs\_done|kafka)\"
  

  Find the logs in the following location to verify that the NetScaler Observability Exporter configuration is applied correctly:

   vi /var/logproxy/lstreamd/conf/lstreamd.conf
  

  If NetScaler Observability Exporter fails, you can collect logs available at the following location and contact NetScaler support.

   /var/crash/ (Loation of the coredump files, if any.)
   /var/ulflog/ (Location of `libulfd` logs and counter details.)
   /var/log  (Location of console logs, lstreamd logs and so on.)
  

For information on NetScaler CPX related troubleshooting, see NetScaler CPX Troubleshooting.