ADC

How to collect performance statistics and event logs

You can collect performance statistics of virtual servers and associated services from an archived newnslog file present in the /var/nslog directory. The newnslog files are interpreted by running /netscaler/nsconmsg.

Collect performance statistics and event logs using the CLI

You can run the nsconmsg command from the NetScaler shell prompt to report events.

At the command prompt, type:

/netscaler/nsconmsg -K /var/nslog/newnslog -d event

Displaying event information
NetScaler V20 Performance Data
NetScaler NS10.5: Build 57.7.nc, Date: May 14 2015, 07:35:21
rtime: Relative time between two records in milliseconds
seqno rtime event-message                         event-time
11648 16310 PPE-0 MonServiceBinding_10.104.20.110:443_(tcp-default)
<!--NeedCopy-->

View the time span covered by a given “newnslog” file

At the command prompt, type:

/netscaler/nsconmsg -K /var/nslog/newnslog -d setime

The current data is appended to the /var/nslog/newnslog file. NetScaler archives the newnslog file automatically every two days by default. To read the archived data, you must extract the archive as shown in the following example:

cd /var/nslog: command to go to a particular directory from NetScaler Shell Prompt.

tar xvfz newnslog.100.tar.gz: command to extract the tar file.

/netscaler/nsconmsg -K newnslog.100 -d setime: Command to check the time span covered by the particular file, in this example newnslog.100.

ls -l: Command checks all the logs file and time stamp associated with those files.

root@NETSCALER# cd /var/nslog root@NETSCALER# ls -l

 wheel    461544 Aug  7  2014 newnslog.1.tar.gz
-rw-r--r--   1 root       wheel    191067 Aug  7  2014 newnslog.10.tar.gz
-rw-r--r--   1 root       wheel  11144873 Apr 26 22:04 newnslog.100.tar.gz
-rw-r--r--   1 root       wheel  11095053 Apr 28 22:04 newnslog.101.tar.gz
-rw-r--r--   1 root       wheel  11114284 Apr 30 22:04 newnslog.102.tar.gz
-rw-r--r--   1 root       wheel  11146418 May  2 22:04 newnslog.103.tar.gz
-rw-r--r--   1 root       wheel  11104227 May  4 22:04 newnslog.104.tar.gz
-rw-r--r--   1 root       wheel  11297419 May  6 22:04 newnslog.105.tar.gz
-rw-r--r--   1 root       wheel  11081212 May  8 22:04 newnslog.106.tar.gz
-rw-r--r--   1 root       wheel  11048542 May 10 22:04 newnslog.107.tar.gz
-rw-r--r--   1 root       wheel  11101869 May 12 22:04 newnslog.108.tar.gz
-rw-r--r--   1 root       wheel  11378787 May 14 22:04 newnslog.109.tar.gz
-rw-r--r--   1 root       wheel  44989298 Apr 11  2014 newnslog.11.gz
<!--NeedCopy-->

Display the time span within a file

Use the nsconmsg command to only display a span of time within the given file, as shown in the following example:

/netscaler/nsconmsg -K /var/nslog/newnslog -s time=22Mar2007:20:00 -T 7 -s ConLb=2 -d oldconmsg

Where,

s : time=22Mar2007:20:00:00 is start at March 22, 2007 at exactly 20:00.

T 7 : Displays seven seconds of data

s : Displays detail level of load balancing statistics.

d : Displays statistical information.

Note:

From ADC release 12.1 you need add at the “time” seconds as well, that is: 22Mar2007:20:00:00

The statistical information provided by the -d oldconmsg parameter is recorded every seven seconds. The following is a sample output.

VIP(10.128.58.149:80:UP:WEIGHTEDRR): Hits(38200495, 18/sec) Mbps(1.02) Pers(OFF) Err(0)
Pkt(186/sec, 610 bytes) actSvc(4) DefPol(NONE) override(0)
Conn: Clt(253, 1/sec, OE[252]) Svr(3)
S(10.128.49.40:80:UP) Hits(9443063, 4/sec, P[2602342, 0/sec]) ATr(5) Mbps(0.23) BWlmt(0 kbits) RspTime(112.58 ms)
Other: Pkt(36/sec, 712 bytes) Wt(10000) RHits(31555)
Conn: CSvr(42, 0/sec) MCSvr(20) OE(16) RP(11) SQ(0)
S(10.128.49.39:80:UP) Hits(9731048, 4/sec, P[2929279, 0/sec]) ATr(9) Mbps(0.27) BWlmt(0 kbits) RspTime(161.69 ms)
Other: Pkt(41/sec, 756 bytes) Wt(10000) RHits(31555)
Conn: CSvr(32, 0/sec) MCSvr(19) OE(13) RP(4) SQ(0)
S(10.128.49.38:80:UP) Hits(9341366, 5/sec, P[2700778, 0/sec]) ATr(4) Mbps(0.27) BWlmt(0 kbits) RspTime(120.50 ms)
Other: Pkt(42/sec, 720 bytes) Wt(10000) RHits(31556)
Conn: CSvr(37, 0/sec) MCSvr(19) OE(13) RP(9) SQ(0)
S(10.128.49.37:80:UP) Hits(9685018, 4/sec, P[2844418, 0/sec]) ATr(3) Mbps(0.23) BWlmt(0 kbits) RspTime(125.38 ms)
Other: Pkt(38/sec, 670 bytes) Wt(10000) RHits(31556)
Conn: CSvr(32, 0/sec) MCSvr(20) OE(10) RP(7) SQ(0)
<!--NeedCopy-->

Note:

The client connection counts of the individual services do not add up to the client connection count of the virtual server. The reason is because of session reuse between the NetScaler appliance and the back-end service.

Virtual Server Output

VIP(10.128.58.149:80:UP:WEIGHTEDRR): Hits(38200495, 18/sec) PHits(5) Mbps(1.02) Pers(OFF) Err(0) LConn_Best [Idx:SubIdx] 0:0 PrimVserverDownBackupHits(0)
Pkt(186/sec, 610 bytes) actSvc(4) DefPol(NONE) override(0) newlyUP(0)
Conn: Clt(253, 1/sec, OE[252]) Svr(3) SQ(Total: 0 OnVserver: 0 OnServices: 0)
slimit_SO: (Sothreshhold: 0 [Ex: 0]  Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 0] Available: 0)
<!--NeedCopy-->

The following list describes the virtual server statistics:

  1. IP (IP address:port:state:Load balancing method): The IP address and port of the Virtual IP address as configured. The virtual server state or virtual IP address is UP, DOWN, or OUT OF SERVICE; Load balancing method configured for the Virtual IP address.
  2. Hits (#):Number of requests that reached the virtual server.
  3. Mbps (#): Total traffic volume on the virtual server (Rx + Tx) converted into Mbits/s.
  4. Pers: Type of persistence is configured.
  5. Err (#): Number of times an error page was generated by the virtual server.
  6. Pkt (#/sec, # bytes): Volume of network traffic (as packets) passing through the virtual server and average packet size flowing through the virtual server.
  7. actSvc(#): Number of active services that are bound to the virtual server.
  8. DefPol (RR): Indicates whether the default load balancing method is active. Default load balancing method is used for some number of initial requests to smooth the behavior of the other methods.
  9. Clt (#, #/sec): Number of current client connections to the virtual server rate.
  10. OE [#]: Number of server connections from the virtual server in open established state.
  11. Svr (#): Number of current server connections from the virtual server.
  12. PHits (#): Number of persistence hits.
  13. SO: Number of times spillover happened.
  14. LConn_Best [Idx:SubIdx] (port:#). The index sub-slot of the best server when the least connection method is used.
  15. PrimVserverDownBackupHits (#): Number of hits to backup the virtual server when the primary server was down.
  16. Override (#): Number of times next best servers was selected based on L2Conn for maxClt.
  17. newlyUP (#): Number of current services which are newly UP.
  18. SQ(Total:OnVserver:OnServices:): Current surge queue length.
  19. slimit_SO: (Sothreshhold:Exclusive:Consumed: [Exclusive:Borrowed:TotActiveConn:] Available: (#)): Exclusive and shared info of shared limit for spillover.

In the preceding output, Svr(3) indicates that the command collects the statistical sample. There are three active connections for the virtual server to the back-end server, even though there are four services in total. When a client establishes a connection with the virtual server, it is not necessary that the client sends or receives any traffic when the command collects the information. Therefore, it is common to see the Svr counter lower than the OE[] number. The Svr counter represents the number of active connections that are actively sending or receiving data. The Subnet IP address (SNIP) is connected to the associated back-end server. And, the NetScaler tracks the virtual server connected to the back-end server and calculates the counter.

Virtual service output

S(10.128.49.40:80:UP) Hits(9443063, 4/sec, P[2602342, 0/sec]) ATr(5) Mbps(0.23) BWlmt(0 kbits) RspTime(112.58 ms) Load(0) LConn_Best [Idx:SubIdx] (C:0; V:0,I:1, B:0, X:0, SI:0)
Other: Pkt(36/sec, 712 bytes) Wt(10000) Wt(Reverse Polarity)(10000) RHits(31555) Conn: CSvr(42, 0/sec) MCSvr(20) OE(16) E(5) RP(11) SQ(0)
slimit_maxClient: (MaxClt: 2 [Ex: 0]  Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 0] Available: 2)
newlyUP_mode: NO, Pending: 0, update: 0x0, incr_time: 0x0, incr_count: 0
<!--NeedCopy-->

The following list describes the service statistics:

  1. S (IP address:port:state): IP address, port, and state of the service such as, DOWN, UP, or OUT OF SERVICE.
  2. Hits (#, P[#]): Number of requests directed to the service, Number of requests directed to the service due to configured server persistence.
  3. ATr (#): Number of active connections to the service.

    Note:

    Active connections have the outstanding request to the service or currently have traffic activity.

  4. Mbps (#.##): Total traffic volume on the Service (Rx + Tx) converted into Mbits/s.
  5. BWlmt (# kbits): Defined bandwidth limit.
  6. RspTime (# ms): Average response time of the service in milliseconds.
  7. Pkt(#/sec, #bytes): Traffic volume in terms of packets per second going to the service; Average size of the packets.
  8. Wt (#): Weight index, used in load balancing algorithm.

    Note:

    If you divide this value by 10,000, then you get the actual configured weight of the service.

  9. RHits (#): Running requests counter used in Round Robin load balancing algorithm.
  10. CSvr (#, #/sec): Number of connections to the service rate.
  11. MCSvr (#): Maximum number of connections to the service.
  12. OE (#): Number of connections to the service in the open and established state.
  13. E (#): Number of connections to the service in the established state.
  14. RP (#): Number of connections to the service, residing in the reuse pool.
  15. SQ (#): Number of connections to the service, waiting in the surge queue.
  16. Load (#): Load on the service.
  17. LConn_Idx: (Current index(#); current virtual index(#),I:(#), base virtual slot index(#), transaction (#), Sub slot index(#)): Index of the server when the least connection method is used.
  18. Wt(Reverse Polarity): Reverse Weight index used in the load balancing algorithm.
  19. slimit_maxClient: (MaxClient [Exclusinve] Consumed: [Exclusive:Borrowed:TotActiveConnection:] Available: (#)): Exclusive and shared information of shared limit for max client.
  20. newlyUP_mode: (No, pending (#), update (#*#), incr_time (#*#),incr_count (#)): Indicates whether the service has newly come up and its stats corresponding to the number of hits allowed on the new service. Also the time when the weights are updated for this service.

Collect performance statistics and event logs using the NetScaler GUI

  1. Navigate to System > Diagnostics > Maintenance > Delete/Download log files.
  2. Select a file and click Download to download the file.
How to collect performance statistics and event logs