Product Documentation
ADC
14.1 - Current Release 13.1 13.0 12.1
View PDF

Logs

June 26, 2024
Contributed by: 
C

Logs refer to the software-generated data that contains information about usage patterns, activities and operations within an operating system, application, server or another device. The files that store these logs are called the log files. Log files are the primary data source for network observability.

This topic covers the following:

Audit logs

Auditing is a methodical examination or review of a condition or situation. The audit logging feature enables you to log the NetScaler states and status information collected by various modules. The log information can be in the kernel and in the user-level daemons.

You can configure NetScaler to store the log information locally on NetScaler or export it to a remote server or both. For more information on audit logs, see Audit logging.

Configure audit logs

For information on how to configure NetScaler for audit logging, see Configuring NetScaler appliance for audit logging.

Sample audit log

The following is a sample audit log:

May 31 02:05:23 <local0.info> 10.106.44.21  05/31/2024:06:05:23 GMT HA1 0-PPE-0 : default API CMD_EXECUTED 7825 0 :  User admuser -
ADM_User NONE - Remote_ip 10.106.44.21 - Command "show ns license" - Status "Success"
<!--NeedCopy-->

The following image describes the audit log:

Audit log format

Export audit logs

You can export audit logs and events from NetScaler to industry standard log aggregator platforms such as Splunk and get meaningful insights. For more information, see Export audit logs and events directly from NetScaler to Splunk.

Management logs

Management logs are the logs that are generated by the processes that are running on the NetScaler management core. (For example, bash, httpd). The messages that are part of the following log files are exported as part of management logs:

  • Shell or platform logs

    • bash.log: The log file is created by the bash process. It records the commands processed by the bash shell.
    • sh.log: The log file is created by the sh process. It records the commands processed by the sh shell.

Access logs

  • auth.log: The log file records all SSHd activity and Console activity.

  • nsvpn.log: The log file records detailed information about Virtual Private Network (VPN) activities and events.

  • httpaccess.log: The log file records any HTTP activity directed to the management IP address is logged. Typically, this log file includes NetScaler GUI access, NITRO API and VPN access.

  • httperror.log: The log file records any HTTP activity directed to the management IP address and resulting in an error due to any timeout (example, PHP maximum timeout reached) or any programming error in the backend. Periodic Apache restarts also get logged into this file.

  • httpaccess-vpn.log: The log file consists of gateway portal endpoints data served by apache that are accessed by users along with date and time.

  • httperror-vpn.log: The log file consists of gateway portal endpoints data served by apache that are accessed by users along with date and time.

NetScaler management logs

  • notice.log: The log file is a syslog data stream and it records the syslog records that are generated according to the “notice” datastream of syslogd.

  • ns.log: The log file contains the NetScaler system log. The logs generated by several modules of NetScaler (for example, load balancing, SSL, app firewall, etc) are written into this file. During export, only the content of ns.log which is written by the management processes are exported. The logs written by the NSPPE processes are not exported.

Management logs are logged to respective log files by the syslogd running on NetScaler which are generated by various processes running on management core. Management logs are exported periodically, typically, every 60 seconds.

Configure export of management logs

You can configure export of management logs using one of the following methods:

  • Configure export management logs to syslog server
  • Configure export management logs to HTTP server or HEC endpoint such as Splunk

Configure export of management logs to Syslog server

To configure the export of management logs, you must perform the following steps:

  1. Create an audit syslog action on NetScaler with the management log option.
  2. Create a syslog audit policy with the syslog action.
  3. Bind the syslog audit policy with the system global entity to enable logging of all NetScaler system events.
Configure syslog audit action

To configure syslog audit action on NetScaler using CLI, run the following command:

add audit syslogAction <name> \(<serverIP> \[-serverPort <port>] -logLevel <logLevel> ... \[-managementlog <managementlog> ...] ... \[-managementloglevel <managementloglevel> ...]\[-transport \( TCP | UDP )])
<!--NeedCopy-->

Example:

add audit syslogAction test 10.106.186.102 -serverPort 514 -logLevel ALL -managementlog SHELL NSMGMT -managementloglevel ALL -transport TCP
<!--NeedCopy-->

In this configuration:

  • name: Name of the syslog action
  • serverIP: IP address of the syslog server.
  • serverPort: Port on which the syslog server accepts connections.
  • logLevel: Audit log level.
  • managementlog: Types of management logs that you must export.
  • managementloglevel: Management log levels that you want to set for export.
  • transport: The transport type used to send audit logs to the syslog server.

Note:

When management logs is enabled, the syslogAction configuration supports only server IP address and port configuration. Domain Based Service (DBS) and load balancing virtual server name configurations are not supported.

For load balancing management logs exported across multiple external syslog servers, you can use the following sample configuration:

add service syslog_server <server_ip> UDP <port>

add service syslog_server1 1.3.4.4 UDP 514

add service syslog_server2 1.3.4.5 UDP 514

add lb vserver lb1 UDP <lb_vip> <lb_port>

bind lb vserver lb1 syslog_server1

bind lb vserver lb1 syslog_server2

In syslogAction, configure the following:

add syslogAction sys1 <server_ip> -serverPort <server_port> -transport UDP -loglevel <loglevel>

add syslogAction sys1 lb_vip -serverPort lb_port -transport UDP -loglevel <loglevel>

To configure syslog audit action on NetScaler using the GUI, perform the following steps:

  1. Navigate to System > Auditing > Syslog > Server tab and click Add.
  2. On the Create Auditing Server page, provide the following details:
    1. Enter a name for the Syslog server.
    2. Select Server IP from the Server Type list and enter the syslog server IP address and port.
    3. Choose the required log levels from the Log Levels, Management Logs and Management Log Levels sections.
  3. Click Create.
Configure syslog audit policy

To configure a syslog audit policy using CLI, run the following command:

add audit syslogPolicy <name> TRUE <syslogAction>

Example:

add audit syslogPolicy test-policy TRUE test

To configure a syslog audit policy using the GUI, perform the following steps:

  1. Navigate to System > Auditing > Syslog > Policies tab and click Add.
  2. On the Create Auditing Syslog Policy page, enter a name, select Advanced Policy, and then select the audit syslog server that you created from the Server list.
Bind audit log policy

To bind the syslog audit log policy to the bind point SYSTEM_GLOBAL using CLI, run the following command:

bind audit syslogGlobal <policyname> -globalBindType SYSTEM_GLOBAL

Example:

bind audit syslogGlobal test-policy -globalBindType SYSTEM_GLOBAL

To bind the syslog audit log policy globally using the GUI, run the following command:

  1. Navigate to System > Auditing > Syslog > Policies tab and select the syslog audit policy that you created.
  2. Right-click on the selected syslog audit policy and click Advanced Policy Global Bindings.
  3. Select the syslog audit policy that you created from the Select Policy list.
  4. Enter the priority in the Priority field.
  5. Select SYSTEM_GLOBAL from the Global Bind Type field and click Bind.
  6. On the Syslog Auditing page, select the syslog audit policy and click Done.

Once the configuration is successful, the management logs are sent as syslog payloads to the external syslog server.

Configure export of management logs to Splunk configured as HTTP server

You can configure Splunk as an HTTP server. In the HTTP server configuration, you can use the HTTP event collector to send management logs over HTTP (or HTTPS) directly to the Splunk platform from your NetScaler. For more information, see Export management logs to Splunk configured as an HTTP server.

Sample management log

The following is a sample management log:

10.222.4.55 10.217.11.103 [04/Oct/2023:18:38:34 +0000] [43533] "GET / HTTP/1.1" 200 20783 "Go-http-client/l.1" "Time: 38848 microsecs"

127.0.0.1 [04/0ct/2023:16:30:00 +0000] [16510] "GET / HTTP/1.1" 200 20783 “curl/7.87.0" "Time: 42015 microsecs"

[Wed Oct 04 16:00:00.093670 2023] [authz_core:error] [pid 7487] [client 127.0.0.1:16161] AH01630: client denied by server configuration: /netscaler/ns_gui/var
<!--NeedCopy-->

The following image describes the management log:

Management log format

The following is a sample bash log:

Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command="PATH=/netscaler:/bin:/sbin:/usr/bin:/usr/sbin:/usr/libexec:/usr/local/bin:/usr/local/sbin:."
Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command="EDITOR=/usr/bin/vi"
Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command="[ -f /var/python/bin/python ]"
Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command="export PATH=$PATH:/var/python/bin"
Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command="[ -f /var/golang/bin/go ]"
Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 return_code="1"
Apr 24 11:21:00 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command=""
Apr 24 11:21:12 <local7.notice> ns bash[3893]: root on /dev/pts/0 shell_command="cat /var/log/bash.log"
<!--NeedCopy-->

The following is a sample httpaccess log:

10.102.201.155 -> 10.102.201.155 - - [10/Jun/2024:23:07:11 +0530] [1571] "GET /nitro/v1/config/route6?format=json&sessionid=[FILTERED] HTTP/1.1" 200 1162 "-" "-" "Time: 9797 microsecs"

10.102.201.155 -> 10.102.201.155 - - [10/Jun/2024:23:07:11 +0530] [1571] "GET /nitro/v1/config/nslicense?format=json&sessionid=[FILTERED] HTTP/1.1" 200 1118 "-" "-" "Time: 7774 microsecs"

10.102.201.155 -> 10.102.201.155 - - [10/Jun/2024:23:07:11 +0530] [1571] "GET /nitro/v1/config/appflowparam?format=json&sessionid=[FILTERED] HTTP/1.1" 200 1643 "-" "-" "Time: 8098 microsecs"

10.102.201.155 -> 10.102.201.155 - - [10/Jun/2024:23:07:12 +0530] [1571] "GET /nitro/v1/config/snmpmib?format=json&sessionid=[FILTERED] HTTP/1.1" 200 347 "-" "-" "Time: 8185 microsecs"

10.102.201.155 -> 10.102.201.155 - - [10/Jun/2024:23:11:16 +0530] [1570] "GET /nitro/v1/config/nslicense?format=json&sessionid=[FILTERED] HTTP/1.1" 200 1118 "-" "-" "Time: 23195 microsecs"

10.102.201.155 -> 10.102.201.155 - - [10/Jun/2024:23:11:17 +0530] [1570] "GET /nitro/v1/config/nslicenseserver?format=json&sessionid=[FILTERED] HTTP/1.1" 200 57 "-" "-" "Time: 8992 microsecs"
<!--NeedCopy-->

The following is a sample httperror log:

[Mon Jun 10 15:00:01.016898 2024] [mpm_prefork:notice] [pid 47106] AH00171: Graceful restart requested, doing restart

[Mon Jun 10 15:00:01.323746 2024] [mpm_prefork:notice] [pid 47106] AH00163: Apache/2.4.56 (Unix) mod_perl/2.0.10 Perl/v5.30.2 configured -- resuming normal operations

[Mon Jun 10 15:00:01.323756 2024] [core:notice] [pid 47106] AH00094: Command line: '/bin/httpd -f /etc/httpd.conf'

[Mon Jun 10 15:30:00.211099 2024] [authz_core:error] [pid 80696] [client 127.0.0.1:31128] AH01630: client denied by server configuration: /netscaler/ns_gui/var

[Mon Jun 10 15:38:06.205307 2024] [authz_core:error] [pid 80696] [client 127.0.0.2:24903] AH01630: client denied by server configuration: /netscaler/ns_gui/var
<!--NeedCopy-->

The following is a sample auth log:

Jun 13 12:03:33 <auth.info> ns login: login on pts/1 as root
Jun 13 12:03:33 <auth.info> ns sshd[17798]: Received disconnect from 10.102.201.155 port 33145:11: Normal Shutdown
Jun 13 12:03:33 <auth.info> ns sshd[17798]: Disconnected from user admuser 10.102.201.155 port 33145
Jun 13 12:03:33 <auth.info> ns sshd[17803]: rexec line 19: Deprecated option UsePrivilegeSeparation
Jun 13 12:03:34 <auth.info> ns sshd[17803]: Accepted keyboard-interactive/NS for admuser from 10.102.201.155 port 65127 ssh2
<!--NeedCopy-->

The following is a sample nsvpn.log:

Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) extract_ldap_attribute: retrieved mail value mail@citrix.com for Complex1, length is 15
Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) receive_ldap_user_search_event: extracted attribute, name: mail, value: mail@citrix.com
Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) receive_ldap_user_search_event: built group string for Complex1 of:group5 group4 group3 g1
Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) receive_ldap_user_search_event: User search succeeded, attempting user authentication(Bind) for <Complex1>
Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) receive_ldap_user_bind_event: Got user bind event.
Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) receive_ldap_user_bind_event: User authentication (Bind event) for user Complex1 succeeded
Jun  6 09:18:16 <local1.notice> ns [1289]: (0-15) send_accept: sending accept to kernel for : Complex1
Jun  6 09:18:16 <local1.info> ns [1289]: (0-15) aaad_alloc_serialize_keyValue_attrs:  2 custom attribute key_len 4  value_len 15
Jun  8 06:21:48 <local1.info> ns [1289]: (0-62) aaad_remove_user_login_failure_info:  sqlite3_step is SUCCESS
<!--NeedCopy-->

The following is a sample httperror-vpn log:

[Thu Jun 06 17:43:38.184137 2024] [core:error] [pid 63478] (63)File name too long: [client 127.0.0.2:42117] AH00036: access to /logon/L                                                                                                                                                       ogonPoint/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaResources/List failed (filesystem path '/var/netscaler/logon/LogonPoint/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa                                                                                                                                                       aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaResources')
[Thu Jun 06 17:44:23.338278 2024] [core:info] [pid 63475] [client 127.0.0.2:65214] AH00128: File does not exist: /var/netscaler/logon/L                                                                                                                                                       ogoqqqqnPoint/Resources/List
[Thu Jun 06 17:44:41.245211 2024] [core:info] [pid 63476] [client 127.0.0.2:56821] AH00128: File does not exist: /var/netscaler/logon/L                                                                                                                                                       og@
[Tue Jun 11 10:10:55.360797 2024] [core:info] [pid 37517] [client 127.0.0.2:53996] AH00128: File does not exist: /netscaler/ns_gui/vpn/                                                                                                                                                       pluginCustomization.json
[Mon Jun 17 14:11:15.816726 2024] [core:info] [pid 63188] [client 127.0.0.2:27594] AH00128: File does not exist: /netscaler/ns_gui/vpn/                                                                                                                                                       pluginCustomization.json
<!--NeedCopy-->

The following is a sample httpaccess-vpn log:

127.0.0.2 - - [20/Jun/2024:08:37:47 +0000] [1508] "GET /logon/LogonPoint/custom/strings.en.js?_=1718869081399 HTTP/1.1" 304 -          "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0" "Time: 529 microsecs"
127.0.0.2 - - [20/Jun/2024:08:37:47 +0000] [1508] "GET /logon/LogonPoint/plugins/ns-gateway/nsg-epa.js HTTP/1.1" 304 - "-" "Mo         zilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0" "Time: 303 microsecs"
127.0.0.2 - - [20/Jun/2024:08:37:47 +0000] [1508] "GET /logon/LogonPoint/plugins/ns-gateway/nsg-setclient.js HTTP/1.1" 304 - "         -" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0" "Time: 311 microsecs"
127.0.0.2 - - [20/Jun/2024:08:37:47 +0000] [1508] "GET /logon/LogonPoint/plugins/ns-gateway/ns-nfactor.js HTTP/1.1" 304 - "-"          "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0" "Time: 315 microsecs"
127.0.0.2 - - [20/Jun/2024:08:37:47 +0000] [1508] "GET /vpn/media/citrixgateway_logo_white.png HTTP/1.1" 304 - "-" "Mozilla/5.         0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0" "Time: 358 microsecs"
<!--NeedCopy-->

Export management logs

You can export management logs from NetScaler to industry standard log aggregator platforms such as Splunk. For more information, see Export management logs directly from NetScaler to Splunk.

Logs
June 26, 2024
Contributed by: 
C
June 26, 2024
Contributed by: 
C

In this article

Site feedback | Your privacy choices footer linkYour Privacy Choices | Privacy and legal terms | Cookie preferences | Consent settings | docs.cloud.com
© 1999- Cloud Software Group, Inc. All rights reserved.