Product Documentation
ADC
14.1 - Current Release 13.1 13.0 12.1
View PDF

Customize logging on the NSWL client system

October 17, 2024
Contributed by: 
C S C

You can customize logging on the NetScaler Web Logging (NSWL) client system by making more modifications to the NSWL client configuration file (log.conf). Use a text editor to modify the log.conf configuration file on the client system.

To customize logging, use the configuration file to define filters and log properties.

  • Log filters. Filter log information based on the host IP address, domain name, and host name of the Web servers.
  • Log properties. Each filter has an associated set of log properties. Log properties define how to store the filtered log information.

Sample configuration file

Following is a sample configuration file:

##########
# This is the NSWL configuration file
# Only the default filter is active
# Remove leading # to activate other filters
##########
##########
# Default filter (default on)
# W3C Format logging, new file is created every hour or on reaching 10MB file size,
# and the file name is Exyymmdd.log
##########
Filter default
begin default
        logFormat               W3C
        logInterval             Hourly
        logFileSizeLimit        10
        logFilenameFormat       Ex%`{`%y%m%d}t.log
end default
##########
# NetScaler caches example
# CACHE_F filter covers all the transaction with HOST name www.netscaler.com and the listed server ip's
##########
#Filter CACHE_F HOST www.netscaler.com IP 192.168.100.89 192.168.100.95 192.168.100.52 192.168.100.53 ON
##########
# netscaler origin server example
# Not interested in Origin server to Cache traffic transaction logging
##########
#Filter ORIGIN_SERVERS IP 192.168.100.64 192.168.100.65 192.168.100.66 192.168.100.67 192.168.100.225 192.168.100.226 192.168.
100.227 192.168.100.228 OFF
##########
# netscaler image server example
# all the image server logging.
##########
#Filter IMAGE_SERVER HOST www.netscaler.images.com IP 192.168.100.71 192.168.100.72 192.168.100.169 192.168.100.170 192.168.10
0.171 ON
##########
# NCSA Format logging, new file is created every day midnight or on reaching 20MB file size,
# and the file name is /datadisk5/netscaler/log/NS<hostname>/Nsmmddyy.log.
# Exclude objects that ends with .gif .jpg .jar.
##########
#begin ORIGIN_SERVERS
#       logFormat               NCSA
#       logInterval             Daily
#       logFileSizeLimit        40
#       logFilenameFormat       /datadisk5/ORGIN/log/%v/NS%`{`%m%d%y}t.log
#       logExclude              .gif .jpg .jar
#end ORIGIN_SERVERS

##########
# NCSA Format logging, new file is created every day midnight or on reaching 20MB file size,
# and the file name is /datadisk5/netscaler/log/NS<hostname>/Nsmmddyy.log with log record timestamp as GMT.
##########
#begin CACHE_F
#       logFormat               NCSA
#       logInterval             Daily
#       logFileSizeLimit        20
#       logFilenameFormat /datadisk5/netscaler/log/%v/NS%`{`%m%d%y}t.log
#       logtime                 GMT
#end CACHE_F

##########
# W3C Format logging, new file on reaching 20MB and the log file path name is
# atadisk6/netscaler/log/server's ip/Exmmyydd.log with log record timestamp as LOCAL.
##########
#begin IMAGE_SERVER
#       logFormat               W3C
#       logInterval             Size
#       logFileSizeLimit        20
#       logFilenameFormat /datadisk6/netscaler/log/%AEx%`{`%m%d%y}t
#       logtime                 LOCAL
#end IMAGE_SERVER

##########
# Virtual Host by Name firm, can filter out the logging based on the host name by,
##########

#Filter VHOST_F IP 10.101.2.151 NETMASK 255.255.255.0
#begin VHOST_F
#       logFormat               W3C
#       logInterval             Daily
#       logFileSizeLimit        10
logFilenameFormat /ns/prod/vhost/%v/Ex%`{`%m%d%y}t
#end VHOST_F

########## END FILTER CONFIGURATION ##########
<!--NeedCopy-->

Create filters

You can use the default filter definition in the configuration file (log.conf), or you can modify the filter or create a filter. You can create more than one log filter.

Note:

Consolidated logging, which logs transactions for which no filter is defined, uses the default filter if it is enabled. Consolidated logging of all servers can be done by defining only the default filter.

If the server hosts multiple websites and each website has its own domain name, and each domain is associated with a virtual server, you can configure Web server logging to create a separate log directory for each website. The following table displays the parameters for creating a filter.

Table 1. Parameters for creating a filter

Parameter Specifies
filterName Name of the filter. The filter name can include alphanumeric characters and cannot be longer than 59 characters. Filter names longer than 59 characters are truncated to 59 characters.
Host name Host name of the server for which the transactions are being logged.
IP ip IP address of the server for which transactions are to be logged (for example, if the server has multiple domains that have one IP address).
IP ip 2…ip n Multiple IP addresses (for example, if the server domain has multiple IP addresses).
ip6 IP IPv6 address of the server for which transactions are to be logged.
IP ip NETMASK mask IP addresses and netmask combination to be used on a subnet.
ON | OFF Enable or disable the filter to log transactions. If no argument is selected, the filter is enabled (ON).

To create a filter, enter the following command in the log.conf file:

  • filter <filterName> <HOST name> | [IP<ip> ] | [IP<ip 2...ip n> ] | <IP ip NETMASK mask> [ON | OFF]

  • filter <filterName> <HOST name> | [IP6 ip/<prefix length>] [ON | OFF]

Create a filter for a virtual server

To create a filter for a virtual server, enter the following command in the log.conf file:

filter <filterName> <VirtualServer IP address>

Example

In the following example, you specify an IP address of 192.168.100.0 and netmask of 255.255.255.0. The filter applies to IP addresses 192.168.100.1 through 192.168.100.254.

Filter F1 HOST www.netscaler.com ON
Filter F2 HOST www.netscaler.com IP 192.168.100.151 ON
Filter F3 HOST www.netscaler.com IP 192.168.100.151 192.165.100.152 ON
Filter F4 IP 192.168.100.151
Filter F5 IP 192.168.100.151 HOST www.netscaler.com OFF
Filter F6 HOST www.netscaler.com HOST www.xyz.com HOST www.abcxyz.com IP 192.168.100.200 ON
Filter F7 IP 192.250.100.0 NETMASK 255.255.255.0
Filter F8 HOST www.xyz.com IP 192.250.100.0 NETMASK 255.255.255.0 OFF
For creating filters for servers having IPv6 addresses.
Filter F9 2002::8/112 ON
Filter F10 HOST www.abcd.com IP6 2002::8 ON

<!--NeedCopy-->

Specify log properties

Log properties are applied to all log entries associated with the filter. The log property definition begins with the keyword BEGIN and ends with END as illustrated in the following example:

BEGIN <filtername>
 logFormat ...
 logFilenameFormat ...
 logInterval ...
 logFileSize ....
 logExclude ....
 logTime ….
 END
<!--NeedCopy-->

Entries in the definition can include the following:

  • LogFormat specifies the Web server logging feature that supports NCSA, W3C Extended, and custom log file formats.

By default, the logformat property is w3c. To override, enter custom or NCSA in the configuration file, for example:

LogFormat NCSA
<!--NeedCopy-->

Note:

For the NCSA and custom log formats, local time is used to time stamp transactions and for file rotation.

  • LogInterval specifies the intervals at which new log files are created. Use one of the following values:

    • Hourly: A file is created every hour.
    • Daily: A file is created every day at midnight. Default value.
    • Weekly: A file is created every Sunday at midnight.
    • Monthly: A file is created on the first day of the month at midnight.
    • None: A file is created only once, when Web server logging starts.</span>

Example:

LogInterval Daily
<!--NeedCopy-->

LogFileSizeLimit specifies the maximum size of the log file in MB. It can be used with any log interval (weekly, monthly, and so on.) A file is created when the maximum file size limit is reached or when the defined log interval time elapses.

To override this behavior, specify the size as the loginterval property so that a file is created only when the log file size limit is reached.

The default LogFileSizeLimit is 10 MB.

Example:

LogFileSizeLimit 35
<!--NeedCopy-->

  • LogFilenameFormat specifies the file name format of the log file. The name of the file can be of the following types:

    • Static: Specifies a constant string that contains the absolute path and file name.

      Dynamic: Specifies an expression containing the following format:

      • Server IP address
      • Date (%{format}t)
      • URL suffix (%x)
      • Host name (%v)

Example:

LogFileNameFormat Ex%`{`%m%d%y}t.log
<!--NeedCopy-->

This command creates the first file name as Exmmddyy.log, then every hour creates a file with a file name: Exmmddyy.log.0, Exmmddyy.log.1,…, Exmmddyy.log.n.

Example:

     LogInterval size
     LogFileSize 100
     LogFileNameFormat Ex%`{`%m%d%y}t
<!--NeedCopy-->

Caution:

The date format %t specified in the LogFilenameFormat command overrides the log interval property for that filter. To prevent a new file being created every day instead of when the specified log file size is reached, do not use %t in the LogFilenameFormat.

  • LogExclude prevents logging of transactions with the specified file name extensions.

Example:

LogExclude.html
<!--NeedCopy-->

This command creates a log file that excludes log transactions for *.html files.

LogTime specifies log time as either GMT or LOCAL.

The defaults are:

  • NCSA log file format: LOCAL
  • W3C log file format: GMT.

Understand NCSA and W3C log formats

The NetScaler supports the following standard log file formats:

  • NCSA Common Log Format
  • W3C Extended Log Format

NCSA Common log format

If the log file format is NCSA, the log file displays log information in the following format:

Client_IP_address -User_Name [Date:Time -TimeZone] "Method Object HTTP_version" HTTP_StatusCode BytesSent
<!--NeedCopy-->

To use the NCSA Common log format, enter NCSA in the LogFormat argument in the log.conf file.

The following table describes the NCSA Common log format.

Argument Specifies
Client_IP_address The IP address of the client computer.
User Name The user name.
Date The date of the transaction.
Time The time when the transaction was completed.
Time Zone The time zone (Greenwich Mean Time or local time).
Method The request method (for example; GET, POST).
Object The URL.
HTTP_version The version of HTTP used by the client.
HTTP_StatusCode The status code in the response.
Bytes Sent The number of bytes sent from the server.

W3C extended log format

An extended log file contains a sequence of lines containing ASCII characters terminated by either a Line Feed (LF) or the sequence Carriage Return Line Feed (CRLF.) Log file generators must follow the line termination convention for the platform on which they are run.

Log analyzers must accept either LF or CRLF form. Each line might contain either a directive or an entry. If you want to use the W3C Extended log format, enter W3C as the Log-Format argument in the log.conf file.

By default, the standard W3C log format is defined internally as the custom log format, shown as follows:

%`{`%Y-%m-%d%H:%M:%S}t %a %u %S %A %p %m %U %q %s %j %J %T %H %+{user-agent}i %+{cookie} i%+{referer}i
<!--NeedCopy-->

You can also change the order or remove some fields in this W3C log format. For example:

logFormat W3C %`{`%Y-%m-%d%H:%M:%S}t %m %U
<!--NeedCopy-->

W3C log entries are created with the following format:

#Version: 1.0
#Fields: date time cs-method cs-uri
#Date: 12-Jun-2001 12:34
2001-06-12 12:34:23 GET /sports/football.html 2001-06-12 12:34:30
GET /sports/football.html
<!--NeedCopy-->

Entries

Entries consist of a sequence of fields relating to a single HTTP transaction. Fields are separated by white space. Citrix recommends the use of tab characters. If a field in a particular entry is not used, a dash (-) marks the omitted field.

Directives

See the Directives table for the information about the logging process. Lines beginning with the pound sign (#) contain directives.

Example:

The following sample log file shows the log entries in W3C Extended log format:

#Version: 1.0
#Fields: time cs-method cs-uri
#Date: 12-Jan-1996 00:00:00
00:34:23 GET /sports/football.html
12:21:16 GET /sports/football.html
12:45:52 GET /sports/football.html
12:57:34 GET /sports/football.html
<!--NeedCopy-->

Fields

The Fields directive lists a sequence of field identifiers that specify the information recorded in each entry. Field identifiers might have one of the following forms:

  • identifier: Relates to the transaction as a whole.
  • prefix-identifier: Relates to information transfer between parties defined by the value prefix.
  • prefix (header): Specifies the value of the HTTP header field header for transfer between parties defined by the value prefix. Fields specified in this manner always have the type.

The following table describes defined prefixes.

Prefix Specifies
c Client
s Server
r Remote
cs Client to server
sc Server to client
sr Server to remote server (prefix used by proxies)
rs Remote server to server (prefix used by proxies)
x Application-specific identifier

Examples:

The following examples are defined identifiers that use prefixes:

cs-method: The method in the request sent by the client to the server.

sc(Referer): The Referer field in the reply.

c-ip: The IP address of the client.

Identifiers

The following table describes the W3C Extended log format identifiers that do not require a prefix.

Identifier Description
date The date on which the transaction was done.
time The time when the transaction is done.
time-taken The time taken (in seconds) for the transaction to complete.
bytes The number of bytes transferred.
cached Records whether a cache hit has occurred. A zero indicates a cache misses.

The following table describes the W3C Extended log format identifiers that require a prefix.

Identifier Description
IP The IP address and the port number.
DNS The DNS name.
status The status code.
comment The comment returned with a status code.
method The method.
url The URL.
url-stem The stem portion of the URL.
url-query The query portion of the URL.

The W3C Extended Log file format allows you to choose log fields. These fields are shown in the following table.

Field Description
Date The date on which the transaction is done.
Time The time when the transaction is done.
Client IP The IP address of the client.
User Name The user name.
Service Name The service name, which is always HTTP.
Server IP The server IP address.
Server Port The server port number
Method The request method (for example; GET, POST).
Url Stem The URL stem.
Url Query The query portion of the URL.
HTTP Status The status code in the response.
Bytes Sent The number of bytes sent to the server (request size, including HTTP headers).
Bytes Received The number of bytes received from the server (response size, including HTTP headers).
Time Taken The time taken for a transaction to complete, in seconds.
Protocol Version The version number of HTTP being used by the client.
User Agent The User-Agent field in the HTTP protocol.
Cookie The Cookie field of the HTTP protocol.
Referer The Referer field of the HTTP protocol.

Create a custom log format

You can customize the display format of the log file data manually or by using the NSWL library. By using the custom log format, you can derive most of the log formats that Apache currently supports.

Create a custom log format by using the NSWL library

Use one of the following NSWL libraries depending on whether the NSWL executable has been installed on a Windows or Solaris host computer:

  • Windows: The nswl.lib library in the \ns\bin directory on the system manager host computer.
  • Solaris: The libnswl.a library in usr/local/netscaler/bin.

  1. Add the following two C functions defined by the system in a C source file:

    ns_userDefFieldName(): This function returns the string that must be added as a custom field name in the log record.

    ns_userDefFieldVal(): This function implements the custom field value, then returns it as a string that must be added at the end of the log record.

  2. Compile the file into an object file.
  3. Link the object file with the NSWL library (and optionally, with third party libraries) to form a new NSWL executable.
  4. Add a %d string at the end of the logFormat string in the configuration file (log.conf).

Example:

##########
# A new file is created every midnight or on reaching 20MB file size,
# and the file name is
/datadisk5/netscaler/log/NS<hostname>/Nsmmddyy.log and create
digital
#signature field for each record.
BEGIN CACHE_F
    logFormat custom "%a - "%{user-agent}i" [%d/%B/%Y %T -%g] "%x"
%s %b%{referrer}i "%{user-agent}i" "%{cookie}i" %d "
    logInterval Daily
    logFileSizeLimit 20
    logFilenameFormat
/datadisk5/netscaler/log/%v/NS%`{`%m%d%y}t.log
END CACHE_F
<!--NeedCopy-->

Create a Custom log format manually

To customize the format in which log file data must appear, specify a character string as the argument of the LogFormat log property definition. The following is an example where character strings are used to create a log format:

LogFormat Custom ""%a - "%{user-agent}i" %[%d/%m/%Y]t %U %s %b %T"
<!--NeedCopy-->
  • The string can contain the “c” type control characters \n and \t to represent new lines and tabs.
  • Use the Esc key with literal quotes and backslashes.

The characteristics of the request are logged by placing % directives in the format string, which are replaced in the log file by the values.

If the %v (Host name) or %x (URL suffix) format specifier is present in a log file name format string, the following characters in the file name are replaced by an underscore symbol in the log configuration file name:

" *. /: < > ? \ |

Characters whose ASCII values lie in the range of 0-31 are replaced by the following:

%<ASCII value of character in hexadecimal>.

For example, the character with ASCII value 22 is replaced by %16.

Caution:

If the %v format specifier is present in a log file name format string, a separate file is opened for each virtual host. To ensure continuous logging, the maximum number of files that a process can have open must be sufficiently large. See your operating system documentation for a procedure to change the number of files that can be opened.

Create Apache log formats

You can derive from the custom logs most of the log formats that Apache currently supports. The custom log formats that match Apache log formats are:

NCSA/combined: LogFormat custom %h %l %u [%t] “%r” %s %B “%{referer}i” “%{user-agent}i”

NCSA/Common: LogFormat custom %h %l %u [%t] “%r” %s %B

Referer Log: LogFormat custom “%{referer}i” -> %U

User agent: LogFormat custom %{user-agent}i

Similarly, you can derive the other server log formats from the custom formats.

Arguments for defining a custom log format

The following table describes the custom log format.

Argument Specifies
%a Remote IPv4 address.
%A Local IPv4 address.
%a6 Remote IPv6 address.
%A6 Local IPv6 address.
%B Bytes sent, excluding the HTTP headers (response size).
%b Bytes received, excluding the HTTP headers (request size).
%d User-defined field.
%K Client port information.
%e1 Value of the first custom HTTP request header.
%e2 Value of the second custom HTTP request header.
%E1 Value of the first custom HTTP response header.
%E2 Value of the second custom HTTP response header. Note: For instructions on how to export customHTTP headers, see Configuring the NetScaler for Web Server Logging
%g Greenwich Mean Time offset (for example, -0800 for Pacific Standard Time).
%h IPv4 address of a remote host.
%h6 IPv6 address of a remote host.
H Request protocol.
% {Foobar}i Contents of the Foobar: header line(s) in the request sent to the server. The system supports the User -Agent, Referer, and cookie headers. The + after the % in this format informs the logging client to use the + as a word separator.
%j Bytes received, including headers(request size).
%J Bytes sent, including headers (response size).
%l Remote log name (from identd, if supplied).
%m Request method.
%M Time taken to serve the request (in microseconds).
% {Foobar}o Contents of Foobar: header line(s) in the reply. USER -AGENT, Referrer, and cookie headers (including set cookie headers) are supported.
%p Canonical port of the server serving the request.
%P The admin partition.
%q Query string (prefixed with a question mark (?) if a query string exists).
%r First line of the request.
%s Requests that were redirected internally, this is the status of the original request.
%t Time, in common log format (Standard English time format).
% {format}t Time, in the form given by format, must be in the strftime(3) format. For format descriptions, see Time Format Definition.
%T Time taken to serve the request, in seconds.
%u Remote user (from auth; may be bogus if return status (%s) is 401).
%U URL path requested.
%v Canonical name of the server serving the request.
%V6 Virtual server IPv6 address in the system, if load balancing, content switching, and/or cache redirection is used.
%D Prints the HTTP transaction ID.
%L Transaction time in milliseconds.
%R HTTP Reason string mapped to status code.
%f Source port logging.
%V Virtual Server IPv4 address.

Notes:

  • The virtual IP address of HTTP transaction is displayed as “-“ in the following scenarios:

    • Transparent mode is enabled
    • Integrated Caching is enabled, where NetScaler responds to client from Cache
    • An error response is generated

  • For instructions on how to export custom HTTP headers, see Configuring the NetScaler for Web Server Logging.

For example, if you define the log format as %+{user-agent}i, and if the user agent value is NetScaler system Web Client, then the information is logged as NetScaler system+Web+Client. An alternative is to use double quotation marks. For example, “%{user-agent}i” logs it as “NetScaler system Web Client.” Do not use the \<Esc\> key on strings from %.. .r, %. . .i and, %. . .o. It complies with the requirements of the Common Log Format. Clients can insert control characters into the log. Therefore, you must take care when working with raw log files.

Time format definition

The following table describes the time format definition to know about the format part of the %{format}t string described in the Custom Log Format table. Values within brackets ([ ]) show the range of values that appear. For example, [1,31] in the %d description in the following table shows %d ranges from 1 to 31.

Argument Specifies
%% The same as %.
%a The abbreviated name of the weekday for the locale.
%A The full name of the weekday for the locale
%b The abbreviated name of the month for the locale.
%B The full name of the month for the locale.
%C The century number (the year divided by 100 and truncated to an integer as a decimal number [1,99]); single digits are preceded by a 0.
%d User-defined field.
%K The century number (the year divided by 100 and truncated to an integer as a decimal number [1,99]); single digits are preceded by a 0.
%e The day of month [1,31]; single digits are preceded by a blank.
%h The abbreviated name of the month for the locale.
%H The hour (24-hour clock) [0,23]; single digits are preceded by a 0.
%I The hour (12-hour clock) [1,12]; single digits are preceded by a 0.
%j The number of the day in the year [1,366]; single digits are preceded by 0.
%k The hour (24-hour clock) [0,23]; single digits are preceded by a blank.
%l The hour (12-hour clock) [1,12]; single digits are preceded by a blank.
%m The number of the month in the year [1,12]; single digits are preceded by a 0.
%M The minute [00,59]; leading 0 is permitted but not required.
%n Inserts a new line.
%p The equivalent of either a.m. or p.m. for the locale.
%r The appropriate time representation in 12-hour clock format with %p
%S The seconds [00,61]; the range of values is [00,61] rather than [00,59] to allow for the occasional leap second and for the double leap second.
%3 The milliseconds [000,999]; the range of values is [000,999].
%6 The microseconds [000000,999999]; the range of values is [000000,999999].
%9 The nanoseconds [000000000,999999999]; the range of values is [000000000,999999999].
%t Inserts a tab.
%u The day of the week as a decimal number [1,7]. 1 represents Sunday, 2 represents Tuesday and so on.
%U The number of the week in the year as a decimal number [00,53], with Sunday as the first day of week 1.

Note:

If you specify a conversion that does not correspond to any of the ones described in the preceding table, or to any of the modified conversion specifications listed in the next paragraph, the behavior is undefined and returns 0.

The difference between %U and %W (and also between modified conversions %OU and %OW) is the day considered to be the first day of the week. Week number 1 is the first week in January (starting with a Sunday for %U, or a Monday for %W). Week number 0 contains the days before the first Sunday or Monday in January for %U and %W.

Display server logs

You can configure an NSWL feature to display server logs on the console or redirect server logs to a directory on the NetScaler appliance.

There are two ways to display logs on the console (standard output): Option 1: Display all logs on the console. Option 2: Display only selected logs on the console with filters with logfilenameformat as STDOUT.

Customize logging on the NSWL client system
October 17, 2024
Contributed by: 
C S C
October 17, 2024
Contributed by: 
C S C

In this article

Site feedback | Your privacy choices footer linkYour Privacy Choices | Privacy and legal terms | Cookie preferences | Consent settings | docs.cloud.com
© 1999- Cloud Software Group, Inc. All rights reserved.