Policy extensions - use cases

Certain customer applications have requirements that cannot be addressed with existing policies and expressions. The policy extension feature enables customers to add customized functions to their applications to meet their requirement.

The following use cases illustrate the addition of new functions using the policy extension feature on the NetScaler appliance.

• Case 1: Custom hash
• Case 2: Collapse double slashes in URLs
• Case 3: Combine headers

Case 1: Custom hash

The CUSTOM_HASH function provides a mechanism to insert any type of hash value in the responses sent to the client. In this use case, the hash function is used to compute the hash of the query string for a rewrite HTTP request and insert an HTTP header named CUSTOM_HASH with the computed value. The CUSTOM_HASH function implements the DJB2 hash algorithm.

Sample Usage of CUSTOM_HASH:

> add rewrite action test_custom_hash insert_http_header "CUSTOM_HASH" "HTTP.REQ.URL.QUERY.CUSTOM_HASH"
<!--NeedCopy-->

Sample Definition of CUSTOM_HASH():

    -- Extension function to compute custom hash on the text

    -- Uses the djb2 string hash algorithm
    function NSTEXT:CUSTOM_HASH() : NSTEXT

        local hash = 5381

        local len = string.len(self)

        for i = 1, len do

             hash = bit32.bxor((hash * 33), string.byte(self, i))

        end

        return tostring(hash)

    end
<!--NeedCopy-->

Line-by-line description of the above sample:

function NSTEXT:CUSTOM_HASH() : NSTEXT

Defines the CUSTOM_HASH() function, with text input and a text return value.

local hash = 5381
local len = string.len(self)

Declares two local variables:

 - hash. Accumulates the compute hash value and is seeded with the number 5381

 - len. Sets to the length of the self input text string, using the built-in string.len() function.

for i = 1, len do
      hash = bit32.bxor((hash * 33), string.byte(self, i))
end

Iterates through each byte of the input string and adds the byte to the hash. It uses the built-in string.byte() function to get the byte and the built-in bit32.bxor() function to compute the XOR of the existing hash value (multiplied by 33) and the byte.

return tostring(hash)

Calls the built-in tostring() function to convert the numeric hash value to a string and returns the string as the value of the function.
<!--NeedCopy-->

Case 2: Collapse double slashes in URLs

Collapsing double slashes in URLs improves the website rendering time, because browsers parse the single slash URLs more efficiently. The single slash URLs also to maintain compatibility with applications that do not accept double slashes. The policy extension feature allows customers to add a function that replaces the double slashes with single slashes in the URLs. The following example illustrates the addition of a policy extension function that that collapses double slashes in URLs.

Sample Definition of COLLAPSE_DOUBLE_SLASHES():

    -- Collapse double slashes in URL to a single slash and return the result
    function NSTEXT:COLLAPSE_DOUBLE_SLASHES() : NSTEXT

        local result = string.gsub(self, "//", "/")

        return result

    end
<!--NeedCopy-->

Line-by-line description of the above sample:

function NSTEXT:COLLAPSE_DOUBLE_SLASHES() : NSTEXT

Declares the COLLAPSE_DOUBLE_SLASHES() function with text input and return.

local result = string.gsub(self, "//", "/")

Declares a local variable named result and uses the built-in string.gsub() function to replace all double slashes with single slashes in the self input text.

The second parameter of string.gsub() is actually a regular expression pattern, although here a simple string is used for the pattern.

return result

Returns the resulting string.
<!--NeedCopy-->

Case 3: Combine headers

Certain customer applications cannot handle multiple headers in a request. Also, parsing of duplicate headers with same header values, or multiple headers with same name but different values in a request, consumes time and network resources. The policy extension feature allows customers to add a function to combine these headers into single headers with a value combining the original values. For example, combining the values of the headers H1 and H2.

Original request:

GET /combine_headers HTTP/1.1
User-Agent: amigo unit test
Host: myhost
H2: h2val1
H1: abcd
Accept: */*
H2: h2val2
Content-Length: 0
H2: h2val3
H1: 1234
<!--NeedCopy-->

Modified request:

GET /combine_headers HTTP/1.1
User-Agent: amigo unit test
Host: myhost
H2: h2val1, h2val2, h2val3
H1: abcd, 1234
Accept: */*
Content-Length: 0
<!--NeedCopy-->

In general, this type of request modification is done using the Rewrite feature, using policy expressions to delineate the part of the request to be modified (the target) and the modification to be performed (the string builder expression). However, policy expressions do not have the ability to iterate over an arbitrary number of headers.

The solution to this problem requires an extension to the policy facility. To do this, we will define an extension function, called COMBINE_HEADERS. With this function, we can set up the following rewrite action:

> add rewrite action combine_headers_act replace 'HTTP.REQ.FULL_HEADER.AFTER_STR("HTTP/1.1rn")' 'HTTP.REQ.FULL_HEADER.AFTER_STR("HTTP/1.1rn").COMBINE_HEADERS'

Here, the rewrite target is HTTP.REQ.FULL_HEADER.AFTER_STR(“HTTP/1.1rn”). The AFTER_STR(“HTTP/1.1rn”) is required because FULL_HEADER includes the first line of the HTTP request (e.g. GET /combine_headers HTTP/1.1).

The string builder expression is HTTP.REQ.FULL_HEADER.AFTER_STR(“HTTP/1.1rn”).COMBINE_HEADERS, where the headers (minus the first line) are fed into the COMBINE_HEADERS extension function, which combines and returns the values for headers.

Sample Definition of COMBINE_HEADERS():

    -- Extension function to combine multiple headers of the same name into one header.



    function NSTEXT:COMBINE_HEADERS(): NSTEXT

        local headers = {} -- headers

        local combined_headers = {} -- headers with final combined values
        -- Iterate over each header (format "name:valuer\r\n")

        -- and build a list of values for each unique header name.

        for name, value in string.gmatch(self, "([^:]+):([^\r\n]*)\r\n") do

            if headers[name] then

                local next_value_index = #(headers[name]) + 1

                headers[name][next_value_index] = value

            else

                headers[name] = {name .. ":" .. value}

            end

        end



        -- iterate over the headers and concat the values with separator ","

        for name, values in pairs(headers) do

            local next_header_index = #combined_headers + 1

            combined_headers[next_header_index] = table.concat(values, ",")

        end



        -- Construct the result headers using table.concat()

        local result_str = table.concat(combined_headers, "\r\n") .. "\r\n\r\n"

        return result_str

    end
<!--NeedCopy-->

Line-by-line description of the above sample:

function NSTEXT:COMBINE_HEADERS(): NSTEXT

Defines the COMBINE_HEADERS extension function, with the text input into the function from the policy expression and a text return type to the policy expression.

local headers = {} -- headers
local combined_headers = {} -- headers with final combined values

Declares local variables headers and combined_headers and initialize these variables to empty tables. headers will be a table of arrays of strings, where each array holds one or more values for a header. combined_headers will be an array of strings, where each array element is a header with its combined values.

for name, value in string.gmatch(self, "([^:]+):([^\r\n\]*)\r\n") do
. . .
end
<!--NeedCopy-->

This generic for loop parses each header in the input. The iterator is the built-in string.gmatch() function. This function takes two parameters: a string to search, and a pattern to use to match pieces of the string. The string to search is supplied by the implicit self parameter, which is the text for the headers input into the function.

The pattern is expressed using a regular expression (regex for short). This regex matches the header name and value for each header, which the HTTP standard defines as name:value\r\n. The parentheses in the regex specify the matching parts to be extracted, so the regex schematic is (match-name):(match-value)\r\n. The match-name pattern needs to match all characters except the colon. This is written [^:]+. [^:] is any character except : and + is one or more repetitions. Similarly, the match-value pattern has to match any characters except the \r\n, so it is written [^\r\n]*. [^\r\n] matches any character except \r and \n and * is zero or more repetions. This makes the complete regex ([^:]+):([^\r\n]*)\r\n.

The for statement uses a multiple assignment to set name and value to the two matches returned by the string.gmatch() iterator. These are implicitly declared as local variables within the body of the for loop.

if headers[name] then
     local next_value_index = #(headers[name]) + 1
     headers[name][next_value_index] = value
else
     headers[name] = {name .. ":" .. value}
end
<!--NeedCopy-->

These statements within the for loop put the header names and values into the headers table. The first time a header name is parsed (say H2: h2val1 in the example input), there is no headers entry for the name and headers[name] is nil.

Since nil is treated as false, the else clause is executed. This sets the headers entry for name to an array with one string value name:value.

Note: The array constructor in the else loop is equivalent to {[1] = name .. “:” .. value}, which sets the first element of the array.) For the first H2 header, it sets headers[“H2”] = {“H2:h2val1”}.

On subsequent instances of a header, (say, H2: h2val2 in the example input). headers[name] is not nil, so the then clause is executed. This determines the next available index in the array value for headers[name], and puts the header value into that index. For the second H2 header, it sets headers[“H2”] = {“H2:h2val1”, “h2val2”}.

for name, values in pairs(headers) do
    local next_header_index = #combined_headers + 1
    combined_headers[next_header_index] = table.concat(values, ",")
end
<!--NeedCopy-->

After the original headers have been parsed and the headers table filled in, this loop builds the combined_headers array. It uses the pairs() function as the for loop iterator.

Each call to pairs() returns the name and value of the next entry in the headers table.

The next line determines the next available index in the combined_headers array, and the next line sets that array element to the combined header. It uses the built-in table.concat() function, which takes as its arguments an array of strings and a string to use as a separator, and returns a string that is the concatenation of the array strings, separated by the separator.

For example, for values = {“H2:h2val1”, “h2val2”}, this produces “H2:h2val1, h2val2”

local result_str = table.concat(combined_headers, "\r\n") .. "\r\n\r\n"
<!--NeedCopy-->

After the combined_headers array are built, it concatenates the elements into one string, and adds a double \r\n that terminates the HTTP headers.

return result_str
<!--NeedCopy-->

Returns a string as the result of the COMBINE_HEADERS extension function.