Streaming support for request processing
NetScaler Web App Firewall supports request side streaming to provide a significant performance boost. Instead of buffering a request, the appliance examines the incoming traffic for security violation such as SQL, cross-site scripting, field consistency, field formats. When the appliance completes processing data for a field, the request is forwarded to the back-end server while the appliance continues to evaluate other fields. This data processing significantly improves the processing time in handling forms have many fields.
Citrix recommends you to enable streaming for payload content greater than 20MB. Also, the back-end server must accept the chunked requests if streaming is enabled.
Note:
Post Body Limit action is always set to block and is applicable for both streaming and non-streaming modes. If the incoming traffic is greater than 20MB, Citrix recommends you configure the
PostBodyLimitto the expected value.
Although the streaming process is transparent to the users, minor configuration adjustments are required due to the following changes:
RegEx Pattern Match: RegEx pattern match is now restricted to 4K for contiguous character string match.
Field Name Match: The Web App Firewall learning engine can only distinguish the first 128 bytes of the name. If a form has multiple fields with names that have an identical string match for the first 128 bytes, the learning engine does not distinguish them. Similarly, the deployed relaxation rule might inadvertently relax all such fields.
Removal of white spaces, percent decoding, Unicode decoding, and charset conversion are done during canonicalization to provide security check inspection. The 128 byte limit is applicable for the canonicalized representation of the field name in UTF-8 character format. The ASCII characters are 1 byte in length but the UTF-8 representation of the characters in some international languages might range from 1 byte to 4 bytes. If each character in a name takes 4 bytes for converting to UTF-8 format, only the first 32 characters in the name might be distinguished by the learned rule.
Field Consistency Check: When you enable Field Consistency, all the forms in the session are stored based on the “as_fid” tag inserted by the Web App Firewall without considering the “action_url.”
- Mandatory Form tagging for Form Field consistency: When the field consistency check is enabled, the form tag must be enabled also. The Field Consistency protection might not work if form tagging is turned off.
- Sessionless Form Field Consistency: The Web App Firewall no longer carries out the “GET” to “POST” conversion of forms when the sessionless field consistency parameter is enabled. The form tag is required for sessionless field consistency also.
- Tampering of as_fid: If a form is submitted after tampering as_fid, it triggers field consistency violation even if no field was tampered. In non-streaming requests, this was allowed because the forms can be validated using the “action_url” stored in the session.
Signatures: The signatures now have the following specifications:
-
Location: It is now a mandatory requirement that location must be specified for each pattern. All patterns in the rule MUST have a
<Location>tag. -
Fast Match: All signature rules must have a fast match pattern. If there is no fast match pattern, an attempt is made to select one if possible. Fast match is a literal string but
PCREcan be used for fast match if they contain a usable literal string. -
Deprecated Locations: Following locations are no longer supported in signature rules.
- HTTP_ANY
- HTTP_RAW_COOKIE
- HTTP_RAW_HEADER
- HTTP_RAW_RESP_HEADER
- HTTP_RAW_SET_COOKIE
cross-site scripting/SQL Transform: Raw data is used for transformation because the SQL special characters such as single quote(‘), backslash (), and semicolon (;)), and cross-site scripting tags are the same and do not require canonicalization of data. Representation of special characters such as HTML entity encoding, percent encoding, or ASCII are evaluated for transform operation.
The Web App Firewall no longer inspects both the attribute name and value for the cross-site scripting transform operation. Now only cross-site scripting attribute names are transformed when streaming is engaged.
Processing cross-site scripting Tags: As part of the streaming changes in NetScaler 10.5.e build and later, the processing of the cross-site scripting tags has changed. In earlier releases, the presence of either open bracket (<), or close bracket (>), or both open and close brackets (<>) was flagged as cross-site scripting Violation. The behavior has changed in 10.5.e build onwards. Presence of only the open bracket character (<), or only the close bracket character (>) is no longer considered as an attack. This is when an open bracket character (<) is followed by a close bracket character (>), the Cross-site scripting attack gets flagged. Both characters must be present in the right order (< followed by >) to trigger the Cross-site scripting violation.
Note:
Change in SQL violation log Message: As part of the streaming changes in the NetScaler release 10.5.e onwards, we now process the input data in blocks. RegEx pattern matching is now restricted to 4K for contiguous character string matching. With this change, the SQL violation log messages might include different information compared to the earlier builds. The keyword and special character in the input are separated by many bytes. The appliance has a track of the SQL keywords and special strings when processing the data, instead of buffering the entire input value. In addition to the field name, the log message includes the SQL keyword, SQL special character, or both the SQL keyword and the SQL special character. The rest of the input is no longer included in the log message, as shown in the following example:
Example:
In 10.5, when the Web App Firewall detects the SQL violation, the entire input string might be included in the following log message:
SQL Keyword check failed for field text="select a name from testbed1\;\(\;\)".*<blocked>
In 11.0, we log only the field name, keyword, and special character (if applicable) in the following log message.
SQL Keyword check failed for field
text="select(;)" <blocked>This change is applicable to requests that contain application/x-www-form-urlencoded, or multipart/form-data, or text/x-gwt-rpc content-types. Log messages generated during processing of JSON or XML payloads are not affected in this change.
RAW POST Body: The security check inspections are always done on the RAW POST body.
Form ID: The Web App Firewall inserted “as_fid” tag, which is a computed hash of the form is longer unique for the user session. It is an identical value for a specific form irrespective of the user or the session.
Charset: If a request does not have a charset, the default charset specified in the application profile is used when processing the request.
Counters:
Counters with a prefix “se” and “appfwreq” are added to track the streaming engine and streaming engine request counters.
nsconsmg -d statswt0 -g se_err_
nsconsmg -d statswt0 -g se_tot_
nsconsmg -d statswt0 -g se_cur_
nsconsmg -d statswt0 -g appfwreq_err_
nsconsmg -d statswt0 -g appfwreq_tot_
nsconsmg -d statswt0 -g appfwreq_cur_
_err counters: indicate the rare event which must have succeeded but failed due to either memory allocation problem or some other resource crunch.
_tot counters: ever increasing counters.
_cur counters: counters indicating current values that keep changing based on usage from current transactions.
Tips:
- The Web App Firewall security checks must work the same as before.
- There is no set ordering for the processing of the security checks.
- The response side processing is not affected and remains unchanged.
- Streaming is not engaged if clientless VPN is used.
Important:
Calculating the Cookie length: In release 10.5.e in addition to NetScaler release 11.0 (in builds before 65.x), Web App Firewall way of processing the cookie header was changed. The appliance evaluated the cookie individually, and if the length of a cookie in the cookie header exceeded the configured length, the Buffer Overflow violation was triggered. As a result, requests blocked in the NetScaler 10.5 version or earlier releases might be allowed. The length of the entire cookie header is not calculated for determining the cookie length. In some situations, the total cookie size might be larger than the accepted value, and the server might respond with “400 Bad Request”.
Note: The change has been reverted. The behavior in NetScaler version 10.5.e to version 59.13xx.e and its subsequent builds is similar to the non-enhancement builds of release 10.5. The entire raw Cookie header is now considered when calculating the length of the cookie. Surrounding spaces and the semicolon (;) characters separating the name-value pairs are also included in determining the cookie length.