NetScaler ingress controller

Remote content inspection or content transformation service using ICAP

November 13, 2024

Contributed by:

The Internet Content Adaptation Protocol (ICAP) is a simple lightweight protocol for running a value-added transformation service on HTTP messages. In a typical scenario, an ICAP client forwards HTTP requests and responses to one or more ICAP servers for processing. The ICAP servers perform content transformation on the requests and send back responses with an appropriate action to take on the request or response.

In a NetScaler setup, NetScaler acts as an ICAP client interoperating with third-party ICAP servers, such as antimalware and Data Loss Protection (DLP). When NetScaler receives incoming web traffic, it intercepts the traffic and uses a Content Inspection policy to evaluate if the HTTP request needs an ICAP processing. If yes, NetScaler decrypts and sends the message as a plain text to the ICAP servers. The ICAP servers run the content transformation service on the request message and send back a response to NetScaler. The modified messages can either be an HTTP request or an HTTP response. If NetScaler interoperates with multiple ICAP servers, NetScaler performs load balancing of ICAP servers. This setup helps when one ICAP server is not sufficient to handle all the traffic load. After the ICAP servers return a modified message, NetScaler forwards the modified message to the back-end origin server.

NetScaler also provides a secured ICAP service if the incoming traffic is of HTTPS type. NetScaler uses an SSL based TCP service to establish a secured connection between NetScaler and the ICAP servers.

In a Kubernetes environment, to enable ICAP on NetScaler through NetScaler Ingress Controller, NetScaler provides the ICAP Custom Resource Definition (CRD). By enabling ICAP, you can perform the following actions:

Block URLs with a specified string
Block a set of IP addresses to mitigate DDoS attacks
Mandate HTTP to HTTPS

After you deploy the ICAP CRD in the Kubernetes cluster, you can define ICAP policies by using the ICAP CRD attributes and also enable audit logs for statistics on NetScaler. For more information on the ICAP feature provided by NetScaler, see ICAP for remote content inspection.

ICAP policies can be broadly classified as request modification and response modification policies.

Request modification: In the request modification (REQMOD) mode, NetScaler forwards the HTTP request received from its client to the ICAP server. For an example request modification CRD resource, see sample policy configurations.

Response modification: In the response modification (RESPMOD) mode, NetScaler sends an HTTP response to the ICAP server (the response sent by NetScaler is typically the response sent by the origin server). For an example response modification CRD resource, see sample policy configurations.

For detailed information on request and response modification policies, see ICAP for remote content inspection.

Deploy the NetScaler ICAP CRD

The NetScaler ICAP CRD deployment YAML file is available in GitHub at the following location: icap-crd-deployment.yaml.

Note:

Ensure that you do not modify the deployment YAML file.

Deploy the ICAP CRD by using the following command:

  kubectl create -f icap-crd-deployment.yaml
  <!--NeedCopy-->

For example,

root@master:~# kubectl create -f icap-crd-deployment.yaml
customresourcedefinition.apiextensions.k8s.io/icap-crd.citrix.com created

ICAP CRD attributes

The ICAP CRD provides attributes to enable content inspection and configure various options required to define the ICAP policies. These CRD attributes correspond to NetScaler commands and attributes respectively.

The following table lists the CRD attributes that you can use to define an ICAP policy. Also, the table lists the corresponding NetScaler commands and attributes.

Adding ICAP servers

You can specify the ICAP servers under the icap-servers object in the ICAP CRD spec. You can specify either one or multiple ICAP servers based on your requirement.

CRD attribute	NetScaler command	NetScaler attribute	Description
ip (Mandatory)	`add service <name> <IP> <serviceType> <port>`	IP	ICAP server IP address
port (Mandatory)	`add service <name> <IP> <serviceType> <port>`	port	The port on which the ICAP server communicates
server-type (Mandatory)	`add service <name> <IP> <serviceType> <port>`	serviceType	The type of ICAP server. Possible values are `TCP` and `SSL_TCP`.

Specifying the back-end services and ingress class

Specify the Services and ingressclass attributes to do the following configuration:

Specify the back-end services for which the ICAP policy needs to be enabled.
Specify the ICAP resources that need to be processed by the ingress controller.

CRD attribute	NetScaler command	Description
Services (Mandatory)	NA	Lists the back-end services for which the ICAP policy must be enabled.
ingressclass (Mandatory)	NA	Specifies the ingress class so that only the ingress controller associated with the specified ingress class processes the resource. Otherwise, all the controllers in the cluster process this CRD resource.

Adding an ICAP profile

ICAP configurations for NetScaler are specified in an entity called the ICAP profile. The profile has a collection of the ICAP settings. The settings include parameters to dynamically generate an ICAP request, receive the ICAP response, and log content inspection data.

CRD attribute	NetScaler command	NetScaler attribute	Description
preconfigured-profile (Optional)	NA	NA	Names of the preconfigured ICAP profile.
direction (Mandatory)	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD )`	`Mode`	ICAP mode of operation. Possible values are `REQUEST` and `RESPONSE`.
uri (Mandatory)	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD )`	uri	URI representing the ICAP service.
preview	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -preview ( ENABLED \\| DISABLED )]`	preview	Enable or disable the preview header with ICAP request. This feature allows an ICAP server to see the beginning of a transaction. Then decide if it wants to opt out of the transaction early, instead of receiving the remainder of the request message.
preview-length	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -previewLength <positive_integer>`	previewLength	Value of the `Preview Header` field. NetScaler uses the smaller value between this set value and the preview size received on OPTIONS.
host-header	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -hostHeader <string>`	hostHeader	ICAP host header.
user-agent-header	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -userAgent <string>`	userAgent	ICAP user agent header.
query-params	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -queryParams <string>`	queryParams	Query parameters to be included with the ICAP request URI. Entered values must be in the `arg=value` format. For more than one parameter, add `&` between the parameters. For example, `arg1=val1&arg2=val2`.
connection-keep-alive	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -connectionKeepAlive ( ENABLED \\| DISABLED )`	connectionKeepAlive	Enable or disable sending Allow: 204 header in ICAP request.
insert-icap-headers	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -insertICAPHeaders <string>`	insertICAPHeaders	Insert custom ICAP headers in the ICAP request to send to the ICAP server. The headers can be static or can be dynamically constructed by using PI policy expression. For example, to send static user agent and client’s IP addresses, the expression can be specified as `"User-Agent: NS-ICAP-Client/V1.0r0-Client-IP: "+CLIENT.IP.SRC+"r0`. NetScaler does not check the validity of the specified header name-value. You must manually validate the specified header syntax.
insert-http-request	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -insertHTTPRequest <string>`	insertHTTPRequest	Exact HTTP request in the form of an expression, which NetScaler encapsulates and sends to the ICAP server. If you set this parameter, the ICAP request is sent using this header only. This attribute can be used when the HTTP header is not available to send or the ICAP server only needs part of the incoming HTTP request. NetScaler does not check the validity of this request. You must manually validate the request.
req-timeout	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -reqTimeout <positive_integer>`	reqTimeout	Time, in seconds, within which the remote server should respond to the ICAP request. If NetScaler does not receive the full response within this time, the specified request timeout action is performed. When the value is configured as zero, the functionality is disabled.
req-timeout-action	`add ns icapProfile <name> -uri <string> -Mode ( REQMOD \\| RESPMOD ) -reqTimeoutAction <reqTimeoutAction>`	reqTimeoutAction	Action to perform if the virtual server representing the remote service does not respond within the timeout value configured. The supported actions are: BYPASS - Ignores the remote server response and sends the request/response to the client/server. If the ICAP response with encapsulated headers is not received within the request-timeout value configured, this option ignores the remote ICAP server response and sends the full request/response to server/client. DROP - Drops the request without sending a response to the user. RESET - Resets the client connection by closing it. The client may then resend the request if desired.

Content inspection policy and action

After you enable the content inspection feature, you must add an ICAP action for handling the ICAP request information. The ICAP profile and services, or load balancing virtual server that are created are bound to the ICAP action.

CRD attribute	NetScaler command	NetScaler attribute	Description
content-inspection-criteria	`add contentInspection policy <name> -rule <expression> -action <string>`	rule	Expression that the policy uses to determine whether to perform the specified action.
default-action	`add contentInspection action <name> -type ICAP -serverName <string> -icapProfileName <string> -ifserverdown <if-server-down>`	undefAction	Action to perform if the result of the policy evaluation is undefined (UNDEF). An UNDEF event indicates an internal error condition. Only the built-in actions before this can be used.
log-action	`add contentinspection policy <name> -rule <expression> -action <string> -logAction <string>`	logAction	Name of the messagelog action to use for requests that match this policy.
operation	`add ContentInspection action <name> -type ICAP -serverip <ip> - serverport <port> -icapProfileName <string>`	type	The type of operation this ICAP action performs. The following actions are available to configure: ICAP - Forwards the incoming request or response to an ICAP server for modification. INLINEINSPECTION- Forwards the incoming or outgoing packets to IPS server for intrusion prevention. MIRROR - Forwards cloned packets for intrusion detection. NOINSPECTION - Does not forward incoming and outgoing packets to the inspection device. NSTRACE - Captures current and further incoming packets on this transaction.
server-failure-action	`add contentInspection action <name> -type ICAP -serverName <string> -icapProfileName <string> -ifserverdown <>`	ifserverdown	Action to perform if the virtual server representing the remote service is not up. The supported actions are: RESET - Resets the client connection by closing it. The client program, such as a browser, handles this action and may inform the user. The client may then resend the request if desired. DROP - Drops the request without sending a response to the user. CONTINUE - Bypasses the content inspection and resumes the traffic flow to client or server.

Goto-priority-expression

The following table provides information about the goto-priority-expression attribute, which is a CRD attribute for binding a group of policies to services.

CRD attribute	NetScaler command	NetScaler attribute	Supported values	Default value
goto-priority-expression	Bind lb vserver	gotoPriorityExpression	NEXT and END	End

How to write a policy configuration

After deploying the ICAP CRD specification provided by NetScaler in the Kubernetes cluster, you can define the policy configuration in a .yaml file. In the .yaml file, use icappolicy in the kind field and based on your requirement specify values to the attributes for policy configuration. For information on mandatory and optional parameters and their descriptions, see the preceding section.

After you deploy the .yaml file, NetScaler Ingress Controller applies the policy configuration on NetScaler.

Guidelines for the policy configuration

If the CRD is associated with a namespace then, by default, the policy is applied to the services associated with the namespace. For example, if you have the same service name associated with multiple namespaces, then the policy is applied to the service that belongs to the namespace associated with the CRD.
If you have defined multiple policies in a single .yaml file, then the first policy configuration defined in the file takes priority and the subsequent policy configurations are applied as per the sequence. If you have multiple policies defined in different files then the first policy configuration defined in the file that you deployed first takes priority.

Guidelines for the usage of Goto-priority-expression

The ICAP policies can be combined as multiple groups by using the NEXT keyword within the goto-priority-expression field.
When the goto-priority-expression field is NEXT within the current policy and if the current policy evaluates to True, the next policy in the group is executed. This process continues with the subsequent policies until a policy with the goto-priority-expression field is set to END.
When the current policy evaluates to FALSE, the goto-priority-expression has no impact, as the policy execution stops at the current policy.
The ICAP policy group within the ICAP policies begins with the policy assigned with goto-priority-expression as NEXT and includes all the consecutive policies until the goto-priority-expression field is assigned to END.
When you group ICAP policies by using goto-priority-expression, the service names bound to the policies within the group should be the same.
The last policy within the ICAP should always have the goto-priority-expression as END.
If the goto-priority-expression field is not specified for a policy, the default value of END is assigned to goto-priority-expression.

Create and verify ICAP policy

Consider a scenario where you want to define a policy in NetScaler to drop all the incoming URLs to a microservice containing the string example. Create a .yaml file called exampleicappolicy.yaml and use the appropriate CRD attributes to define the ICAP policy as follows:

apiVersion: citrix.com/v1beta1
kind: icappolicy
metadata:
  name: exampleicappolicy
spec:
  ingressclass: "cic-vpx"
  services:
    - "frontend"
  icap-servers:
    servers:
      - ip: "192.168.1.1"
        port: 1344
    server-type: "TCP"
  icap:
  - direction: "REQUEST"
    profile:
      preview: "ENABLED"
      preview-length: 1024
      uri: "http://icap.example.com/reqmod"
      host-header: "icap.example.com"
      user-agent-header: "testAgent"
      query-params: "arg1=val1&arg2=val2"
      connection-keep-alive: "ENABLED"
      req-timeout: 30
      req-timeout-action: "BYPASS"
    content-inspection-criteria: "HTTP.REQ.URL.CONTAINS(\"example\")"
    default-action: "DROP"
    goto-priority-expression: "END"
    operation: "ICAP"
    server-failure-action: "CONTINUE"
<!--NeedCopy-->

After you deploy the .yaml file by running the command kubectl create -f exampleicappolicy.yaml, NetScaler Ingress Controller applies the policy configuration on NetScaler.

On the master node in the Kubernetes cluster, you can verify the status of the applied ICAP policy CRD by running the following command:

  kubectl get icappolicies.citrix.com exampleicappolicy 
  <!--NeedCopy-->

You can view the status by running the following command:

kubectl get icappolicies.citrix.com exampleicappolicy 
NAME               STATUS    MESSAGE
exampleicappolicy   Success   CRD Activated

If there are issues while creating or applying the CRD, the same can be debugged by using the ingress controller logs.

  kubectl logs <nsic pod name>
  <!--NeedCopy-->

Also, you can verify whether the configuration is applied on the NetScaler by using the following steps.

Log on to the NetScaler CLI.
Use the following command to verify if the configuration is applied to NetScaler:

  show ns icapProfile | grep exampleicappolicy
  show icapProfile <icapProfile-name>
  show run | grep -i icap
  <!--NeedCopy-->

icap

Sample policy configurations

ICAP policy in response mode

In this example, the ICAP policy is applied to the response traffic. In other words, the policy will inspect and potentially modify the responses sent from the ICAP server to NetScaler.

kubectl apply -f - <<EOF 
apiVersion: citrix.com/v1beta1
kind: icappolicy
metadata:
  name: exampleicappolicy
spec:
  ingressclass: "cic-vpx"
  services:
    - "frontend"
  icap-servers:
    servers:
      - ip: "192.168.1.1"
        port: 1344
    server-type: "TCP"
  icap:
  - direction: "RESPONSE"
    profile:
      preview: "ENABLED"
      preview-length: 1024
      uri: "http://icap.example.com"
      user-agent-header: "testAgent"
      query-params: "arg1=val1&arg2=val2"
      connection-keep-alive: "ENABLED"
      req-timeout: 30
      req-timeout-action: "BYPASS"
    content-inspection-criteria:  "HTTP.RES.HEADER(\"Location\").CONTAINS(\"example\")"
    default-action: "DROP"
    goto-priority-expression: "END"
    operation: "ICAP"
    server-failure-action: "CONTINUE"
EOF
<!--NeedCopy-->

Pipelining multiple ICAP servers

In this example, we use multiple ICAP servers for processing requests.

kubectl apply -f - <<EOF 
apiVersion: citrix.com/v1beta1
kind: icappolicy
metadata:
  name: exampleicappolicy
spec:
  ingressclass: "cic-vpx"
  services:
    - "frontend"
  icap-servers:
    servers:
      - ip: "192.168.1.1"
        port: 1344
      - ip: "192.168.1.2"
        port: 1344
      - ip: "192.168.1.3"
        port: 1344
    server-type: "TCP"
  icap:
  - direction: "RESPONSE"
    profile:
      preview: "ENABLED"
      preview-length: 1024
      uri: "http://icap.example.com"
      user-agent-header: "testAgent"
      query-params: "arg1=val1&arg2=val2"
      connection-keep-alive: "ENABLED"
      req-timeout: 30
      req-timeout-action: "BYPASS"
    content-inspection-criteria:  "HTTP.RES.HEADER(\"Location\").CONTAINS(\"example\")"
    default-action: "DROP"
    goto-priority-expression: "END"
    operation: "ICAP"
    server-failure-action: "CONTINUE"
EOF
<!--NeedCopy-->

Multiple policy configurations

You can add multiple policy configurations in a single .yaml file and apply the policies to NetScaler. You need to add separate sections for each policy configuration as shown in the example here.

kubectl apply -f - <<EOF 
apiVersion: citrix.com/v1beta1
kind: icappolicy
metadata:
  name: exampleicappolicy1
spec:
  ingressclass: "cic-vpx"
  services:
    - "frontend"
  icap-servers:
    servers:
      - ip: "192.168.1.1"
        port: 1344
    server-type: "TCP"
  icap:
    - direction: "RESPONSE"
      profile:
        preview: "ENABLED"
        preview-length: 1024
        uri: "http://icap.example.com"
        user-agent-header: "testAgent"
        query-params: "arg1=val1&arg2=val2"
        connection-keep-alive: "ENABLED"
        req-timeout: 30
        req-timeout-action: "BYPASS"
      content-inspection-criteria: "HTTP.RES.HEADER(\"Location\").CONTAINS(\"example\")"
      default-action: "DROP"
      goto-priority-expression: "NEXT"
      operation: "ICAP"
      server-failure-action: "CONTINUE"
---
apiVersion: citrix.com/v1beta1
kind: icappolicy
metadata:
  name: exampleicappolicy2
spec:
  ingressclass: "cic-vpx"
  services:
    - "frontend"
  icap-servers:
    servers:
      - ip: "192.168.1.1"
        port: 1344
    server-type: "TCP"
  icap:
    - direction: "RESPONSE"
      profile:
        preview: "ENABLED"
        preview-length: 1024
        uri: "http://icap.example.com"
        user-agent-header: "testAgent"
        query-params: "arg1=val1&arg2=val2"
        connection-keep-alive: "ENABLED"
        req-timeout: 30
        req-timeout-action: "BYPASS"
      content-inspection-criteria: "HTTP.RES.HEADER(\"Location\").CONTAINS(\"sample\")"
      default-action: "DROP"
      goto-priority-expression: "END"
      operation: "ICAP"
      server-failure-action: "CONTINUE"
EOF
<!--NeedCopy-->

The official version of this content is in English. Some of the Cloud Software Group documentation content is machine translated for your convenience only. Cloud Software Group has no control over machine-translated content, which may contain errors, inaccuracies or unsuitable language. No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, suitability, or correctness of any translations made from the English original into any other language, or that your Cloud Software Group product or service conforms to any machine translated content, and any warranty provided under the applicable end user license agreement or terms of service, or any other agreement with Cloud Software Group, that the product or service conforms with any documentation shall not apply to the extent that such documentation has been machine translated. Cloud Software Group will not be held responsible for any damage or issues that may arise from using machine-translated content.

Was this helpful

Remote content inspection or content transformation service using ICAP

November 13, 2024

Contributed by:

November 13, 2024

Contributed by:

Remote content inspection or content transformation service using ICAP

Deploy the NetScaler ICAP CRD

ICAP CRD attributes

Adding ICAP servers

Specifying the back-end services and ingress class

Adding an ICAP profile

Content inspection policy and action

Goto-priority-expression

How to write a policy configuration

Guidelines for the policy configuration

Guidelines for the usage of Goto-priority-expression

Create and verify ICAP policy

Sample policy configurations

ICAP policy in response mode

Pipelining multiple ICAP servers

Multiple policy configurations

In this article