NetScaler® ingress controller

Rate limiting in Kubernetes using NetScaler

In a Kubernetes deployment, you can rate limit the requests to the resources on the back end server or services using rate limiting feature provided by the ingress NetScaler.

NetScaler provides a Kubernetes CustomResourceDefinitions (CRDs) called the Rate limit CRD that you can use with the NetScaler Ingress Controller to configure the rate limiting configurations on the NetScalers used as Ingress devices.

Apart from rate limiting the requests to the services in a Kubernetes environment, you can use the Rate limit CRD for API security as well. The Rate limit CRD allows you to limit the REST API request to API servers or specific API endpoints on the API servers. It monitors and keeps track of the requests to the API server or endpoints against the allowed limit per time slice and hence protects from attacks such as the DDoS attack.

You can enable logging for observability with the rate limit CRD. Logs are stored on NetScaler which can be viewed by checking the logs using the shell command. The file location is based on the syslog configuration. For example, /var/logs/ns.log.

Rate limit CRD definition

The Rate limit CRD spec is available in the NetScaler Ingress Controller GitHub repo at: ratelimit-crd.yaml. The Rate limit CRD provides attributes for the various options that are required to define the rate limit policies on the Ingress NetScaler that acts as an API gateway.

Rate limit CRD attributes

Note:

In NetScaler Ingress Controller 4.2.26 and later releases, attributes of ratelimit CRD schema are updated to be under the ratelimits spec. Also, new rate limit attributes are introduced. In previous releases, ratelimit CRD schema was defined directly under spec.

The following table lists the various attributes provided in the Rate limit CRD:

CRD attribute Description
ingressclass Specifies the ingress class so that only the ingress controller associated with the specified ingress class processes the resource. Otherwise, all the controllers in the cluster will process this resource.
servicenames The list of Kubernetes services to which you want to apply the rate limit policies.
selector_keys The traffic selector keys that filter the traffic to identify the API requests against which the throttling is applied and monitored.
  Note: The selector_keys is an optional attribute. You can choose to configure zero, one or more of the selector keys. If more than one selector keys are configured then it is considered as a logical AND expression.
  path: An array of URL path prefixes that refer to a specific API endpoint. For example, /api/v1/products/.
  method: An array of HTTP methods. Allowed values are GET, PUT, POST, DELETE, HEAD, OPTIONS, TRACE, CONNECT, PATCH, or UNKNOWN_METHOD.
  header_name: HTTP header that has the unique API client or user identifier. For example, X-apikey which comes with a unique API-key that identifies the API client sending the request.
  per_client_ip: Allows you to monitor and apply the configured threshold to each API request received per unique client IP address.
ratelimits A list of rate limit configurations that define the limits and the action to take when a limit is exceeded. This attribute is separate from selector_keys, which only selects the traffic to which the limits apply. For the attributes supported under each entry, see ratelimits attributes.
streams A list of stream identifier configurations used to collect traffic analytics and tracking for observability. For the attributes supported under each stream identifier, see Stream identifier attributes.

Note:

You must specify at least one of ratelimits or streams.

ratelimits attributes

The following table lists the attributes that you can configure under each entry in the ratelimits list. The req_threshold attribute is required.

CRD attribute Description
req_threshold The maximum number of requests that are allowed in the given time slice (request rate). This attribute is required.
timeslice The time interval specified in milliseconds (multiple of 10), during which the requests are monitored against the configured limits. If not specified it defaults to 1000 milliseconds.
limittype It allows you to configure the type of throttling algorithm that you want to use to apply the limit. Supported algorithms are BURSTY and SMOOTH. The default is SMOOTH.
mode The metric to which the limit is applied. Allowed values are REQUEST_RATE, CONNECTION, and TOKEN_RATE. Use TOKEN_RATE to enforce token-based rate limiting for AI backends.
alertsintimeslice The number of alerts to raise within a time slice.
throttle_action It allows you to define the throttle action that needs to be taken on the traffic that is throttled for crossing the configured threshold.
  DROP: Drops the requests above the configured traffic limits.
  RESET: Resets the connection for the requests crossing the configured limit.
  RESPOND: Responds with the standard “429 Too many requests” response.
  NOOP: Takes no action on the requests crossing the configured limit.
redirect_url This attribute is an optional attribute that is required only if throttle_action is configured with the value REDIRECT.
logpackets Enables audit logs. It contains the logexpression and loglevel attributes, both of which are required when logpackets is used.
logexpression Specifies the default-syntax expression that defines the format and content of the log message.
loglevel Specifies the severity level of the log message that is generated.

Stream identifier attributes

The following table lists the attributes that you can configure under each entry in the streams list. The sort attribute is required.

CRD attribute Description
interval The number of minutes of data to use when calculating session statistics such as the number of requests, bandwidth, and response times. The supported range is 1–10080.
sampleCount The size of the sample from which to select a request for evaluation. To evaluate all requests, set the sample count to 1. The supported range is 1–65535.
sort Sorts the stored records by the specified statistics column, in descending order. Allowed values are REQUESTS (default), CONNECTIONS, RESPTIME, BANDWIDTH, RESPTIME_BREACHES, TOKENS, and NONE. This attribute is required.
snmpTrap Enables or disables the SNMP trap for the stream identifier. Allowed values are ENABLED and DISABLED.
appflowLog Enables or disables AppFlow® logging for the stream identifier. Allowed values are ENABLED and DISABLED.
trackAckOnlyPackets Tracks ACK-only packets. Applicable only when packet rate limiting is used. Allowed values are ENABLED and DISABLED.
trackTransactions Tracks transactions exceeding the configured threshold. Allowed values are RESPTIME, TOKENS, and NONE. When set to TOKENS, the transaction threshold attributes do not apply.
maxTransactionThreshold The maximum per-transaction value of the tracked metric. The minimum value is 0.
minTransactionThreshold The minimum per-transaction value of the tracked metric. The minimum value is 0.
acceptanceThreshold The ratio of non-breaching transactions to total transactions, expressed as a percentage. A maximum of 6 decimal places is supported.
breachThreshold The breaching transactions threshold calculated over the interval. The minimum value is 0.
log The location where objects collected on the identifier are logged. Allowed values are SYSLOG and NONE.
logInterval The time interval in minutes for logging the collected objects. It must be greater than or equal to the interval of the stream identifier. The supported range is 1–10080.
logLimit The maximum number of objects to be logged in the log interval. The supported range is 1–1000.

Deploy the Rate limit CRD

Perform the following to deploy the Rate limit CRD:

  1. Download the CRD (ratelimit-crd.yaml).

  2. Deploy the Rate limit CRD using the following command:

    kubectl create -f ratelimit-crd.yaml
    

    For example,

    root@master:~# kubectl create -f ratelimit-crd.yaml
    
    customresourcedefinition.apiextensions.k8s.io/ratelimits.citrix.com created
    
    root@master:~# kubectl get crd
    
    NAME CREATED AT
    ratelimits.citrix.com 2019-08-27T01:06:30Z
    

How to write a rate-based policy configuration

After you have deployed the CRD provided by NetScaler in the Kubernetes cluster, you can define the rate-based policy configuration in a .yaml file. In the .yaml file, use ratelimit in the kind field and in the spec section add the Rate limit CRD attributes based on your requirement for the policy configuration.

After you deploy the .yaml file, the NetScaler Ingress Controller applies the rate-based policy configuration on the Ingress NetScaler device.

Following are some examples for rate limit policy configurations.

Limit API requests to configured API endpoint prefixes

Consider a scenario wherein you want to define a rate-based policy in NetScaler to limit the API requests to 15 requests per minute from each unique client IP address to the configured API endpoint prefixes. Create a .yaml file called ratelimit-example1.yaml and use the appropriate CRD attributes to define the rate-based policy as follows:

apiVersion: citrix.com/v1beta1
kind: ratelimit
metadata:
  name: throttle-req-per-clientip
spec:
  servicenames:
    - frontend
  selector_keys:
   basic:
    path:
     - "/api/v1/products"
     - "/api/v1/orders/"
    per_client_ip: true
  ratelimits:
    - req_threshold: 15
      timeslice: 60000
      throttle_action: "RESPOND"
      logpackets:
        logexpression: "http.req.url"
        loglevel: "INFORMATIONAL"
<!--NeedCopy-->

Note:

You can initiate multiple Kubernetes objects for different paths that require different rate limit configurations.

After you have defined the policy configuration, deploy the .yaml file using the following command:

root@master:~#kubectl create -f ratelimit-example1.yaml
ratelimit.citrix.com/throttle-req-per-clientip created

The NetScaler Ingress Controller applies the policy configuration on the Ingress NetScaler device.

Limit API requests to calender APIs

Consider a scenario wherein you want to define a rate-based policy in a NetScaler to limit the API requests (GET or POST) to five requests from each API client identified using the HTTP header X-API-Key to the calender APIs. Create a .yaml file called ratelimit-example2.yaml and use the appropriate CRD attributes to define the rate-based policy as follows:

apiVersion: citrix.com/v1beta1
kind: ratelimit
metadata:
  name: throttle-calendarapi-perapikey
spec:
  servicenames:
    - frontend
  selector_keys:
    basic:
      path:
        - "/api/v1/calender"
      method:
        - "GET"
        - "POST"
      header_name: "X-API-Key"
  ratelimits:
    - req_threshold: 5
      throttle_action: "RESPOND"
      logpackets:
        logexpression: "rate exceeded, you may want to configure higher limit"
        loglevel: "INFORMATIONAL"
<!--NeedCopy-->

After you have defined the policy configuration, deploy the .yaml file using the following command:

root@master:~#kubectl create -f ratelimit-example2.yaml
ratelimit.citrix.com/throttle-req-per-clientip created

The NetScaler Ingress Controller applies the policy configuration on the Ingress NetScaler device.

Rate limiting in Kubernetes using NetScaler