AI Gateway profile
The aigatewayprofile custom resource definition (CRD) lets you define an AI Gateway profile on NetScaler® using Kubernetes-native configuration. An AI Gateway profile carries the settings that NetScaler applies when it processes AI traffic between your applications and large language model (LLM) backends.
You can use the aigatewayprofile CRD to:
- Apply a frontend profile to the client-facing load balancing virtual server that receives inference requests.
- Apply a backend profile to the service groups that connect to the upstream LLM providers, with a per-model token quota and a quota refresh frequency.
- Reference the authentication token (API key) for an AI endpoint from a Kubernetes Secret.
- Reuse a preconfigured, built-in AI Gateway profile that already exists on NetScaler.
A profile is attached to a GatewayClass and is consumed by an aigatewayroute through its aigatewayprofiles field.
Note:
The
preconfiguredandconfigfields are mutually exclusive. You must specify one of them.
Deploy the aigatewayprofile CRD
Download the aigatewayprofile CRD and deploy it using the following command:
kubectl create -f https://raw.githubusercontent.com/citrix/citrix-k8s-ingress-controller/master/crd/aigateway/aigatewayprofile-crd.yaml
<!--NeedCopy-->
Note:
Do not modify the CRD deployment YAML file.
aigatewayprofile CRD attributes
The following table lists the attributes available under spec for the aigatewayprofile CRD.
| Attribute | Description | Supported values |
|---|---|---|
gatewayClassName |
Name of the GatewayClass that this AI Gateway profile is applied to. The profile is applied only when this value matches a GatewayClass managed by the controller. This field is required.
|
String |
preconfigured |
Name of a preconfigured, built-in AI Gateway profile on NetScaler to use. Mutually exclusive with config. |
String |
config |
Contains all the parameters related to NetScaler for a fully custom AI Gateway profile. Mutually exclusive with preconfigured. |
Object |
config attributes
When you define a custom profile, the following attributes are available under spec.config.
| Attribute | Description | Supported values |
|---|---|---|
profiletype |
Type of AI Gateway profile. This value is immutable after it is set. |
backend, frontend
|
endpointtype |
Type of the AI endpoint. This value is immutable after it is set. | azureopenai |
tokenquota |
Token quota limit for backend profiles. Applicable only when profiletype is backend. |
Integer (0–2147483647) |
quotarefreshfrequency |
Quota refresh rate, in minutes. Applicable only when profiletype is backend. |
Integer (1–65535) |
authtoken |
Authentication token or API key for the AI Gateway endpoint, sourced from a Kubernetes Secret. This value is immutable after it is set. Applicable only when profiletype is backend. |
Object |
authtoken attributes
| Attribute | Description | Supported values |
|---|---|---|
name |
Name of the Kubernetes Secret object that contains the authentication token. This field is required. | String matching ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
|
namespace |
Namespace of the Kubernetes Secret object. Defaults to the namespace of the AI Gateway profile. | String matching ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$
|
How to write the profile configuration
In the aigatewayprofile YAML definition, set the kind as aigatewayprofile. In the spec section, add the gatewayClassName and either a config block (for a custom profile) or a preconfigured value (to reuse a built-in profile).
Keep the following guidelines in mind:
- Specify exactly one of
configorpreconfigured. - Set
tokenquotaonly on profiles whereprofiletypeisbackend. - The
profiletype,endpointtype, andauthtokenvalues cannot be changed after the profile is created. - Store the API key in a Kubernetes Secret and reference it through
authtoken.
Sample profile configurations
Frontend profile
The following configuration creates a client-facing AI Gateway profile.
apiVersion: citrix.com/v1
kind: aigatewayprofile
metadata:
name: frontend-profile
namespace: default
spec:
gatewayClassName: aigwclass
config:
profiletype: frontend
endpointtype: azureopenai
<!--NeedCopy-->
Backend profile with a token quota
The following configuration creates a backend AI Gateway profile that enforces a token quota that is refreshed every 60 minutes.
apiVersion: citrix.com/v1
kind: aigatewayprofile
metadata:
name: backend-profile
namespace: default
spec:
gatewayClassName: aigwclass
config:
profiletype: backend
endpointtype: azureopenai
tokenquota: 100000
quotarefreshfrequency: 60
authtoken:
name: azure-openai-key
namespace: default
<!--NeedCopy-->
Preconfigured profile
The following configuration reuses a built-in AI Gateway profile that already exists on NetScaler.
apiVersion: citrix.com/v1
kind: aigatewayprofile
metadata:
name: builtin-profile
namespace: default
spec:
gatewayClassName: aigwclass
preconfigured: default-ai-frontend
<!--NeedCopy-->